TL;DR: Big Tech steals all of human knowledge, packages it all up into models and sells it back to you, making billions off other people’s work. The heist makes the Napster case look insignificant in comparison. Bernie Sanders thinks AI company should be charged for what they stole.
The video’s title is terrible and it’s complete gibberish to me, but the video is interesting.
FWIW, Ari Melber is right on the big AI steal.
I bought an AI rig this week and I installed Ollama and a few different models for coding and for general queries, and they basically have very decent and accurate answers for almost everything I ask them - particularly technical points. Clearly they contains a wealth of knowledge from other computer professionals, yet the models work totally offline.
In other words, this machine lets me tap into everybody’s know-how in my field without even being online. It’s like having the internet at home. Amazing! But that also means nobody got paid for what’s inside the models.
I am curious about the specs/cost of your rig. Mind if you share?
Not at all.
It’s a Gigabyte AI Top PC. It’s one of several such mini-PCs based on the Nvidia Blackwell GB-10 processor, in the emerging category of “personal AI supercomputer”. It’s not cheap but it’s not out of reach either.
I got my company to buy me this thing on the argument that the company shouldn’t use cloud services if possible on our sensitive code, and American cloud services even less, for obvious political and moral reasons. I argued that we should investigate the capabilities of this machine - or one like it - either to use as a shared AI server, or as an individual machine for all the developers here to use as locally-hosted AI coding assistants. My pitch made sense to my boss, so the company bought me this machine for me to investigate.
I’ve been using it since Monday for development and other things, such as translations, computer vision, general chatty thing, but also for traditional applications such as CAD, Blender, video editing and gaming too 🙂 I’ve been trying different models for different things, see how they perform, how snappy they are… I’m impressed enough that I plan on buying one for me and my wife at home. That one or maybe a slightly cheaper equivalent based on the same processor / memory, but without the connectors I don’t need. It costs a little over 5 grand.
Cool, thanks for the info. When I saw the memory bandwidth for the integrated systems like these, I thought perhaps it wouldn’t be worth it over a custom build with a couple of 5090s (2.5x the price before the memory price hike). What models have you thrown at it? What’s your tokens/second? The local-sized models keep getting more capable.
My potato 2060 plus 64GB of system RAM is enough to crawl along with some of the MoE models, but I’ve got projects that need way more compute (would like to content filter my torrent crawler database, finetune of one of the small Qwens to produce my novel programming language, ML experiments that currently take days/weeks to iterate, etc). Maybe an integrated system like yours is worth it after all.