"I think the scenario being described may have a lot of validity: a lot of folks, for a wide variety of reasons, will want to have their own local processing power for inference models rather than cloud LLM inference. And that it appears that the memory and processing power and power efficiency of doing that will be in reach of a very large population. If not quite now, then very soon. I for one will switch to that configuration as soon as I can get a local model that will do local image analysis and tagging without undue hassle. I've already looked into it, but the hassle hurdle is high for now."

According to Cult of Mac, the new Mac Studio with an M3 Ultra chip, which supports up to 512GB of unified memory, is the easiest and cheapest way to run powerful, cutting-edge LLMs on your own hardware. The latest DeepSeek v3 model, which sent shockwaves through the AI space for its comparable performance to ChatGPT, can run entirely on a single Mac.

There’s a huge advantage to running these models locally. You don’t have to pay any money using a cloud-based AI service; the only cost is the price of the machine and its electricity. There isn’t any network latency, usage limits or security concerns that you get working across the internet on someone else’s server. Its unclear how many parameters it will support.

Bottom line: Mac mini is a machine that fits comfortably on your desk, rather than a server farm; it costs the same as a used Honda Civic, not a new Lamborghini. And multiple Mac Minis can be daisy-chained to run even more complex models, so it is an easily scalable solution. Sort of like adding hard drives to a server for more space. This will likely be a growth area for Mac sales as data privacy remains a concern for many AI users.

cultofmac.com - Mac studio AI performance

abromber