Apple introduces M6 and M5 Ultra

Apple never needed to participate in the AI race to zero. Because they were already at the finish line years ago building their own chips that can run large >100B parameter AI models locally.

It's possible that they're working on their own LLM that's going to work very well on their chips, and possibly outperform anything out there when they do release it.

10 years ago 32GB ram laptops sounded too much. 8 was enough. These days even I would get that much ram since it’s soldered. 64GB is higher end.

In a few years we should see such high end hardware commonplace. Working with a local LLM to get work done is the ideal way to go which has mostly hardware limitation as of now that gets solved in due time.

Ten years ago I got 64gb of ram in my laptop, same as I have now. I bought both for business and personal use. System ram capacity hasn't changed much in 10 years.

It makes me curious how old you were 10 years ago.

We were definitely outliers that long ago. I put 64 GB in a MacBook Pro back in 2019, and that was (a) overkill for everything I ever ran on that machine, and (b) stupidly expensive by 2019 standards (albeit almost affordable by 2026 standards)

Yep, Siri AI; they’re doing it in public.

The iPhone 15 was almost entirely marketed based upon AI (I would say fraudulently so, advertising features they still haven't delivered), and a huge portion of the OS work was on local AI or AI integration.

And for that matter Apple has been dumping enormous sums into their own AI development. Their failure to have a lot to show for it doesn't void the fact that they tried really, really hard.

It's bizarre how often this "Apple sat on the sidelines and let the AI people fight...so smart!" narrative appears on HN. Apple hasn't gone down the path of spending hundreds of billions on nvidia GPU data centres, but they absolutely tried really hard to matter in AI.

I mean this is not nvidia based right? It's all custom? So we can use it under Asahi perhaps?

I want to get something for my company to run local models, wondering what would be a good option.

I just want my butt ugly repairable beast machine to do the same trick. Why is my ram not unified? I have an iGPU in my server, but it can't access the 64 GB ram (I got last year for 150 euro) directly or something? It's on the CPU right? Why did only Apple go for this architecture? So many questions...

Linux runs very well in a VM on macOS. There are many good options for this, some free and open source (QEMU, UTM, Lima, Colima), some proprietary (VMware Fusion, Parallels).

But Linux in a VM doesn't get access to the real GPU, so model performance is limited. Those running on the CPU perform well, and those needing the GPU don't.

However, macOS on M-series macs is excellent for local models. (Maybe not as excellent as a box full of the best nVidia GPUs, but still excellent).

So if you're getting Apple hardware, like Linux, and want to run all of it locally, a fine setup for a machine to run local models, with agentic characteristics:

- macOS running one of the many local model runners. I used to use Ollama and Whisper, and now use llama.cpp instead of Ollama. Others use LM Studio, oMLX, etc. Provide HTTP endpoints to access the models.

- Linux in a VM for overall control and orchestration, with standard VM settings, and bridged networking so it appears as its own machine on your network. Also, in here provide a robust shared file server for shared state. Use this VM as your desktop and primary access to the machine, if you like Linux.

- Linux in a VM to launch ephemeral, volatile containers, with the containers using a memory-only tmpfs overlay on top of a read-only Linux filesystem in a VM disk image, with tools in this filesystem. Alternatively, a writable Linux filesystem in a VM disk image, with disk buffering set to use macOS host buffering and discard fsync requests. These settings optimise for container disk performance for data that's only ephemeral which will be deleted soon or on system shutdown. (You can combined both VMs, but need to use two VM disks to get equivalent behaviour, and be careful about VM disk configuration of the two disks.)

- Containers spawned within that second Linux VM can be spawned very quickly and run quickly, so are ideal for LLM agents that need a quick sandbox. These sandboxes generally run faster than a macOS sandbox, despite being on the same machine with VM overhead, because Linux is faster at some things. Teach the LLMs to store files and memories they want to keep in the shared file server.

But you get a generic computer and much more RAM.

And you lose a couple of organs.

It would take Apple one or two engineers to make Linux life much easier on macs. But Linux is outside their walled garden so it's ignored.

My Mac Mini is strictly a headless server for llama.cpp.

I use a Linux workstation.

If I were limited to use Mac hardware , I would install Linux in VMware Fusion and work from there.

My old 3090 is typically significantly faster (almost 2x token/s) than my M4 Max 128GB machine, as long as the model fits in the 24GB of VRAM.

In most situations it's a better idea to just buy tokens. But there are definitely cases when that's not an option. And then a machine like the M5 Ultra can allow you to do things locally for a fairly limited budget. And in a simpler package to manage than a machine with multiple GPUs.

* the iPhone * the iPad * apple watch * airpods * unified memory laptops and computers

Those are all products that either created a category or changed that industry.

They weren't the first smart phone, tablet, smart watch, or true wireless earbuds. They did a damn fine job making each of those though. I am typing this on a my work macbook wearing AirPods, and AppleWatch, listening to audio on my iPhone. Apple does a really good job with their products.

Realizing how surrounding by Apple I am...

FWIW, Ollama, LM Studio and Lemonade (and oMLX) also wrap Apple's MLX framework.

It's the Hardware, Software and Services in combination. None would work without the other (to reach the scale apple is)

More specifically, they're a hardware dongle company first

not getting on that bandwagon but wasn't that not the most demanding game as its a just a nonstop cutscene.

It's a very deliberate choice when if it doesn't make sense to gamers.

* it's critically acclaimed (86 metacritic, 10/10 IGN)

* whatever person decided this likely knows nothing about video games

* most importantly it's a modern game in UE5 that's COMING NATIVELY TO MAC, including to the App Store

What would you have put?

In US its $4000 upgade so $25 for 1GB.

Also:

> 512GB memory option for M5 Ultra coming late October

New Hampshire, Oregon, Montana, Alaska, Delaware.

So, on the mini the RAM upgrade runs at 25$ per GB on all tiers, the same as the Studio therefore the upgrade to 512 will probably cost 6400$.

The fully maxed out Apple Studio then will be 24699$. It's 17199$ if you don't upgrade the storage(1TB).

Nevertheless I itch to have one :)

EDIT: or buy AAPL. If I had bought Apple stock instead of buying a Mac LC II in 1992, then I would have about $2 million in Apple stock.

Unless you need privacy for your inference this instant, paying for credits can get 80 to 90 percent of people everything they need.

Of course if you do need that privacy, then forking the $25K over to Apple is a no brainer.

There are both cheaper and faster options out there.

If treasuries “fail” we have a different class of problem.