Apple spreads MLX inference across Macs
Apple demonstrates Muse Spark Open running locally on four M5 Ultra Mac Studios.
In this article
The measured configuration
Apple’s experiment splits each model layer across machines, rather than assigning successive layers to different computers. Its benchmark uses an 8,192-token context, NVFP4 precision, 128 generated tokens and a batch size of one. The command explicitly enables experimental fast CPU–GPU synchronization with MLX_METAL_FAST_SYNCH=1. This describes Apple’s measurement configuration; CurioFuse has not independently tested it.
A warning and connection requirements
The connection manual recommends leaving fast synchronization unset: it is disabled by default and can deadlock, leaving the graphics processor stuck. For communication, its JACCL backend uses Thunderbolt 5 RDMA, available from macOS 26.2. Enabling RDMA requires local macOS Recovery access and a subsequent reboot; remote activation is unavailable. Every pair of Macs needs a direct cable connection. The configuration helper uses SSH to inspect that network, supplies setup commands, and generates the host file used to launch distributed work.
Managing prompt memory
The package documentation explains another practical control: prompt processing happens in steps. Smaller steps reduce peak memory while reading the prompt, at some cost to processing speed. Its step-size option applies to generation, chat, serving and benchmarking commands, so this trade-off concerns input processing rather than a promised overall performance gain.
Sources(3)
- Experimental configurationChecked:
- Connection requirementsChecked:
- Prompt memoryChecked:
