Local AI

If you've been living under a rock, Apple's MLX team has been quietly turning Macs into fine-tuning boxes. I ran a 7B LoRA on my M4 overnight — no cloud GPU, no rented H100, no surprise invoice.

The Setup

MLX's LoRA tool ships as a CLI inside mlx-lm. It eats JSONL chat data, attaches LoRA to the attention projections, and trains against the unified memory pool. On an M4 Max with 64GB you can comfortably touch 8B. On a base M4 with 16GB, stick to ~3B.

pip install mlx-lm

# data/train.jsonl — one object per line:
# {"messages":[{"role":"user","content":"..."},{"role":"assistant","content":"..."}]}

mlx_lm.lora   --model mlx-community/Meta-Llama-3.1-8B-Instruct-4bit   --train   --data ./data   --iters 600   --batch-size 2   --lora-layers 16   --learning-rate 1e-4

The Money Pattern

This is the script I actually used on the Aidxn Design support corpus. Run it before bed, wake up to an adapter, fuse it back into the base for distribution. The fused model runs in Ollama or LM Studio without anyone knowing it was trained on a laptop.

# evaluate the adapter
mlx_lm.lora   --model mlx-community/Meta-Llama-3.1-8B-Instruct-4bit   --adapter-path ./adapters   --test   --data ./data

# fuse the LoRA into a standalone model for shipping
mlx_lm.fuse   --model mlx-community/Meta-Llama-3.1-8B-Instruct-4bit   --adapter-path ./adapters   --save-path ./aidxn-support-8b

The Catch

You're stuck under about 13B if you want comfortable iteration. 70B fine-tunes still need real GPUs — MLX can do them with QLoRA tricks but the wall time becomes "leave for a weekend". Also throughput on Mac is lower than a 4090; this is a "while you sleep" workflow, not a "ten experiments before lunch" one.

The Verdict

For 3-8B adapters trained on small private datasets — exactly the shape most product fine-tunes have — an M-series Mac is now a legitimate training box. No cloud bill, no data leaving the laptop, no procurement ticket. Plug in the dataset, run mlx_lm.lora, ship the adapter.