Local AI

Spoiler: the "AI runs in the cloud" era is quietly ending. Ollama just crossed one million monthly active users and nobody put out a press release because the team is too busy shipping.

The Setup

If you've been living under a rock, Ollama is the docker-for-LLMs tool everyone uses but nobody talks about. One binary, one command, and you're running Llama 3 on your laptop with zero config. It quietly became the default way developers run local models.

{`# Install on Mac, Linux, or Windows
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run a model
ollama pull llama3
ollama run llama3 "Explain Astro server islands"`}

The Money Pattern

The killer feature is the OpenAI-compatible API on localhost. Swap one base URL and your existing OpenAI SDK code points at a model running on your M4 Max. No rewrites, no new clients, no vendor lock-in.

{`curl http://localhost:11434/api/generate -d '{
  "model": "llama3",
  "prompt": "Why did the developer move to the Gold Coast?",
  "stream": false
}'`}

The Catch

The GUI situation is still embarrassing. The official desktop app is fine but feels like a placeholder. The model library curation lags behind Hugging Face by weeks, so the bleeding-edge stuff lands elsewhere first.

Also: nobody warned you that pulling a 70B model would eat 40GB of disk. You will run out of space. This is fine.

The Verdict

One million users is the moment local AI stopped being a hobbyist toy. If you're still shipping every prompt to OpenAI for tasks Llama 3 can handle, you're burning money. Install Ollama today, point your existing code at localhost, do not @ me.