Local AI

If you've been living under a rock, Jan is the open-source ChatGPT-alike for local models that doesn't look like a 2009 forum. Spoiler: it's the first local AI app I've left running in the dock without flinching.

The Setup

Jan is an Electron app with a real designer behind it. It runs llama.cpp under the hood, supports OpenAI/Anthropic remote keys if you want them, and ships an OpenAI-compatible local API server on port 1337. Behold: a local ChatGPT clone that doesn't make you want to close your laptop.

{`{
  "id": "llama3-70b-q4",
  "name": "Llama 3 70B Instruct",
  "model": "llama-3-70b-instruct.Q4_K_M.gguf",
  "engine": "nitro",
  "parameters": {
    "ctx_len": 8192,
    "temperature": 0.7,
    "top_p": 0.9,
    "ngl": 100
  }
}`}

The Money Pattern

Point any OpenAI client at `http://localhost:1337/v1` and Jan is your backend. I've got Cursor on an M4 Mac talking to a local Qwen model through Jan, and it's a genuinely usable coding loop with zero API spend.

{`from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:1337/v1",
    api_key="jan-doesnt-need-a-key",
)

resp = client.chat.completions.create(
    model="llama3-70b-q4",
    messages=[{"role": "user", "content": "refactor this astro component"}],
    stream=True,
)
for chunk in resp:
    print(chunk.choices[0].delta.content or "", end="", flush=True)`}

The Catch

The model catalog is smaller than LM Studio's — fewer one-click GGUFs, less granular quant selection. Extensions/plugins are still maturing. And the auto-update cycle on the desktop app is aggressive enough that you'll see a "new version" toast every other launch. Minor stuff, but real.

The Verdict

If you want a local AI app you can keep open without aesthetic shame, Jan is the pick. It's the closest thing the open-source community has to a polished consumer product. Install it, point Cursor at it, never pay for a cheap coding assistant again.