Applied LLM / Agents

Cipher

A terminal coding agent that runs on a local fine-tuned model, or falls back to a frontier model when you want one.

Year: 2026
Stack: Python · Anthropic API · Ollama (local) · Fine-tuning

The idea

Most coding assistants are someone else’s service running on someone else’s hardware, billing you per token and seeing every line of your code. Cipher is an experiment in the opposite: a coding agent you own end to end, that can run entirely on a local, fine-tuned model, and reaches for a frontier model only when you decide the task warrants it.

In active development. The repository is private and intentionally not linked.

Architecture

Prompt terminal

Agent loop tools · plugins

Backend local Ollama · or Claude

Tool calls read · write · run

Response session-logged

Fig. 1 · Request flow (cost tracking and auto-compaction run across the whole loop)

The piece I care most about is the model-agnostic backend. The same agent loop drives either a local model served through Ollama or the Anthropic API behind a single switch, so the local fine-tune handles the routine work for free and offline, and a frontier model is one environment variable away when a task needs the extra horsepower.

Decisions worth surfacing

Auto-compaction instead of context cliffs. Long agent sessions blow through a context window fast. Cipher watches the window and, once it crosses a configurable fraction (85% by default), compacts the conversation rather than truncating it, so a session can run long without falling off a cliff or silently dropping early context.

Cost and budget as first-class state. Every session tracks token spend against an optional hard budget. When you’re routing some calls to a paid API, knowing what a run costs, and being able to cap it, is the difference between a tool you trust to run autonomously and one you babysit.

Plugins and an autonomous mode. Tooling is extensible through a plugin directory, and an opt-in autonomous mode lets the agent carry a task across multiple steps without a human in every loop, bounded by the same budget and compaction guards.

Status

Cipher is a personal R&D project for learning where a small, locally fine-tuned model is genuinely good enough for day-to-day coding work and where a frontier model still earns its cost, with the switch between them under my control rather than a vendor’s.