building-a-coding-agent-from-scratch-course

https://github.com/user-attachments/assets/f8711e5b-05e2-4c31-aa7f-355374c332d8

Try the finished agent first — 5 minutes, $0:

git clone https://github.com/decodingai-magazine/building-a-coding-agent-from-scratch-course.git
cd building-a-coding-agent-from-scratch-course
make install
cp .env.example .env   # set LLM API key
uv run decode

Then type /demo- and pick a demo. See what they do below. Full setup guide.

📖 About This Course

In LangChain's Terminal-Bench experiment, changing only the harness (with the same model) moved a coding agent from ~30th place into the top 5: the harness, not the model, is what makes a coding agent good.

The agent is ~20 lines. The course is everything else, known as the harness.

agent = Agent(
    build_model(settings.llm_provider),        # gemini | openrouter | modal
    deps_type=AgentDeps,                       # cwd, event sink, permission gate
    output_type=[str, DeferredToolRequests],   # final answer, or tools paused for approval
)
register_tools(agent)                          # read, edit, bash, grep, ...

async with agent.iter(prompt, message_history=history) as run:
    async for node in run:                     # model request → tool calls → repeat
        stream_events(node)

That's the entire tool-calling agent. Everything else in this repo (the tools, skills, permission layer, sandbox, steering queue, memory, compaction, session recording & replay, remote execution, subagent fan-out, and evals) is the harness. That's what you're here to build.

We spent months under the hood of Claude Code (via its leaked source), OpenCode, Pi, and Aider, then distilled what we learned into 8 articles and 6 videos where you'll build decode, your own coding agent, from scratch. One headless core hooked to two modes: an interactive TUI and Modal serverless functions running N copies in parallel, fired by CLI, webhook, or cron.

[!WARNING] Building a coding harness from scratch can make you dangerously good at building any other AI product, whether in finance, medicine, or e-commerce.

🎮 See It Work

The finished agent ships with demo skills under .decode/skills/. Open the TUI, type /demo-, pick one, and watch the harness you're about to build do real work:

🤖 You'll Walk Away Knowing How To

• Design a coding agent harness from scratch (article 1 · video 1)
• Implement the coding agent loop as a headless harness (article 2 · video 1)
• Attach the headless harness to multiple interfaces: a terminal UI, a CLI, or even a remote runtime (article 6 · video 4)
• Execute the agent's tools within local Docker or remote Modal sandboxes (article 3 · video 2)
• Host open-source models as SGLang servers on Modal (article 2 · video 1)
• Implement guardrails by adding a permission layer (article 1 · video 1)
• Build essential context engineering techniques: memory, compaction, skills (article 4 · video 3)
• Hook up an LSP server for faster feedback loops (article 4 · video 3)
• Implement a configurable agents catalog: build, plan, code reviewer, and exploration agents (article 5)
• Deploy the headless harness on Modal, triggering remote agents via the CLI, a webhook, or a cron job (article 6 · video 4)
• Spawn parallel subagents via fan-out strategies (article 5)
• Add observability (article 2 · video 1)
• Design an eval harness for benchmarking the agent and checking for regressions (article 7 · video 5)
• Organically grow your regression suite from failed agent traces (article 8 · video 6)
• Reproduce agent failures and check for regressions when changing your prompts or models by replaying traces with Kitaru (article 8 · video 6)

Tech Stack

The code is written in Python, with the following frameworks and libraries:

• Agent Framework: Pydantic AI
• LLM Providers: Modal (open weights you serve yourself via SGLang), OpenRouter (open weights as a service), or Gemini (proprietary).
• Session Recording & Replays: Kitaru
• Observability & Evals: Opik
• Sandboxing: local Docker & remote Modal sandboxes
• Deploying: Modal as headless agents (fired by hand, by cron, or by webhook)

Otherwise, we build all the functionality from scratch to teach you the foundations that last, not frameworks that abstract away the hard parts.

📬 Learn Harness Engineering

Join 45k+ engineers subscribed to the Decoding AI Magazine to learn to build coding agents from scratch.

💡 The code tells you what. The lessons tell you why.

For the full experience, go through the articles and videos that cover what the code can't. The why behind every decision.

• What the essential components of a coding agent are, and what is optional. (article 1 · video 1)
• Why we have a headless harness and two interface modes: TUI + Remote. (article 1 · video 1)
• Why we plugged in 9 tools, no more, no less. (article 2 · video 1)
• What guardrails are actually useful. (article 3 · video 2)
• Why compaction fires at ~80% of the window instead of at the limit. (article 4 · video 3)
• Why subagents are context engineering: scoped windows instead of one bloated context. (article 5)
• Why your agents should keep working after you close your laptop lid. (article 6 · video 4)
• Why you need both benchmarks and regression tests. (article 7 · video 5)
• Why we record every run, and what a replay buys you that a re-run doesn't. (article 8 · video 6)

📚 Course Outline

🛠️ Keep Building From Scratch

Enjoyed building Claude Code from scratch? Keep going with CodeCrafters, where you can rebuild real developer tools like Redis, Git, a Shell, or an HTTP Server through step-by-step challenges with automated tests, in your language of choice. Signing up is free, and my affiliate link gets you 40% off if you upgrade. Or start with their free build-your-own-x guides.

👥 Who Should Join?

Engineers who learn by building. You finish with a working coding agent that teaches you harness engineering patterns to steal for your own agentic applications.

Best for ML/AI engineers who want to level up their craft and for software engineers and data scientists who want to transition into building agentic systems from scratch.

🎓 Prerequisites

Category	Requirements
Skills	- Python (Intermediate) - LLMs & agents (Beginner)
Hardware	Any modern machine will do. No GPU required, as we run all the LLMs in the cloud.
Level	Intermediate (but with a little sweat and patience, anyone can do it)
Time	~4–8 hours for the whole course — 4 if you read and watch, 6–8 if you run everything.

💰 Cost Structure

Running the code costs $0 if you stick to free tiers:

Service	Cost
Gemini API (default provider — easy setup, but limited API requests)	free tier (Google AI Studio)
Modal (recommended provider, remote sandbox, remote agents)	$30 free credits — enough to run the course
OpenRouter (alternative provider)	$0 on :free models (optional $10 credit raises the daily cap)
Opik (tracing + evals)	free tier
Kitaru (recording + replays)	free — the OSS server runs on your laptop (make kitaru-local)

Only reading? Everything's free!

⚙️ How It Works

As an open-source course, it is entirely self-paced and based on this repository, plus the attached lessons that walk you through the code. No paywall. No platform.

Read the lessons on the Decoding AI Magazine, watch the videos on the Decoding AI Channel, run the code on your own machine, break it, fix it, and learn from the process.

🏗️ Project Structure

One Python package; each module maps to one part of the architecture:

.
├── docs/
│   ├── adr/                  # Architecture Decision Records — the "why" of every choice
│   └── glossary.md           # one canonical name per concept
├── evals/                    # benchmark tasks + regression cases + the Opik harness
├── scripts/                  # operator scripts: Kitaru bootstrap + eval table generation
├── tests/{unit,integration}/ # mirrors src/ 1:1; milestone capstones prove each milestone
└── src/decode/
    ├── cli.py                # Click entrypoint → launches the TUI
    ├── tui/                  # input: prompt_toolkit · output: Rich
    ├── harness/              # message queue + priority gate around the loop
    ├── agent/                # the Pydantic-AI ReAct loop (LLM ⇄ tools)
    ├── agents/               # agents catalog: Build / Plan / Code-Reviewer + Explore subagent
    ├── tools/                # file I/O, bash, web, todo, skills dispatch, LSP, ask_user
    ├── permissions/          # allow/ask/deny · modes · settings.json
    ├── sandbox/              # bash + file tools seam: none (host) / docker / modal
    ├── services/lsp/         # hand-rolled stdio LSP client (ty)
    ├── runtime/              # plain headless decode run + the Recording Seam
    ├── remote/               # decode remote deploy|run|attempts|logs — the Modal Headless App (CLI, webhook, cron)
    ├── context/              # compaction + conversation log (JSONL)
    ├── memory/               # AGENTS.md / MEMORY.md loading + write-back
    ├── observability/        # Opik tracing
    └── config/, entities/    # settings singleton · shared models

🚀 Running the Code

The guides live under running_the_code/. Follow them in order; each ends with a link to the next:

Guide	What's inside
00_troubleshooting.md	Every known failure, and its fix
01_install_and_usage.md	Start here: install, one key, first session
02_modal_endpoints.md	Serving open models on Modal
03_sandboxing.md	Docker (local) / Modal (remote) sandboxing + the sandbox git token
04_deploy.md	The headless harness on Modal — CLI, webhook, cron
05_evals.md	Benchmarks and regression cases on Opik
06_evals_replays.md	Kitaru on your laptop: record, replay & the full evals loop

🤝 Sponsors

🗣️ Questions and Troubleshooting

Open a GitHub issue for course questions, setup trouble, or concept clarifications. Known gotchas are documented in the running_the_code/ guides.

❓ FAQ

Do I need a paid API key? No. The default Gemini provider has a free tier, OpenRouter routes across :free models, and Modal gives $30 in credits — see Cost Structure.

Why Python and not TypeScript or Go? Accessibility: our audience knows Python. The course focuses on the design decisions, which transfer to any language.

Why build from scratch instead of extending Pi, DeepAgents, or an existing harness? Because adding custom logic to an existing harness is the easy part. Knowing what to add requires understanding the internals. Those fundamentals are what still make AI engineers valuable. Build a coding agent once and you're equipped to build a custom agent for any use case.

🥂 Contributing

Found a bug and know the fix? Fork, fix, run make ci (no API key needed), and open a pull request. Future readers will thank you 🤗

👨‍🏫 Course Author

📬 Learn Harness Engineering

Join 45k+ engineers subscribed to the Decoding AI Magazine to learn to build coding agents from scratch.

⭐ One More Thing

If you found this course useful, consider starring the repository so others can find it too.

License

Released under Apache-2.0. Clone, fork, and build on it; keep the LICENSE and credit this repo.