Code mode lost its first race. Then we gave it muscle memory

Mark Hetherington

Member of Technical Staff

TL;DR

Get started!

I've known the team behind Blacksmith since before it had a name, and in all that time I could never answer one question:

What would I even do there?

I am a product guy. CI is infrastructure.

Then Aayush, one of the co-founders, sketched my own day back at me. I work on my laptop. I push, and CI rebuilds a context I already had. I hand the same repo to an agent, and it starts cold on a third machine. Three environments, one codebase, all pretending to be the same machine. That was the lightbulb: every inch of that gap is a product problem.

codesmith is where that journey starts for me: a general-purpose coding agent built on the same components your CI already runs on. Closing that gap is bigger than any one feature. But one requirement showed up immediately. Like everything Blacksmith ships, codesmith had to be fast. The story of making it fast starts with a race we lost.

Making codesmith fast.

Code mode changes how agents call MCP tools: the connectors they use to reach apps like Linear. Traditionally, the agent is handed raw JSON tool definitions:

{
  "name": "list_issues",
  "inputSchema": {
    "type": "object",
    "properties": { ... }
  }
},
{
  "name": "list_comments",
  "inputSchema": {
    "type": "object",
    "properties": { ... }
  }
}

In code mode, it sees the same tools as code:

class Linear {
  list_issues(input): ...
  list_comments(input): ...
}

The full tool catalog does not sit in the model's context window. The agent searches for the tools it needs, sees their interfaces as TypeScript, and calls them through one execute tool inside a sandbox.

This matters because models have trained on billions of lines of TypeScript. Tool calling is new by comparison.

And because the tools are just code, the promise is that the model can replace the call, wait, call loop with one small script that does the whole job. The script runs inside the sandbox, and only the final result comes back to the model.

Here is one Linear task routed two ways:

Find my 8 most recently updated open Linear issues, fetch comments for each one separately, report counts and latest authors.

First, the classic loop. Watch where every payload goes:

Chat dataflow · classic loop

classic loop

Pause

chat

find my 8 latest issues + comments for each

list_issues→ raw JSON · 8 issues

list_comments#1→ raw JSON

list_comments#2→ raw JSON

list_comments#3→ raw JSON

list_comments#4→ raw JSON

list_comments#5→ raw JSON

list_comments#6→ raw JSON

list_comments#7→ raw JSON

list_comments#8→ raw JSON

Here are your 8 issues, with comment counts and latest authors.

dataflow
tasklist_issuesanswerlist_comments ×8model context52.8k+0k

data packet
tool call 8 issues → comments → report

The chat logs each call with a one-line result, but the full raw payload still detours through model context, which swells 52.8k → 101.1k (+48k).

In that loop, every raw payload crossed model context. Code mode keeps the same work inside a sandbox instead, so only the summary crosses back:

Chat dataflow · code mode

code mode

Pause

chat

find my 8 latest issues + comments for each

Ran code linear

const issues = await linear.list_issues()
return Promise.all(issues.map(i =>
  linear.list_comments(i.id)))

Running in sandbox…

summary · one compact result
8 issues · comment counts · latest authors

Here are your 8 issues, with comment counts and latest authors.

dataflow
sandboxtasklist_issuessummarizeanswerlist_comments ×8model context44.2k+0k

data packet
tool call sandbox 8 issues → comments → report

The chat stays quiet while payloads circulate in the sandbox; one summary crosses into model context, 44.2k → 60.1k (+16k).

The race

Fewer turns, smaller context. Code mode should win every time. So I raced both approaches on that exact task.

Code mode lost.

Race replay · code vs classic

Pause

LLMTool calls
Orchestration
Wasted guess script

8 linear issues + comments per issue

wall clock · ×8 speed
100.6s

Code mode

100.6s

Classic tool calls

77.5s

0s 30s 60s 90s116.6s

Wasted guess scripts flip red as they finish, and the green row is the one that got there first. Wall times and phase totals are measured; per-turn splits are illustrative.

Metrics · code vs classic — Blacksmith

Code mode

116.6s
Wall time

60.1k
Context

8
Turns

$1.91
Cost

Classic tool calls

77.5s
Wall time

101.1k
Context

6
Turns

$2.01
Cost

Why it lost

This is the part that hurt to discover: most MCP servers ship input schemas and nothing else. The model knows exactly what to send, and nothing about what comes back. Which makes sense. The classic loop never needed more: raw results land in context and the model reads them. Code mode cares a lot. The script has to name fields the model has never seen.

What Linear ships:

list_issues(input): unknown

Faced with unknown, the model had to guess the return shape. It guessed wrong. Authors came back wrong, IDs came back null, and it took three extra scripts to learn the real fields: the turns that flip red in the replay above. Every wasted script is a full extra turn, and every turn drags everything with it: more model time, more streaming, more overhead. That's how classic pulled ahead.

But then I looked at the context

Even while losing, code mode did the one thing it promised: it kept context small. The savings start at turn 0, before a single result comes back. The classic loop has to carry every Linear tool schema in the model's context, so it starts at 52.8k tokens. Code mode hides them all behind one execute tool and starts at 44.2k. That's 9k saved for just showing up.

Then the gap widens. The classic loop pushed every raw Linear payload into model context and grew 48k tokens. Code mode digested the payloads in the sandbox and returned just the mapped summary. It only grew 16k. Even losing, it was cheaper: $1.91 against classic's $2.01.

Context per turn · cold vs classic

Context per turn

40k 60k 80k 100k 120k turn 0
turn 1
turn 2
turn 3
turn 4
turn 5
turn 6

Cold code mode (k tokens)
Classic tool calls (k tokens)

Context carried into each model turn (input + cache, k tokens). A flat tail means the run finished early. Source: per-turn token usage from the session logs.

Giving it a muscle memory

Take a step back, though. Every MCP call produces a result whose shape the runtime can inspect. What if we just remembered it?

We call it Muscle Memory: a shape cache that infers a small schema from each result, field names and broad types only, never values. Shapes merge across calls, survive across sessions, and show up in the next tool search. After the first session, the agent stops thinking about the API and just moves.

Here is the mechanism at work. On the left, a users session: the calls the model makes and what comes back. On the right, the only thing the cache keeps:

Shape cache inspector — Blacksmith

Pause

agent session · first run

Find my 8 most recently updated open issues, fetch the comments on each, and report counts and latest authors.

linear.list_issues

({ state: "open", first: 8 })

← 8 issues returned

{ id: , title: , status: }

shape cache · names & types only

list_issues(input): { issues: Array<{ id: string title: string status: string }> }

list_comments(input): unknown

Call 1. Eight issues come back. The cache keeps field names and broad types only, never a value.

Before the cache learned, the model saw:

list_issues(input): unknown

After the first run, the model sees:

list_issues(input): { issues: Array<{ id, title, status, … }> }

The rematch

Okay, rematch. Same prompt, same rules. The cold code run and classic loop stay on the board. This time, code mode starts warm: the model knows the shapes. One script, first try, no guessing.

Race replay · warm / cold / classic

Pause

LLMTool calls
Orchestration
Wasted guess script

8 linear issues + comments per issue

wall clock · ×8 speed
100.6s

Warm code mode

56.6s

Cold code mode

100.6s

Classic tool calls

77.5s

0s 30s 60s 90s116.6s

Metrics · rematch — Blacksmith

Warm code mode

56.6s
Wall time

58k
Context

6
Turns

$1.06
Cost

Cold code mode

116.6s
Wall time

60k
Context

8
Turns

$1.91
Cost

Classic tool calls

77.5s
Wall time

101k
Context

6
Turns

$2.01
Cost

57 seconds. Half the cold run, 27% faster than classic, with the smallest context of the three! And a bloated context is not free. It compounds: every turn re-sends everything that came before it, so classic processed 373k cumulative tokens and cost $2.01. Warm processed 267k and cost $1.06. Same question, same answer, half the price.

Context per turn · rematch

Context per turn

40k 60k 80k 100k 120k turn 0
turn 1
turn 2
turn 3
turn 4
turn 5
turn 6

Warm code mode (k tokens)
Cold code mode (k tokens)
Classic tool calls (k tokens)

Context carried into each model turn (input + cache, k tokens). A flat tail means the run finished early. Source: per-turn token usage from the session logs.

Code mode was not inherently slow. It was missing the return-shape knowledge the classic loop never needed. The first run is practice, and every run after it plays from muscle memory: fastest, smallest context, half the price.

The race it lost was just the warmup. So is this post. Stay tuned.