While I'm here - I recently optimized our PR review skill to make better use of the KV cache.
Previously, it loaded up the diff, persona and prompt, and wrote those into the first message for each of 6 sub-agent reviewers. The prompt was templated with the persona name, so was slightly different for each reviewer.
My optimized version had a common first message with diff + prompt, and a script to run to atomically claim a persona. It also runs the first reviewer before the rest to warm the KV cache, and the agent doesn't launch the rest of the subagents until the first persona has been claimed (which means that the LLM is running, and therefore the KV cache is warm). The agent has a script it runs which does the waiting for it.
Agents 2-6 only run once the KV cache is warm, and because the intro message including the diff is shared, it's there in the cache. Only the persona file is different.
In my testing this brought down the cost of a review by ~half, though of course this depends on how big the diff is, how many agent are launched, and what model is used.
1 days ago [-]
_aavaa_ 1 days ago [-]
What is there to talk about the KV Cache, they’re handing off to a different model, I thought that you can’t reuse KV cache between entirely different models?
resonious 1 days ago [-]
Why does KV cache matter if they show it's cheaper anyway?
draw_down 1 days ago [-]
[dead]
nxtfari 2 days ago [-]
This is really smart, like the author said, old idea but cleverly applied.
In case anyone wants a summary: don’t one shot, don’t use plan mode and hand off the plan to cheap executors, ask the frontier model to explore, create a todo list, and then start when it feels confident; stop it after first code edit, then prefill the context to cheap executor to continue.
brookst 1 days ago [-]
How is that different than just having the frontier model write a build plan and store it, then have a cheaper model execute? That’s pretty normal practice.
The build plan should be better than the context that generated it since it strips out wrong turns and other noise. I think?
xutopia 1 days ago [-]
The article actually explains it better than I can but essentially you do not want the agent to get desperate which causes it to burn extra tokens. Planning phase doesn't test assumptions by going straight in the code and testing out parts of the ideas to execute so it has to consider way more in planning than it would if it could go quicker into the coding phase.
brookst 1 days ago [-]
Hmm. When I have Opus or Fable write build plans, then invariably invariably read code extensively, but it’s true they don’t make changes and run tests.
I guess the way to formalize this would be to add a “make minimal change to confirm approach” instruction to the build planning prompt. Probably can even parallelize that so as the build planning prompt iterates subagents get launched to validate, similar to what research modes do.
I’m a little skeptical, but will give it a try.
diarized 1 days ago [-]
Would it be `/superpowers:brainstorming` + `/superpowers:writing-plans` on stronger model and `/superpowers:subagent-driven-development` on smaller model?
hombre_fatal 1 days ago [-]
The article isn't about plan vs no plan but rather that instead of a weak hand-off of "here's a plan describing what I hope works" you can do a much stronger hand-off with "here's an approach that survived first contact with the code".
Imanari 1 days ago [-]
So its basically (a plan + a bunch of file reads + a first edit) injected into the context of a cheap model so the cheap model does not feel like it has to re-read the files and just continues executing and editing?
trash_cat 1 days ago [-]
Yes this is correct. It essentially biases the cheaper model into procedural action grounded by the frontier model. To put it mode generally, it injects the cheap model context with more useful information, where it might not need to read all the files again.
swingboy 1 days ago [-]
Refilling the context of the cheap executor == just switching the model mid-conversation instead of /clear’ing and passing some plan doc?
Too 2 days ago [-]
> The mistake is upstream of the architecture diagram. People price agents the way they price people: senior time is expensive, so minimize senior involvement.
> But the expensive part of an agent's day is not the fixing, building, or even the thinking. Opus fixing things does not cost money. Opus reading things costs money.
Heh, another rediscovery that agents and humans are alike. By the time I've researched a Jira ticket and made it unambiguous enough to outsource, I might as well have written the code myself instead.
wordpad 2 days ago [-]
You are more proficient at writing code than ai specs.
You could get better/faster at writing good specs and its a higher cap skill now.
For low risk changes you could let llm write its own spec from high level requirements and just validate its assumptions/design decisions.
unholiness 1 days ago [-]
No mention of KV cache, one of the biggest reasons not to switch models mid-stream. Once you pay, say, 50k input and 50k output Opus 4.8 tokens, you don't pay token cost for cache reads of the 100k context while it builds. Switch to another model, you'll start with the cost of 100k input tokens on the smaller model to get that context loaded and its unique KV cache set.
The post may have some real insight here, where this 100k working context is actually better than trying to summarize it's findings into a plan. It's also right that the handoff is a perfect time to edit the context (removing planning instructions). But it doesn't mention it's a trade-off: the plan is smaller, so it's a cheaper "on-boarding" of the next model. Seems quite plausible that this is worth it for 1-off tasks. If it's right, this is basically "plans are useless, planning is everything" for LLMs.
My problem is, I think the plans are useful. I want to review and edit them. I want them to give context for the upcoming code review (even if humans aren't reviewing). LLMs are notoriously bad at explaining why they're doing something in the moment. Humans are notoriously bad at accepting there's no reason why. I think the humans have it right here, and to bridge this gap want my PRs, my docs, and my comments teeming with reasons why. Plans help with that.
CBLT 1 days ago [-]
Not by name, sure, but it does mention that you're going to be reading the same context into the second model. It specifically refutes your point that you're only reading the smaller plan:
> Opus reads base.py, signing.py, the test file (twenty cards of gray), then writes its plan and leaves. And what's the first thing Flash does with that beautiful document? It re-reads base.py and the test file, because a plan is not a file and you cannot edit prose. The gray reads just keep stacking, first at Opus prices, then again at Flash prices. There is no version of this where a second reader is the cost optimization.
resonious 1 days ago [-]
The article shows that it's still cheaper to pay for the switch than it is to let the frontier model do all the work. At least in benchmarks.
If you want to interact with plans then I think this technique just isn't for you.
figmert 2 days ago [-]
The reason I use plan then implement is because I can adjust the plan, whereas if I get it to implement straight away, it might (and often does) make the wrong decisions that will be harder to adjust, or I'd have to adjust it after the fact.
canpan 2 days ago [-]
One big pain point is not seeing the thinking trace. I noticed using pi with a self hosted model that I could spot, stop and correct my prompt much faster. With claude etc I have to first wait for it to think 3 minutes and then notice it got completely off the path I wanted.
I still use a plan, because my local model is not as smart as opus, but I can iterate much faster.
WilcoKruijer 2 days ago [-]
I’ve seen people have success figuring out model reasoning by adding a required `reasoning` parameter to tool calls. Might be worth experimenting with this for the `write` tool call in a coding agent harness.
embedding-shape 2 days ago [-]
Also "divide and conquer" is still applicable. Tired of waiting 10 minutes for the model to come up with a plan that ends up being wrong? Divide the problem into pieces and attack the pieces individually, worst case scenario the model spends 2 minutes and got some detail wrong, but way easier to correct the tiny stuff.
Once you've done with the pieces, do one "integrate them together" part, then you have a fully formed plan with less chance of wrong stuff in it. Yes, this requires a bit more interactivity than "prompt model then come back after making and drinking a coffee", but personally I prefer that.
carterschonwald 2 days ago [-]
this so true. its really hard to make sure a model isnt going off the rails if i dont see full cot. the fact that oai and anthropic models hide it now has made them less reliable. which is a shame
trollbridge 2 days ago [-]
The great news is you can use DS-V-Pro, MiMo-V2.5-Pro, GLM-5.2, or K3 and see everything.
I tend to use 5.6-Sol now more for one shot type of tasks where all I want is the answer and I’m not going to read the reasoning.
carterschonwald 1 days ago [-]
5.6 sol is pretty good, its definitely very very well tuned, i'm not quite sure which of the others i should use near term, but theyre doing great work
_ink_ 2 days ago [-]
> Below: the share of runs that went poking around the web for it. Filthy cheaters!
I can understand that in their benchmark setting they wouldn't want the model to find an existing solution. But in my day to day work, wouldn't I want the model to search online for the best solution? Am I not shooting myself in the foot by preventing it?
gillesjacobs 1 days ago [-]
In ML, you want to test general capability of a model (generalizability), because you want it to perform well on unseen tasks. In that benchmark, the literal reference is leaking through web search, the agent can see the matching real codebase online and the commits so that's test set leakage. I know no programmer that was ever paid to rewind an existing codebase to a previous commit and implement a feature/fix a bug that exists in the next commits.
andreyvit 2 days ago [-]
It's hard to argue with the numbers, but starting with a (mostly true!) “research is the most expensive part” premise, this strikes me as an odd direction to go to optimize costs:
1. As others pointed out, we feed all the same research turns to a smaller model, so we pay the uncached price for all of them.
2. During research, the model typically reads more code than is relevant, to figure out what is relevant and what is not. If that's the expensive part, we keep paying for those turns with the most expensive model?
3. There is no quality comparison of the resulting code. Same plan != same code, and AI tokens during the initial implementation phase isn't the only cost attributable to the task.
I do the opposite:
1. Outsource research to a subagent, or several parallel subagents. Let them output the relevant code paths only. Using gpt-5.6-terra-high on Codex and sonnet on Claude. Merge results into a single per-task research file. This saves tokens and context window of the bigger model, and avoids re-researching after compactions and in subagents.
2. Use the smartest agent (Sol xhigh ultra / Fable max) for both planning and execution. Tell it to use the research file where possible.
3. Switch to dumber agents for verbose substeps, e.g. gpt-5.6-luna-medium / sonnet is enough to drive browser use.
Sadly, got no numbers to back this approach.
PS: Of course, different task complexities and codebase complexities demand different approaches. The most expensive part is actually the code review step, which they don't mention/have at all. We should come up with some sort of complexity grading, so that discussions like this can be contextualized.
pmontra 2 days ago [-]
I've been doing more or less this (from the post) for months:
> plan deeply, then capture the plan as a todo list, then start.
I do the plan deeply phase by writing a 50 line specification myself and then asking Claude to evaluate it, find what I missed, ask questions. I answer the question and we iterate until I know that we have what I had in mind to do.
Then I ask it to write the implementation plan, that I might end up reviewing because it always asks some more question. Finally it implements the steps.
If the planning was good the implementation is fast.
kimonsodu 1 days ago [-]
[flagged]
wrs 2 days ago [-]
Lately I'm trying a variation on this: Have the main agent make a phased implementation plan, then for each phase, have it start an implementation subagent with a focused prompt, then review that agent's work in the main session. The theory being that the main session still contains all the research, but it can review just the diff rather than have the entire implementation session in context as well.
The post doesn't include a review in the cost comparison, but I find that immediately doing a review catches lots of mistakes.
sgc 2 days ago [-]
Actually it does include an important level of review. They say "init a TODO list with a validation step for each item", and then the last several sections of the execution are within the TODO - where it is validating the code.
To me that was the heart of the article and of what they did - create a self-validating todo list that deterministically forces the agents to stay on task, and then trim out just the right extraneous context (the actual planning) without messing with the read-file context which is where they found subagents burning most of their tokens.
Just a few days ago I was speculating about forking context to implement instead of spawning subagents here on hn. This is similar to that just several more steps more sophisticated. You could even fork the trimmed context here for very large tasks.
wrs 1 days ago [-]
It wasn’t clear to me (and still isn’t, after slogging through the inhuman article again) that the verification here is done by a separate agent from the implementation. That’s a second trimming step, but in reverse: at that point you want the new code in the context, but not the implementation reasoning.
sgc 1 days ago [-]
No idea, but yes it seems like good practice to trim context there as well. A good candidate for v2 if it's not present.
At the same time, even if that is not done, the todo is both specification and tests. If it passes those when it should not, then either you approved something you should not have, or else there should only be smaller improvements to be made. I like the concept enough and I am not very concerned about this, so I will give it a test on a project I have had on backburner for a bit.
bensyverson 2 days ago [-]
Yes, this is a core part of my workflow; use the smartest model (e.g. Fable) to generate a large hierarchical implementation plan, then clear the context and have a lesser model (e.g. Sonnet) track and execute the plan [0], often in parallel.
It all depends on how much work you're doing; if it's a simple task, there's virtually no need for plans at all; just have the smart agent do it. But if the work is going to take 5, 10 or 100 sessions, there's real value in the "smart agent plans, cheap agent executes" model.
This is exactly what the post argues against though, as it leads to higher overall cost.
bensyverson 2 days ago [-]
Yes, and I'm arguing that the post is over-simplifying. As I mention, for small plans, it's probably easier to have the big model just do it. But for large, multi-session plans, it's cheaper to let smaller models execute the bulk of the work.
markust 1 days ago [-]
Perhaps, there's a small change that might work better here - if we use a different agent (by creating custom agents) for plan and implementation, then the switch to the implementation agent wouldn't end up re-reading a lot as the new agent would have fresh empty context.
I implemented this[1] from one of the posters on HN and I think it'd work well with this problem too.
Would be great if you could point out if my understanding is wrong.
The issue with this is just how much context it consumes. Generating the plan could easily get me to 30% of a 1M context window—the point at which I normally clear. Most of that context fill is not necessary for execution, and the model degrades as context fills. So starting from 30% (or passing a fresh agent all of that context) is not ideal. But maybe I'm misunderstanding what you're proposing?
markust 14 hours ago [-]
Yes, sorry let me explain a bit more.
The idea is that we have custom agents each having a model version of their own. So, if we have a planning agent that uses Opus and a worker agent that uses Sonnet, then the planning agent writes the plan and hands it over to the worker agent. This way, the handover here is not going to retain the context.
When the worker agent picks it up, it'd have a new context and only the todo list from the plan, so it shouldn't be reading all the files again.
wrs 1 days ago [-]
My main point was not about the implementation stage, but the verification stage. Letting the smarter agent verify the work of the cheap agent finds a lot of little oversights and sloppiness that I used to have to find myself. So I’d be interested in that cost comparison.
bensyverson 1 days ago [-]
I think it would depend on how you measure it. Adding a review step will definitely increase cost, but theoretically that cost is lower than re-doing the subagent's implementation later when you discover it's broken.
pqdbr 2 days ago [-]
I’m using Claude code dynamic workflow like this. I tell Fable to use a workflow. He is the planner, orchestrator and reviewer. Opus agents are implementers. Works unbelievably well.
tern 1 days ago [-]
I do the same, but delegate to codex instead via the /smux skill. It's expensive, but I get one long-running Claude context window and one frequently compacting codex, and this saves me from needing to re-gather context all the time.
Per the OP's technique, I'll use Fable for intensive planning moments, then switch back to opus when things are going well.
oliver236 2 days ago [-]
can you explain this in a bit more detail? super interested
pqdbr 1 days ago [-]
It's literally all there is to it. Write your prompt normally. then, at the last paragraph, you write:
Use a workflow to implement this. You (Fable) are the orchestrator, planner and reviewer. Opus agents are implementers.
That's all there's to it. it will create the dynamic workflow - has a nice interface native to Claude Code - and do all the coordination to deliver what you asked.
danielciocirlan 2 days ago [-]
“He”
brabel 2 days ago [-]
In many languages that’s the only way to express that since there is no “it”. It doesn’t mean speakers of such languages anthropofy everything and I think that’s the case here.
llbbdd 2 days ago [-]
I call him "my son"
ghiculescu 2 days ago [-]
> We upstreamed it to omp
For those not sufficiently cracked... what is omp? Is there a way I can have this command or workflow in Cursor / Claude Code?
Sounds like a consulting slaughterhouse. Picture Java Enterprise Solutions. As we all know, the pinnacle of software engineering.
swiftcoder 2 days ago [-]
Hah. Yeah. Weirdly, most software managers also seem to believe this whole "senior writes a plan, junior implements" is cost-effective (it typically isn't, one should invest in hiring/promoting more seniors).
2 days ago [-]
nchmy 2 days ago [-]
This entire argument rests upon the fact that Gemini Flash ain't cheap.
Try plan with opus 4.8 and then implement with Deepseek v4 flash - its 35x cheaper for reads, 90x cheaper writes, and 18x cheaper cache reads.
Or plan with Deepseek Pro, or even Flash itself. I've been impressed with both.
pdyc 2 days ago [-]
+1. i have similar workflow and i use local models on igpu so token cost is free and electricity cost is in cents.
spuz 2 days ago [-]
I cannot imagine local models running on an igpu could get anything close to either a useful plan or execution of a plan. I've tested Qwen 3.5 27B locally and its solutions to coding problems are usually flawed and running in thinking mode is too slow and that's on a discrete GPU. How do you get anything useful done on a model that runs on an igpu?
nchmy 1 days ago [-]
yeah, i dont understand, whatsoever, the folks who are running models locally other than for privacy/curiosity. It is surely FAR slower, more expensive (electricity + hardware costs), and worse quality. Better to outsource to providers who have optimized everything and costs are distributed/amortized across all users.
Perhaps someday local llms will be sufficient for coding etc, but it's going to be a while.
leobg 2 days ago [-]
I guess Anthropic knows this. They show you this warning:
---
Switch model?
Your next response will be slower and use more tokens
This conversation is cached for the current model. Switching to Opus 4.8 (1M context) means the full history gets re-read on your next message.
---
Reminds me a bit of airline booking sites ("We noticed you didn't add insurance").
unholiness 1 days ago [-]
I mean, this is real. Your KV cache lives for an hour since the last token (on anthropic pro/Max at least, today at least, this has degraded in the past). So once you pay, say, 50k input and 50k output Opus tokens, you don't pay for cache reads of the 100k context while it builds. Switch to another model, you'll start with the cost of 100k input tokens on the smaller model to get that context loaded and its unique KV cache set.
The post may have some real insight here, where this 100k working context is actually better than trying to summarize it's findings into a plan. It's also right that the handoff is a perfect time to edit the context (removing planning instructions). But it doesn't mention it's a trade-off: the plan is smaller, so it's a cheaper "on-boarding" of the next model. Send quite plausible that this is worth it for 1-off tasks. If it's right, this is basically "plans are useless, planning is everything" for LLMs.
My problem is, I think the plans are useful. I want to review and edit them. I want them to give context for the upcoming code review (even if humans aren't reviewing). LLMs are notoriously bad at explaining why they're doing something in the moment. Humans are notoriously bad at accepting there's no reason why. I think the humans have it right here, and to bridge this gap want my PRs, my docs, and my comments teeming with reasons why.
(NB: ranted to long, reposting at top level)
pistoriusp 2 days ago [-]
Apparently "/prewalk" is built into https://github.com/can1357/oh-my-pi, which I guess is a lot like "Oh My ZSH?" Pi with batteries included.
There are a lot of interesting ideas in it, mostly none that I have applied to my own workflows. Curious if anyone has used it?
AshamedBadger56 2 days ago [-]
I've played around with all the popular agent CLI's and I've landed on omp/ohmypi as my favorite for now. It's definitely the opposite of a lean install like Pi, but I find it works a lot better than OpenCode. It's also very very active, with a lot of cool new ideas like this prewalk in it.
heisenbit 2 days ago [-]
I‘m really wondering about plan and then agent mode switch in Copilot even with the same model. The switch swaps the prompt so all the learnings during planning which are in the cache will get invalidated.
2 days ago [-]
ursuscamp 1 days ago [-]
Can someone explain to me the difference between this approach and using planning with a larger model, then just switching to a small model for implementation without clearing the context? I understand that it specifically does the first edit as well either way the larger model. Is there some other difference I am missing here?
trash_cat 1 days ago [-]
When a frontier makes a succesfull edit based on the plan that it made, it leaves an procedural trace in turn biases the NEXT model, low cost model, straight into procedural action. The cheaper model doesn't need to reread everything again because it has enough information from the frontier model to complete the task. A simple "plan" of what needs to be done does not carry this information.
actionfromafar 1 days ago [-]
If I understand correctly, switching to a small model makes the small model read the context again.
rwc 2 days ago [-]
Just specify that the more efficient model should only be used if the cost of doing the task would be less than using the higher horsepower model. Fable has no problem routing based on that instruction.
ricardobeat 2 days ago [-]
I just did this five minutes ago. Started a task with Kimi K3, noticed cost going up, switched to Minimax M3 continuing from the same context. Really easy to do with Crush/OpenCode.
It works quite well most of the time. For very deeply technical tasks, where exploration involves multiple agents (and blowing up context limits a few times), I'll have a plan written down to disk that will be much longer than 2k tokens.
rolymath 1 days ago [-]
Wow. Someone using Crush in the wild. What do you think of it? I'm having trouble switching to it because the rest work fine, but would love to actually adopt it.
ricardobeat 19 hours ago [-]
It has the most user-friendly UI of them all, a really usable scroll view. Automatically picks up LSPs. Has a well maintained provider registry. Looks nice and runs fast.
The main downside is that subagents cannot have edit tools, which kills any agent orchestration possibilities. I use it for most one-shot / single task prompts using open models.
Incipient 2 days ago [-]
Wait this is new? Large context models figure out the architecture of the request. Miss sized models plan out each requirement, and smaller models implement a tightly detailed task? Isn't this the standard approach?
benrutter 2 days ago [-]
> Miss sized models plan out each requirement, and smaller models implement a tightly detailed task? Isn't this the standard approach?
It is, but the article is suggesting something different. The central idea is to use /prewalk to load slightly edited context from the more expensive model into the one, so that the cheaper model doesn't burn tokens reading all the files again.
kademolu 1 days ago [-]
Good stats but how does this work with models like Claude where a lot of the context or relevant information is stored in its "memory", other coding agents can't use it.
CBLT 1 days ago [-]
Memory is a harness feature, not a model feature.
The author is probably only evaluating their own harness with different models, so Claude-Code-specifics like memory aren't part of their evaluation.
walthamstow 1 days ago [-]
You turn auto memory off in Claude Code and manage the context yourself
pjerem 1 days ago [-]
I use GLM 5.2 for everything and it's dirt cheap anyway.
Almondsetat 2 days ago [-]
Did they also use Gemini Flash to write this article? Because, frankly, it's unbearable
IshKebab 2 days ago [-]
Very clearly. Even if you have no qualms with AI I don't understand why people think writing in this cliched style is ok.
On the other hand people thought it was fine to use cliched titles like "The unreasonable effectiveness of..", "... for fun and profit", "The rise and fall of ...", "All you need is ...", etc. etc.
So I guess there's just a large portion of writers that are immune to cliche? Pretty annoying anyway.
egamirorrim 2 days ago [-]
Isn't part of this the inevitable cache invalidation that comes from switching between providers (Opus to Gemini)?
Aeolun 2 days ago [-]
Yeah, but you only pay the switch on the much cheaper model, the first bit is done with smaller context on the expensive model, so you still save money in the end.
maelito 1 days ago [-]
This first chart is very hard to read, in dark mode and thin lines and text.
pphysch 2 days ago [-]
> Any agent, any model, any scaffold: the bill is essentially O(reads)
Therefore token sellers have incentive to produce verbose, unreadable code
pixl97 2 days ago [-]
Whichever token seller defects can capture a larger chunk of the market with slimmer code.
pphysch 2 days ago [-]
Do you think the average vibe coder or AI-smitten suit cares about slim code? A lot of folks still think "LOC = good" in 2026. It will be a hard sell for the practical engineer.
Even if they understand "slim code = fewer tokens = less spend" they still have to grok that less code is not less functionality.
pietje 1 days ago [-]
They are supply constrained at the moment so they still have incentive to keep it short
viccis 2 days ago [-]
From what I can tell, I have a similar workflow. I get Fable 5 / Sol on High to talk to me about requirements until it's ready to design. I then have it break the design into tickets. I use kata, an agent-oriented issue tracker. If it ever starts to get bloated, I'll just make my own as it doesn't need too many features. Anyway, I tell it to include sufficient context in each such ticket to be picked up by a new implementation agent. Explicit user stories, Cucumber style acceptance criteria, etc.
When it's done, I switch to a smaller model and tell it to start a /goal of calling `kata ready` to get tickets ready to be worked. Work one ticket on each goal iteration, committing changes when done. Stop when all the tickets are either closed out or are blocked on actions from me.
It works fantastically. I can get entire (relatively straightforward) iOS apps done in under my $20/month five hour session window.
I get even better results if I do the QA session with the frontier model with two output artifacts: an implementation spec and a design prompt for Claude Design. I push the design prompt through Claude Design, tweak the results, and get a design spec. When I have the frontier model do the planning, I have it read the implementation spec from before, along with the design spec's overview file.
When I do it this way, I've done A/B tests between Fable 5 and Sol, and the apps they wrote were basically identical. It's so much cheaper than the "let loose the subagent fleet!" form of context management.
I haven't even added any kind of frontier model validation cycle into this loop yet. Everyone keeps talking about a subagent flow in which the big boy reviews the work of the drones, but I've found that if it encodes its acceptance criteria well enough, they do a satisfactory job of it themselves.
potus_kushner 2 days ago [-]
this article is full of missing text like this:
" of Opus at of the cost, the speed, points over oneshot Flash."
is it just me or is that page using some brand new js feature or chromium-only hack?
1 days ago [-]
peheje 2 days ago [-]
[flagged]
luciana1u 2 days ago [-]
[dead]
Bolwin 2 days ago [-]
This is a lot of words to say "switch models instead of writing a plan file and starting a new session" which I do anyway.
That said, it takes me a while to reads plans and often I'll take a break, by when the cache has expired. At that point it may be better to start over
I want an expert opinion, if I just wanted to ask an LLM I have my own.
How can this article not mention the KV cache even once?
https://news.ycombinator.com/item?id=48886741
Previously, it loaded up the diff, persona and prompt, and wrote those into the first message for each of 6 sub-agent reviewers. The prompt was templated with the persona name, so was slightly different for each reviewer.
My optimized version had a common first message with diff + prompt, and a script to run to atomically claim a persona. It also runs the first reviewer before the rest to warm the KV cache, and the agent doesn't launch the rest of the subagents until the first persona has been claimed (which means that the LLM is running, and therefore the KV cache is warm). The agent has a script it runs which does the waiting for it.
Agents 2-6 only run once the KV cache is warm, and because the intro message including the diff is shared, it's there in the cache. Only the persona file is different.
In my testing this brought down the cost of a review by ~half, though of course this depends on how big the diff is, how many agent are launched, and what model is used.
In case anyone wants a summary: don’t one shot, don’t use plan mode and hand off the plan to cheap executors, ask the frontier model to explore, create a todo list, and then start when it feels confident; stop it after first code edit, then prefill the context to cheap executor to continue.
The build plan should be better than the context that generated it since it strips out wrong turns and other noise. I think?
I guess the way to formalize this would be to add a “make minimal change to confirm approach” instruction to the build planning prompt. Probably can even parallelize that so as the build planning prompt iterates subagents get launched to validate, similar to what research modes do.
I’m a little skeptical, but will give it a try.
> But the expensive part of an agent's day is not the fixing, building, or even the thinking. Opus fixing things does not cost money. Opus reading things costs money.
Heh, another rediscovery that agents and humans are alike. By the time I've researched a Jira ticket and made it unambiguous enough to outsource, I might as well have written the code myself instead.
You could get better/faster at writing good specs and its a higher cap skill now.
For low risk changes you could let llm write its own spec from high level requirements and just validate its assumptions/design decisions.
The post may have some real insight here, where this 100k working context is actually better than trying to summarize it's findings into a plan. It's also right that the handoff is a perfect time to edit the context (removing planning instructions). But it doesn't mention it's a trade-off: the plan is smaller, so it's a cheaper "on-boarding" of the next model. Seems quite plausible that this is worth it for 1-off tasks. If it's right, this is basically "plans are useless, planning is everything" for LLMs.
My problem is, I think the plans are useful. I want to review and edit them. I want them to give context for the upcoming code review (even if humans aren't reviewing). LLMs are notoriously bad at explaining why they're doing something in the moment. Humans are notoriously bad at accepting there's no reason why. I think the humans have it right here, and to bridge this gap want my PRs, my docs, and my comments teeming with reasons why. Plans help with that.
> Opus reads base.py, signing.py, the test file (twenty cards of gray), then writes its plan and leaves. And what's the first thing Flash does with that beautiful document? It re-reads base.py and the test file, because a plan is not a file and you cannot edit prose. The gray reads just keep stacking, first at Opus prices, then again at Flash prices. There is no version of this where a second reader is the cost optimization.
If you want to interact with plans then I think this technique just isn't for you.
I still use a plan, because my local model is not as smart as opus, but I can iterate much faster.
Once you've done with the pieces, do one "integrate them together" part, then you have a fully formed plan with less chance of wrong stuff in it. Yes, this requires a bit more interactivity than "prompt model then come back after making and drinking a coffee", but personally I prefer that.
I tend to use 5.6-Sol now more for one shot type of tasks where all I want is the answer and I’m not going to read the reasoning.
I can understand that in their benchmark setting they wouldn't want the model to find an existing solution. But in my day to day work, wouldn't I want the model to search online for the best solution? Am I not shooting myself in the foot by preventing it?
1. As others pointed out, we feed all the same research turns to a smaller model, so we pay the uncached price for all of them.
2. During research, the model typically reads more code than is relevant, to figure out what is relevant and what is not. If that's the expensive part, we keep paying for those turns with the most expensive model?
3. There is no quality comparison of the resulting code. Same plan != same code, and AI tokens during the initial implementation phase isn't the only cost attributable to the task.
I do the opposite:
1. Outsource research to a subagent, or several parallel subagents. Let them output the relevant code paths only. Using gpt-5.6-terra-high on Codex and sonnet on Claude. Merge results into a single per-task research file. This saves tokens and context window of the bigger model, and avoids re-researching after compactions and in subagents.
2. Use the smartest agent (Sol xhigh ultra / Fable max) for both planning and execution. Tell it to use the research file where possible.
3. Switch to dumber agents for verbose substeps, e.g. gpt-5.6-luna-medium / sonnet is enough to drive browser use.
Sadly, got no numbers to back this approach.
PS: Of course, different task complexities and codebase complexities demand different approaches. The most expensive part is actually the code review step, which they don't mention/have at all. We should come up with some sort of complexity grading, so that discussions like this can be contextualized.
> plan deeply, then capture the plan as a todo list, then start.
I do the plan deeply phase by writing a 50 line specification myself and then asking Claude to evaluate it, find what I missed, ask questions. I answer the question and we iterate until I know that we have what I had in mind to do.
Then I ask it to write the implementation plan, that I might end up reviewing because it always asks some more question. Finally it implements the steps.
If the planning was good the implementation is fast.
The post doesn't include a review in the cost comparison, but I find that immediately doing a review catches lots of mistakes.
To me that was the heart of the article and of what they did - create a self-validating todo list that deterministically forces the agents to stay on task, and then trim out just the right extraneous context (the actual planning) without messing with the read-file context which is where they found subagents burning most of their tokens.
Just a few days ago I was speculating about forking context to implement instead of spawning subagents here on hn. This is similar to that just several more steps more sophisticated. You could even fork the trimmed context here for very large tasks.
At the same time, even if that is not done, the todo is both specification and tests. If it passes those when it should not, then either you approved something you should not have, or else there should only be smaller improvements to be made. I like the concept enough and I am not very concerned about this, so I will give it a test on a project I have had on backburner for a bit.
It all depends on how much work you're doing; if it's a simple task, there's virtually no need for plans at all; just have the smart agent do it. But if the work is going to take 5, 10 or 100 sessions, there's real value in the "smart agent plans, cheap agent executes" model.
[0]: https://github.com/bensyverson/jobs
[1]: https://www.stavros.io/posts/how-i-write-software-with-llms/
When the worker agent picks it up, it'd have a new context and only the todo list from the plan, so it shouldn't be reading all the files again.
Per the OP's technique, I'll use Fable for intensive planning moments, then switch back to opus when things are going well.
Use a workflow to implement this. You (Fable) are the orchestrator, planner and reviewer. Opus agents are implementers.
That's all there's to it. it will create the dynamic workflow - has a nice interface native to Claude Code - and do all the coordination to deliver what you asked.
For those not sufficiently cracked... what is omp? Is there a way I can have this command or workflow in Cursor / Claude Code?
I tried going to the homepage (https://stencil.so/). It didn't really help.
Sounds like a consulting slaughterhouse. Picture Java Enterprise Solutions. As we all know, the pinnacle of software engineering.
Try plan with opus 4.8 and then implement with Deepseek v4 flash - its 35x cheaper for reads, 90x cheaper writes, and 18x cheaper cache reads.
Or plan with Deepseek Pro, or even Flash itself. I've been impressed with both.
Perhaps someday local llms will be sufficient for coding etc, but it's going to be a while.
--- Switch model? Your next response will be slower and use more tokens
This conversation is cached for the current model. Switching to Opus 4.8 (1M context) means the full history gets re-read on your next message. ---
Reminds me a bit of airline booking sites ("We noticed you didn't add insurance").
The post may have some real insight here, where this 100k working context is actually better than trying to summarize it's findings into a plan. It's also right that the handoff is a perfect time to edit the context (removing planning instructions). But it doesn't mention it's a trade-off: the plan is smaller, so it's a cheaper "on-boarding" of the next model. Send quite plausible that this is worth it for 1-off tasks. If it's right, this is basically "plans are useless, planning is everything" for LLMs.
My problem is, I think the plans are useful. I want to review and edit them. I want them to give context for the upcoming code review (even if humans aren't reviewing). LLMs are notoriously bad at explaining why they're doing something in the moment. Humans are notoriously bad at accepting there's no reason why. I think the humans have it right here, and to bridge this gap want my PRs, my docs, and my comments teeming with reasons why.
(NB: ranted to long, reposting at top level)
There are a lot of interesting ideas in it, mostly none that I have applied to my own workflows. Curious if anyone has used it?
It works quite well most of the time. For very deeply technical tasks, where exploration involves multiple agents (and blowing up context limits a few times), I'll have a plan written down to disk that will be much longer than 2k tokens.
The main downside is that subagents cannot have edit tools, which kills any agent orchestration possibilities. I use it for most one-shot / single task prompts using open models.
It is, but the article is suggesting something different. The central idea is to use /prewalk to load slightly edited context from the more expensive model into the one, so that the cheaper model doesn't burn tokens reading all the files again.
The author is probably only evaluating their own harness with different models, so Claude-Code-specifics like memory aren't part of their evaluation.
On the other hand people thought it was fine to use cliched titles like "The unreasonable effectiveness of..", "... for fun and profit", "The rise and fall of ...", "All you need is ...", etc. etc.
So I guess there's just a large portion of writers that are immune to cliche? Pretty annoying anyway.
Therefore token sellers have incentive to produce verbose, unreadable code
Even if they understand "slim code = fewer tokens = less spend" they still have to grok that less code is not less functionality.
When it's done, I switch to a smaller model and tell it to start a /goal of calling `kata ready` to get tickets ready to be worked. Work one ticket on each goal iteration, committing changes when done. Stop when all the tickets are either closed out or are blocked on actions from me.
It works fantastically. I can get entire (relatively straightforward) iOS apps done in under my $20/month five hour session window.
I get even better results if I do the QA session with the frontier model with two output artifacts: an implementation spec and a design prompt for Claude Design. I push the design prompt through Claude Design, tweak the results, and get a design spec. When I have the frontier model do the planning, I have it read the implementation spec from before, along with the design spec's overview file.
When I do it this way, I've done A/B tests between Fable 5 and Sol, and the apps they wrote were basically identical. It's so much cheaper than the "let loose the subagent fleet!" form of context management.
I haven't even added any kind of frontier model validation cycle into this loop yet. Everyone keeps talking about a subagent flow in which the big boy reviews the work of the drones, but I've found that if it encodes its acceptance criteria well enough, they do a satisfactory job of it themselves.
That said, it takes me a while to reads plans and often I'll take a break, by when the cache has expired. At that point it may be better to start over