How to Fix Your Context

This seems to be an important article.

However it uses various terms that I'm not sure of the definition for.

E.g. the word "Tool".

How can I learn about the definitions of these words? Will reading prior articles in the series help? Please advise.

   you> what's going on?
   > It's going great --- how can I help you today?

   you> [json blob of available "tools": "ls", "grep", "cat"]
   you> what's going on?
   > [json blob selecting "ls"]
   (you) presumably run "ls"
   you> [json blob of "ls" output]
   > [json blob selecting "cat foo.c"]
   (you) dump "foo.c"
   you> [json blob of "cat foo.c"]
   > I can see that we're in a C project that does XYZ...

This is all true, and we've prototyped a number of these things at my current startup. You need to be pretty considered about implementing them.

For a counter-example, consider Claude Code:

- 1 long context window, with (at most) 1 sub-agent

- same tools available at all times, and to sub-agent (except: spawning a sub-sub-agent)

- Full context stays in conversation, until you hit the context window limit. Compaction is automatic but extremely expensive. Quality absolutely takes a dive until everything is re-established.

- Deterministic lookup of content. Claude reads files with tools, not includes random chunks from RAG cosine similarity.

I could go on. In my experience, if you're going to use these techniques 1) maybe don't and 2) turn up the determinism to 11. Get really specific about _how_ you're going to use, and why, in a specific case.

For example, we're working on code migrations [0]. We have a tool that reads changelogs, migration guides, and OSS source. Those can be verbose, so they blow the context window on even 200k models. But we're not just randomly deleting things out of the "plan my migration" context, we're exposing a tool that deliberately lets the model pull out the breaking changes. This is "Context Summarization," but before using it, we had to figure out that _those_ bits were breaking the context, _then_ summarizing them. All our attempts at generically pre-summarizing content just resulted in poor performance because we were hiding information from the agent.

[0] https://tern.sh

This isn't going to be much of a problem for long. I'm wrapping up on an agent context manager that gives effectively infinite context while producing ~77% better results than naive vector+BM25 baseline on my benchmark suite.

> When prompting DeepSeek-v3, the team found that selecting the the right tools becomes critical when you have more than 30 tools. Above 30, the descriptions of the tools begin to overlap, creating confusion. Beyond 100 tools, the model was virtually guaranteed to fail their test. Using RAG techniques to select less than 30 tools yielded dramatically shorter prompts and resulted in as much as 3x better tool selection accuracy.

> For smaller models, the problems begin long before we hit 30 tools. One paper we touched on last post, “Less is More,” demonstrated that Llama 3.1 8b fails a benchmark when given 46 tools, but succeeds when given only 19 tools. The issue is context confusion, not context window limitaions.

High number of tools is a bit of a "smell" to me and often makes me wonder if the agent doesn't have too much responsibility. A bit like a method with so many parameters, it can do almost anything.

Have folks had success with agents like that? I found the fewer tools the better, e.g. <10 "ballpark".