How do you scale to hundreds of tools without overwhelming the LLM?

hard

Answer

$(1) Retrieval-based tool selection: embed tool descriptions in a vector DB; at each step, retrieve the top-10 most relevant tools and inject only those in the prompt.$
$(2) Hierarchical: a top-level agent chooses a category ('search / write / analyze'), then a sub-agent with tools in that category.$
$(3) Dynamic tool schemas that generate lazily.$
$(4) Fine-tuning a tool-selection classifier separately from the answer LLM.$
$Cost: retrieval adds a step, but keeps prompt small (context bloat = accuracy loss beyond ~50 tools).$

Check yourself — multiple choice

All tools in one prompt
Retrieve top-k tools per step + hierarchical delegation + tool-selection classifier — keeps prompt small (>50 tools bloats context)
Random subset
Impossible

Many tools: retrieve top-k per step + hierarchical + tool-selection classifier.

#agents#tools#retrieval

214 interview questions in this topic.