Tool granularity in LLM agent design

How splitting an agent's tools between database reads and computation over the cached result set halved latency on parameter-change turns.

Give an agent one tool that answers the whole question and it cannot adapt when the question changes slightly. Give it twenty primitives and it spends its turns assembling them, badly. The nine tools here landed where they did for a specific reason: four of them touch the database and four of them are forbidden to.

The problem

An agent's tool list is an API design problem wearing a different hat. Each tool is a function the model calls without reading documentation beyond a short description, with arguments it inferred from a sentence someone typed.

Coarse tools are reliable and rigid. A single analyze_facility(location, capacity) always works, and it cannot answer "what if we weighted fire higher" without redoing everything.

Fine tools are flexible and fragile. Expose query_clusters, join_layer, filter_by_slope, sort_by, sum_column and the model can compose anything, including several things that are wrong. Every additional call is another place to hallucinate an argument, and every composition step is latency.

Neither failure is about the model's capability. Both are about where the boundary sits.

What the split actually is

The nine tools divide by what they touch, not by what they mean.

Four read. Four compute on what was read. One resolves a place name.

The bottom three never issue a query. They operate on the result set that retrieve_clusters already pulled and left in session state. That is the whole design, and everything useful follows from it.

Why the boundary is drawn there

Retrieval is the expensive step and it is the only expensive step. Pulling several thousand clusters across three joined layers is where the seconds go. Sorting those clusters by a score and taking them until demand is met is arithmetic over a list already in memory.

Once the boundary is drawn at retrieval, a second turn that changes only a parameter can skip everything above the line.

Turn 1  estimate_demand → estimate_radius → retrieve_clusters → build_supply_curve
        18.2 s

Turn 2  "what if fire mattered more"
        build_supply_curve   (cached clusters, new weighting)
        7.2 s

Measured directly, that is a 2.0× reduction. Without the split, turn two re-runs retrieval and takes an estimated 14.1 seconds.

The split also makes the expensive step cacheable in a way that a merged tool would not be. Results are cached in Redis with different lifetimes by kind: 24 hours for demand, since a megawatt figure converts to tonnes the same way today and tomorrow; one hour for regional summaries; 30 minutes for cluster sets. Those lifetimes only make sense because each tool returns one kind of thing. A tool that both retrieved and computed would have to be cached at the shortest lifetime of anything inside it.

What the model is allowed to decide

Tool granularity is only half of it. The other half is how much the model infers rather than asks.

For the common case the tool sequence is prompted rather than discovered: demand, then radius, then retrieval, then supply curve. The model is not reasoning its way to that order on each query. A deterministic pipeline is auditable, and for decision support that matters more than flexibility.

Autonomy is concentrated at three points instead.

Parameter inference. "Fire risk matters" becomes a specific weighting. "Maximize biomass" selects one silvicultural treatment over another. "Power plant" versus "CHP" sets a conversion efficiency. The user never sees a parameter name and never types a number.

Exception recognition. Deciding when enough context is present to skip a clarification turn.

Session continuity. Deciding that a parameter changed but the location did not, so the cached cluster set still applies. This is the decision the tool split exists to enable.

Removing the inference mappings and requiring explicit specification drops estimated completion from 96.4% to somewhere between 78 and 85%. That figure is estimated from query-level dependency rather than measured by rerunning the benchmark, so treat it as a direction rather than a number.

Where it breaks

Cache invalidation is manual and rule-based. The agent decides that a location change invalidates the cluster set and a weighting change does not. That rule is correct for the parameters currently exposed. Add a parameter that silently affects retrieval, such as a maximum slope filter, and the rule is quietly wrong, serving a stale cluster set with no error. Nothing in the system detects this. It is a rule someone has to keep true by hand.

Nine tools fit in a system prompt. Ninety would not. This split works partly because the whole tool list is small enough for the model to hold at once. A larger surface would need retrieval over the tools themselves, which is a different problem.

Two of 55 benchmark queries stalled on clarification. Both were cases where the agent asked rather than guessed. That is the failure mode you want, but from the user's side an agent that asks a question instead of answering one has still failed the turn.

The read and compute boundary assumes retrieval dominates. It does here, by a wide margin. In a system where the computation were the expensive part, this split would buy nothing and the tools should be drawn somewhere else entirely.

More on what the system can and cannot answer at /docs/capabilities-overview.

References

1. Yao, S. et al. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. ICLR.

2. Schick, T. et al. (2024). Toolformer: Language Models Can Teach Themselves to Use Tools. Advances in Neural Information Processing Systems, 36, 68539–68551.

3. Redis Ltd. Redis. Documentation.