Tool routing and the supervisor pattern

Decoupling model logic from action execution using central orchestration hubs.

Why it matters

Allowing an LLM to freely invoke long-running tasks can lead to runaway loops and unpredictable state mutation. The supervisor pattern introduces an authoritative orchestrator node that interprets intent, selects the single most efficient specialized worker node or tool, and cleanly absorbs the resulting output back into the graph state.

The supervisor pattern puts one orchestrator node in charge of deciding which tool runs next, instead of letting the LLM free-call long-running actions. The supervisor reads state and routes; workers execute and return.

function route(state: State): Tool {
  if (state.containsPII)        return 'humanReview';   // safety first
  if (state.question.includes('connected to')) return 'graphQuery';
  if (state.confidence < 0.55)  return 'ticketLookup';
  if (state.spendUsd > budget)  return 'cheapModel';    // cost guard
  return 'vectorSearch';
}

Safety routes come before optimisation routes. Notice humanReview is checked first. A production supervisor needs at least one guardrail branch (PII, irreversible action, low confidence) that fires regardless of how 'efficient' another path looks. Happy-path-only routing is how agents take unsafe actions confidently.

Bounded autonomy. By funnelling tool choice through one node, you cap loops, log every decision in one place, and can enforce policy (allow-lists, cost ceilings) centrally. Compare that to scattering tool calls across many prompts, where no single place can stop a runaway.

Cost is a routing dimension. Add a branch that downshifts to a cheaper model or a retrieval-only answer once a per-request budget is exceeded, so cost control is a first-class decision, not a surprise on the invoice.

The supervisor is an emergency dispatcher. Callers (the LLM's intent) don't drive the ambulances themselves — the dispatcher assesses each call, sends the right unit, and always escalates a life-threatening case to a human, no matter how busy the queue.

• No safety branch. Symptom: unsafe actions on edge cases. Fix: guardrail routes checked first.
• LLM free-calling tools. Symptom: runaway loops. Fix: centralise tool choice in the supervisor.
• No cost ceiling. Symptom: budget blowouts. Fix: a spend-aware downgrade branch.
• Unlogged routing. Symptom: undebuggable choices. Fix: log every supervisor decision + state.