AI Systems · Architecture

Function-Calling and Tool-Use Systems

Function calling architecture for LLM tool-use systems: schema registry, execution sandbox, idempotency, write confirmations, traces, and key failure modes.

Read time
16 min
Level
Advanced
Updated
2026-08-25
Components
Schema registry · Execution sandbox · Idempotency layer
The short answer

A function-calling system is the production layer that lets an LLM select tools from a versioned schema registry, execute them in a sandbox with idempotency keys, and apply confirmation gates on writes — while every call is traced, authorized, and bounded by timeout and retry policy.

Key takeaways

  • 01

    Tool schemas are an API you own: version them, validate arguments server-side, and never trust the model to honor a description-only constraint.

  • 02

    Execute tools in a sandbox with least privilege; the model proposes calls, the runtime decides whether they run.

  • 03

    Idempotency keys on every side effect turn retries and double-emits into no-ops instead of duplicate payments or tickets.

  • 04

    Reads can be autonomous; writes, sends, and payments pause for confirmation unless a narrow, tested policy says otherwise.

  • 05

    Most tool-use failures are schema drift, missing authz, and unbounded retries — not the model picking the wrong function name.

01

Function calling architecture: registry to runtime

Function calling architecture is how an LLM tool use system actually runs in production: a registry of tools, a model that emits structured calls, a runtime that authorizes and executes them, and a trace of every attempt. "Agent tools" are not prompt text. They are typed functions with owners, scopes, side-effect class, and SLAs, exposed to one or more models through a single gateway.

Keep the model on the proposing side of a hard boundary. It may choose a tool and fill arguments; it may not hold credentials, talk to the network, or skip validation. That split is what makes the system auditable when a call goes wrong.

  • Registry: name, JSON schema, side-effect class, owner, version
  • Authorizer: identity, allow-list, argument constraints
  • Executor: sandbox, timeout, retries, idempotency
  • Trace: call, args (redacted), result, latency, cost
02

Schema registry and argument validation

The schema registry is the source of truth for what the model is allowed to request. Each tool publishes a JSON Schema (or equivalent) for arguments, a side-effect flag (read, write, send, payment), and a version. The prompt the model sees is compiled from the registry, so a retired tool disappears from both docs and runtime in one change.

Validate arguments on the server against the schema before execution. Enum constraints, ID formats, and max lengths in the description are hints; they are not enforcement. Reject unknown fields. Pin tool versions per agent so a schema change cannot silently alter a live workflow mid-flight.

03

Execution sandbox, authz, and retries

Run tools with least privilege in a sandbox: scoped credentials, network allow-lists, timeouts, and output size caps. Authz is evaluated at call time against the end user's identity, not a shared service account that can see every record. A research tool and a refund tool should not share a credential.

Retries belong to the runtime. Use bounded exponential backoff on idempotent reads; on writes, retry only with the same idempotency key. Surface structured errors to the model (not stack traces) so it can correct arguments without guessing. Kill the call at the timeout — hanging tools stall agents more often than wrong tools do.

04

Idempotency and confirmation gates

Every side-effecting tool takes an idempotency key derived from the conversation or task ID plus tool name plus a canonical argument hash. The executor stores the first result and returns it on duplicates. Without that, a model that "tries again" creates two tickets, two emails, or two charges.

Confirmation gates sit in front of irreversible tools. The runtime pauses, presents a human-readable diff of the proposed call, and proceeds only on approval (or on a documented auto-approve policy with a tight allow-list). Reads stay autonomous. Mixing those two classes in one un-gated toolkit is how agents send mail while "looking something up".

05

Failure modes and when not to use tool calling

Typical failures: the model invents a tool name, omits a required field, passes a plausible but unauthorized ID, or loops on a tool that always errors. Schema drift — prompt catalog and runtime registry out of sync — produces calls that look valid to the model and 404 at execution. Unbounded agent loops turn a flaky API into a cost incident.

Do not put a tool behind the model when a deterministic workflow already exists. If the path is "always create a ticket from this form," a rules engine is cheaper and safer. Function calling earns its complexity when the next action is genuinely conditional on unstructured input. Also skip it when you cannot sandbox the side effect at all.

  • Hallucinated tools or arguments that bypass client-side hints
  • Shared over-privileged credentials
  • Duplicate side effects without idempotency keys
  • Retry storms and missing confirmation on writes
06

Evaluating a tool-use system

Golden cases should cover tool selection, argument validity, refusal to call unauthorized tools, and end-state correctness after execution. Deterministic graders can check schema validity, allow-list hits, and idempotent replay. Trace every call with redacted arguments so you can replay incidents. Gate registry changes in CI: a new required field is a breaking change for every agent that uses that tool.

Frequently asked questions

What is function calling architecture?

Function calling architecture is the production path from a versioned tool schema, through model-proposed calls, to authorized, sandboxed execution with traces. The model selects a tool and fills arguments; the runtime validates, enforces policy, executes, and records the result. Descriptions in the prompt are not a security boundary.

What is an LLM tool use system?

An LLM tool use system is the registry, authorizer, sandbox, and trace around agent tools. It decides which functions exist, who may call them, how arguments are checked, how writes are confirmed, and how failures retry. Without that layer, "tools" are just text the model can ignore or invent.

Why sandbox tool execution?

Because the model is untrusted at execution time. A sandbox applies least-privilege credentials, network allow-lists, timeouts, and output caps so a bad argument cannot reach the rest of the estate. Authz is checked per call against the end user. Shared god-mode service accounts undo the rest of the design.

When do writes need confirmation?

Any irreversible action — sending a message, mutating a system of record, charging money — should pause for HITL agent approval unless a narrow, tested auto-approve policy says otherwise. Reads can run autonomously. Confirmation is a runtime gate, not a polite instruction in the tool description.

How do you make tool calls idempotent?

Pass an idempotency key on every side-effecting call, store the first result, and return it on duplicates. Derive the key from task ID, tool name, and canonical arguments so model retries and network doubles do not create two tickets or two payments. Reads may retry freely; writes retry only with that key.

Keep reading

ArchitectureMulti-Agent Orchestration SystemsQualityLLM Evaluation Systems (Evals)TrainingRL Environments for Agent TrainingArchitectureAgent Memory SystemsArchitectureAgentic RAG SystemsArchitectureComputer-Use Agent SystemsArchitectureGraphRAG SystemsArchitectureHuman-in-the-Loop AI SystemsArchitectureHybrid Retrieval and Re-ranking SystemsQualityLLM Guardrail SystemsInfrastructureLLM Inference and Serving SystemsInfrastructureLLM Observability and TracingInfrastructureMCP Tool Gateway SystemsArchitectureModel Routing and Fallback SystemsTrainingPEFT and Fine-Tuning PipelinesArchitecturePermissioned Retrieval SystemsQualityPrompt Injection Defense SystemsArchitectureRAG ArchitectureArchitectureRealtime Voice AI SystemsQualityAI Red-Teaming SystemsArchitectureStructured Generation SystemsTrainingSynthetic Data Generation Systems

Building one of these systems?

We help teams design, build, and validate production AI systems — orchestration, evals, and training environments included.

FAQ

Working with us

How soon can AI systems work start?

Typically within a week or two of a scope being agreed. The first delivery is deliberately a small batch so you can check the output against your expectations before volume ramps.

What do you need from our team?

One process owner who knows the workflow, one engineer with access to the systems involved, and a weekly 45-minute review. No standing committee, and no requirement for an ML specialist on your side.

Who owns the output and the data?

You do. Datasets, labels, weights, evaluation suites and runbooks are yours and are handed over at the end. Your data trains your models only, with zero-retention provider settings by default.

Can you scale volume up quickly if we need it?

Yes, and the quality bar holds because the rubric and gold set are already agreed by that point. Ramping is a staffing question, not a re-scoping one, so it usually takes days rather than a new engagement.

We already have a vendor for this. Why switch?

Often you should not. The cases where teams move to us are when they cannot get a quality number out of their current vendor, or when the work is delivered as an opaque batch with no trace of how disagreements were resolved.

What happens after the engagement ends?

We stay on-call for 30 days at no extra cost, then move to an optional support retainer. Most teams also keep a quarterly evaluation review with us to catch drift early.

Copyright © 2026
ReinforcedX, Inc.
All rights reserved