Agent Tool Contracts That Survive a Model Swap

Agent Tool Contracts That Survive a Model Swap

Most teams wire tool calls straight into the LLM prompt and hope the next model version keeps the same behavior. That rarely holds. When you swap models, argument formats shift, required fields change, and error handling diverges. Agent tool contracts fix this by putting a stable layer between the agent and the model.

Direct tool calling breaks on upgrades

We have seen production agents stop working after a routine model update. The old model returned arguments that matched our internal functions. The new model added extra fields or changed date formats. The agent then failed on the first real request.

Fixing each failure one by one takes weeks. Each model swap restarts the same cycle.

Defining agent tool contracts

An agent tool contract is a JSON Schema that declares exactly what the model must return for each tool. It lists required fields, types, and allowed values. The contract sits in front of the model output and rejects anything that does not match before the code ever runs the tool.

We keep the contract versioned and stored alongside the agent code. When a new model arrives, we run the same test set against the contract. If compliance stays above 95 percent, we allow the swap. Below that threshold, we fix the prompt or the contract before proceeding.

Tradeoffs in enforcement

Two approaches appear most often in our work. The first uses a strict schema validator that rejects any deviation and forces the model to retry. The second uses a lighter schema plus runtime checks that attempt to coerce values and log mismatches for later review.

Strict validation catches bad output early but increases token use on retries. The lighter approach reduces immediate failures yet lets malformed calls reach downstream systems more often. We choose strict validation when the tool performs writes or payments. We accept the lighter version only for read-only queries where partial results can be discarded.

A common failure mode

One model we tested began returning null for optional fields that the contract marked as required. The agent retried three times, then fell back to a default that produced incorrect results in the UI. The root cause was that the model treated the field as optional despite the schema.

We now add an explicit instruction in the system prompt that lists every required field and states that null is never acceptable. We also added a contract test that sends the exact prompt the model saw during the failure and checks that the output passes validation on the first attempt.

Keeping contracts current

Contracts drift when new tool features are added without updating the schema. We require any pull request that changes a tool signature to include an updated contract and at least three new test cases. The CI job fails if the contract and code disagree.

This rule keeps the contract and implementation aligned without requiring constant manual review.

Final takeaway

Put the JSON Schema contract between the model and every tool call, then enforce it on every model change.