Tool Use and Agent Reliability: Why Loops Fail Quietly

LLM agents that call tools don't fail because the model can't read the API spec. They fail because the loop has no stop condition that works, no evidence trail a human can verify at two AM, and no evaluation harness that catches degradation before a customer does.

Every LLM Ops team ships agentic tools knowing this will happen. The difference between a one-hour debug session and a three-day outage is how you measure your agent's reliability today.

Silent failure is the default mode

A tool-calling agent exits with code zero, returns an answer that *sounds* right, and nobody notices. Then they do it again at 3 AM during peak traffic — longer this time, more tokens burned, same wrong result. Here are the modes:

Confident hallucinated outputs. The model formats a valid JSON payload, calls the tool (or fakes the response), and declares success. The answer reads coherently. The data is fabricated. This is the most common failure mode for agents in their first quarter of production use — and it's also the hardest to detect because nothing in the output looks broken.

Polite infinite loops. The tool returns a signal the model interprets as "not done yet" when it actually means "try a different approach." The agent retries with marginally different parameters, burns tokens, hits rate limits, tries again. It is not crashing. It is *working*. This pattern often surfaces after fine-tuning or context-window upgrades that subtly change how the model parses error responses — the fix for one bug becomes the cause of another.

Cascade drift. One agent calls a second agent's tool with slightly wrong schema parameters. The callee accepts them (because it doesn't validate the way you expect), produces partial output, and passes it downstream. Each hop in the chain looks reasonable in isolation. By hop four, nothing matches the original intent. No alert fires because every component reports "success."

The dead tool that nobody removed. An API endpoint was deprecated months ago. The agent calls it anyway because the schema reference was never updated. The service returns a 404, the agent catches the exception, retries with exponential backoff, and you have a denial-of-service incident caused by code that stopped working for anyone who wasn't watching.

Why standard evals miss these failures

Benchmarks like MMLU or GSM8K tell you very little about agentic reliability. They measure isolated correctness on static problems with clean inputs. An agent's output depends on:

  1. The environment. Every tool call changes state. Two runs with identical prompts produce different trajectories if the database had a row inserted mid-run.
  2. Temporal drift. The model you tested in July may behave differently than the one you're running in October — not because of weight changes, but because your APIs evolved while nobody updated the eval suite.
  3. Evaluation bias. LLM-as-judge setups (Claude or GPT evaluating Claude's own tool calls) have documented agreement rates under 70% on complex reasoning tasks. The judge and the agent are both likely to make the *same* mistake, so your eval reports "pass" while production suffers.

What actually works in production

Trace-based evaluation with stop conditions. Every tool call needs a trace entry: what was sent, what response arrived, how long it took, whether the result matched an expected schema. If no trace exists, you don't have an agent — you have a chatbot hoping for the best. Stop conditions must be *observable*: max steps, latency caps, or a confidence threshold that triggers human handoff. Without these, the loop is infinite by definition.

Evaluation harnesses that validate trajectories, not just endpoints. Tools like EleutherAI's lm-evaluation-harness now support plugin-based evaluation of agent actions. The point isn't to judge whether the final answer is right — it's to measure whether *each step* in the reasoning chain was valid. A correct answer reached through a flawed trajectory (wrong tool called, skipped validation, assumed permissions) is wrong for compliance reasons even if the output looks fine.

HITL gates at high-decision points. Not every step needs human review. But any action that modifies external state — write, delete, transfer, deploy — should pass through a checkpoint where a person (or an independent system the human trusts) validates the decision before execution. This is not about slowing down agents; it's about ensuring there's something in the loop when things go wrong, so "something went wrong" doesn't mean "we don't know it went wrong for three days."

Regular evaluation sweeps. The model didn't change. Your API did. Every integration update to a downstream tool or schema requires re-running your agent eval harness. Automate this on CI — if someone merges a breaking change to an API that your agent calls, the eval suite should fail before the code reaches any environment where a real person notices.

The MeltingFace way

At MeltingFace — an LLM Ops research consultancy — we forbid silent loops because they're indistinguishable from correct behavior until it costs too much to investigate. Our Presence living lab implements HITL as the publish button: agents run, trace everything, flag uncertainty, and hand off when the confidence curve flattens without a resolution. No unsupervised deployment of tools that touch external state. No agent budget to spend. Just traces, evals, and a person with clearance.

Presence is our live testing ground for these patterns; the blog posts here are observations from running them at scale — or learning what happens when you don't.

If you're running agents that call tools in production and can't describe how you'd know if a loop went quiet tomorrow, start with the trace. Everything else follows.

---

_MeltingFace is an LLM Ops research consultancy. Presence is our HITL living lab — not a chatbot vendor. Explore our approach to LLM Operations or learn more about our services._

MeltingFace