AI Forge · Practical AI & coding

Never trust an LLM's output directly

The validation layer that goes between the model and the code that believes it

The failure shows up in nearly every agent codebase: the code receives a model response, trusts the JSON inside it, and reads result.items[0].id — which throws at 2 a.m. because the model returned {"result": null} on an edge case. The model didn't hallucinate the content. It hallucinated the structure.

Why structured-output mode isn't the whole fix

Constrained JSON modes are genuinely useful — use them. But they don't close the gap, for two reasons:

• JSON-valid is not semantically valid. A response can match your schema exactly and still be wrong: a UUID field holding a made-up id that fails the database lookup, a confidence field of 847 where the code expects a 0–1 float. The schema enforces types, not meaning.
• Not every call uses it. Chain-of-thought steps, tool-call parsing, and models without native JSON mode all hand you free text. You still have to parse that robustly.

The pattern: parse, validate, classify

Every model call runs through three stages, and the caller is forced to handle the outcome:

1. Parse — pull the structure out of the text. Models wrap JSON in prose or code fences; extract it before JSON.parse, and guard the empty-response case first.
2. Validate — assert the parsed structure against a real schema (Zod, or equivalent), so a shape that merely looks right can't slip through.
3. Classify — return a discriminated union — { ok: true, data } or { ok: false, reason, raw } — so the calling code physically cannot read .data without first checking .ok. The type system enforces the error handling.

Retry only what's worth retrying

Not every failure is permanent. A malformed-JSON attempt often succeeds on retry; an empty response usually means something else is broken, so don't retry it. The lever that actually moves the number: feed the validation error back into the next prompt — "your last response failed validation: <error>" — rather than blindly re-rolling. Telling the model what was wrong gets a valid output far more often than hoping the dice land better.

The one-line rule

Treat every model response as untrusted input from the network: parse it defensively, validate it against a schema, and make the failure path a value the caller must handle — never an exception you discover in production at 2 a.m.