Beyond Structured Outputs: The Validation Layer LLMs Still Need

Every team that generates content with an LLM writes the same defensive code
twice. First a prompt that ends in "respond with JSON in this exact shape."
Then, a week later, the parser that copes with what actually comes back: a
stray code fence, a preamble sentence, a key named "answer" instead of
"correct_answer", a number arriving as a string.

So when constrained decoding — "structured outputs" — went generally available
on our cloud LLM provider, the obvious move looked obvious. We were about to
multiply our generation surface by a large factor, which multiplies every parse
failure along with it. Hand the provider a JSON Schema, let the decoder refuse
to leave the grammar, and delete the tolerant parsing.

We did delete it. What we nearly also deleted was the semantic validator
sitting behind it — and that would have been a real bug in a feature that
scores people on the answers it generates. Here is why the two look like one
job and aren't.

01 / THE PROBLEM — ASKING IS NOT THE SAME AS ENFORCING

A prompt is a request. A grammar is a constraint. That distinction sounds
academic until you look at where the JSON is decided.

With a prompt, the model samples freely and hopefully lands on valid JSON;
your parser is the thing that copes when it doesn't. With constrained decoding,
the provider compiles your schema into a grammar and, at each step, masks out
any token that would take the output off a valid path. Malformed JSON stops
being a category of failure, because it is no longer reachable.

Constrained decoding removes the box you wrote by hand.
It does not remove the box below it.

The temptation is to read "the output is now schema-valid" as "the output is
now valid." Those are different claims, and the second one is the one your
product actually needs.

THE TRAP — A SCHEMA LANGUAGE IS A SUBSET

Providers don't implement all of JSON Schema. They implement a subset of Draft
2020-12, chosen for what can be compiled into a deterministic grammar. On ours,
the unsupported list includes:

• String constraints — minLength, maxLength: not supported.
• Numeric constraints — minimum, maximum, multipleOf: not supported.
• minItems — honoured only for the values 0 and 1.
• Recursive schemas, external $ref, and additionalProperties set to anything
  but false.

Now put our real rules next to that list. We generate multiple-choice care
questions, and the content is unpublishable unless:

1. There are 2–6 options, no duplicates.
2. Each option is at most 60 characters (it has to fit a phone).
3. The explanation is at least 60 characters (a one-word explanation teaches
   nothing).
4. correct_answer is character-identical to one of the options.

Every single one of those lives in the unsupported half. Rule 1 needs
minItems/maxItems at values above 1. Rules 2 and 3 need string bounds. Rule 4
isn't a schema question at all — no keyword anywhere says "this string must
equal a member of that array."

The grammar owns SHAPE. The validator owns MEANING.
The bottom row owns itself.

Read the table top to bottom and the split names itself: shape goes in the
schema, meaning stays in code. Anything counted, measured, compared across
fields, or checked for distinctness is meaning.

THE APPROACH — DELETE THE PARSER, KEEP THE JUDGE

That split told us exactly what to remove. Gone: fence-stripping,
brace-hunting, key-alias guessing, string-to-int coercion, and the
default-filling that quietly turned a half-answer into a whole one. Roughly a
third of the file, and the third that was hardest to reason about, because
"tolerant" and "wrong" are neighbours.

The schema we send is deliberately boring — shape only:

{
  "type": "object",
  "additionalProperties": False,
  "required": ["question", "options", "correct_answer", "explanation"],
  "properties": {
    "question":       {"type": "string"},
    "options":        {"type": "array", "items": {"type": "string"}},
    "correct_answer": {"type": "string"},
    "explanation":    {"type": "string"},
  },
}

And the validator behind it stayed intact — same rules, now the only thing
standing between a generation and a published question:

def unpublishable(fields) -> bool:
    options = fields["options"]
    if not MIN_OPTIONS <= len(options) <= MAX_OPTIONS:   # minItems can't say this
        return True
    if len(set(options)) != len(options):                # nor uniqueness
        return True
    if any(len(o) > MAX_OPTION_CHARS for o in options):  # nor maxLength
        return True
    if len(fields["explanation"]) < MIN_EXPLANATION:
        return True
    return fields["correct_answer"] not in options       # nor cross-field agreement

Three operational details are worth knowing before you ship this, because none
of them are in the happy path:

• An unsupported keyword is a 400 at request time, not a relaxation. That is
  the good failure — a schema written from memory breaks the call loudly
  instead of silently accepting a 12-option question. But it means the
  keyword list is a build-time dependency, not a footnote.
• The first request for a new schema compiles a grammar, and compilation can
  take minutes. Compiled grammars are then cached for about a day. Our
  generation is asynchronous and never sits on a page load, so a cold schema
  is invisible. Put a constrained call on a synchronous read path and that
  same fact is a latency incident.
• Feature interactions are real. On our provider, constrained decoding is
  mutually exclusive with citations, and is rejected outright on one of the
  API surfaces. Check the matrix, not just the announcement.

THE RESULT — AND ONE STUB WE DELIBERATELY KEPT USELESS

Malformed-JSON failures went to zero, and refusals now arrive with a named
reason from the validator instead of a shrug from the parser. That was the
expected win.

The unexpected part came from our offline path. When no model is configured —
local development, and any test run without live credentials — calls fall
through to a stub. Before structured outputs, that stub emitted an echo which
happened to be unparseable as a question. Nobody designed that, but it had
become load-bearing: it meant a development run physically could not seed real
content with filler.

Handing that stub a schema would have destroyed the property. It would have
started emitting well-formed, publishable-looking content into a real bank.

So the stub is now schema-shaped on purpose and refusable on purpose. It walks
the schema, knows nothing about our domain, and fills every string with the
same placeholder. Which means every array of strings comes out as a list of
duplicates — and a duplicated choice is precisely what a multiple-choice
validator must reject, because the correct answer becomes ambiguous. The
offline path exercises the entire pipeline end to end and still cannot publish
a single row.

An unusable generation publishes nothing.
Filler in a scored bank is worse than an empty slot.

A well-formed answer just became the cheap part. Deciding whether it's a GOOD
answer is still entirely yours.

TAKEAWAYS

• "Schema-valid" and "valid" are different claims. Before you plan a
  validation layer away because the platform now enforces schemas, read which
  keywords that platform actually implements.
• Split on shape versus meaning. Keys, types, enums, closed objects → the
  grammar. Counts, lengths, distinctness, cross-field agreement → your code.
• Cross-field rules are never schema rules. "This field must match a member
  of that array" has no keyword in any dialect. If your domain has one of
  these, you have a validator, permanently.
• Test with output that is valid-but-refusable. A fixture that's malformed
  only proves the parser works. A fixture that's perfectly shaped and
  semantically wrong is what proves the validator does.
• Keep constrained calls off synchronous paths until the grammar is warm.
  Asynchronous generation absorbs a cold compile; a page load does not.

A NOTE ON WHERE THIS CAME FROM

Hoomanely builds technology for pet parents — connected devices, health
signals, and AI that turns all of it into guidance a household can act on. Our
mission is straightforward: help people give their animals longer, healthier,
better-understood lives.

This particular piece of plumbing sits under a feature that generates daily
care questions written for one specific animal — its breed, age, life stage,
declared conditions — and then scores the owner's answer. That last detail is
why the validator is a product-safety layer rather than developer hygiene. In
general contexts, a reader can often tell when a model is wrong. In health
contexts they usually cannot; that asymmetry is well documented, and it is the
whole reason a confidently well-formed wrong answer is more dangerous than a
malformed one.

Constrained decoding made our output well-formed. Keeping the validator is what
keeps it answerable for.