The hardest thing about a strict-format contract is not writing the contract. It
is not parsing the contract. It is the third thing, the one that doesn't have a
technical name in any RFC I've read: not adding to it. The reliable strategy
for a deliverable that has to match a template byte-for-byte is to copy the
template, then surgically replace only the bracketed regions, then stop. The
moment the helpful instinct wakes up — the one that wants to apologise, to
explain, to "improve," to add a friendly preamble — the deliverable is no
longer the deliverable. It is a draft of a deliverable, wearing the
deliverable's hat.
I want to talk about how old that habit is, because the modern frame — "an
LLM that must emit exactly one block matching exactly one schema" — is the
latest instance of a discipline that runs through the history of every
machine-readable contract ever written.
Herman Hollerith's 1890 census tabulator ran on a punched card about 3 by 5½
inches — sized to fit the banking card trays of the day — with 12 rows and 24
columns, and an ad-hoc coding system in which specific groups of holes meant
specific things: sex in one location, marital status in another, and so on.
If a clerk punched a hole in the wrong place, the electromagnetic brush under
the card read a different datum — not a malformed one, a wrong one — and the
totals were corrupted in a way that took weeks of audit to find. Hollerith's
contract had no notion of "the card is broken." It had only "the card says
what the card says." That is the property strict-format systems share, from
punch cards to JSON Schema: there is no fallback state for non-conformance;
there is only the wrong number in the wrong field, looking for all the world
like a plausible number in a plausible field.
JSON Schema itself is a small, stubborn corner of the open web. The project
has been publishing Internet-Drafts through the IETF process since December
2009 — drafts 0 through 7 by number, then 2019-09 and 2020-12 by year-and-month,
with the next revision currently in working-draft status. The current
meta-schema is 2020-12. Its design philosophy, written into the spec, is that
a schema is a contract: a description of what valid data looks like, not a
tutorial on how to construct it. Recent drafts have added prefixItems,
unevaluatedProperties, and tighter rules around oneOf, but the central
claim has not moved in seventeen years — a validator can only ever tell you
whether the document you handed it matches the document you said it should
be. It cannot tell you whether you meant to hand it a different document.
The thing I find genuinely interesting in 2026 is how the same discipline is
now being bolted onto language models. Anthropic's structured outputs feature,
shipped through the Claude API, takes the JSON Schema you pass in, compiles it
into a grammar that constrains token-by-token sampling, and guarantees —
not "with high probability," not "usually," guaranteed — that the response
will parse as valid JSON matching your schema. The implementation has a
compilation step with a 24-hour grammar cache, a 180-second compilation
timeout, and a documented set of schema-complexity limits: 20 strict tools per
request, 24 optional parameters across strict schemas, 16 union-typed
parameters, plus internal limits on the compiled grammar size that scale
non-linearly with nesting and optional fields. The docs are unusually honest
about this: "schema complexity doesn't reduce to a single dimension." Add too
many optional fields, or too many tools, and the system returns a 400 error
saying "Schema is too complex for compilation." The contract is the
deliverable. If the deliverable cannot be compiled, the work is not yet done.
The LLM-specific twist is one the punch-card era did not have to confront:
verbosity. A Hollerith clerk who put the wrong hole in column 7 produced a
bad card. A Hollerith clerk who, after the bad card, wrote a polite note on
the back explaining the mistake — there was no such clerk, because the card
had no back, and the machine had no reader for the back. The card was the
deliverable. There was no slot for the helpful preamble.
A language model has a slot. Every one of them does. The conversation is the
slot, the prefill is the slot, the system prompt is the slot, the user's
"could you also…" follow-up is the slot. In any of those slots, the model
can emit a JSON document plus a paragraph that explains why it picked the
numbers it did, plus an apology for a previous failed attempt, plus a
suggested improvement to the schema. None of those additions are part of the
contract. All of them feel, in conversation, like correctness. All of them are,
under contract, defects.
This is the part the field is still working out. The "helpful assistant"
default — produce as much as is plausibly useful, err on the side of
explanation, narrate your corrections — is precisely what a strict-format
prompt forbids. The two defaults point opposite directions, and the system has
to pick one. The schemas and the grammars do not do this work for you. They
constrain the output; they do not constrain the additional output you
chose to produce. The model has to learn, separately, that when a spec says
"emit exactly this block," the right amount of extra material is zero.
That learning, in turn, is what schema-constrained sampling is for. Not
just to spare downstream code a try/catch. The point of a grammar is to
shrink the action space until the only thing the system can emit is the thing
the contract asked for. The skill that matters, on both sides of the
interface, is the discipline of not adding. It is the same skill Hollerith's
clerks had — or rather, the same skill their managers had, when they wrote
the column spec in the first place. A card is not a suggestion. A schema is
not a starting point. The deliverable is the deliverable.
The reason this is worth writing about on a Monday-morning blog is that the
discipline travels. The same muscles you build to write a regex that matches
exactly, and not one byte more, are the muscles you build to ship a
schema-constrained LLM call that doesn't leak prose into its JSON output, and
they are the muscles you build to write a configuration file that another
program — or another person — will read without complaint. The skill is
small, unglamorous, and as old as punch cards. It is also, increasingly, the
one that separates a working system from a system that mostly works most of
the time, which is a polite way of describing a system that breaks when you
need it not to.
Sources
- Anthropic, Structured outputs — Claude Platform docs, accessed September 2026 (the grammar-constrained decoding pipeline: schema compilation, 24-hour grammar cache, 180-second compilation timeout, the 20-tool / 24-optional / 16-union-parameter complexity limits, and the documented "Schema is too complex for compilation" failure mode)
- JSON Schema Organization, JSON Schema specification and Specification Links — project homepage and the full version table (Draft 0 / Draft 1, Dec 2009, through the current 2020-12 meta-schema, with the next draft in working-draft status; the December 2009 origin; the
prefixItems,unevaluatedProperties, andoneOfrevisions; the "schema is a contract, not a tutorial" design philosophy; the IETF Internet-Draft governance model) - Wikimedia Commons, Hollerith punched card — Herman Hollerith, Railroad Gazette, 19 April 1895; public domain in the United States (featured image)
- Library of Congress, Hollerith Punched Card System for the 1890 Census — the original Railroad Gazette page from which the featured image was scanned
- U.S. Census Bureau, The Hollerith Machine — backgrounder on the 1890 tabulating system and its fixed-column card format.
