Structured Data: Schemas and Honest Checks
Ask Claude for JSON in a fixed shape, check the shape with a real program, and then check the content against the source, because a perfectly shaped reply can still be wrong.
In this lesson you will
- Describe a schema with fields, types and allowed values
- Ask for a fixed shape with a way out for missing values
- Check the shape by program
- Check the content against the source
About 24 minutes
Meera wants invoices turned into a spreadsheet
Meera receives dozens of supplier invoices as text. She wants each one turned into a tidy row: supplier name, amount, currency and whether it has been paid. That means getting Claude to answer in a fixed shape, usually JSON, so that a spreadsheet or a program can read it. This lesson is about asking for structured output, checking that the shape is right, and, just as important, checking that the content is true. The first is easy to automate. The second is the part people forget.
Ask for the shape you want
Lesson 5 showed a strict check of JSON replies. Now we describe the shape properly. A schema is a description of the data you expect: which fields exist, which are required, what type each has, and which values are allowed. Anthropic’s developer documentation describes a structured outputs feature for the API that constrains replies to follow a schema you provide, and says it is meant for jobs such as extracting data from text, and that availability depends on the model and platform. In an ordinary chat you do not get that guarantee, so you ask clearly and check the reply yourself. Either way the habit is the same: say exactly which fields you want, with an example.
Read the invoice text below and reply with JSON only, with exactly these keys: name (text), amount (number), currency (one of GBP, EUR, USD), paid (true or false). If a value is not in the text, use null for it. Do not add any other keys.
Two different checks
A schema check asks: is this reply the right shape? A source check asks: is what it says true to the document? A reply can pass the first and fail the second, which is why reports on the feature point out that structured outputs make the format reliable, not the facts. A perfectly formatted wrong number is the most dangerous kind, because nothing looks wrong. The program below does both checks on six saved example replies to one invoice. The replies are written by hand as examples, not produced by Claude.
import json
SOURCE = "Invoice from Brightway Print for 240.50 GBP, paid on 3 June."
SCHEMA = {"name": str, "amount": (int, float), "currency": ["GBP", "EUR", "USD"], "paid": bool}
def check_schema(raw):
try:
d = json.loads(raw)
except json.JSONDecodeError:
return None, "not JSON"
if set(d) - set(SCHEMA):
return None, "extra key " + sorted(set(d) - set(SCHEMA))[0]
for k, rule in SCHEMA.items():
if k not in d:
return None, "missing " + k
v = d[k]
if isinstance(rule, list):
if v not in rule:
return None, k + " not allowed"
elif isinstance(v, bool) != (rule is bool) or not isinstance(v, rule):
return None, k + " wrong type"
return d, "schema ok"
def check_source(d):
problems = []
if d["name"] not in SOURCE:
problems.append("name")
if f"{d['amount']:.2f}" not in SOURCE:
problems.append("amount")
if d["currency"] not in SOURCE:
problems.append("currency")
return problems
# Saved EXAMPLE replies (not real Claude replies)
REPLIES = [
'{"name": "Brightway Print", "amount": 240.5, "currency": "GBP", "paid": true}',
'{"name": "Brightway Print", "amount": 420.5, "currency": "GBP", "paid": true}',
'{"name": "Brightway Print", "amount": 240.5, "currency": "GBP"}',
'{"name": "Brightway Print", "amount": 240.5, "currency": "GBP", "paid": true, "note": "ok"}',
'{"name": "Brightway Print", "amount": "240.50", "currency": "GBP", "paid": true}',
'Here is the data: Brightway Print, 240.50 GBP',
]
for i, raw in enumerate(REPLIES, 1):
d, msg = check_schema(raw)
if d is None:
print(f"reply {i}: {msg}")
else:
bad = check_source(d)
print(f"reply {i}: {msg}, " + ("matches source" if not bad else "NOT in source: " + ", ".join(bad)))
reply 1: schema ok, matches source
reply 2: schema ok, NOT in source: amount
reply 3: missing paid
reply 4: extra key note
reply 5: amount wrong type
reply 6: not JSON
Read the results one at a time. Reply 1 passes both checks. Reply 2 is the lesson in one line: the shape is perfect, but the amount is 420.50 while the invoice says 240.50, two digits swapped, so only the source check catches it. Reply 3 is missing a required key, reply 4 has an extra key, reply 5 gives the amount as text instead of a number, and reply 6 is not JSON at all because Claude added a friendly sentence. Those four are shape problems, and a program can reject them instantly and ask again. Reply 2 is a content problem, and only a comparison with the source, by you or by code, can find it.
A sensible routine
- Describe the fields, types and allowed values, and say what to do when something is missing, such as use null.
- Give one worked example of a correct reply, so the shape is unmistakable.
- Check the shape by program, and reject and re-ask when it fails.
- Check the content against the source: amounts, dates and names should appear in the text.
- Keep a human check for anything that matters, such as payments.
Letting Claude fill a required field when the information is missing. If a field is demanded and the document does not contain it, a guess is likely. Always give a way out, such as null or not stated, and then treat any filled value as something to verify.
Types matter. The amount 240.50 as a number and the text 240.50 look alike on screen, but a spreadsheet treats them differently. Say which you want.
Practice for this lesson
Check what you learned
Five quick questions. Pick an answer and you will see straight away whether it is right and why.
What does a schema describe?
- The colour of the page
- How long a reply is
- Which fields, types and values are expected
- Which model to use
A schema is a description of the data shape you expect.
Why did reply 2 need the source check?
- It was not JSON
- It had an extra key
- It was missing a key
- Its shape was right but the amount was wrong
Only a comparison with the invoice shows 420.50 does not match 240.50.
What can structured outputs make reliable?
- That every fact is true
- A valid shape, not true facts
- Nothing at all
- That nothing is missing
Format can be constrained, but the values can still be wrong.
What should you offer when a field may be missing?
- A request to guess
- A longer prompt
- A way out such as null
- Nothing
A forced field invites guessing; null gives an honest option.
Which reply had the wrong type?
- Reply 5, where the amount was text
- Reply 1
- Reply 4
- Reply 3
The amount was given as 240.50 in quotes, which is text, not a number.
Summary
- A schema says which fields, types and values you expect.
- Format can be checked by program; a valid shape is not a true answer.
- Always compare values with the source.
- Offer null or not stated for missing values.
Questions
Is structured output the same as a schema check in chat?
No. Anthropic describes a structured outputs feature for the API that constrains replies to a schema, depending on model and platform. In an ordinary chat you ask clearly and check the reply yourself.
Why not trust a reply that passes the schema?
The schema only checks shape. Commentary on the feature notes that values can still be wrong, so compare names, amounts and dates with the source.
Are the replies in the program real?
No. They are saved examples written by hand to show each kind of problem. Your own replies will differ.
