BeginnerLesson 7 of 306 min read

Research, Learning and Checking Facts

Learn why Claude can make confident mistakes, a four-step method for learning with it, how to ground answers in your own sources, and how to flag unsupported numbers with a real checker.

In this lesson you will

  • Explain hallucinations
  • Use a four-step method for learning
  • Ground answers in a source
  • Flag claims that need checking

About 22 minutes

A confident answer with a wrong date

Noor asks Claude for a short history of her town’s library for a school project. The answer is fluent, detailed and confident. She copies the opening year into her work. Her teacher circles it in red: the library opened decades earlier. Claude had produced something that sounded right and was not. This kind of error has a name, hallucination: a reply that contains made-up or incorrect information, written in the same fluent style as correct information. It is the most important thing to understand about using AI to learn.

Why it happens

Claude writes by predicting plausible text. When it knows something well, plausible and true usually match. When it does not, for a small local fact, a recent event, an exact quote, a specific number or a source, it can still write something plausible, and plausible is not the same as true. Anthropic’s own help pages say that Claude can occasionally produce incorrect or misleading responses, including quotes that look authoritative but are not grounded in fact, and that for current events it may not have the latest information. This is a limit of the technology, not a sign of a broken tool, and it does not make Claude useless. It means you need a method.

A four-step method for learning with Claude

  1. Ask for the idea first. Claude is excellent at explaining concepts and giving examples. Ask it to explain, compare and quiz you.
  2. Give it the sources. For facts, paste the source text and say ‘use only this text’. Anthropic’s guidance on reducing made-up answers recommends restricting Claude to the documents you give it.
  3. Allow doubt. Tell it it may say ‘I am not sure’ and should not guess.
  4. Check the checkable. Names, dates, numbers, quotes and links need a second source.
Use only the text below to answer. If the text does not say, reply "not in the text". Do not use anything you already know.

<text>
[paste source]
</text>

Question: When did the library open, and how many branches does it have?

Anthropic’s guidance also suggests asking Claude to pull out word-for-word quotes first when working with long documents, and then to answer using those quotes, which makes each claim traceable. If it cannot find a quote for a claim, it should withdraw the claim. These are habits you can borrow even for short texts.

A simple fact-checker

You can also check a reply against a source with a little code. The program below takes a short source text about a library and a reply that claims things about it. It pulls out every number in the reply and checks whether each one also appears in the source. A number missing from the source is not necessarily false, but it is unsupported, so you must check it. The program cannot understand meaning. It only compares numbers, so it will miss wrong names and wrong claims with no numbers in them.

import re

source = ("The Parkside Library opened in 1958. It holds 12,400 books across 3 branches. "
          "The reading club began in 2019 and meets every Wednesday.")
reply = ("The Parkside Library opened in 1985 and now holds 12,400 books across 5 branches. "
         "Its reading club began in 2019 and has 40 members.")

def numbers(text):
    return re.findall(r"\d[\d,]*", text)

src = set(numbers(source))
claims = numbers(reply)
print("numbers in the reply:", len(claims))
for n in claims:
    status = "in the source" if n in src else "NOT in the source"
    print(f"  {n}: {status}")
unsupported = [n for n in claims if n not in src]
print()
print("to check:", ", ".join(unsupported))
print("share supported:", f"{100 * (len(claims) - len(unsupported)) // len(claims)}%")
numbers in the reply: 5
  1985: NOT in the source
  12,400: in the source
  5: NOT in the source
  2019: in the source
  40: NOT in the source

to check: 1985, 5, 40
share supported: 40%

The reply contains 5 numbers. The 12,400 books and the 2019 start of the reading club both appear in the source, so they pass. But three do not: 1985, 5 and 40. The library opened in 1958 according to the source, so 1985 is a transposed year, a classic slip. The source says 3 branches, but the reply says 5. And the 40 members are not mentioned anywhere. The program’s summary says 1985, 5, 40 need checking, and only 40 per cent of the numbers are supported. Now look at the other half of the lesson. The program did not tell you what is true. It only told you where to look. That is what a good check does.

What Claude is good for in learning

  • Explaining a concept in three different ways until one clicks.
  • Giving worked examples and practice questions, and marking your answers.
  • Summarising a text you give it, and asking you questions about it.
  • Pointing out what you might have missed, so you know what to look up.

What to double-check

  • Dates, numbers, names and spellings of people and places.
  • Quotes and where they come from. Never use a quote you have not seen in the original.
  • Links and references. Claude can invent plausible-looking ones.
  • Anything recent, local or specialist, or anything with health, money or legal consequences.
Common mistake: self-checking is not enough

Asking Claude to check its own answer and treating the reply as proof. A second reading can catch some errors, and asking it to find supporting quotes is useful, but it is not independent verification. For important facts, go to a source you trust.

Open the sources

When Claude can search the web in your version, it may show sources. Open them. A source can be misread or summarised badly, and the original often has context the summary lacks.

Ask Claude a question about your local area, then check three details of its answer against a reliable source. Count how many were right. Then paste a source and ask the same question using only the source, and compare.

Practice for this lesson

Check what you learned

Five quick questions. Pick an answer and you will see straight away whether it is right and why.

What is a hallucination in this context?

  1. A picture
  2. A slow reply
  3. A fluent reply containing made-up or incorrect information
  4. A spelling mistake only

It is wrong or invented content written in the same confident style as correct content.

Which kind of information is risky to trust without checking?

  1. Dates, numbers, quotes and links
  2. The word hello
  3. A greeting
  4. A colour

Specific facts and references are the usual places for errors.

What does restricting Claude to your source do?

  1. Makes it longer
  2. Switches language
  3. Reduces invented facts
  4. Hides errors

Anthropic’s guidance suggests using only provided documents to keep answers grounded.

What did the program do with numbers?

  1. Flagged those not in the source for checking
  2. Decided which were true
  3. Deleted them
  4. Added more

It only shows where to look, not what is true.

Is asking Claude to double-check itself enough for important facts?

  1. Yes, always
  2. No, use a trusted source
  3. Only on Mondays
  4. Only for dates

Self-checking can catch some errors but it is not independent verification.

Summary

  • Claude can write wrong things fluently, so you need a method.
  • Ask for ideas, supply sources, allow doubt and check the checkable.
  • Restricting answers to your source reduces invented facts.
  • A check shows where to look, not what is true.

Next lesson

Lesson 8 covers Projects, artifacts and keeping work organised.

Questions

Why does Claude sometimes get facts wrong?

It writes plausible text, and for small, local, recent or specific facts, plausible can differ from true. Anthropic's help pages say it can occasionally be incorrect or misleading.

Can I trust Claude's sources?

Open and check them. Summaries can miss context, and references can be wrong or invented.

Is it good for studying?

Yes, for explanations, examples and practice questions. For facts you will be marked on, confirm with your course materials.

Progress is saved in this browser.