Long Documents: Chunk, Map and Combine
Handle text too long for one go: cut it into chunks with overlap, run the same prompt on each, combine the results, and watch a real program lose a fact at a boundary and then find it again.
In this lesson you will
- Cut a long text into overlapping chunks
- Run the same prompt on every chunk
- Combine the results from quotes, not memory
- Check the seams for lost facts
About 24 minutes
A report too long for one go
Priya has a ninety-page annual report and needs a one-page summary for the trustees. Even if the whole report fits in the conversation, a very long input is harder to handle well, and in many setups it may not fit at all. So she splits it into parts, summarises each part, and then combines the summaries. This is the chunk-and-combine method, sometimes called map and reduce. It works well, and it has one classic way to fail quietly: a fact that is cut in half by the boundary between two chunks.
The method in four moves
- Chunk. Cut the text into parts that are comfortably small, at natural breaks such as headings where you can.
- Overlap. Let each chunk repeat the last few words of the one before it, so that anything near a boundary appears whole in at least one chunk.
- Map. Give each chunk the same prompt: for example, list every date, amount and decision, with the exact words.
- Reduce. Put the lists together, remove duplicates, and ask for the final summary from the combined list, not from memory.
Anthropic’s guidance for long material suggests putting the document first and the question after it, and asking Claude to pull out relevant quotes before answering. Both fit here: the map step is a quote-extraction step, and the reduce step works only from those quotes. Anything that never makes it into a map list cannot reach the summary, so the quality of the whole method depends on the boundaries.
Watch a fact fall through the gap
The program below takes a short made-up notice, cuts it into chunks of eight words, and uses a pattern to pick out the seven facts it contains: dates, times and amounts. It does this twice, once with no overlap and once with an overlap of three words. A pattern stands in for the map step, so the effect of the boundary is easy to see and count.
import re
text = ("The fair is on Saturday 17 June. Stalls open at 9 am. Each stall pays 12 pounds. "
"Tea is served by the church group. The total budget is 400 pounds and it cannot be raised. "
"Posters must be printed by Friday 9 June. Volunteers meet at 8 am. "
"Parking is free at the school. The raffle closes at 3 pm.")
words = text.split()
FACT = re.compile(r"(?:Saturday|Friday) \d+ June|\d+ pounds|\d+ [ap]m")
def chunks(words, size, overlap):
out, i = [], 0
while i < len(words):
out.append(words[i:i + size])
if i + size >= len(words):
break
i += size - overlap
return out
truth = set(FACT.findall(text))
print("facts in the whole text:", len(truth))
for size, overlap in [(8, 0), (8, 3)]:
parts = chunks(words, size, overlap)
found = set()
for part in parts:
found |= set(FACT.findall(" ".join(part)))
sent = sum(len(p) for p in parts)
print(f"size {size}, overlap {overlap}:")
print(f" chunks {len(parts)}, words sent {sent}")
print(f" found {len(found & truth)} of {len(truth)}")
print(" missed:", sorted(truth - found) or "none")
facts in the whole text: 7
size 8, overlap 0:
chunks 8, words sent 60
found 6 of 7
missed: ['12 pounds']
size 8, overlap 3:
chunks 12, words sent 93
found 7 of 7
missed: none
Without overlap the program finds 6 of 7 facts and misses 12 pounds. The reason is plain once you see it: the number 12 ended one chunk and the word pounds began the next, so neither chunk contained the whole fact. With an overlap of three words every fact appears whole in some chunk, and the program finds 7 of 7. The price is more work: 60 words sent without overlap, and 93 with it. That is the trade. A little duplication costs a little more, and it buys back the facts that sit on the seams.
Checks that catch what chunking loses
- Count the facts. If you can count the dates or amounts in the original, count them in the combined list.
- Spot-check the seams. Pick two boundaries and read the text either side yourself.
- Keep source labels. Ask each map step to say which chunk or page a fact came from, so you can trace it.
- Ask for gaps. In the reduce step, say: if two chunks disagree, show both and do not choose.
Summarising summaries of summaries. Each round loses detail and can add small errors. Do one map round and one reduce round where you can, and go back to the quotes when something looks odd.
Where the full document fits comfortably, giving it to Claude whole is often simpler, and then you can ask for quotes with page numbers. Chunking is for when it does not fit, or when you want a separate answer for each section.
Practice for this lesson
Check what you learned
Five quick questions. Pick an answer and you will see straight away whether it is right and why.
Why add overlap between chunks?
- To make chunks shorter
- So facts near a boundary appear whole in some chunk
- To avoid the map step
- To change the topic
Overlap repeats words at the seam so a cut fact is not lost.
In the program, what was missed with no overlap?
- The date of the fair
- Every time
- 12 pounds, cut across two chunks
- Nothing was missed
The number 12 ended one chunk and pounds began the next.
What is the map step?
- The same prompt run on each chunk
- Drawing a map
- Joining the lists
- Deleting the chunks
Map means giving every chunk the same task, such as listing facts with quotes.
What is the cost of overlap?
- Facts are lost
- The text changes
- More words are sent
- Nothing at all
The program sent more words with overlap than without.
Why avoid summaries of summaries?
- Each round loses detail and can add errors
- They are always longer
- They are not allowed
- They need no checking
Every extra round drifts further from the source.
Summary
- Chunk, overlap, map, then reduce.
- A fact cut in half by a boundary can be lost; overlap fixes this.
- Combine from extracted quotes, not from memory.
- Count facts and spot-check seams.
Questions
How big should a chunk be?
Small enough to handle well and big enough to keep context, such as a section or a few pages. Natural breaks such as headings are better than arbitrary cuts.
Do I always need chunking?
No. If the whole document fits comfortably, giving it whole is often simpler. Chunking helps when it does not fit, or when you want section by section answers.
Does the program use Claude?
No. A pattern stands in for the map step so you can see and count the boundary effect exactly. With Claude the map step would be a prompt.
