Capstone: build and document a prompt system
Finish the course with a real capstone: brief, versioned template, test set, runner and report, safety notes and a README anyone can follow.
In this lesson you will
- Plan a capstone with all six parts
- Build a test set and runner
- Write a clear README
- Present results honestly
About 20 minutes
Bring it all together
Over thirty lessons you have learned to describe goals, give context and examples, fix formats, chain steps, validate output, test, manage context and run projects. The capstone is where those parts become one piece of work that someone else could pick up and use. Ayesha, a final-year student, builds hers for a campus helpdesk. She does not present a single clever prompt. She presents a template, a test set, a runner, a report and a README, and a visiting lecturer says it is the first prompt work he has seen that he could hand to a colleague on Monday.
What your capstone must include
| Part | What it shows |
|---|---|
| 1. Brief | The problem, users, success and limits |
| 2. Template | A versioned prompt with rules and examples |
| 3. Test set | At least 12 cases: typical, awkward, trap |
| 4. Runner and report | Results per case, regressions, summary |
| 5. Safety notes | Data, human review, injection and failure handling |
| 6. README | How someone else uses, tests and updates it |
Step 1: Choose a real, small problem
Pick something you understand and that has clear good and bad outputs: reply drafts, meeting-note summaries, quiz generation, product descriptions, email triage, study-guide creation. Avoid anything where a wrong output could cause serious harm. Keep the scope small enough to finish in a week or two.
Step 2: Write the brief and criteria
Use lesson 29: half a page of brief, five requirements, four numbered acceptance criteria, one out-of-scope line. Write them before building.
Step 3: Build the template
Use the sections from earlier lessons: role, goal, context, rules, examples, output format, uncertainty rule. Include a version number and a short note on what it was tested on and when.
Step 4: Build the test set and runner
Create at least 12 cases, with outputs from your chosen tool saved in a spreadsheet. The program below shows a tiny version of the runner and report: a template record, three test cases (typical, awkward and trap), a grader and a JSON report. The test outputs are written examples, but the grading logic and the printed report are real:
import json
# Capstone: a tiny documented prompt system - template, tests, runner, report
TEMPLATE = {
"name": "reply-drafter", "version": "1.2",
"rules": ["Apologise once", "Offer one next step", "Under 60 words", "Never promise a refund"],
}
TESTS = [
{"id": "T1", "kind": "typical", "output": "We are sorry your parcel is late. We are checking with the courier and will update you tomorrow.", "must": ["sorry", "checking"]},
{"id": "T2", "kind": "awkward", "output": "Sorry about the mix-up. We will send a replacement this week.", "must": ["sorry", "replacement"]},
{"id": "T3", "kind": "trap", "output": "We are sorry. A refund is guaranteed.", "must": ["sorry"], "forbidden": ["refund is guaranteed"]},
]
def grade(t):
text = t["output"].lower()
issues = [f"missing '{w}'" for w in t["must"] if w not in text]
issues += [f"forbidden '{w}'" for w in t.get("forbidden", []) if w in text]
if len(text.split()) > 60:
issues.append("too long")
return issues
report = {"template": TEMPLATE["name"], "version": TEMPLATE["version"], "results": {}}
for t in TESTS:
report["results"][t["id"]] = {"kind": t["kind"], "issues": grade(t)}
passed = sum(1 for r in report["results"].values() if not r["issues"])
report["summary"] = f"{passed}/{len(TESTS)} passed"
print(json.dumps(report, indent=2))
{
"template": "reply-drafter",
"version": "1.2",
"results": {
"T1": {
"kind": "typical",
"issues": []
},
"T2": {
"kind": "awkward",
"issues": []
},
"T3": {
"kind": "trap",
"issues": [
"forbidden 'refund is guaranteed'"
]
}
},
"summary": "2/3 passed"
}
The trap case fails because the output promises a refund. That is the point of a trap: it shows a rule being tested. In your capstone, fix the prompt, bump the version to 1.3, and run all cases again to check for regressions.
Step 5: Safety notes
- What data enters the tool, and is that allowed?
- Where is the human checkpoint?
- How are injected instructions in user text handled (lesson 23)?
- What happens when validation fails after retries (lesson 24)?
- What are the known limits?
Step 6: The README
Write for a person who has never met you. Here is a model README for the example:
# Reply Drafter (v1.2)
## What it does
Drafts short, polite customer replies for a person to review.
## Who it is for
Support staff. Drafts are never sent without review.
## Rules
Apologise once, offer one next step, under 60 words, never promise a refund.
## How to use
1. Paste the customer message and order facts.
2. Run the template.
3. Check the draft against the rules; edit; send.
## Tests
See tests.csv (typical, awkward and trap cases). Re-run after every change.
## Known limits
Cannot decide refunds. May miss unusual requests. Needs review.
## Change log
v1.2 - added trap test for refund promises.
v1.1 - shortened rules.
v1.0 - first version.
Step 7: Present it
- State the problem in one sentence.
- Show one before-and-after example, labelled as an example.
- Show the test results, including the failure you found and fixed.
- Name the limits honestly.
- Offer the README and files.
Self-review checklist
- Could a stranger reproduce my results from the README?
- Do I have at least one trap case that once failed?
- Do I record the tool and the date I tested on?
- Is every claim in my report backed by a test result?
- Have I removed private data?
Where to go next
Keep your capstone alive. Add a test every time something goes wrong, re-run after tool updates, and build two more pieces in different areas to form a small portfolio. If you want to go further, learn the basics of Python or a spreadsheet for running tests, and explore the automation and agents course on this site.
Practice
Complete your capstone and ask one other person to follow your README without your help. Everything they get stuck on is a missing line in the README or a missing rule in the template.
Typical gaps found by a second person: the input format is not shown, the expected output is not shown, and the ‘known limits’ section is missing. Add each answer to the README and note the change in the change log.
Presenting only your best outputs. The failures you found and fixed are the strongest evidence that your system works.
Check what you learned
Five quick questions. Pick an answer and you will see straight away whether it is right and why.
Which is NOT a part of the capstone?
- A single clever prompt with no tests
- Brief
- Test set
- README
The capstone is a system, not a trick.
Why include a trap case?
- To make it harder to read
- To test that rules hold
- To shorten the set
- To avoid tests
It proves a rule works.
What should you do after fixing a failure?
- Delete the test
- Skip checks
- Change the goal
- Re-run all cases and bump the version
Check for regressions.
Who is the README for?
- Only you
- Someone who has never met you
- The AI tool
- No one
It makes the work reusable.
What is the strongest evidence a system works?
- Only perfect samples
- A long prompt
- A nice title
- Failures found, fixed and re-tested
Honest testing builds trust.
Summary
- A capstone is a system: brief, template, tests, report, safety, README.
- Keep trap cases and fix failures with versions.
- Write for a stranger and let someone test your README.
- Keep improving after you finish.
Questions
How long should the capstone take?
A small, focused project takes one to two weeks of part-time work. Keep the scope small.
Can I use any AI tool for the capstone?
Yes. Record which tool and date you tested on, and follow its terms and your data rules.
What if my system fails some tests at the end?
Report them honestly with their causes and the planned fixes. That is better than hiding them.
