Intermediate Project: A Shop Assistant With Lookup, Checks and Logs
Combine lookup, reply checks, handover, logging and measurement into one small shop assistant, test it on ten saved questions and plan a safe launch.
In this lesson you will
- Combine lookup, checks, logs and handover
- Measure outcomes on saved questions
- Spot the misses that point to new notes
- Plan a safe launch with alerts and review
About 28 minutes
Putting the intermediate ideas to work
Ravi now has the parts: branches, schedules, duplicates, stages, lookups, checks, measures, logs and a tool registry. In this project he builds a small shop assistant that answers customer questions from his own notes. It must look up the right note, check its reply, hand over when unsure, keep a log and be measured on saved questions. Nothing here needs a real AI model, so you can see every moving part. If you later add an AI step to write friendlier wording, the same frame, checks and tests apply around it.
The design on one page
- Question arrives. The workflow receives the text of a customer question.
- Look up. Find the note with the most shared words. If fewer than two words are shared, hand over.
- Write the reply. Use fixed wording that quotes the note, so no promise can be invented.
- Check the reply. If it contains a banned word, hand over instead.
- Log it. Record the first words of the question and what happened.
- Measure. Run saved questions with known answers and count the outcomes.
Notice that a person remains in the picture in two places: the handover for anything uncertain, and the weekly review of the log. The assistant never sends a refund promise or invents a delivery time, because it can only quote what the notes say.
Run the assistant on ten saved questions
The program below holds seven short notes, one of which is about refunds, and ten saved questions with the note a person would pick, or none. It runs every question through the assistant and counts what happened.
import re
docs = {
"d1": "Delivery takes three working days after you approve the proof.",
"d2": "We print flyers on thick or thin paper. Thick paper costs more.",
"d3": "You can change an order before the proof is approved. After that, changes start a new order.",
"d4": "Our shop is open from nine to five on weekdays.",
"d5": "Pay by card or bank transfer. Invoices are due in fourteen days.",
"d6": "Business cards come in packs of one hundred.",
"d7": "A refund request is handled by Ravi in person.",
}
stop = set("a an the is are do does did how i you we our my to of in on at or and by for it can what when after before more long".split())
def words(t):
return {w for w in re.findall(r"[a-z]+", t.lower()) if w not in stop}
banned = ["guarantee", "definitely", "100%", "refund"]
log = []
def assistant(q):
mine = words(q)
scores = {d: len(mine & words(t)) for d, t in docs.items()}
best = max(scores, key=scores.get)
if scores[best] < 2:
log.append((q[:18], "escalated"))
return None, "ask a person"
reply = "From our shop notes: " + docs[best]
if any(b in reply.lower() for b in banned):
log.append((q[:18], "blocked"))
return None, "ask a person"
log.append((q[:18], "answered " + best))
return best, reply
# SAVED questions and the document a person would pick (None = nobody knows)
tests = [
("How many days does delivery take once I approve the proof?", "d1"),
("Is thick paper dearer than thin paper?", "d2"),
("Can I change my order after the proof is approved?", "d3"),
("When do you close?", "d4"),
("Can I pay by bank transfer or card?", "d5"),
("How many business cards come in a pack?", "d6"),
("Do you print on glass?", None),
("Can I request a refund for an order?", None),
("What are the due days for invoices?", "d5"),
("Where is your shop?", None),
]
results = [(want, assistant(q)[0]) for q, want in tests]
as_person = sum(want == got for want, got in results)
escalated = sum(got is None for _, got in results)
wrong = sum(got is not None and got != want for want, got in results)
missed = sum(got is None and want is not None for want, got in results)
print("questions:", len(tests))
print("handled as a person would:", as_person)
print("answered from the notes:", len(tests) - escalated)
print("escalated to a person:", escalated)
print("answerable but escalated:", missed)
print("answered wrongly:", wrong)
print("blocked by the reply check:", sum(s == "blocked" for _, s in log))
print("log lines written:", len(log))
questions: 10
handled as a person would: 9
answered from the notes: 6
escalated to a person: 4
answerable but escalated: 1
answered wrongly: 0
blocked by the reply check: 1
log lines written: 10
Out of 10 questions, the assistant handled 9 as a person would. It answered 6 from the notes and handed 4 to a person. Of those handovers, 1 was for a question the notes could answer: ‘When do you close?’. The note says the shop is open from nine to five and never uses the word close, so word matching found nothing. A person answers it in a minute, but it shows what to fix: add a note that uses the words customers use. There were 0 wrong answers, which is the number to watch most closely. The reply check blocked 1 reply. That was the refund question: the note matched, but its wording contained a banned word, so the assistant handed over instead of answering a money question. The log recorded 10 lines, one per question, which is what lets Ravi review the week.
Be honest about the test
These ten questions were written by someone who knew the notes, so they are friendlier than real customers will be. Real questions will include typing mistakes, two questions in one message and topics the notes never mention. Before relying on the assistant, collect real questions, label them by hand, and measure precision and recall as lesson 17 described, with a fresh set for the final check. Expect worse numbers, and read every miss, because the misses tell you what notes to write.
Checklist before switching on
- Run on real, anonymised questions, not only on your own examples.
- Set the shared-word line with the saved questions in front of you, and favour handing over.
- Keep personal details out of the log, and decide how long to keep it.
- Put an alert on a rising handover count and on a stopped workflow.
- Name the workflow with the four-part pattern, add a version and write the changelog line.
- Choose who reads the handover queue each day, and how fast they must reply.
Where this leads
The Advanced level goes further: connecting several steps into longer chains, building more capable agents, testing and improving them methodically and running automations as part of your working life. Everything there rests on what you have just practised: small steps, clear rules, honest measurement and a person who stays in charge.
Declaring victory because ten hand-picked questions passed. A small, friendly test proves the wiring works, not that real customers will be served well.
The notes, questions and results are made up for a pretend shop, and the assistant is a simple teaching model with no AI service involved. Real results will differ, and no promises about savings or accuracy are made.
Practice for this lesson
Check what you learned
Five quick questions. Pick an answer and you will see straight away whether it is right and why.
Why does the assistant quote the note in fixed wording?
- So it cannot invent a promise
- To make replies longer
- To speed up the log
- To avoid using notes
It can only repeat what the notes say.
Why was the refund question handed to a person?
- No note matched
- The note contained a banned word, so the reply check blocked it
- The log was full
- The question was too long
A money question should not be answered automatically.
What did the question “When do you close?” show?
- The assistant answered wrongly
- The log was empty
- The shop was closed
- Word matching misses notes that use different words
The note says open from nine to five, never close.
Which number should you watch most closely?
- Log lines written
- Wrong answers given
- Notes written
- Questions asked
A wrong answer costs more than a handover.
Why is the ten-question test not enough?
- Ten is too many
- Labels are unnecessary
- Logs hide errors
- It was written by someone who knew the notes
Real customers write messier questions.
Summary
- Answer only from notes, check the reply and hand over when unsure.
- Log every question and review the log weekly.
- Watch wrong answers most closely.
- Test on real, fresh questions before relying on it.
Questions
Do I need an AI model for this project?
No. The project uses simple lookup and fixed wording. An AI step could later write friendlier text, but the same checks and tests would apply around it.
How do I decide the shared-word line?
Run your saved questions at different lines and count handovers and wrong answers. Favour handing over when in doubt.
Are these results real?
No. The notes, questions and counts are made up, and the assistant is a teaching model.
