Adding an AI Step: Classify, Summarise and Draft
Add an AI step to a workflow safely: see how confidence scores route sure answers through and doubtful ones to a person.
In this lesson you will
- Name three jobs an AI step does well
- Use a confidence line to route work
- Read a labelled example reply with care
- Keep people in charge of sending and deciding
About 22 minutes
Rules run out, and then what?
Ravi’s rules from lesson 1 work well for tidy emails, but real customers write messily. One writes, ‘Price for the order, the invoice and the receipt’. Another says, ‘My order arrived broken and I need a receipt’. These mix several topics, and keyword rules struggle. This is where an AI step can help. An AI step is a box in your workflow that sends some text to an AI model and receives text back. It can sort a message into a category, summarise a long email in two lines or draft a reply. It is flexible, but unlike a rule it is not guaranteed to be right, so it needs a safety plan.
Three jobs an AI step does well
- Classify. Put a message into one of a few labels.
- Summarise. Shrink a long message into a short one.
- Draft. Write a first version of a reply for a person to check.
All three produce text that a person can quickly check, which is why they are good first AI steps. Notice that none of them decides anything that cannot be undone.
Confidence and routing
A smart habit is to ask the step not just for an answer but for how sure it is, then to send only the confident answers straight through and the doubtful ones to a person. The program below uses a small stand-in classifier. It is not a real AI model. It simply counts keywords, which lets us run it here and see exactly how a confidence score is made: the share of keyword hits that belong to the winning label. A real AI step would give you a label and often a confidence figure, or a reason, in its own way. The routing logic around it, though, is the same.
# A STAND-IN for an AI classifier: counts keywords. It is not a real AI model.
words = {
"sales": ["quote", "price", "how much", "flyers", "banner"],
"billing": ["invoice", "payment", "overdue", "receipt"],
"support": ["order", "late", "broken", "missing", "where is"],
}
def classify(text):
low = text.lower()
score = {k: sum(w in low for w in v) for k, v in words.items()}
total = sum(score.values())
if total == 0:
return "other", 0.0
best = max(score, key=score.get)
return best, score[best] / total
mails = [
"How much for 200 flyers and a banner?",
"Invoice 118 is overdue, sorry",
"Where is my order? It is late",
"Thanks, the cards look great",
"Price for the order, the invoice and the receipt",
"My order arrived broken and I need a receipt",
]
threshold = 0.7
auto = review = 0
for m in mails:
label, conf = classify(m)
route = "auto" if conf >= threshold else "review"
auto += route == "auto"
review += route == "review"
print(f"{label} {conf:.2f} {route}: {m[:24]}")
print("auto:", auto, "| review:", review)
sales 1.00 auto: How much for 200 flyers
billing 1.00 auto: Invoice 118 is overdue,
support 1.00 auto: Where is my order? It is
other 0.00 review: Thanks, the cards look g
billing 0.50 review: Price for the order, the
support 0.67 review: My order arrived broken
auto: 3 | review: 3
Out of six sample emails, 3 were confident enough to go straight through and 3 went to review. The thank-you note contained none of the keywords, so it went to review as ‘other’. The mixed messages show the idea best. ‘Price for the order, the invoice and the receipt’ touches three topics, so its best label, billing, had a confidence of only 0.50, and it was held for a person. ‘My order arrived broken and I need a receipt’ leaned towards support with a confidence of 0.67, still under the line of 0.70 that the program uses, so it went to review as well. Where to draw that line is a choice. A higher line sends more mail to people, which costs time but lowers risk. A lower line saves time but lets more mistakes through.
What a real AI step might return
For a message like the mixed one above, a real AI step might return something like the block below. Treat it only as an illustration, because the exact wording, format and quality depend on the tool, its settings and your instructions, and they will differ each time.
Label: billing
Reason: the message mentions an invoice and a receipt, but also an order.
Suggested reply: Thanks for getting in touch. I will check the invoice and send the receipt today.
Notice the suggested reply. It is a draft for a person, not a message to send automatically. A draft that sounds sure of itself can still be wrong, for example by promising a receipt that the shop cannot issue.
Starting rules for AI steps
- Let AI steps label, summarise and draft, and let people send and decide.
- Ask for a confidence or a reason, and route doubtful cases to a person.
- Keep a sample of results to check by eye every week.
- Expect different wording each time, and design around that.
- Never send private details to a service you have not checked.
Trusting a confident tone. An AI step can sound certain and still be wrong. Check a sample of its work regularly, especially after you change instructions.
The classifier above is a stand-in written for teaching, not a real AI model, and all emails are made up. Which AI features are available, and how they behave, depends on your tool, plan and region. No model names, prices or limits are quoted here.
Practice for this lesson
Check what you learned
Five quick questions. Pick an answer and you will see straight away whether it is right and why.
Which is a good first job for an AI step?
- Send refunds automatically
- Delete old accounts
- Classify, summarise or draft for a person to check
- Change prices
These produce text a person can check quickly and nothing irreversible.
What is a confidence line (threshold) for?
- Sending sure answers through and doubtful ones to a person
- Making the reply longer
- Choosing the font
- Saving money
It splits results into auto and review.
What happens if you raise the confidence line?
- More mail is sent automatically
- More mail goes to people, with lower risk
- The AI gets smarter
- Nothing changes
A higher line is stricter: costs time, lowers risk.
Why is the sample AI reply labelled “yours will differ”?
- It is always the same
- It is illegal to copy
- It is a secret
- Real wording and quality vary by tool, settings and instructions
AI replies are not identical from one run or tool to the next.
Is the stand-in classifier in the lesson a real AI model?
- Yes, it is the latest model
- No, it only counts keywords to show the routing idea
- Yes, it learns from you
- It is a spreadsheet
It is a teaching tool for confidence and routing.
Summary
- AI steps classify, summarise and draft; people send and decide.
- A confidence line splits auto from review.
- Replies vary, so design around different wording.
- Check a sample of results regularly.
Questions
Is an AI step always better than a rule?
No. Rules are predictable and free of surprises. AI steps handle messy language but can be wrong, so many workflows use rules first and AI for what is left.
Where does the confidence number come from?
It depends on the tool. Some steps can be asked to state a confidence or a reason, but treat it as a hint, not a guarantee.
Do I need a particular AI service?
No. The ideas in this lesson apply to any service. Check current features, plans and privacy terms for the one you choose.
