Responsible Data: Privacy, Small Groups and Bias (an Outline)
Share data more safely with pseudonyms and small-group suppression, and ask who is missing from the numbers. Not legal advice.
In this lesson you will
- Collect and keep only what you need
- Explain pseudonymisation and its limits
- Suppress small groups in a table
- Ask who is missing from the data
About 28 minutes
Data about people
The workshop’s file is about people: who came, what they paid, how they rated the service. A neighbourhood newsletter wants to publish “what the workshop found”. Omar is ready to email the whole file. Leila stops him with a simple question: “Does the newsletter need to know who each person is?” Analysis changes lives when it is shared, and the more detailed the data, the more care it needs. This lesson is an outline of good habits. It is not legal advice, and the rules that apply to you depend on where you live and what kind of data you hold, so check the guidance of your local data protection authority.
Collect and keep only what you need
The cheapest way to protect data is not to have it. Before collecting a column, ask what question it answers. Do you need a customer’s full name to count visits? Do you need a birth date, or would an age band do? The same goes for how long you keep it. Data you no longer need is a risk with no benefit. People often call this idea data minimisation, and it is a good habit even where no rule requires it.
Pseudonyms are not anonymity
A common step is to replace names with codes, such as C1 and C2, and keep the list that links codes to names somewhere else, safe and separate. This is called pseudonymisation. It reduces harm if the file leaks, because the file alone does not show names. But it is not the same as anonymising the data. Anyone holding the key list can undo it. And even without the list, a visit on a particular date to fix a particular bike might still point to one person in a small community. In many data protection regimes, pseudonymised data is still treated as personal data, so the safe default is to treat it with care.
import pandas as pd
visits = pd.read_csv("clean.csv")
people = pd.read_csv("customers.csv")
people["customer"] = people["customer"].replace({"Hanna Lee": "Hana Lee"})
order = visits["customer"].drop_duplicates().to_list()
ids = {name: "C" + str(i) for i, name in enumerate(order, 1)}
shared = visits.assign(who=visits["customer"].map(ids)).drop(columns=["customer"])
print("columns shared:", list(shared.columns))
print("names left in shared file:", int(shared.astype(str).apply(lambda c: c.str.contains("Asha|Ben|Dina")).sum().sum()))
print("key kept apart:", len(ids), "names")
joined = visits.merge(people, on="customer", how="left")
by_area = joined.groupby("area").agg(people=("customer", "nunique"), visits=("visit_id", "count"))
for area, row in by_area.iterrows():
if row["people"] < 3:
print(area + ": hidden, fewer than 3 people")
else:
print(area + ": " + str(int(row["people"])) + " people, " + str(int(row["visits"])) + " visits")
rated = joined.assign(has=joined["rating"].notna()).groupby("source")["has"].agg(["sum", "size"])
for name, row in rated.iterrows():
print(name + ": " + str(int(row["sum"])) + " of " + str(int(row["size"])) + " rated")
columns shared: ['visit_id', 'date', 'bike', 'repair', 'minutes', 'cost', 'rating', 'source', 'who']
names left in shared file: 0
key kept apart: 8 names
Dock: 3 people, 13 visits
Hill: 3 people, 15 visits
Park: hidden, fewer than 3 people
friend: 6 of 7 rated
online: 8 of 9 rated
passing: 6 of 9 rated
poster: 13 of 13 rated
The shared file keeps these columns: the visit number, date, bike, repair, minutes, cost, rating, source, and a code for who. A scan finds 0 names left in it, and the key linking the 8 customers to their codes was not written into the shared file. A scan for known names is a weak check, though, because it cannot find nicknames or details that identify a person without naming them. Read the file as a stranger would.
Small groups can give people away
The second part of the program counts people and visits by area, after joining the customers table. Two areas are shown. The third, Park, is hidden, fewer than 3 people. The rule is a simple version of a method used by statistical offices, called small cell suppression: do not publish a figure about a group so small that readers could work out who is in it. If only two customers live in Park, then “Park customers paid more than average” is a statement about two named neighbours. The threshold of three here is a teaching choice. Real organisations choose larger thresholds, and many suppress counts below five or more.
Suppression has a catch. If you hide one cell but publish the totals, a reader can subtract the other cells and recover the hidden one. Think about the whole table, not each number in isolation.
Who is missing, and who is over-represented?
Bias is another way that data harms people. It is not about hiding anything, but about whose voice is in the data. The last lines of the program count how many visits were rated, by how the customer found the workshop. Passers-by left 6 of 9 ratings, and poster customers left 13 of 13. If the newsletter reports “average rating 3.88”, it reports the views of people who were handed a card. Customers who walked in off the street and did not rate are underrepresented. The lesson from the previous one applies here: a gap that is linked to who the person is can bias the answer.
A short checklist
- Purpose. Write one sentence: what question does this analysis answer, and who will see the results?
- Minimise. Drop columns you do not need before sharing.
- Separate the key. Keep the list linking codes to names apart from the data, with fewer people able to open it.
- Suppress small groups and check that totals do not give them away.
- Ask who is missing and say so in the report.
- Check the rules that apply to you, and ask for advice when the data is sensitive.
Sensitive information, such as health, money or children’s details, calls for far more care than a bicycle repair record. If you work with it, learn the rules for your place before collecting anything.
Believing that removing names makes data safe to share. Dates, places and unusual details can still point to one person. Think like a stranger who knows the neighbourhood.
The customers, areas and ratings are made up, and the three-person limit is a teaching choice, not a rule. This is an outline and not legal advice.
Practice for this lesson
Check what you learned
Five quick questions. Pick an answer and you will see straight away whether it is right and why.
What is the cheapest way to protect data?
- To store it twice
- To rename the file
- Not to collect what you do not need
- To print it
Data you never hold cannot leak.
What does pseudonymisation do?
- Replaces names with codes while the key is kept apart
- Makes data impossible to link
- Deletes the data
- Encrypts the report
It reduces harm but is not the same as anonymising.
Why was Park hidden in the area table?
- It had no visits
- It was a mistake
- Fewer than three people live there, so figures could identify them
- Park is not an area
Small cell suppression protects people in small groups.
What catch does suppression have?
- Totals can let a reader work out a hidden cell
- It deletes the file
- It changes names
- It is always illegal
Think about the whole table, not each number alone.
What is the risk if only people handed a card rated the workshop?
- The average is always too high
- The average reflects some customers more than others
- The file is smaller
- The ratings are anonymous
Gaps linked to who the person is can bias the answer.
Summary
- Minimise what you collect and keep.
- Pseudonyms reduce harm but do not make data anonymous.
- Hide small groups, and watch for totals that give them away.
- Ask who is missing, and check the rules that apply to you.
Questions
Is pseudonymised data personal data?
Often yes. Where data protection law applies, data that can be linked back to a person with extra information is usually still treated as personal data. This is an outline, so check the guidance for your own place.
What group size is safe?
There is no universal number. Organisations choose thresholds from the risk. Three is a teaching example. Many suppress counts below five or more, and for sensitive topics the limit is higher.
Does this apply to my small project?
The habits do: minimise, separate, suppress small groups and ask who is missing. Whether specific legal rules apply depends on what data you hold and where you are, so look up the local guidance.
