Files: reading, writing and working with paths
Save and load data with open() and the with statement, loop over lines, append, use pathlib and handle encodings.
In this lesson you will
- Read and write text files safely with with
- Choose between r, w and a modes
- Work with paths using pathlib
- Turn file lines into dictionaries and lists
About 20 minutes
Programs that remember
Every program you have written so far forgot everything the moment it ended. Close the window and the score, the list, the names are gone. Real programs remember: they read a settings file, save your work, load a list of customers, write a log. All of that means working with files. This lesson shows you how to read and write them safely, which opens the door to almost every useful program.
Writing a text file
with open("notes.txt", "w") as f:
f.write("Learn Python\n")
f.write("Practise every day\n")
print("File saved")
File saved
open(name, mode) opens a file. The mode says what you intend to do: "w" to write (and replace anything already there), "r" to read, "a" to append to the end. The with statement is the safe way to use files: Python closes the file for you at the end of the indented block, even if an error happens halfway. Always use with.
Write \n at the end of each line you save. write() does not add a new line by itself, unlike print().
Reading a file
with open("notes.txt", "w") as f:
f.write("Learn Python\nPractise every day\nBuild a project\n")
with open("notes.txt", "r") as f:
text = f.read() # the whole file as one string
print(text)
print(len(text), "characters")
Learn Python
Practise every day
Build a project
48 characters
with open("notes.txt", "w") as f:
f.write("Learn Python\nPractise every day\nBuild a project\n")
with open("notes.txt") as f: # "r" is the default mode
for line in f: # one line at a time: memory friendly
print(line.strip())
with open("notes.txt") as f:
lines = f.readlines() # a list of lines
print(len(lines), lines[0])
Learn Python
Practise every day
Build a project
3 Learn Python
Looping with for line in f is the best habit: it reads one line at a time, so even a file of several gigabytes works. Each line still carries its trailing \n, so use .strip() to remove it.
Appending
with open("log.txt", "w") as f:
f.write("Start\n")
with open("log.txt", "a") as f: # a = append: add to the end
f.write("Something happened\n")
f.write("Done\n")
print(open("log.txt").read())
Start
Something happened
Done
Opening a file with "w" when you meant "a". Write mode erases the existing content immediately, before you write anything. If you want to add to a file, use append mode.
Where is the file? Paths and pathlib
A bare name like "notes.txt" means “in the folder where the program is running”. For other places you give a path. The pathlib module makes paths pleasant and works the same on Windows, Mac and Linux.
from pathlib import Path
folder = Path("data")
folder.mkdir(exist_ok=True) # create the folder if needed
file = folder / "scores.txt" # the / joins path parts
file.write_text("Sara,90\nAli,72\n") # write in one line
print(file)
print(file.exists())
print(file.name, file.suffix, file.stem)
print(file.read_text())
for p in folder.iterdir(): # list a folder
print("Found:", p.name)
data/scores.txt
True
scores.txt .txt scores
Sara,90
Ali,72
Found: scores.txt
write_text and read_text are shortcuts for small files. For big files, or when you need to go line by line, use open.
Checking, copying and deleting
from pathlib import Path
import shutil
Path("a.txt").write_text("hello")
print(Path("a.txt").exists())
shutil.copy("a.txt", "b.txt") # copy
Path("b.txt").rename("c.txt") # rename or move
print(sorted(p.name for p in Path(".").glob("*.txt"))) # find files by pattern
Path("a.txt").unlink() # delete a file
Path("c.txt").unlink()
print(sorted(p.name for p in Path(".").glob("*.txt")))
True
['a.txt', 'c.txt']
[]
Reading structured lines: a small data file
Path = __import__("pathlib").Path
Path("scores.txt").write_text("Sara,90\nAli,72\nMina,85\n")
scores = {}
with open("scores.txt") as f:
for line in f:
name, mark = line.strip().split(",")
scores[name] = int(mark)
print(scores)
best = max(scores, key=scores.get)
print(f"Best: {best} with {scores[best]}")
print(f"Average: {sum(scores.values()) / len(scores):.1f}")
{'Sara': 90, 'Ali': 72, 'Mina': 85}
Best: Sara with 90
Average: 82.3
This is the everyday pattern: read each line, split it into pieces, convert the pieces, and store them in a list or dictionary. In the JSON and CSV lesson you will meet better formats for exactly this job.
Encoding: handling any language
text = "سلام دنیا, héllo, 你好"
with open("multi.txt", "w", encoding="utf-8") as f:
f.write(text)
with open("multi.txt", encoding="utf-8") as f:
print(f.read())
سلام دنیا, héllo, 你好
Computers store text as numbers using an encoding. UTF-8 can represent every language, including Urdu, Arabic, Chinese and emoji. Add encoding="utf-8" whenever you open a text file, so a program written on one computer reads correctly on another.
Binary files
data = bytes([72, 101, 108, 108, 111])
with open("raw.bin", "wb") as f: # wb = write binary
f.write(data)
with open("raw.bin", "rb") as f: # rb = read binary
print(f.read())
b'Hello'
What if the file does not exist?
with open("missing.txt") as f:
print(f.read())
Traceback (most recent call last):
...
FileNotFoundError: [Errno 2] No such file or directory: 'missing.txt'
Trying to read a file that is not there raises FileNotFoundError. The next lesson shows how to catch errors like this and respond gracefully. For now, you can check first with Path("missing.txt").exists().
Write the numbers 1 to 5, one per line, to a file called numbers.txt. Then read the file back and print the sum of the numbers.
Show one possible answer
with open("numbers.txt", "w") as f:
for n in range(1, 6):
f.write(f"{n}\n")
total = 0
with open("numbers.txt") as f:
for line in f:
total += int(line)
print("Sum:", total)
Sum: 15
Count how many lines and how many words are in a text file. Create a small file first so the example is self-contained.
Hint: split() with no argument splits on any whitespace.
Show one possible answer
with open("poem.txt", "w") as f:
f.write("Roses are red\nViolets are blue\nPython is fun\nAnd so are you\n")
lines = words = 0
with open("poem.txt") as f:
for line in f:
lines += 1
words += len(line.split())
print(lines, "lines,", words, "words")
4 lines, 13 words
Files can be missing, empty or edited by hand. Check that lines have the pieces you expect before you use them. The error handling lesson shows the tools.
Check what you learned
Five quick questions. Pick an answer and you will see straight away whether it is right and why.
Which mode adds to the end of a file without erasing it?
"w""a""r""x"
a is append mode.
Why use the with statement?
- It is faster
- It encrypts the file
- It is required for reading
- It closes the file automatically, even after an error
with guarantees the file is closed.
What does opening a file with "w" do to existing content?
- It erases it immediately
- It keeps it
- It appends
- It locks it
Write mode truncates the file.
What does Path("data") / "a.txt" build?
- A division result
- An error
- The path data/a.txt
- A list
The slash joins path parts.
Which encoding handles every language?
- ascii
- latin-1
- none
- utf-8
UTF-8 represents all characters.
Summary
- Always use with open(…).
- r reads, w replaces, a appends.
- pathlib builds and checks paths for any system.
- Use encoding=”utf-8″ for text.
Questions
Where does my file get saved?
In the folder the program runs from, the working directory, unless you give a different path. Print Path.cwd() to see it.
Do I need to close files?
Not if you use with. It closes the file for you.
How do I read a huge file?
Loop over it line by line with for line in f, instead of calling read().
