IntermediateLesson 11 of 306 min read

Files: reading, writing and working with paths

Save and load data with open() and the with statement, loop over lines, append, use pathlib and handle encodings.

In this lesson you will

  • Read and write text files safely with with
  • Choose between r, w and a modes
  • Work with paths using pathlib
  • Turn file lines into dictionaries and lists

About 20 minutes

Programs that remember

Every program you have written so far forgot everything the moment it ended. Close the window and the score, the list, the names are gone. Real programs remember: they read a settings file, save your work, load a list of customers, write a log. All of that means working with files. This lesson shows you how to read and write them safely, which opens the door to almost every useful program.

Writing a text file

with open("notes.txt", "w") as f:
    f.write("Learn Python\n")
    f.write("Practise every day\n")

print("File saved")
File saved

open(name, mode) opens a file. The mode says what you intend to do: "w" to write (and replace anything already there), "r" to read, "a" to append to the end. The with statement is the safe way to use files: Python closes the file for you at the end of the indented block, even if an error happens halfway. Always use with.

Good to know

Write \n at the end of each line you save. write() does not add a new line by itself, unlike print().

Reading a file

with open("notes.txt", "w") as f:
    f.write("Learn Python\nPractise every day\nBuild a project\n")

with open("notes.txt", "r") as f:
    text = f.read()          # the whole file as one string

print(text)
print(len(text), "characters")
Learn Python
Practise every day
Build a project

48 characters
with open("notes.txt", "w") as f:
    f.write("Learn Python\nPractise every day\nBuild a project\n")

with open("notes.txt") as f:        # "r" is the default mode
    for line in f:                   # one line at a time: memory friendly
        print(line.strip())

with open("notes.txt") as f:
    lines = f.readlines()            # a list of lines
print(len(lines), lines[0])
Learn Python
Practise every day
Build a project
3 Learn Python

Looping with for line in f is the best habit: it reads one line at a time, so even a file of several gigabytes works. Each line still carries its trailing \n, so use .strip() to remove it.

Appending

with open("log.txt", "w") as f:
    f.write("Start\n")

with open("log.txt", "a") as f:     # a = append: add to the end
    f.write("Something happened\n")
    f.write("Done\n")

print(open("log.txt").read())
Start
Something happened
Done
Common mistake

Opening a file with "w" when you meant "a". Write mode erases the existing content immediately, before you write anything. If you want to add to a file, use append mode.

Where is the file? Paths and pathlib

A bare name like "notes.txt" means “in the folder where the program is running”. For other places you give a path. The pathlib module makes paths pleasant and works the same on Windows, Mac and Linux.

from pathlib import Path

folder = Path("data")
folder.mkdir(exist_ok=True)               # create the folder if needed

file = folder / "scores.txt"              # the / joins path parts
file.write_text("Sara,90\nAli,72\n")     # write in one line

print(file)
print(file.exists())
print(file.name, file.suffix, file.stem)
print(file.read_text())

for p in folder.iterdir():                # list a folder
    print("Found:", p.name)
data/scores.txt
True
scores.txt .txt scores
Sara,90
Ali,72

Found: scores.txt

write_text and read_text are shortcuts for small files. For big files, or when you need to go line by line, use open.

Checking, copying and deleting

from pathlib import Path
import shutil

Path("a.txt").write_text("hello")

print(Path("a.txt").exists())
shutil.copy("a.txt", "b.txt")             # copy
Path("b.txt").rename("c.txt")             # rename or move
print(sorted(p.name for p in Path(".").glob("*.txt")))   # find files by pattern

Path("a.txt").unlink()                    # delete a file
Path("c.txt").unlink()
print(sorted(p.name for p in Path(".").glob("*.txt")))
True
['a.txt', 'c.txt']
[]

Reading structured lines: a small data file

Path = __import__("pathlib").Path
Path("scores.txt").write_text("Sara,90\nAli,72\nMina,85\n")

scores = {}
with open("scores.txt") as f:
    for line in f:
        name, mark = line.strip().split(",")
        scores[name] = int(mark)

print(scores)
best = max(scores, key=scores.get)
print(f"Best: {best} with {scores[best]}")
print(f"Average: {sum(scores.values()) / len(scores):.1f}")
{'Sara': 90, 'Ali': 72, 'Mina': 85}
Best: Sara with 90
Average: 82.3

This is the everyday pattern: read each line, split it into pieces, convert the pieces, and store them in a list or dictionary. In the JSON and CSV lesson you will meet better formats for exactly this job.

Encoding: handling any language

text = "سلام دنیا, héllo, 你好"
with open("multi.txt", "w", encoding="utf-8") as f:
    f.write(text)

with open("multi.txt", encoding="utf-8") as f:
    print(f.read())
سلام دنیا, héllo, 你好

Computers store text as numbers using an encoding. UTF-8 can represent every language, including Urdu, Arabic, Chinese and emoji. Add encoding="utf-8" whenever you open a text file, so a program written on one computer reads correctly on another.

Binary files

data = bytes([72, 101, 108, 108, 111])
with open("raw.bin", "wb") as f:       # wb = write binary
    f.write(data)

with open("raw.bin", "rb") as f:       # rb = read binary
    print(f.read())
b'Hello'

What if the file does not exist?

with open("missing.txt") as f:
    print(f.read())
Traceback (most recent call last):
  ...
FileNotFoundError: [Errno 2] No such file or directory: 'missing.txt'

Trying to read a file that is not there raises FileNotFoundError. The next lesson shows how to catch errors like this and respond gracefully. For now, you can check first with Path("missing.txt").exists().

Practice

Write the numbers 1 to 5, one per line, to a file called numbers.txt. Then read the file back and print the sum of the numbers.

Show one possible answer
with open("numbers.txt", "w") as f:
    for n in range(1, 6):
        f.write(f"{n}\n")

total = 0
with open("numbers.txt") as f:
    for line in f:
        total += int(line)
print("Sum:", total)
Sum: 15
Practice

Count how many lines and how many words are in a text file. Create a small file first so the example is self-contained.

Hint: split() with no argument splits on any whitespace.

Show one possible answer
with open("poem.txt", "w") as f:
    f.write("Roses are red\nViolets are blue\nPython is fun\nAnd so are you\n")

lines = words = 0
with open("poem.txt") as f:
    for line in f:
        lines += 1
        words += len(line.split())
print(lines, "lines,", words, "words")
4 lines, 13 words
Never trust a file’s contents

Files can be missing, empty or edited by hand. Check that lines have the pieces you expect before you use them. The error handling lesson shows the tools.

Check what you learned

Five quick questions. Pick an answer and you will see straight away whether it is right and why.

Which mode adds to the end of a file without erasing it?

  1. "w"
  2. "a"
  3. "r"
  4. "x"

a is append mode.

Why use the with statement?

  1. It is faster
  2. It encrypts the file
  3. It is required for reading
  4. It closes the file automatically, even after an error

with guarantees the file is closed.

What does opening a file with "w" do to existing content?

  1. It erases it immediately
  2. It keeps it
  3. It appends
  4. It locks it

Write mode truncates the file.

What does Path("data") / "a.txt" build?

  1. A division result
  2. An error
  3. The path data/a.txt
  4. A list

The slash joins path parts.

Which encoding handles every language?

  1. ascii
  2. latin-1
  3. none
  4. utf-8

UTF-8 represents all characters.

Summary

  • Always use with open(…).
  • r reads, w replaces, a appends.
  • pathlib builds and checks paths for any system.
  • Use encoding=”utf-8″ for text.

Next lesson

Next: errors and exceptions, keeping your program calm when things go wrong.

Questions

Where does my file get saved?

In the folder the program runs from, the working directory, unless you give a different path. Print Path.cwd() to see it.

Do I need to close files?

Not if you use with. It closes the file for you.

How do I read a huge file?

Loop over it line by line with for line in f, instead of calling read().

Progress is saved in this browser.