AdvancedLesson 24 of 305 min read

Testing with pytest

Write automated tests with pytest: asserts, expected errors, parametrize, fixtures, tmp_path and monkeypatch.

In this lesson you will

  • Write and run tests with pytest
  • Read a failing test
  • Use parametrize and fixtures
  • Test code that depends on time or files

About 24 minutes

Code you can trust

You fix a bug on Monday. On Friday another change quietly breaks the same thing, and a customer finds it before you do. Professionals defend against this with automated tests: small programs that check your code still does what it should. You run them in seconds, any time, and a red failure tells you exactly what broke. The most popular tool is pytest, installed with pip install pytest.

Your first test

def add(a, b):
    return a + b

def divide(a, b):
    if b == 0:
        raise ValueError("Cannot divide by zero")
    return a / b
from calc import add, divide
import pytest

def test_add():
    assert add(2, 3) == 5

def test_add_negative():
    assert add(-1, 1) == 0

def test_divide():
    assert divide(10, 4) == 2.5

def test_divide_by_zero():
    with pytest.raises(ValueError):
        divide(1, 0)
$ python -m pytest -q
....                                                                     [100%]
4 passed in 0.01s
  • Test files are named test_*.py; test functions are named test_*.
  • An assert that is true passes; a false one fails the test.
  • pytest.raises checks that an error happens when it should.
  • Each dot is a passing test; F marks a failure.

When a test fails

A failing test is useful information. Pytest shows the values that disagreed.

def average(numbers):
    return sum(numbers) / (len(numbers) + 1)    # a hidden bug
from calc import average

def test_average():
    assert average([2, 4, 6]) == 4
$ python -m pytest -q
F                                                                        [100%]
=================================== FAILURES ===================================
_________________________________ test_average _________________________________

    def test_average():
>       assert average([2, 4, 6]) == 4
E       assert 3.0 == 4
E        +  where 3.0 = average([2, 4, 6])

test_calc.py:4: AssertionError
=========================== short test summary info ============================
FAILED test_calc.py::test_average - assert 3.0 == 4
1 failed in 0.01s

The output shows the test name, the exact line, and what the code produced compared to what you expected. Fix the code (remove the + 1), run again, and it turns green.

Test many cases with parametrize

def is_palindrome(s):
    s = "".join(ch.lower() for ch in s if ch.isalnum())
    return s == s[::-1]
import pytest
from text import is_palindrome

@pytest.mark.parametrize("word, expected", [
    ("level", True),
    ("Python", False),
    ("A man, a plan, a canal: Panama", True),
    ("", True),
])
def test_is_palindrome(word, expected):
    assert is_palindrome(word) is expected
$ python -m pytest -v
============================= test session starts ==============================
rootdir: /your/project
collecting ... collected 4 items

test_text.py::test_is_palindrome[level-True] PASSED                      [ 25%]
test_text.py::test_is_palindrome[Python-False] PASSED                    [ 50%]
test_text.py::test_is_palindrome[A man, a plan, a canal: Panama-True] PASSED [ 75%]
test_text.py::test_is_palindrome[-True] PASSED                           [100%]

============================== 4 passed in 0.01s ===============================

Fixtures: shared setup

class Account:
    def __init__(self, balance=0):
        self.balance = balance

    def deposit(self, amount):
        if amount <= 0:
            raise ValueError("amount must be positive")
        self.balance += amount

    def withdraw(self, amount):
        if amount > self.balance:
            raise ValueError("insufficient funds")
        self.balance -= amount
import pytest
from bank import Account

@pytest.fixture
def account():
    return Account(100)          # a fresh account for every test

def test_deposit(account):
    account.deposit(50)
    assert account.balance == 150

def test_withdraw(account):
    account.withdraw(30)
    assert account.balance == 70

def test_overdraw(account):
    with pytest.raises(ValueError, match="insufficient"):
        account.withdraw(500)

def test_tmp_file(tmp_path):
    f = tmp_path / "note.txt"
    f.write_text("hi")
    assert f.read_text() == "hi"
$ python -m pytest -q
....                                                                     [100%]
4 passed in 0.01s

A fixture is a function marked @pytest.fixture. Name it as a parameter and pytest calls it and passes in the result. Every test gets a brand-new object, so tests never affect each other. The built-in tmp_path fixture hands you an empty temporary folder.

Testing without the real thing: monkeypatch

import datetime

def greeting():
    hour = datetime.datetime.now().hour
    return "Good morning" if hour < 12 else "Good afternoon"
import greeter

class FakeNow:
    def __init__(self, hour):
        self.hour = hour

def test_morning(monkeypatch):
    class FakeDT:
        @staticmethod
        def now():
            return FakeNow(9)
    monkeypatch.setattr(greeter.datetime, "datetime", FakeDT)
    assert greeter.greeting() == "Good morning"

def test_afternoon(monkeypatch):
    class FakeDT:
        @staticmethod
        def now():
            return FakeNow(15)
    monkeypatch.setattr(greeter.datetime, "datetime", FakeDT)
    assert greeter.greeting() == "Good afternoon"
$ python -m pytest -q
..                                                                       [100%]
2 passed in 0.01s

Code that depends on the clock, the network or a database is hard to test because the answer changes. monkeypatch replaces that dependency temporarily, so the test is predictable. The change is undone automatically after each test.

What makes a good test

  1. Arrange: set up the data.
  2. Act: call the code once.
  3. Assert: check the result.
  4. Keep each test small and about one behaviour.
  5. Name it like a sentence: test_overdraw_fails.
  6. Test the edge cases: empty input, zero, negative numbers, very large values.
  7. Tests must be independent and able to run in any order.

Useful commands

pytest                       # run everything
pytest -v                    # one line per test
pytest -x                    # stop at the first failure
pytest test_calc.py          # one file
pytest -k "divide"           # only tests with this word in the name
pip install pytest-cov
pytest --cov=.               # how much code the tests cover
Good to know

When you fix a bug, first write a test that fails because of it, then fix the code. The bug can never silently return. This habit alone makes you a much safer programmer.

Common mistake

Writing tests that depend on each other, or on today’s date, the internet or random numbers. A test that fails sometimes is worse than no test, because people stop trusting it. Fix the inputs: fixtures, seeds and monkeypatch.

Practice

Write a function fizzbuzz(n) and the pytest tests for it (3 gives “Fizz”, 5 gives “Buzz”, 15 gives “FizzBuzz”, 7 gives “7”). Describe how you would run them.

Show one possible answer
def fizzbuzz(n):
    if n % 15 == 0:
        return "FizzBuzz"
    if n % 3 == 0:
        return "Fizz"
    if n % 5 == 0:
        return "Buzz"
    return str(n)

# in test_fizz.py:
# import pytest
# @pytest.mark.parametrize("n, expected", [(3, "Fizz"), (5, "Buzz"), (15, "FizzBuzz"), (7, "7")])
# def test_fizzbuzz(n, expected):
#     assert fizzbuzz(n) == expected
#
# Run with: python -m pytest -v
Practice

Which edge cases would you test for a function average(numbers)? List four.

Show one possible answer
# 1. an empty list (it should raise a clear error, not crash with ZeroDivisionError)
# 2. a list with one number
# 3. negative numbers
# 4. very large numbers, or floats where rounding matters (use pytest.approx)
Start small

You do not need a huge test suite. Even five tests covering the most important functions give you a safety net. Add one more test every time you find a bug.

Check what you learned

Five quick questions. Pick an answer and you will see straight away whether it is right and why.

How must a pytest test function be named?

  1. Starting with check_
  2. Ending in _test only
  3. Starting with test_
  4. Any name

pytest finds functions beginning test_.

What does pytest.raises(ValueError) check?

  1. That the code never fails
  2. That output is printed
  3. That a file exists
  4. That the code raises ValueError

It passes only if that error occurs.

What does a fixture do?

  1. Deletes tests
  2. Provides shared setup to tests
  3. Formats code
  4. Installs packages

Tests ask for it by parameter name.

What is parametrize for?

  1. Naming tests
  2. Skipping tests
  3. Running one test with many inputs
  4. Timing tests

It avoids repeating similar tests.

A good test should be:

  1. Small and independent
  2. Long and dependent
  3. Random
  4. Hidden

One behaviour per test, no dependency on other tests.

Summary

  • Tests are small functions starting with test_ that use assert.
  • pytest.raises checks expected errors.
  • parametrize and fixtures remove repetition.
  • Write a test when you fix a bug.

Next lesson

Next: regular expressions, a powerful language for finding patterns in text.

Questions

How much of my code should be tested?

Aim to cover important logic and every bug you have fixed. 100 percent is not required.

Is unittest different from pytest?

unittest is Python's built-in framework. pytest does the same job with simpler, shorter tests, which is why most people prefer it.

Where do tests live?

Usually in a tests folder beside your code, or as test_*.py files next to the modules they check.

Progress is saved in this browser.