AdvancedLesson 21 of 305 min read

Decorators: adding powers to functions

Understand closures and write decorators for timing, logging, caching, retrying and permissions, with and without options.

In this lesson you will

  • Explain that functions are objects
  • Write a decorator with a wrapper
  • Use functools.wraps and lru_cache
  • Create decorators that take options

About 24 minutes

Adding powers to a function without touching it

Imagine your program has forty functions, and your boss says: “Log every time one of them runs, and tell me how long it took.” Editing all forty would be dull and risky. A decorator lets you add that behaviour once and attach it to any function with a single line. Decorators are everywhere in professional Python: web frameworks use them to define pages, testing tools use them to mark tests, and libraries use them to add caching. To understand them, we start with one idea you already know: functions are values.

Step 1: functions are objects

def greet(name):
    return f"Hello, {name}"

say = greet                 # no brackets: we are not calling it, just giving it a second name
print(say("Sara"))

def run_twice(func, value):
    return [func(value), func(value)]

print(run_twice(greet, "Ali"))
Hello, Sara
['Hello, Ali', 'Hello, Ali']

Step 2: a function that builds a function

def make_multiplier(n):
    def multiply(x):
        return x * n          # remembers n from the outside
    return multiply

double = make_multiplier(2)
triple = make_multiplier(3)
print(double(10), triple(10))
20 30

Step 3: your first decorator

def shout(func):
    def wrapper(*args, **kwargs):
        result = func(*args, **kwargs)
        return result.upper() + "!"
    return wrapper

@shout
def greet(name):
    return f"hello {name}"

print(greet("sara"))
HELLO SARA!

The line @shout above def greet is shorthand for greet = shout(greet). The name greet now points to wrapper, which calls the original function and then changes the result. *args, **kwargs means “accept any arguments and pass them on”, so the decorator works for any function.

A useful decorator: timing

import time

def timed(func):
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        elapsed = time.perf_counter() - start
        print(f"{func.__name__} took {'under a second' if elapsed < 1 else 'a while'}")
        return result
    return wrapper

@timed
def add_up(n):
    return sum(range(n))

print(add_up(1_000_000))
add_up took under a second
499999500000

Keep the original name: functools.wraps

from functools import wraps

def logged(func):
    @wraps(func)                       # copies the name and docstring
    def wrapper(*args, **kwargs):
        print(f"Calling {func.__name__} with {args} {kwargs}")
        return func(*args, **kwargs)
    return wrapper

@logged
def add(a, b):
    """Add two numbers."""
    return a + b

print(add(2, b=3))
print(add.__name__, "-", add.__doc__)
Calling add with (2,) {'b': 3}
5
add - Add two numbers.
Good to know

Without @wraps, add.__name__ would say wrapper, which confuses debugging tools and documentation. Add it every time.

Decorators that take options

from functools import wraps

def repeat(times):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            result = None
            for _ in range(times):
                result = func(*args, **kwargs)
            return result
        return wrapper
    return decorator

@repeat(3)
def say(text):
    print(text)

say("Python!")
Python!
Python!
Python!

When a decorator has options, you need one more layer. repeat(3) runs first and returns the real decorator. Read it from the outside in: options, then the function, then the arguments.

Real uses

from functools import wraps, lru_cache

# 1. caching: remember answers to avoid repeating work
@lru_cache(maxsize=None)
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)

print(fib(80))
print(fib.cache_info().hits > 0)

# 2. permission check
def require_admin(func):
    @wraps(func)
    def wrapper(user, *args, **kwargs):
        if user != "admin":
            raise PermissionError(f"{user} is not allowed")
        return func(user, *args, **kwargs)
    return wrapper

@require_admin
def delete_all(user):
    return "everything deleted"

print(delete_all("admin"))
try:
    delete_all("guest")
except PermissionError as e:
    print("Blocked:", e)
23416728348467685
True
everything deleted
Blocked: guest is not allowed
import time
from functools import wraps

def retry(attempts):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            for i in range(1, attempts + 1):
                try:
                    return func(*args, **kwargs)
                except ConnectionError as e:
                    print(f"Attempt {i} failed: {e}")
            raise ConnectionError("All attempts failed")
        return wrapper
    return decorator

calls = {"n": 0}

@retry(3)
def flaky():
    calls["n"] += 1
    if calls["n"] < 3:
        raise ConnectionError("network down")
    return "success"

print(flaky())
Attempt 1 failed: network down
Attempt 2 failed: network down
success

Decorating classes: the built-ins

class Circle:
    def __init__(self, r):
        self.r = r

    @property
    def area(self):
        return round(3.14159 * self.r ** 2, 2)

    @staticmethod
    def unit():
        return Circle(1)

    @classmethod
    def from_diameter(cls, d):
        return cls(d / 2)

print(Circle(2).area, Circle.unit().r, Circle.from_diameter(8).r)
12.57 1 4.0
Common mistake

Forgetting to return the result from the wrapper. Then every decorated function quietly returns None. Always write return func(*args, **kwargs) or store and return the result.

Common mistake

Forgetting the brackets on a decorator with options: writing @repeat instead of @repeat(3). Python then passes the function as the option and everything breaks in a confusing way.

Practice

Write a decorator count_calls that counts how many times a function has been called, storing the count in wrapper.calls. Call a function three times and print the count.

Show one possible answer
from functools import wraps

def count_calls(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        wrapper.calls += 1
        return func(*args, **kwargs)
    wrapper.calls = 0
    return wrapper

@count_calls
def hello():
    return "hi"

hello(); hello(); hello()
print(hello.calls)
3
Practice

Write a decorator validate_positive that raises ValueError if any numeric argument is negative. Test it on a function that calculates a rectangle’s area.

Hint: any(a < 0 for a in args)

Show one possible answer
from functools import wraps

def validate_positive(func):
    @wraps(func)
    def wrapper(*args):
        if any(a < 0 for a in args):
            raise ValueError("Arguments must not be negative")
        return func(*args)
    return wrapper

@validate_positive
def area(w, h):
    return w * h

print(area(3, 4))
try:
    area(-1, 5)
except ValueError as e:
    print("Error:", e)
12
Error: Arguments must not be negative
Do not decorate everything

Decorators are best for cross-cutting jobs that apply to many functions: logging, timing, caching, permissions, retrying. If the extra behaviour belongs to only one function, put it inside that function.

Check what you learned

Five quick questions. Pick an answer and you will see straight away whether it is right and why.

What does @shout above a function mean?

  1. A comment
  2. The same as greet = shout(greet)
  3. A loop
  4. An import

The decorator replaces the function with its wrapper.

Why use *args, **kwargs in a wrapper?

  1. To make it faster
  2. To print them
  3. To avoid return
  4. So it accepts any arguments

It forwards whatever arguments the caller gave.

What does functools.wraps do?

  1. Keeps the original name and docstring
  2. Speeds up the code
  3. Adds logging
  4. Creates a class

It copies metadata onto the wrapper.

A decorator with options needs how many nested layers?

  1. One
  2. Two
  3. Three
  4. Four

Options, decorator, wrapper.

Which decorator remembers results of earlier calls?

  1. property
  2. staticmethod
  3. wraps
  4. lru_cache

lru_cache caches return values.

Summary

  • A decorator takes a function and returns a new one.
  • @name means func = name(func).
  • Use *args, **kwargs and @wraps in wrappers.
  • Options add one more layer.

Next lesson

Next: context managers, the clean way to set things up and always tidy them away.

Questions

When would I write my own decorator?

When many functions need the same extra behaviour, such as logging, timing, permission checks or caching.

Can I stack several decorators?

Yes. Place them on separate lines; the one nearest the function is applied first.

What is a closure?

An inner function that remembers variables from the function that created it.

Progress is saved in this browser.