Decorators: adding powers to functions
Understand closures and write decorators for timing, logging, caching, retrying and permissions, with and without options.
In this lesson you will
- Explain that functions are objects
- Write a decorator with a wrapper
- Use functools.wraps and lru_cache
- Create decorators that take options
About 24 minutes
Adding powers to a function without touching it
Imagine your program has forty functions, and your boss says: “Log every time one of them runs, and tell me how long it took.” Editing all forty would be dull and risky. A decorator lets you add that behaviour once and attach it to any function with a single line. Decorators are everywhere in professional Python: web frameworks use them to define pages, testing tools use them to mark tests, and libraries use them to add caching. To understand them, we start with one idea you already know: functions are values.
Step 1: functions are objects
def greet(name):
return f"Hello, {name}"
say = greet # no brackets: we are not calling it, just giving it a second name
print(say("Sara"))
def run_twice(func, value):
return [func(value), func(value)]
print(run_twice(greet, "Ali"))
Hello, Sara
['Hello, Ali', 'Hello, Ali']
Step 2: a function that builds a function
def make_multiplier(n):
def multiply(x):
return x * n # remembers n from the outside
return multiply
double = make_multiplier(2)
triple = make_multiplier(3)
print(double(10), triple(10))
20 30
Step 3: your first decorator
def shout(func):
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
return result.upper() + "!"
return wrapper
@shout
def greet(name):
return f"hello {name}"
print(greet("sara"))
HELLO SARA!
The line @shout above def greet is shorthand for greet = shout(greet). The name greet now points to wrapper, which calls the original function and then changes the result. *args, **kwargs means “accept any arguments and pass them on”, so the decorator works for any function.
A useful decorator: timing
import time
def timed(func):
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = func(*args, **kwargs)
elapsed = time.perf_counter() - start
print(f"{func.__name__} took {'under a second' if elapsed < 1 else 'a while'}")
return result
return wrapper
@timed
def add_up(n):
return sum(range(n))
print(add_up(1_000_000))
add_up took under a second
499999500000
Keep the original name: functools.wraps
from functools import wraps
def logged(func):
@wraps(func) # copies the name and docstring
def wrapper(*args, **kwargs):
print(f"Calling {func.__name__} with {args} {kwargs}")
return func(*args, **kwargs)
return wrapper
@logged
def add(a, b):
"""Add two numbers."""
return a + b
print(add(2, b=3))
print(add.__name__, "-", add.__doc__)
Calling add with (2,) {'b': 3}
5
add - Add two numbers.
Without @wraps, add.__name__ would say wrapper, which confuses debugging tools and documentation. Add it every time.
Decorators that take options
from functools import wraps
def repeat(times):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
result = None
for _ in range(times):
result = func(*args, **kwargs)
return result
return wrapper
return decorator
@repeat(3)
def say(text):
print(text)
say("Python!")
Python!
Python!
Python!
When a decorator has options, you need one more layer. repeat(3) runs first and returns the real decorator. Read it from the outside in: options, then the function, then the arguments.
Real uses
from functools import wraps, lru_cache
# 1. caching: remember answers to avoid repeating work
@lru_cache(maxsize=None)
def fib(n):
return n if n < 2 else fib(n - 1) + fib(n - 2)
print(fib(80))
print(fib.cache_info().hits > 0)
# 2. permission check
def require_admin(func):
@wraps(func)
def wrapper(user, *args, **kwargs):
if user != "admin":
raise PermissionError(f"{user} is not allowed")
return func(user, *args, **kwargs)
return wrapper
@require_admin
def delete_all(user):
return "everything deleted"
print(delete_all("admin"))
try:
delete_all("guest")
except PermissionError as e:
print("Blocked:", e)
23416728348467685
True
everything deleted
Blocked: guest is not allowed
import time
from functools import wraps
def retry(attempts):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
for i in range(1, attempts + 1):
try:
return func(*args, **kwargs)
except ConnectionError as e:
print(f"Attempt {i} failed: {e}")
raise ConnectionError("All attempts failed")
return wrapper
return decorator
calls = {"n": 0}
@retry(3)
def flaky():
calls["n"] += 1
if calls["n"] < 3:
raise ConnectionError("network down")
return "success"
print(flaky())
Attempt 1 failed: network down
Attempt 2 failed: network down
success
Decorating classes: the built-ins
class Circle:
def __init__(self, r):
self.r = r
@property
def area(self):
return round(3.14159 * self.r ** 2, 2)
@staticmethod
def unit():
return Circle(1)
@classmethod
def from_diameter(cls, d):
return cls(d / 2)
print(Circle(2).area, Circle.unit().r, Circle.from_diameter(8).r)
12.57 1 4.0
Forgetting to return the result from the wrapper. Then every decorated function quietly returns None. Always write return func(*args, **kwargs) or store and return the result.
Forgetting the brackets on a decorator with options: writing @repeat instead of @repeat(3). Python then passes the function as the option and everything breaks in a confusing way.
Write a decorator count_calls that counts how many times a function has been called, storing the count in wrapper.calls. Call a function three times and print the count.
Show one possible answer
from functools import wraps
def count_calls(func):
@wraps(func)
def wrapper(*args, **kwargs):
wrapper.calls += 1
return func(*args, **kwargs)
wrapper.calls = 0
return wrapper
@count_calls
def hello():
return "hi"
hello(); hello(); hello()
print(hello.calls)
3
Write a decorator validate_positive that raises ValueError if any numeric argument is negative. Test it on a function that calculates a rectangle’s area.
Hint: any(a < 0 for a in args)
Show one possible answer
from functools import wraps
def validate_positive(func):
@wraps(func)
def wrapper(*args):
if any(a < 0 for a in args):
raise ValueError("Arguments must not be negative")
return func(*args)
return wrapper
@validate_positive
def area(w, h):
return w * h
print(area(3, 4))
try:
area(-1, 5)
except ValueError as e:
print("Error:", e)
12
Error: Arguments must not be negative
Decorators are best for cross-cutting jobs that apply to many functions: logging, timing, caching, permissions, retrying. If the extra behaviour belongs to only one function, put it inside that function.
Check what you learned
Five quick questions. Pick an answer and you will see straight away whether it is right and why.
What does @shout above a function mean?
- A comment
- The same as greet = shout(greet)
- A loop
- An import
The decorator replaces the function with its wrapper.
Why use *args, **kwargs in a wrapper?
- To make it faster
- To print them
- To avoid return
- So it accepts any arguments
It forwards whatever arguments the caller gave.
What does functools.wraps do?
- Keeps the original name and docstring
- Speeds up the code
- Adds logging
- Creates a class
It copies metadata onto the wrapper.
A decorator with options needs how many nested layers?
- One
- Two
- Three
- Four
Options, decorator, wrapper.
Which decorator remembers results of earlier calls?
propertystaticmethodwrapslru_cache
lru_cache caches return values.
Summary
- A decorator takes a function and returns a new one.
- @name means func = name(func).
- Use *args, **kwargs and @wraps in wrappers.
- Options add one more layer.
Questions
When would I write my own decorator?
When many functions need the same extra behaviour, such as logging, timing, permission checks or caching.
Can I stack several decorators?
Yes. Place them on separate lines; the one nearest the function is applied first.
What is a closure?
An inner function that remembers variables from the function that created it.
