IntermediateLesson 13 of 306 min read

Modules, packages, pip and virtual environments

Import standard-library modules, write your own, install packages with pip and keep projects tidy with virtual environments.

In this lesson you will

  • Import and explore modules
  • Write your own modules and packages
  • Install packages with pip
  • Use a virtual environment and requirements.txt

About 21 minutes

Do not reinvent the wheel

Someone has already written code to read a spreadsheet, send an email, resize an image, and pick a random number. Python’s great strength is that you can borrow that code with a single line. A file of Python code you can borrow from is called a module; a folder of modules is a package. In this lesson you will use Python’s built-in modules, install other people’s packages, and split your own program into several files.

Importing from the standard library

import math
import random
import statistics

print(math.sqrt(81))
print(math.pi)

random.seed(1)
print(random.randint(1, 6))
print(random.choice(["red", "green", "blue"]))

scores = [72, 91, 45, 68, 88]
print(statistics.mean(scores))
print(statistics.median(scores))
9.0
3.141592653589793
2
blue
72.8
72

Python ships with hundreds of modules called the standard library. Nothing to install; you just import them. Here are a few worth knowing:

Module What it is for
math Square roots, trigonometry, constants
random Random numbers and choices
statistics Mean, median, standard deviation
datetime Dates and times
json, csv Reading and writing common data formats
pathlib, os, shutil Files, folders and the operating system
collections Counter, defaultdict, deque
re Regular expressions
sys Python itself and command-line arguments
urllib, http Basic web access
sqlite3 A small built-in database

Different ways to import

import math                     # use as math.sqrt(...)
print(math.sqrt(16))

from math import sqrt, pi       # use sqrt(...) directly
print(sqrt(25), round(pi, 2))

import random as rnd            # a shorter alias
print(rnd.Random(5).randint(1, 100))

from collections import Counter
print(Counter("banana"))
4.0
5.0 3.14
80
Counter({'a': 3, 'n': 2, 'b': 1})
Common mistake

Using from module import *. It dumps every name into your program, makes it unclear where things come from, and can silently overwrite your own names. Import what you need, by name.

Exploring a module

import math

print(type(math))
print([name for name in dir(math) if not name.startswith("_")][:8])
print(math.floor.__doc__)
<class 'module'>
['acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'cbrt']
Return the floor of x as an Integral.

This is the largest integer <= x.

dir(module) lists everything inside, and help(module.function) prints its documentation. The official documentation at docs.python.org has a page for every standard module with examples.

Writing your own module

Any Python file is a module. Imagine two files in the same folder: one with reusable helpers, one that uses them.

"""Small helper functions for money."""

TAX_RATE = 0.17

def add_tax(price):
    return price * (1 + TAX_RATE)

def money(amount):
    return f"Rs {amount:,.2f}"
import helpers
from helpers import money

print(helpers.add_tax(100))
print(money(helpers.add_tax(1000)))
print(helpers.TAX_RATE)
117.0
Rs 1,170.00
0.17

When Python sees import helpers, it looks for helpers.py in the same folder, runs it once, and gives you access to its names. This is how real programs are organised: each file has one clear job.

The __name__ guard

def shout(text):
    return text.upper() + "!"

def main():
    print(shout("running directly"))

if __name__ == "__main__":
    main()
import tools
print(tools.shout("imported, so main() did not run"))
IMPORTED, SO MAIN() DID NOT RUN!

When a file is run directly, Python sets its __name__ to "__main__". When it is imported, __name__ is the module’s name. The guard if __name__ == "__main__": lets a file work both as a program and as a module, without running the program part on import.

Packages: folders of modules

def discount(price, percent):
    return price - price * percent / 100
def money(x):
    return f'Rs {x:,.0f}'
from shop import prices, format
from shop.prices import discount

print(discount(2000, 25))
print(format.money(prices.discount(2000, 25)))
1500.0
Rs 1,500

A folder containing Python files (and usually an empty __init__.py) is a package. You reach inside with dots: shop.prices. Bigger projects are organised into packages like this.

Installing other people’s packages with pip

Beyond the standard library lies the Python Package Index (PyPI), with hundreds of thousands of free packages. The tool that installs them is pip, which comes with Python. You run it in the terminal, not inside Python.

pip install requests          # install a package
pip install requests==2.32.3   # a specific version
pip list                       # what is installed
pip show requests              # details about one package
pip uninstall requests
pip install --upgrade requests

Popular packages include requests (web access), pandas (data tables), numpy (fast numbers), matplotlib (charts), flask and django (websites), pillow (images) and pytest (testing). On some systems the command is pip3 or python -m pip.

Virtual environments: a clean room for each project

Imagine two projects: one needs version 1 of a package, the other needs version 2. If everything is installed in one place, they fight. A virtual environment is a private, isolated copy of Python and its packages for one project.

  1. Open a terminal in your project folder.
  2. Create it: python -m venv .venv
  3. Activate it. On Windows: .venv\Scripts\activate. On Mac or Linux: source .venv/bin/activate. Your prompt now starts with (.venv).
  4. Install what the project needs: pip install requests
  5. Save the list of packages: pip freeze > requirements.txt
  6. Someone else (or you, on another computer) rebuilds it with pip install -r requirements.txt.
  7. Leave it with deactivate.
Good to know

Make one virtual environment per project, and add .venv to your .gitignore so you never upload it. The requirements.txt file is what you share.

Where does Python look for modules?

import sys

print(type(sys.path), len(sys.path) > 0)
print(sys.version_info.major)
<class 'list'> True
3

When you import something, Python searches the folders in sys.path: first your script’s folder, then the standard library, then installed packages. A frequent beginner trap follows from this: never name your own file the same as a module you want to import. A file called random.py would hide the real random module.

Command-line arguments with sys.argv

import sys

print(sys.argv)         # a list: the script name first, then any arguments
['-c']

If you run python greet.py Sara, then sys.argv is ["greet.py", "Sara"]. For serious command-line tools, the standard argparse module builds help messages and validates options for you.

Practice

Use the random and statistics modules: seed the generator with 10, roll a six-sided die 10 times, and print the rolls and their mean.

Show one possible answer
import random
import statistics

random.seed(10)
rolls = [random.randint(1, 6) for _ in range(10)]
print(rolls)
print(statistics.mean(rolls))
[5, 1, 4, 4, 5, 1, 2, 4, 4, 3]
3.3
Practice

Use collections.Counter to find the three most common letters in the word “mississippi”.

Show one possible answer
from collections import Counter

print(Counter("mississippi").most_common(3))
[('i', 4), ('s', 4), ('p', 2)]
Read before you install

Anyone can publish a package, and typing the wrong name can install a look-alike. Check the package name, its download count and its page on pypi.org before you install anything new.

Check what you learned

Five quick questions. Pick an answer and you will see straight away whether it is right and why.

What is a module?

  1. A kind of loop
  2. A Python file you can import
  3. A variable type
  4. A comment

Any .py file can be imported as a module.

Which tool installs packages from PyPI?

  1. npm
  2. git
  3. make
  4. pip

pip is Python’s package installer.

Why use a virtual environment?

  1. To make Python faster
  2. To keep each project’s packages separate
  3. To hide code
  4. To save disk space

Each project gets its own isolated set of packages.

What does if __name__ == "__main__": do?

  1. Imports the file
  2. Defines a class
  3. Stops the program
  4. Runs code only when the file is executed directly

It stops that code running when the file is imported.

Which file lists a project’s packages?

  1. requirements.txt
  2. main.py
  3. readme.md
  4. packages.exe

pip freeze > requirements.txt creates it.

Summary

  • import gives you other people’s code.
  • Any .py file is a module; a folder of them is a package.
  • pip installs from PyPI; use a virtual environment per project.
  • requirements.txt makes a project reproducible.

Next lesson

Next: classes and objects, building your own types.

Questions

pip is not recognised. What now?

Python was probably not added to PATH. Try python -m pip install ..., or reinstall Python with the Add to PATH box ticked.

What is the difference between a module and a package?

A module is one file. A package is a folder of modules.

Do I need a virtual environment for small scripts?

Not for scripts using only the standard library, but it is a good habit as soon as you install packages.

Progress is saved in this browser.