Package HowTo

This is a brief introduction to python packages. How to make them, how to install them, how to upload them, how to maintain them, how to work with them. For details consult https://packaging.python.org and other sites.

Why packages?

When working on your project, you typically end up with some scripts, functions and classes that are of more general interest. As a first step, you make some modules, i.e. separate python files, where you collect this code. These modules can be easily imported from other scripts in the very same directory.

For example, consider a module addition.py and a script analyze.py both in the same directory:

├── addition.py
└── analyze.py

In addition.py we define a function add_two():

def add_two(x):
    return x + 2

We can use this function in analyze.py by importing it from addition.py:

from addition import add_two

x = 5
y = add_two(x)

This works, however, only if scripts and modules are in the same directory (or if modules are in sub-directories). To make modules available in other places of your file system or even for other people, you need to turn the modules into packages.

Minimal package

First we make a project directory for our new package. Usually the name of the project directory is the same as the one of the package. Here, however, we call the proejct directory packagehowto and the name of the package numerix.

A package is a directory, and the name of the package is the name of this directory, here numerix. Inside the package directory we put all the modules, for now just addition.py.

What makes this directory a package is the presence of a file named __init__.py. This file is executed when the package is imported. For now we leave it empty.

The package directory resides in the src/ directory of the project:

packagehowto/
├── pyproject.toml
└── src/
    └── numerix
        ├── __init__.py
        └── addition.py

A package also needs a pyproject.toml file. This file contains some metadata about the package and informations about how to build a package. As the bare minimum the content of the pyproject.toml file specifies setuptools to be used for building the project:

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

With this pyproject.toml file, and addition.py and __init__.py in the src/numerix directory, you can install the numerix package on your machine and import it from wherever you want.

To install the project, go to the project root, here packagehowto/ and run from your shell

pip3 install .

This installs the package of the current project folder . somewhere in your home directory. From anywhere in your home file system you now can import this package. The import line of our analyze.py file needs to look like this:

from numerix.addition import add_two

From the addition module of the numerix package the function add_two is imported.

You would need to reinstall the package whenever you change your package, for exmple when you add a new function or when you just fix some package code. This is tedious, and that is why there is a -e option for pip install. So install your package with

pip3 install -e .

Then all future changes on your package are immediately available without the need to reinstall the package again.

The __init__.py file

This file is your package. In fact, you could write a package that only has an __init__.py file in the package directory. All code you want to make available in you package could be place in the __init__.py file. For example, if you define a add_four() function in __init__.py like this:

def add_four(x):
    return x + 4

then it can be imported directly from the package:

from numerix import add_four

Alternatively, you can define all your functions in module files, like we did with the function add_two() in the addition.py module. In the __init__.py file you can import this function and this way make it available directly from the package. With this line in __init__.py

from .addition import add_two

(note the . in front of addition!) you can import add_two() without specifying the module:

from numerix import add_two

Multiple modules within a package

You can have as many modules as you need in your package. Let's add a module numbers.py to the numerix package

    └── numerix
        ├── __init__.py
        ├── addition.py
        └── numbers.py

with the following content:

from .addition import add_two

def three():
    return add_two(1)

This makes a function three() available that can be imported via

from numerix.numbers import three

Note, that the numbers.py module imports a function from the addition.py module via a relative import with the . in front of addition.

Distribute your package

Full pyproject.toml file...

README.md

LICENSE

pypi.org

python3 -m build

python3 -m twine upload dist/*

token

Package version

Unit tests

pytest

coverage

pytest -v --cov-report html --cov numerix tests/

Documentation