Testing

Contents

Testing#

Any developer is constantly running tests to make sure the written code behaves as expected. But instead of manually executing the code and comparing the results to expectations, it is much more efficient to automate this process. This can range from small unit tests that check individual components (like functions) to larger integration tests that are run to test how multiple parts of the program integrate together. Often a small set of tests can already go a long way. Some software engineers even plead for test-driven development (TDD), where development starts by writing tests representing the functional requirements. Writing the code itself comes next and must be written to make all tests pass.

In addition to cross-checking that the code produces the expected results, having a test suite proves to be a valuable asset when refactoring the code, as the passing of all tests is a strong indicator that the refactoring did not break anything. And if something is nevertheless found to have changed inadvertently, add a corresponding test to catch the regression next time. Last but not least, a test suite helps when ensuring the software runs on different platforms and with a certain range of package versions. Even adding a couple of basic tests to your program plants the seed for further tests to be added later on.

assert#

The assert statement can be a quick and convenient way to insert simple checks into a program. It raises an exception if the asserted expression is false.

number = 3
assert number > 0

A second argument is accepted to specify the exception message.

number = 3
assert number %% 2 == 0, 'The number must be even'

Traceback (most recent call last):
  File "assert_even.py", line 2, in <module>
    assert number % 2 == 0, "The number must be even"
           ^^^^^^^^^^^^^^^
AssertionError: The number must be even

It’s even possible to run the code with all assert statements removed with the PYTHONOPTMIZE flag -O. This allows to have best performance in case the assertions are only used in development.

$ python -O script.py

doctest#

The doctest module searches for code snippets in the docstrings and executes them as tests. This introduces tests with minimal effort and keeps them right next to the corresponding code, thereby ensuring that the tests get updated as the code evolves.

def square(x):
    """
    Computes the second power of a given number.

    >>> square(-3)
    -9
    """
    return x**2

$ python -m doctest -v doctest_square.py
Trying:
    square(-3)
Expecting:
    -9
**********************************************************************
File "doctest_square.py", line 8, in doctest_square.square
Failed example:
    square(-3)
Expected:
    -9
Got:
    9
1 item had no tests:
    doctest_square
**********************************************************************
1 item had failures:
   1 of   1 in doctest_square.square
1 test in 2 items.
0 passed and 1 failed.
***Test Failed*** 1 failure.

pytest#

The third-party pytest is a widely used alternative to the built-in unittest. Its numerous features can be used to write extensive test suites. The functionality can be further extended via a large list of plugins, for instance to also run a linter/formatter against the code. Most of the advanced code editors offer a tight integration with such testing frameworks.

By convention all tests are placed in a tests/ directory next to the actual code to be tested.

├── mypackage/
│   ├── __init__.py
│   └── functions.py
└── tests/
    └── test_square.py

Where functions.py contains the definition of the square function (see above) and __init__.py is an empty file to define mypackage as Python package. The name of all test files and functions start with test_ so that pytest can discover and run them.

from pytest import approx
from mypackage.functions import square


def test_square_negative_integer():
    assert square(-1) == 1
    assert square(-3) == 9


def test_square_rational():
    assert square(1 / 5) == approx(1 / 25)

Notice how we have made use of pytest.approx to test for almost equality of floats within a given tolerance.

Finally run the test suite with pytest.

$ python -m pytest -v tests
==================================== test session starts ====================================
collected 2 items

tests/test_square.py::test_square_negative_integer PASSED                             [ 50%]
tests/test_square.py::test_square_rational PASSED                                     [100%]

===================================== 2 passed in 0.00s =====================================

When faced with larger test suites one can run specific tests to speed things up.

$ python -m pytest -q tests/test_square.py::test_square_negative_integer
.                                                                                    [100%]
1 passed in 0.00s
Contents