pytest is an awesome Python test framework. According to its homepage:
pytest is a mature full-featured Python testing tool that helps you write better programs.
Pytests may be written either as functions or as methods in classes – unlike unittest, which forces tests to be inside classes. Test classes must be named “Test*”, and test functions/methods must be named “test_*”. Test classes also need not inherit from unittest.TestCase or any other base class. Thus, pytests tend to be more concise and more Pythonic. pytest can also run unittest and nose tests.
pytest provides many advanced test framework features:
- Parameterized tests
- Fixtures for setup/cleanup (builtin, custom, and classic)
- Mocking modules and environments
- Marking tests with attributes
- Plugins and hooks
pytest is actively supported for both Python 2 and 3.
Use pip to install the pytest module. Optionally, install other plugins as well.
pip install pytest pip install pytest-cov pip install pytest-xdist pip install pytest-bdd
The modules containing pytests should be named “test_*.py” or “*_test.py”. While the pytest discovery mechanism can find tests anywhere, pytests must be placed into separate directories from the product code packages. These directories may either be under the project root or under the Python package. However, the pytest directories must not be Python packages themselves, meaning that they should not have “__init__.py” files. (My recommendation is to put all pytests under “[project root]/tests”.) Test configuration may be added to configuration files, which may go by the names “pytest.ini”, “tox.ini”, or “setup.cfg”.
[project root directory] |‐‐ [product code packages] |-- [test directories] | |-- test_*.py | `-- *_test.py `-- [pytest.ini|tox.ini|setup.cfg]
example-py-pytest |-- com.automationpanda.example | |-- __init__.py | |-- calc_class.py | `-- calc_func.py |-- tests | |-- test_calc_class.py | `-- test_calc_func.py |-- README.md `-- pytest.ini
The pytest.ini file is simply a configuration file stub. Feel free to add contents for local testing needs.
The com.automationpanda.example.calc_func module contains basic math functions.
The calc_func tests located in tests/test_calc_func.py are written as functions. Test functions are preferable to test classes when testing functions without side effects.
The divide-by-zero test uses pytest.raises:
And the min/max tests use parameterization:
The com.automationpanda.example.calc_class module contains the Calculator class, which uses the math functions from calc_func. Keeping the functional spirit, the private _do_math method takes in a reference to the math function for greater code reusability.
While tests for the Calculator class could be written using a test class, pytest test functions are just as capable. Fixtures enable a more fine-tuned setup/cleanup mechanism than the typical xUnit-like methods found in test classes. Fixtures can also be used in conjunction with parameterized methods. The tests/test_calc_class.py module is very similar to tests/test_calc_func.py and shows how to use fixtures for testing a class.
Personally, I prefer to write pytests as functions because they are usually cleaner and more flexible than classes. Plus, test functions appeal to my affinity for functional programming.
Basic Test Execution
pytest has a very powerful command line for launching tests. Simply run the pytest module from within the project root directory, and pytest will automatically discover tests.
# Find and run all pytests from the current directory python -m pytest # Run pytests under a given path python -m pytest # Run pytests in a specific module python -m pytest tests/test_calc_func.py # Generate JUnit-style XML test reports python -m pytest --junitxml=[path-to-file] # Get command help python -m pytest -h
The terminal output looks like this:
python -m pytest =============================== test session starts =============================== platform darwin -- Python 2.7.13, pytest-3.0.6, py-1.4.32, pluggy-0.4.0 rootdir: /Users/andylpk247/Programming/automation-panda/python-testing-101/example-py-pytest, inifile: pytest.ini plugins: cov-2.4.0 collected 25 items tests/test_calc_class.py ............. tests/test_calc_func.py ............ ============================ 25 passed in 0.11 seconds ============================
pytest also provides shorter “pytest” and “py.test” command that may be run instead of the longer “python -m pytest” module form. However, the shorter commands do not append the current path to PYTHONPATH, meaning modules under test may not be importable. Make sure to update PYTHONPATH before using the shorter commands.
# Update the Python path PYTHONPATH=$PYTHONPATH:. # Discover and run tests using the shorter command pytest
To run code coverage with the pytest-cov plugin module, use the following command. The report types are optional, but all four types are show below. Specific paths for each report may be appended using “:”.
# Run tests with code coverage python -m pytest [test-path] [other-options] \ --cov= \ --cov-report=annotate \ --cov-report=html \ --cov-report=term \ --cov-report=xml
Code coverage output on the terminal (“term” cov-report) looks like this:
python -m pytest --cov=com --cov-report=term ============================= test session starts ============================== platform darwin -- Python 3.6.5, pytest-3.0.6, py-1.4.32, pluggy-0.4.0 rootdir: /Users/andylpk247/Programming/automation-panda/python-testing-101/example-py-pytest, inifile: pytest.ini plugins: cov-2.4.0 collected 25 items tests/test_calc_class.py ............. tests/test_calc_func.py ............ ---------- coverage: platform darwin, python 3.6.5-final-0 ----------- Name Stmts Miss Cover --------------------------------------------------------------- com/__init__.py 0 0 100% com/automationpanda/__init__.py 0 0 100% com/automationpanda/example/__init__.py 0 0 100% com/automationpanda/example/calc_class.py 21 0 100% com/automationpanda/example/calc_func.py 12 0 100% --------------------------------------------------------------- TOTAL 33 0 100% ========================== 25 passed in 0.11 seconds ===========================<span id="mce_SELREST_start" style="overflow:hidden;line-height:0;"></span>
Parallel testing is vital for more intense testing, such as web testing. The pytest-xdist plugin makes it possible both to scale-up tests by running more than one test process and to scale-out by running tests on other machines. (As a prerequisite, machines need rsync and SSH.) The command below shows how to run multiple test sub-processes; refer to official documentation for multi-machine setup
python -m pytest -n 4 ============================= test session starts ============================== platform darwin -- Python 3.6.5, pytest-3.0.6, py-1.4.32, pluggy-0.4.0 rootdir: /Users/andylpk247/Programming/automation-panda/python-testing-101/example-py-pytest, inifile: pytest.ini plugins: xdist-1.22.2, forked-0.2, cov-2.4.0 gw0  / gw1  / gw2  / gw3  scheduling tests via LoadScheduling ......................... ========================== 25 passed in 1.30 seconds ===========================
Pros and Cons
I’ll say it again: pytest is awesome. It is a powerful test framework with many features, yet its tests are concise and readable. It is very popular and actively supported for both versions of Python. It can handle testing at the unit, integration, and end-to-end levels. It can also be extended with plugins, notably ones for code coverage, parallel execution, and BDD. The only challenge with pytest is that advanced features (namely fixtures) have a learning curve.
My recommendation is to use pytest for standard functional testing in Python. It is one of the best and most popular test frameworks available, and it beats the pants off of alternatives like unittest and nose. pytest is my go-to Python framework, period.
Update: On 4/21/2018, I added pytest-xdist and pytest-bdd plugins, and I made some cosmetic changes.