Our aim is that all new functionality is adequately tested. Adding tests for existing functionality is highly recommended before any major reimplementation (refactoring, etcetera).
We use py.test for (unit) testing. You need at least pytest >= 1.0.0, but pytest >= 2.1 is strongly recommended.
To run tests in the current directory and its subdirectories:
$ py.test # runs all tests
$ py.test storage/test_dtd.py # runs just a single test module
We use several py.test features to simplify testing, and to suppress errors in circumstances where the tests cannot possibly succeed (limitations of tests and missing dependencies).
Pytest allows tests, test classes, and modules to be skipped or marked as “expected to fail” (xfail). Generally you should skip only if the test cannot run at all (throws uncaught exception); otherwise xfail is preferred as it provides more test coverage.
Use the builtin importorskip() function to skip a test module if a dependency cannot be imported:
from pytest import importorskip
importorskip("vobject")
If vobject can be imported, it will be; otherwise it raises an exception that causes pytest to skip the entire module rather than failing.
Use the skipif decorator to mark tests to be skipped unless certain criteria are met. The following skips a test if the version of mymodule is too old:
import mymodule
@pytest.mark.skipif("mymodule.__version__ < '1.2'")
def test_function():
...
In Python 2.6 and later, you can apply this decorator to classes as well as functions and methods.
It is also possible to skip an entire test module by creating a pytestmark static variable in the module:
# mark entire module as skipped for py.test if no indexer available
pytestmark = pytest.mark.skipif("noindexer")
Use the xfail decorator to mark tests as expected to fail. This allows you to do the following:
The simplest form is the following:
from pytest import pytest.mark
@mark.xfail
def test_function():
...
You can also pass parameters to the decorator to mark expected failure only under some condition (like skipif), to document the reason failure is expected, or to actually skip the test:
@mark.xfail("sys.version_info >= (3,0)") # only expect failure for Python 3
@mark.xfail(..., reason="Not implemented") # provide a reason for the xfail
@mark.xfail(..., run=False) # skip the test but still regard it as xfailed
The builtin deprecated_call() function checks that a function that we run raises a DeprecationWarning:
from pytest import deprecated_call
def test_something():
deprecated_call(function_to_run, arguments_for_function)
The recwarn plugin allows us to test for other warnings. Note that recwarn is a funcargs plugin, which means that you need it in your test function parameters:
def test_example(recwarn):
# do something
w = recwarn.pop()
# w.{message,category,filename,lineno}
assert 'something' in str(w.message)