Hypothesis

Property testing is a testing method that does not check whether specific inputs lead to specific outputs, but instead generates random inputs, runs the programme with all of these inputs, and then verifies the validity of this property.

In Python, you can use Hypothesis to generate such inputs parametrically, enabling you to quickly find errors in your tests.

See also

The Jupyter Tutorial describes how Hypothesis can also be used in Jupyter Notebooks.

Installation

Linux/macOS

$ uv add --group tests hypothesis

Windows

C:> uv add --group tests hypothesis

Alternatively, Hypothesis can also be installed using extensions, for example:

Linux/macOS

$ uv add --group tests "hypothesis[numpy, pandas]"

Windows

C:> uv add --group tests "hypothesis[numpy, pandas]"

Example using strategies and given

1. First, we import sample data for floats and lists from hypothesis.strategies. To be able to apply this sample data to our test function, we also import hypothesis.given:

   import pytest
   from hypothesis import given
   from hypothesis.strategies import floats, lists
   

2. For our test, we will now use hypothesis.given as a dekorator to convert the test function into a parameterised one, which is then executed with a wide range of suitable data:

   @given(lists(floats(allow_nan=False, allow_infinity=False), min_size=1))
   def test_mean(ls):
       mean = sum(ls) / len(ls)
       assert min(ls) <= mean <= max(ls)
   

3. Finally, we run the test:

   Linux/macOS

   $ uv run pytest docs/test/test_hypothesis.py
   ============================= test session starts ==============================
   platform darwin -- Python 3.13.0, pytest-9.0.3, pluggy-1.6.0
   rootdir: /Users/veit/cusy/trn/python-basics-tutorial-de
   plugins: hypothesis-6.152.1
   collected 1 item
   
   test_hypothesis.py F                                                     [100%]
   
   =================================== FAILURES ===================================
   __________________________________ test_mean ___________________________________
   
       @given(lists(floats(allow_nan=False, allow_infinity=False), min_size=1))
   >   def test_mean(ls):
   
   test_hypothesis.py:6:
   _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
   
   ls = [9.9792015476736e+291, 1.7976931348623157e+308]
   
       @given(lists(floats(allow_nan=False, allow_infinity=False), min_size=1))
       def test_mean(ls):
           mean = sum(ls) / len(ls)
   >       assert min(ls) <= mean <= max(ls)
   E       assert inf <= 1.7976931348623157e+308
   E         +  where 1.7976931348623157e+308 = max([9.9792015476736e+291, 1.7976931348623157e+308])
   
   test_hypothesis.py:8: AssertionError
   ---------------------------------- Hypothesis ----------------------------------
   Falsifying example: test_mean(
       ls=[9.9792015476736e+291, 1.7976931348623157e+308],
   )
   =========================== short test summary info ============================
   FAILED test_hypothesis.py::test_mean - assert inf <= 1.7976931348623157e+308
   ============================== 1 failed in 0.44s ===============================
   

   Windows

   C:> uv run pytest docs/test/test_hypothesis.py
   ============================= test session starts ==============================
   platform darwin -- Python 3.13.0, pytest-9.0.3, pluggy-1.6.0
   rootdir: C:\Users\veit\python-basics-tutorial-de
   plugins: hypothesis-6.152.1
   collected 1 item
   
   test_hypothesis.py F                                                     [100%]
   
   =================================== FAILURES ===================================
   __________________________________ test_mean ___________________________________
   
       @given(lists(floats(allow_nan=False, allow_infinity=False), min_size=1))
   >   def test_mean(ls):
   
   test_hypothesis.py:6:
   _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
   
   ls = [9.9792015476736e+291, 1.7976931348623157e+308]
   
       @given(lists(floats(allow_nan=False, allow_infinity=False), min_size=1))
       def test_mean(ls):
           mean = sum(ls) / len(ls)
   >       assert min(ls) <= mean <= max(ls)
   E       assert inf <= 1.7976931348623157e+308
   E        +  where 1.7976931348623157e+308 = max([9.9792015476736e+291, 1.7976931348623157e+308])
   
   test_hypothesis.py:8: AssertionError
   ---------------------------------- Hypothesis ----------------------------------
   Falsifying example: test_mean(
       ls=[9.9792015476736e+291, 1.7976931348623157e+308],
   )
   =========================== short test summary info ============================
   FAILED test_hypothesis.py::test_mean - assert inf <= 1.7976931348623157e+308
   ============================== 1 failed in 0.44s ===============================
   

   In the list [9.9792015476736e+291, 1.7976931348623157e+308], the calculation of the mean gives inf, and inf is not less than the larger of the two numbers.

Example with regular expressions

1. In the following example, we attempt to extract the username and domain with a regular expression from an email address using:

   import re
   
   
   def parse_email(email):
       result = re.match(r"(?P<username>\w+).(?P<domain>[\w\.]+)", email).groups()
       return result
   

2. Now let’s write a test called test_parse_email() to check our function. We’ll use the emails strategy from Hypothesis as our input values. As a result, we expect, for example, that for veit@cusy.io, the username is veit and the domain is cusy.io.

3. In our test, we assume, on the one hand, that two entries are always returned and that the second entry contains a full stop (.):

   from hypothesis import given
   from hypothesis.strategies import emails
   
   
   @given(emails())
   def test_parse_email(email):
       result = parse_email(email)
       # print(result)
       assert len(result) == 2
       assert "." in result[1]
   

4. Now let’s run the test:

   $ uv run pytest docs/test/test_emails.py
   ============================= test session starts ==============================
   platform darwin -- Python 3.13.0, pytest-9.0.3, pluggy-1.6.0
   rootdir: /Users/veit/cusy/trn/python-basics-tutorial-de
   configfile: pyproject.toml
   plugins: hypothesis-6.152.1
   collected 1 item
   
   docs/test/test_emails.py F                                               [100%]
   
   =================================== FAILURES ===================================
   _______________________________ test_parse_email _______________________________
     + Exception Group Traceback (most recent call last):
     |   File "/Users/veit/cusy/trn/python-basics-tutorial-de/docs/test/test_emails.py", line 12, in test_parse_email
     |     def test_parse_email(email):
     |                    ^^^
     |   File "/Users/veit/cusy/trn/python-basics-tutorial-de/.venv/lib/python3.13/site-packages/hypothesis/core.py", line 2264, in wrapped_test
     |     raise the_error_hypothesis_found
     | ExceptionGroup: Hypothesis found 2 distinct failures. (2 sub-exceptions)
     +-+---------------- 1 ----------------
       | Traceback (most recent call last):
       |   File "/Users/veit/cusy/trn/python-basics-tutorial-de/docs/test/test_emails.py", line 16, in test_parse_email
       |     assert '.' in result[1]
       | AssertionError: assert '.' in '0'
       | Falsifying example: test_parse_email(
       |     email='0/0@A.AC',
       | )
       +---------------- 2 ----------------
       | Traceback (most recent call last):
       |   File "/Users/veit/cusy/trn/python-basics-tutorial-de/docs/test/test_emails.py", line 13, in test_parse_email
       |     result = parse_email(email)
       |   File "/Users/veit/cusy/trn/python-basics-tutorial-de/docs/test/test_emails.py", line 8, in parse_email
       |     result = re.match(r"(?P<username>\w+).(?P<domain>[\w\.]+)", email).groups()
       |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       | AttributeError: 'NoneType' object has no attribute 'groups'
       | Falsifying example: test_parse_email(
       |     email='/@A.AC',
       | )
       +------------------------------------
   =========================== short test summary info ============================
   FAILED docs/test/test_emails.py::test_parse_email - ExceptionGroup: Hypothesis found 2 distinct failures. (2 sub-exceptions)
   ============================== 1 failed in 0.14s ===============================
   

   The email address provided by Hypothesis, 0/0@A.ac, shows that our regular expression in the parse_email() method is not yet sufficient. We will therefore adjust our regular expression and then run the test again:

   --- /home/runner/work/python-basics-tutorial/python-basics-tutorial/docs/test/test_emails.py
   +++ /home/runner/work/python-basics-tutorial/python-basics-tutorial/docs/test/test_emails_2.py
   @@ -5,7 +5,9 @@
    
    
    def parse_email(email):
   -    result = re.match(r"(?P<username>\w+).(?P<domain>[\w\.]+)", email).groups()
   +    result = re.match(
   +        r"(?P<username>[\.\w\-\!~#$%&\|{}\+\/\^\`\=\*']+).(?P<domain>[\w\.\-]+)", email
   +    ).groups()
        return result
    
    
   

   $ uv run pytest docs/test/test_emails_2.py
   ============================= test session starts ==============================
   platform darwin -- Python 3.13.0, pytest-9.0.3, pluggy-1.6.0
   rootdir: /Users/veit/cusy/trn/python-basics-tutorial-de
   configfile: pyproject.toml
   plugins: hypothesis-6.152.1
   collected 1 item
   
   docs/test/test_emails_2.py .                                             [100%]
   
   ============================== 1 passed in 0.29s ===============================
   

Third-party extensions

There are a number of open-source libraries that extend Testing capabilities. Some of these are listed on Third-party extensions; you can find more on PyPI by searching for keywords or using the framework classifier.