Most tests you have written answer a narrow question: does the code produce the expected output for this one input? That is example-based testing, and it has a structural weakness — the tests only cover the cases you happened to imagine. The three-space string, the cart with twenty identical items, the payload nested six levels deep: if you never thought of them, no test checks them.
Property-based testing flips the question around. Instead of asserting outputs for hand-picked inputs, you describe laws your code must obey for any input — encoding and decoding must round-trip, a discount must never increase a total, a stack’s depth must never lie — and a tool generates hundreds of inputs specifically to break those laws. In the Python ecosystem that tool is Hypothesis, and it is the rare testing library that routinely finds bugs in code whose tests all pass.
This post is example-first. We will start with a function whose hand-written tests pass and whose property test fails, learn to state properties worth testing, build custom input generators, run a realistic shopping-cart discount check, and finish with stateful testing for classes that remember what happened to them.
From Examples to Properties
Here is a function that collapses runs of consecutive spaces into a single space, together with the kind of test most of us write for it. Both examples pass — and so would every example you would plausibly add by hand, because a pair of spaces is easy to imagine. The property test at the bottom says something different: whatever the input is, the result may not contain two consecutive spaces.
from hypothesis import given
from hypothesis import strategies as st
def squeeze_spaces(text: str) -> str:
"""Collapse every run of consecutive spaces into a single space."""
return text.replace(" ", " ")
def test_squeeze_spaces_examples():
# Example-based: passes, because we only tried cases we thought of.
assert squeeze_spaces("a b") == "a b"
assert squeeze_spaces(" leading") == " leading"
@given(st.text())
def test_squeeze_spaces_never_leaves_double_spaces(text):
result = squeeze_spaces(text)
assert " " not in resultThe property test fails on the first run. Hypothesis reports a falsifying example, and thanks to automatic shrinking it is not the forty-character blob of unicode it first stumbled on — it is three literal spaces, ' '. A single replace pass turns three spaces into two, and no example-based suite catches that, because nobody writes a triple-space test by hand. The fix is mechanical (loop until stable, or use a regex to match runs of spaces), but the insight came free from the generator. If you want to reproduce this whole loop yourself, the quick-start guide takes about ten minutes.
Properties Worth Testing
Once you stop thinking in examples, the real question becomes: what laws does my code obey? Four shapes cover most of the value.
- Round-trips — serialize/deserialize, encode/decode, save/load: feeding the output back in must return the original value.
- Invariants — conditions that hold after any operation: totals never go negative, sizes never drop below zero, counts always match the ledger.
- Idempotence — applying the operation twice must equal applying it once: sorting, normalization, cache warming.
- Oracle comparisons — check a fast implementation against a slow, obviously correct one and demand the same answer.
The JSON round-trip below is the classic: a recursive strategy assembles arbitrarily nested values, and json.loads(json.dumps(value)) == value must hold for every one of them. The sorting test checks idempotence, and the digit counter is checked against a regex as an oracle — two independent implementations agreeing across thousands of generated inputs.
import json
import re
from hypothesis import given
from hypothesis import strategies as st
# Arbitrarily nested JSON-compatible values.
json_values = st.recursive(
st.none() | st.booleans() | st.integers() | st.text()
| st.floats(allow_nan=False, allow_infinity=False),
lambda children: st.lists(children, max_size=3)
| st.dictionaries(st.text(), children, max_size=3),
max_leaves=25,
)
@given(json_values)
def test_json_round_trip(value):
assert json.loads(json.dumps(value)) == value
@given(st.lists(st.integers()))
def test_sorting_is_idempotent(numbers):
once = sorted(numbers)
assert sorted(once) == once
def count_digits(text: str) -> int:
return sum(1 for ch in text if ch in "0123456789")
@given(st.text())
def test_count_digits_matches_regex_oracle(text):
assert count_digits(text) == len(re.findall(r"[0-9]", text))Custom Strategies with st.composite
Real code does not take bare integers; it takes carts, invoices, and API payloads. Hypothesis calls input generators strategies, and the built-ins — st.integers(), st.text(), st.lists(...), st.sampled_from(...) — compose into larger ones. When you need a domain-specific shape, st.composite hands your function a draw argument: call draw on other strategies, assemble the results, and return the object. The decorated function becomes a first-class strategy you can mix into anything else.
from hypothesis import strategies as st
@st.composite
def cart_items(draw):
"""Generate one cart item: a name, a price in cents, a quantity."""
return {
"name": draw(st.text(min_size=1, max_size=30)),
"price_cents": draw(st.integers(min_value=1, max_value=100_000)),
"quantity": draw(st.integers(min_value=1, max_value=10)),
}
# Composes like any built-in strategy:
carts = st.lists(cart_items(), max_size=20)
# Peek interactively while designing (never inside tests):
# cart_items().example()A Worked Example: Shopping-Cart Discounts
Money is where property testing earns its keep, because floats and currency are a notorious pair. The function below applies a percentage discount to a cart subtotal and promises a result in whole cents. The implementation is exactly the kind of thing a hurried review waves through. Prices are integers in cents, so the subtotal is exact — which means any drift in the output is the function’s fault, and the property test will say so.
from hypothesis import example, given
from hypothesis.strategies import st
@st.composite
def cart_items(draw):
return {
"name": draw(st.text(min_size=1, max_size=30)),
"price_cents": draw(st.integers(min_value=1, max_value=100_000)),
"quantity": draw(st.integers(min_value=1, max_value=10)),
}
def discounted_total_cents(items, tier_pct: int):
"""Apply a percentage discount to the cart subtotal, in integer cents."""
subtotal = sum(i["price_cents"] * i["quantity"] for i in items)
return subtotal * (1 - tier_pct / 100) # silently returns a float
@given(st.lists(cart_items(), max_size=20))
@example([{"name": "shrunk case", "price_cents": 3, "quantity": 1}])
def test_discounted_total_stays_in_whole_cents(items):
subtotal = sum(i["price_cents"] * i["quantity"] for i in items)
total = discounted_total_cents(items, 10)
assert total == int(total) # whole cents only, no float drift
assert 0 <= total <= subtotal # a discount never adds moneyThis fails within a few generated examples. A one-item cart priced at 3 cents yields 3 * 0.9 = 2.7, which is not a whole number of anything, and shrinking pins the failure down to exactly that minimal cart. The fix is integer arithmetic — subtotal * (100 - tier_pct) // 100 — after which the property holds for every input. Note the @example(...) decorator: it forces one specific input through the test on every run, turning the shrunken counterexample into a permanent regression check even though the generator’s random sequence changes from run to run.
Stateful Testing with RuleBasedStateMachine
Everything so far tests pure functions, but plenty of bugs live in sequences: the pop that happens before its push, the refund that happens twice. This is what stateful testing is for. You declare a class inheriting from RuleBasedStateMachine; its rules are randomly interleaved steps, and its invariants are checked before and after every step. A Bundle links steps together — values produced by one rule become available as inputs to later rules, which is how you express “only pop items that were actually pushed.”
from hypothesis import strategies as st
from hypothesis.stateful import (
Bundle,
RuleBasedStateMachine,
initialize,
invariant,
rule,
)
class Stack:
def __init__(self):
self._items = []
def push(self, value):
self._items.append(value)
def pop(self):
return self._items.pop()
def depth(self):
return len(self._items)
class StackMachine(RuleBasedStateMachine):
values = Bundle("values")
@initialize()
def start_empty(self):
self.stack = Stack()
self.pushes = 0
self.pops = 0
@rule(target=values, value=st.integers())
def push(self, value):
self.stack.push(value)
self.pushes += 1
return value # stored in the bundle for later rules
@rule(value=values)
def pop(self, value):
# Only runs when the bundle is non-empty, so pop is always safe.
self.stack.pop()
self.pops += 1
@invariant()
def depth_tracks_history(self):
assert self.stack.depth() == self.pushes - self.pops
TestStackMachine = StackMachine.TestCase # collected by pytest as-isThe invariant runs around every push and pop, so any sequence that corrupts the depth count fails immediately — and the traceback includes the full step history that produced the failure, which is often worth more than the assertion itself. StackMachine.TestCase is a ready-made unittest class, so the machine drops into an existing suite with no glue code.
Running Under pytest
Integration is deliberately invisible. Hypothesis ships a pytest plugin, so any test_ function decorated with @given is collected and run like an ordinary test, and failures print the falsifying example next to the shrunk minimal case. Failed examples are also saved to a .hypothesis/ directory and replayed first on subsequent runs, so a bug you fixed six weeks ago stays fixed. Cost is tuned with @settings:
"""test_text_utils.py — run with: pytest -q"""
from hypothesis import given, settings
from hypothesis import strategies as st
def normalize_whitespace(text: str) -> str:
return " ".join(text.split())
@given(st.text())
@settings(max_examples=200, deadline=None)
def test_normalize_whitespace_is_idempotent(text):
result = normalize_whitespace(text)
assert "\t" not in result and "\n" not in result
assert normalize_whitespace(result) == resultIn CI, pytest --hypothesis-show-statistics prints how many examples each test ran and how long shrinking took — the two dials you will actually adjust when a suite gets slow.
When Not to Reach for Hypothesis
Property testing is a tool, not a religion, and some work is a poor fit for it.
- Known edge cases from the spec deserve plain deterministic tests. If the requirements say an empty coupon code raises
ValueError, a two-line test states that more clearly than any generated run, and its failure message needs no shrinking to interpret. - Slow, IO-heavy suites are the hard no. Generation multiplies whatever each example costs, so a property test hitting a real database or HTTP endpoint a hundred times per run will wreck your build. Keep generated tests pointed at pure, fast code — parsers, serializers, calculations, state machines — and handle boundaries with contracts or mocks.
- Outputs with no crisp law (visual layout, prose, anything judged by a human) have nothing for a property to assert.
Wrapping Up
The pattern in every example here is the same: stop enumerating inputs, start stating laws, and let the generator do the enumerating. Round-trips, invariants, idempotence, and oracle comparisons will cover an outsized share of your logic; st.composite models the domain-specific rest; and RuleBasedStateMachine extends the whole idea to anything with state. The project is developed in the open on GitHub, installation is one pip install hypothesis away, and the best first step is rewriting one round-trip test you already trust — then watching what falls out.