pytest_test_categories.timing¶

Time limit definitions and validation for test categories.

Test sizes are DEFINITIONS, not configurable options. This follows Google’s “Software Engineering at Google” philosophy where test sizes have fixed meanings:

Small tests (< 1s): Fast unit tests without external dependencies
Medium tests (< 5 min): Integration tests with local services
Large tests (< 15 min): Full system/E2E tests
XLarge tests (< 15 min): Extended tests with same limits as large

If a test exceeds its category’s time limit, the correct action is to RECATEGORIZE the test to a larger size, not extend the limit.

Attributes¶

`LARGE_LIMIT`
`MEDIUM_LIMIT`
`SMALL_LIMIT`
`TIME_LIMITS`
`XLARGE_LIMIT`

Exceptions¶

`PerformanceBaselineViolationError`	Exception raised when a test exceeds its custom performance baseline.
`TimingViolationError`	Exception raised when a test exceeds its time limit.

Classes¶

TimeLimit

Fixed time limit for a test size category.

Functions¶

`get_limit`(size)	Get the fixed time limit for a test size.
`validate`(size, duration[, test_nodeid])	Validate a test's duration against its size's fixed time limit.
`validate_with_baseline`(size, duration, baseline[, ...])	Validate a test's duration against a custom baseline or category limit.

Module Contents¶

exception pytest_test_categories.timing.PerformanceBaselineViolationError(test_size, test_nodeid, baseline_limit, category_limit, actual)[source]¶

Bases: Exception

Exception raised when a test exceeds its custom performance baseline.

This exception is raised when a test’s execution time exceeds a custom performance baseline that is stricter than the category’s default limit. Custom baselines allow performance-critical tests to fail early when they regress, even if they’re still within the category limit.

The error message includes: - Error code [TC007] - Test identification (nodeid, size category) - Baseline vs actual duration - Category limit for context - Why performance baselines matter - Remediation suggestions - Documentation link

test_size[source]¶: The test’s size category.

test_nodeid[source]¶: The pytest node ID of the failing test.

baseline_limit[source]¶: The custom performance baseline in seconds.

category_limit[source]¶: The category’s default time limit in seconds.

actual[source]¶: The actual test duration in seconds.

Example

>>> raise PerformanceBaselineViolationError(
...     test_size=TestSize.SMALL,
...     test_nodeid='tests/test_critical.py::test_fast_path',
...     baseline_limit=0.1,
...     category_limit=1.0,
...     actual=0.25
... )

Initialize a performance baseline violation error.

Parameters:

test_size (pytest_test_categories.types.TestSize) – The test’s size category.
test_nodeid (str) – The pytest node ID of the failing test.
baseline_limit (float) – The custom performance baseline in seconds.
category_limit (float) – The category’s default time limit in seconds.
actual (float) – The actual test duration in seconds.

actual[source]¶

baseline_limit[source]¶

category_limit[source]¶

test_nodeid[source]¶

test_size[source]¶

exception pytest_test_categories.timing.TimingViolationError(test_size, test_nodeid, limit, actual)[source]¶

Bases: Exception

Exception raised when a test exceeds its time limit.

This exception is raised when a test’s execution time exceeds the configured time limit for its size category.

The error message includes: - Error code [TC006] - Test identification (nodeid, size category) - Timing details (limit vs actual duration) - Why timing limits matter - Remediation suggestions - Documentation link

test_size[source]¶: The test’s size category.

test_nodeid[source]¶: The pytest node ID of the failing test.

limit[source]¶: The time limit in seconds.

actual[source]¶: The actual test duration in seconds.

Example

>>> raise TimingViolationError(
...     test_size=TestSize.SMALL,
...     test_nodeid='tests/test_slow.py::test_compute',
...     limit=1.0,
...     actual=2.5
... )

Initialize a timing violation error.

Parameters:

test_size (pytest_test_categories.types.TestSize) – The test’s size category.
test_nodeid (str) – The pytest node ID of the failing test.
limit (float) – The time limit in seconds.
actual (float) – The actual test duration in seconds.

actual[source]¶

limit[source]¶

test_nodeid[source]¶

test_size[source]¶

class pytest_test_categories.timing.TimeLimit(/, **data)[source]¶

Bases: pydantic.BaseModel

Fixed time limit for a test size category.

This is an immutable value object representing a test size’s time limit. Time limits are fixed definitions based on Google’s test size standards, not configurable options.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:: data (Any)

limit: Annotated[float, Field(gt=0)][source]¶

model_config[source]¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pytest_test_categories.timing.get_limit(size)[source]¶

Get the fixed time limit for a test size.

Parameters:: size (pytest_test_categories.types.TestSize) – The test size category.
Returns:: The TimeLimit for the given test size.
Return type:: TimeLimit

Example

>>> get_limit(TestSize.SMALL).limit
1.0
>>> get_limit(TestSize.MEDIUM).limit
300.0

pytest_test_categories.timing.validate(size, duration, test_nodeid='')[source]¶

Validate a test’s duration against its size’s fixed time limit.

Test sizes have fixed time limits that are not configurable. If a test exceeds its limit, the correct action is to recategorize the test to a larger size, not extend the limit.

Parameters:

size (pytest_test_categories.types.TestSize) – The test size category.
duration (float) – The actual test duration in seconds.
test_nodeid (str) – Optional pytest node ID for enhanced error messages.

Raises:

TimingViolationError – If the test exceeds its time limit.

Return type:

None

Example

>>> validate(TestSize.SMALL, 0.5)  # Passes (0.5s < 1s limit)
>>> validate(TestSize.SMALL, 2.0)  # Raises TimingViolationError
>>> validate(TestSize.SMALL, 2.0, test_nodeid='tests/test_slow.py::test_compute')

pytest_test_categories.timing.validate_with_baseline(size, duration, baseline, test_nodeid='')[source]¶

Validate a test’s duration against a custom baseline or category limit.

When a custom baseline is provided, the test must complete within that stricter limit. If the baseline is exceeded, a PerformanceBaselineViolationError is raised. If no baseline is provided, falls back to standard category limit validation.

The baseline must be less than or equal to the category’s time limit. This ensures that custom baselines are always stricter than the default.

Parameters:

size (pytest_test_categories.types.TestSize) – The test size category.
duration (float) – The actual test duration in seconds.
baseline (float | None) – Optional custom performance baseline in seconds. If None, uses the category’s default time limit.
test_nodeid (str) – Optional pytest node ID for enhanced error messages.

Raises:

PerformanceBaselineViolationError – If duration exceeds the custom baseline.
TimingViolationError – If duration exceeds category limit (when no baseline).
ValueError – If baseline exceeds the category’s time limit.

Return type:

None

Example

>>> # Test with custom baseline
>>> validate_with_baseline(TestSize.SMALL, 0.05, baseline=0.1)  # OK
>>> validate_with_baseline(TestSize.SMALL, 0.15, baseline=0.1)  # Raises PerformanceBaselineViolationError

>>> # Test without baseline (uses category limit)
>>> validate_with_baseline(TestSize.SMALL, 0.5, baseline=None)  # OK
>>> validate_with_baseline(TestSize.SMALL, 1.5, baseline=None)  # Raises TimingViolationError

pytest_test_categories.timing.LARGE_LIMIT[source]¶

pytest_test_categories.timing.MEDIUM_LIMIT[source]¶

pytest_test_categories.timing.SMALL_LIMIT[source]¶

pytest_test_categories.timing.TIME_LIMITS[source]¶

pytest_test_categories.timing.XLARGE_LIMIT[source]¶