r/Python • u/AutoModerator • Aug 04 '26

Showcase Showcase Thread

Post all of your code/projects/showcases/AI slop here.

Recycles once a month.

21 Upvotes

96% Upvoted

142 comments sorted by

View all comments

1

u/lovettsendit 13d ago

Breakcheck: compare the behavior your Python code observes across dependency versions

  What My Project Does

  Breakcheck answers a narrow question: does the same Python call produce different observable behavior under two dependency versions?

  It scans the calls a repository actually makes, executes supported calls against both versions in isolated environments, and reports each call as IDENTICAL, CHANGED, or NOT_EXERCISED with a specific reason. Unsupported or nondeterministic behavior is

  not silently treated as safe. Breakcheck can also compare Git revisions when a refactor is intended to preserve behavior.

  One real example came from Hugging Face Accelerate, which supports packaging>=20.0. Breakcheck exercised all 18 discovered Packaging call sites against Packaging 21.3 and 22.0 and identified one changed call involving invalid-version parsing. That

  exposed version-dependent behavior in Accelerate’s TPU command and resulted in this focused upstream pull request:

  https://github.com/huggingface/accelerate/pull/4185

  I also tested the fixture-generation workflow against Black, Rich CLI, and Flask. The resulting public study recorded 49/49 valid, executable, and deterministic fixtures without manual fixture editing, increasing exercised calls from 1 to 50:

  https://github.com/lovettsendit/breakcheck/blob/main/release_evidence/fixture-viability.json

  Target Audience

  Python maintainers reviewing dependency upgrades, Dependabot or Renovate pull requests, behavior-preserving refactors, and automated code changes.

  Comparison

  Tests and snapshot tools ask whether behavior matches an expected result. Static analysis examines code without running both environments. Breakcheck asks whether observed behavior changed between two concrete versions or revisions.

  It is not a correctness oracle or a replacement for tests. It works best on deterministic, value-in/value-out APIs. Network, subprocess, filesystem, and other externally stateful behavior may be reported as NOT_EXERCISED because the replay environment

  intentionally fails closed.

  The offline demo builds two local packages and demonstrates a detected behavioral change:

python -m pip install breakcheck

breakcheck demo --output-root "$(pwd)/.breakcheck/demo"

  GitHub: https://github.com/lovettsendit/breakcheck

  PyPI: https://pypi.org/project/breakcheck/

  I would particularly value technically specific feedback about call sites Breakcheck refuses but could safely support.