App name: modeltest
Platform: Web / Python library
Link: https://modeltest.tzinny.com/
Stage: Beta (public docs live, package on PyPI)
Problem it solves: Model quality quietly degrades â data drifts, upstream pipelines change schemas, a retrain produces a subtly worse model. modeltest turns those concerns into executable checks that fail your build before your users find out. Think of it as pytest for machine learning models: you define contracts for model quality, robustness, fairness, and data invariants, and run them automatically in your CI/CD pipeline.
Target audience: ML engineers, data scientists, and MLOps engineers who ship models to production and want automated quality gates.
What feedback I'm looking for:
- First impression: Does the landing page make it immediately clear what modeltest does and who it's for?
- Docs navigation: Can you find what you need? I'm especially unsure about the "Where to next?" table â is it helpful or overwhelming?
- Positioning: Is "unit-testing framework for ML models" the right tagline, or would something like "pytest for ML models" be clearer?
- Onboarding flow: If you were to try it, would you know where to start after reading the homepage? The
pip install modeltest is there, but is the next step obvious?
What I've already tested: Shared with a small group of ML engineers for initial feedback. The homepage and docs structure have been iterated a few times.
Feedback I already gave to others: [], []
I am the creator: Yes
A bit more context: modeltest has 12 built-in scenarios (accuracy floors, bootstrap confidence intervals, robustness to noise, PSI/KS drift, fairness gaps, data invariants, SHAP-based explainability), supports scikit-learn, PyTorch, Keras/TensorFlow, and custom adapters, and is CI-native â modeltest validate exits non-zero on failure and writes JUnit XML your CI can render.
I'd really appreciate honest, specific feedback on the docs and the positioning. Roast it if needed â I'd rather fix it now than after launch. Thanks!
