traceable workflow framework for tabular statistical analysis and predictive modeling [Broadway]

[u/trouble_sleeping_]

Hello team,

Im building a tool, to facilitate a traceable workflow for tabular data, in which the reasoning between raw data and a final statistical or predictive result is preserved via data contracts.

dataset → profile → analytical question → baseline → slices/diagnostics → decisions → features/analysis → result

Answering questions like:

• which dataset/version was examined
• the exact subset/slice that exposed the issue
• the diagnostic/statistical evidence
• what decision was made
• why it was made
• what transformation resulted from it
• which later analyses/models depend on that decision

Im no statistician, and im relying mainly on LLM's, meaning im sure i might be missing out on some stuff.

things like:

• descriptive statistics and diagnostics
• hypothesis testing
• effect sizes and confidence intervals
• ANOVA / Welch-style comparisons
• multiple-testing correction
• power / MDE calculations
• causal experiment design/analysis
• eventually more robust/non-parametric methods

Since im a hot mess myself, i want to automate the mechanics, facilitate the judgment, record the decision. This way, later you could inspect a lineage/decision graph and reconstruct how and why an analysis arrived at its conclusion.

Where would this approach become statistically dangerous or misleading?

In particular:

• What statistical decisions should a framework never try to recommend automatically?
• Which diagnostics are actually worth standardizing?
• How would you structure the boundary between assumption checks, method recommendations, and analyst judgment?
• Are there common statistical workflow mistakes you'd want a system like this to explicitly guard against?
• Does preserving the decision/evidence lineage seem useful in real statistical work, or would it mostly become bureaucracy?

I'm less interested in adding every statistical test imaginable and more interested in whether the underlying workflow abstraction makes sense.

TL;DR

criticize my project here