noisefloor
Is this number real, or is it noise?
Community: Submitted by a user or imported; check the owner before granting accessOnlineNo sign-inGlobalFreeRead-only
What it can do
- Ab Test: Can you call a winner on an A/B test yet? Uses anytime-valid confidence sequences, so it is SAFE TO RUN AFTER EVERY OBSERVATION — peeking does not inflate the false-positive rate the way a t-
- Did It Change: Did a metric actually change, or is the move noise? Detects both rises AND collapses against the metric's own history, with a stated false-alarm rate and no assumption about the distrib
- Forecast Next: What should the next reading be, and within what range? Range adapts to the metric's recent volatility and stays valid even when the metric shifts.
What data it sees
Do you need an account
No: the server works without sign-in
Is this number real, or is it noise? Peek-safe A/B tests you can look at as often as you like, two-sided change detection, honest forecasts with a scored record, and a guard for metrics that only moved because their sample size did. Zero dependencies.
Server tool list (6)
Raw names from tools/list. Only developers need these.
| ab_test | Can you call a winner on an A/B test yet? Uses anytime-valid confidence sequences, so it is SAFE TO RUN AFTER EVERY OBSERVATION — peeking does not inflate the false-positive rate the way a t-test or z-test does. |
| did_it_change | Did a metric actually change, or is the move noise? Detects both rises AND collapses against the metric's own history, with a stated false-alarm rate and no assumption about the distribution. |
| forecast_next | What should the next reading be, and within what range? Range adapts to the metric's recent volatility and stays valid even when the metric shifts. |
| real_or_sampling | Did the metric move, or did the sample size underneath it move? Run this before reporting any RATE as a change — conversion rates, error rates and click-through all shift when the denominator shifts, for reasons that have nothing to do with the thing being measured. |
| score_forecasts | How good would these forecasts actually have been? Grades every prediction the tool would have made over the history, using only what was known at the time, and reports calibration plus the worst misses. |
| which_metrics_matter | You watch many metrics; which genuinely stand out? Controls the false discovery rate across all of them at once, which per-metric thresholds do not: forty metrics each alerting wrongly 5% of the time means two false alarms every round. |