Graphorin API reference v0.7.0
Graphorin API reference / @graphorin/evals / / RegressionOptions
Interface: RegressionOptions
Defined in: packages/evals/src/types.ts:84
Regression-detection options.
Stable
Properties
| Property | Modifier | Type | Description | Defined in |
|---|---|---|---|---|
maxAvgDurationIncreaseMs? | readonly | number | Maximum allowed increase in avgDurationMs before it counts as a regression. Opt-in: defaults to Infinity (gate off) because absolute wall-clock budgets are environment-sensitive (workstation baseline vs CI runner, real LLM-latency jitter). Pass a finite ms budget to enable an absolute duration gate; leave unset to ignore duration entirely. | packages/evals/src/types.ts:96 |
maxAvgScoreDrop? | readonly | number | Minimum drop in average score per scorer that counts as a regression. | packages/evals/src/types.ts:88 |
maxPassRateDropPct? | readonly | number | Minimum drop in pass-rate (in percentage points) that counts as a regression. | packages/evals/src/types.ts:86 |
requireSignificance? | readonly | boolean | E8 (evals-05/08): when true, a pass-rate-drop finding is only kept if McNemar's paired test over the shared cases rejects "no real change" at significanceAlpha - a fixed percentage tolerance is blind to sample size (a 5pp drop is one case in a 20-case suite). Off by default so existing gates keep their exact behavior; the computed pValue is attached to the finding either way whenever both reports carry per-case results. | packages/evals/src/types.ts:106 |
significanceAlpha? | readonly | number | Significance level for requireSignificance. Default 0.05. | packages/evals/src/types.ts:108 |