Pilot proof sheet · concept
Client A: 60-day pilot on a travel and lodging marketplace queue
Pilot 2026-07-03 to 2026-08-31; baseline 2026-06-03 to 2026-07-02. Precision and recall are always shown together. Held values are withheld.
Fictional demo. The client, events, reviewers and values are invented to illustrate the format. Not a GEN result.
| Tier | Metric | With GEN | Baseline | Change | Status |
|---|---|---|---|---|---|
| 1 | Alert precision and recall120 adjudicated items: 58 high-priority alerts, 59 actual positives (TP 50, FP 8, FN 9, TN 53) | 86.2% / 84.7% | 57.1% / 60.0% | +29.1 pts / +24.7 pts | Approved |
| 1 | Analyst minutes per decision60 timed decisions with GEN; 30 without | 16.5 min | 24 min | −7.5 min (−31%) | Approved |
| 1 | Report-ready rate60 audited decisions in the pilot | 88.3% | 63.3% | +25.0 pts | Approved |
| 1 | Time to NCMEC report18 submitted reports with GEN; 10 without | — | — | — | HeldSubmission timestamps come from the client export and are not yet reconciled with CyberTipline receipts |
| 1 | Proactive-rate uplift42 confirmed items with GEN; 20 without | 69.0% | 30.0% | +39.0 pts | Approved |
| 2 | Partner-confirmed next action11 eligible referrals | 72.7% | — | — | Approved |
| 2 | Analyst exposure30 confirmed cases with GEN; 12 without | 8.4 per case | 18.9 per case | −10.5 per case (−56%) | Approved |
Definitions
- Alert precision and recall
- Precision = adjudicated true positives ÷ high-priority alerts. Recall = adjudicated true positives ÷ all actual positives in the same adjudicated sample. Always shown together. Holds for this adjudicated sample only; not prevalence or field performance elsewhere.
- Analyst minutes per decision
- Median active analyst minutes from intake to documented disposition, with GEN (pilot) vs without GEN (baseline). Time-tracked sample of decisions; not a headcount saving until the client confirms its staffing model.
- Report-ready rate
- Decisions passing the audit checklist (provenance, rationale, reviewer, timestamp) ÷ sampled decisions. Checklist completeness, not a legal opinion on any filing.
- Time to NCMEC report
- Median hours from first detection to a submitted NCMEC CyberTipline report. Measures the platform’s own submission step; says nothing about what happens after NCMEC receives a report.
- Proactive-rate uplift
- Share of confirmed trafficking items found by GEN before any user report, pilot vs baseline (after Meta’s “proactive rate”). Detection share within this platform’s confirmed items; not a count of harm prevented.
- Partner-confirmed next action
- Referrals with a documented, partner-acknowledged next step ÷ eligible referrals. Survivors and investigations are “supported”, never “rescued”: a next step is not an outcome.
- Analyst exposure
- Items a human analyst had to view ÷ confirmed cases, pilot vs baseline. A workload proxy: fewer items viewed, not a mental-health result.
Run this on your own queue
A scoped 6-week pilot: one queue, a baseline taken at onboarding, an adjudicated sample, and a proof sheet like this one approved by your reviewer before anything is shared. Human outcomes are not claimed.