Why this matters
A criterion is a checkable rule plus a static source. If the source is wrong or the rule is vague, every review that uses it is weaker. Feedback should target the rule and citation, not “make the AI nicer.”
Where to look
Public browser
Browse packs and sources anytime: /criteria. Same files the product loads.
Structured QC form
When we open a draft pack for review, use the criteria review form: /review. Comment by criterion #; submit returns notes to the team.
Contact
For ongoing expert relationships: /contact — name the pack, criterion id/ref, and proposed source wording when you can.
What useful feedback looks like
- Pack + ref — e.g. quality-event / 01 Root-cause adequacy
- Issue type — wrong citation, too broad, double-counts another rule, missing common failure mode
- Proposed fix — better source text or narrower description
- What not to ask — “have the model invent the CFR number each run” (we will not)
Helpful
- “DI3 source should point at EU GMP Ch. 4 §4.8; current anchor is fine but add …”
- “This SOP rule double-counts version control already in ref 02”
- “Allowed verdicts for signature presence must never include PASS”
Less helpful
- “Make it stricter” with no criterion named
- “Add a readiness score”
- Unsourced novel regulations for the model to free-write
Git is the audit trail. When a criterion changes, the change is a commit. Past reviews stay reproducible against the pack version used at the time.
Start with the live packs.