Reviewer Methodology & Goals
How external reviewers test Quintace strategy outputs — short version.
📋 Master tracker (canonical reviewer input)
"QuintAce — QA findings" Google Doc →
Two tabs:
- Strategy Issues — cross-cutting findings that aren't tied to one article (UTGvBB systematic divergence, hygiene checks, RV v2 vs GTOW). Logged here first; if a single article surfaced the issue, it's still cross-cutting so it lives in this tab.
- Articles — per-article review status (in-review, KILL, approved, awaiting re-look). Detailed comments still go in each article's own review doc (linked below).
The SolverQA hub page (this one) is downstream of the master tracker. Reviewer notes land in the doc first; triage verdicts surface here as R-findings.
The motion
Reviewers are assigned Quintace articles to read and check for correctness. Anything that looks off becomes a comment in the per-article review doc, AND a row in the master tracker's "Articles" tab (status line) or "Strategy Issues" tab (if cross-cutting). Direct browsing of Range Viewer V2 is optional and secondary; the article is the primary test surface.
Reviewer's job ends at "documented finding." Quintace team triages.
Two review goals
Dev strategy grid = prod strategy grid = Range Viewer V2 = published article numbers. When they disagree, the model isn't wrong — the pipe is.
Once Goal A is locked, RVV2 is the ground truth. Reviewers compare RVV2 outputs against PioSolver, GTO Wizard, MonkerSolver, GTO+ on matched spots.
Articles currently in review
Status source: Articles tab of the QuintAce QA findings master tracker. Detailed per-article comments live in the per-article review docs (linked in the table below).
| # | Article | Reviewer | Review doc | Phase | Detailed status |
|---|---|---|---|---|---|
| 1 | seidman-easy-game-reexamined | Timothy Ulmer | link | WIP — AI rewrite | v2.0 fixed, awaiting re-look |
| 2 | jrb-dumbest-hands | Timothy Ulmer | link | WIP — AI rewrite | Voice + structural redesign |
| 3 | ivey-3-defining-hands | Timothy Ulmer | link | WIP — AI rewrite | Voice + asset pass |
| 4 | adjusting-to-donk-bets-in-srp | Timothy Ulmer | link | WIP — AI rewrite | KILL pending |
| 5 | uri-squid-invisible-ante | Timothy Ulmer | link | WIP — AI rewrite | Voice approved · needs Squid-expert reviewer |
| 6 | stuck-series / jrb-foldmarketoonist partner cut | GTOLab | article | Reviewer assigned | Awaiting GTOLab review |
| 7 | every-format-every-game / mariano-stand-up-acesmarketoonist partner cut | GTOLab | article | Reviewer assigned | Awaiting GTOLab review |
| 8 | every-format-every-game / hcl-five-way-stand-upmarketoonist partner cut | GTOLab | article | Reviewer assigned | Awaiting GTOLab review |
Articles #1–5 are WIP — pending AI rewrite & release before Tim Ulmer can continue. Articles #6–8 are Reviewer assigned (GTOLab) — awaiting initial review pass. Phase resets to "ready for review" once a rewrite ships, or advances to "review complete" once a reviewer delivers findings.
Reviewer workflow
- Read the assigned article (from the Articles tab of the master tracker).
- Flag specific cited numbers, claims, or hand examples that look off — anchor each comment to the paragraph or table.
- For anything you're not sure about, cross-check the spot against an external solver (Pio / GTOW / Monker) and record both numbers + the solver's config (rake, blinds, preflop range).
- Drop detailed comments in the per-article review doc (linked in the Articles tab).
- If the finding is cross-cutting (not specific to this article — e.g. a systematic UTGvBB divergence or a hygiene-level concern), add a row in the Strategy Issues tab of the master tracker instead.
- Update the article's status line on the master tracker's Articles tab (WIP / awaiting re-look / KILL / approved). Done.
Triage verdicts (assigned by Quintace, not reviewer)
| Verdict | Meaning |
|---|---|
| SERVING | Surface bug — article on stale model, dev≠prod, aggregation off. Model is fine; pipe is broken. (Goal A failure) |
| REFERENCE-CONFIG | Architectural choice (e.g., universal model has no preflop tree abstraction). Not a bug, documented design decision. |
| MODEL | Real strategy disagreement worth investigating. |
| B1 KNOWN-FAILURE | Aligns with an internal-consistency invariant — captured for property-test signal on retrain. |
| FALSE-POSITIVE | External solver was configured differently than Quintace, or reviewer compared against a stale article instead of RVV2. |
Lessons from past reviews
- Stale articles can fool reviewers. The Seidman K72r article said 59% c-bet; current model says 79%; GTOW says 88%. The reviewer correctly flagged it — but the headline gap (29pp) was a tooling artifact. Pre-publish gate: spot-check 1-2 cited frequencies on the current model before shipping any article.
- "Sounds AI" is real. When LLM does both the analytical wrapping AND the personal voice, articles draw this flag. Keep LLM in analytical voice (boxed annotations); let the coach own first-person prose.
- Premise validation matters. The donk-bets article studied BB-leading SRPs that almost never happen at equilibrium (0% reach). Methodology gate: verify the scenario has > 5% reach before drafting.
- Reviewers can be wrong too. Tim Ulmer's "Quintace too sticky" headline partly reversed sign when proper aggregation was applied. Not a problem — that's what triage is for.
Tools & links
- Master tracker (canonical): QuintAce — QA findings Google Doc (Strategy Issues + Articles tabs)
- Range Viewer V2 (RVV2): ground truth surface — v2.preview.aceguardianrl.com/rangeviewer/v2
- Findings folder:
engineering-department/gameplay-ai/projects/external-solver-benchmark/findings/ - Reviewer findings on the hub: /solver-qa/reviewers/
- Full Solver QA catalog: /solver-qa/