RFC Review: Unified Agent Quality Scorecard — Phase 2: AI Auto-Scoring & In-Room Scorecard
Companion review for
rfc-phase-2-auto-scoring-and-in-room-scorecard.md, produced by therfc-reviewerskill. Lives beside the RFC; valid only for the RFC revision inreviewed_rfc_last_updated.
Executive Summary
- Overall Score:
8.5/10(R2, after applying R1 fixes; R1 scored8.0) - Rating:
Agentic-Ready - RFC Type:
full-stack - Sub-Type:
new-feature(BE scoring pipeline is net-new; FE is a mode extension of the existing panel) - Assessment Confidence:
High(both layers grounded with file:line evidence; three verified-absents checked) - Applied Caps/Gates:
none— no category below 5.0; no cross-layer contract mismatch found; deploy order specified. 9.0 withheld: CNT at 7.5 (< 8.5 bar) and the inbound contract is PROPOSED pending DSAI. - Implementation Readiness Verdict: PROCEED with notes — build-executable behind the flag; org enablement gates on the DSAI contract (REV-8), GA gates on Infosec/G7.
- Report Path:
chatbot/unified-agent-scorecard/rfcs/rfc-phase-2-auto-scoring-and-in-room-scorecard-review.md - RFC Author: Dimas Fauzi Hidayat (DRI) / rfc-starter | Reviewed: 2026-07-17
An agent can implement both layers from this RFC without a clarification meeting: the schema
is DDL-level, both extended endpoints carry exact JSON, the 16-chunk plan has per-chunk
commands and assertable acceptance, and every "existing code" claim is file:line-verified.
The biggest strength is the grounding discipline — the RFC corrects five PRD premises against
the real repos (segment source, existing is_use_ai, actor key, RBAC scope, the resolve-guard
regression) instead of inheriting them. The biggest structural risk is external: the inbound
evaluation contract is explicitly PROPOSED and DSAI has not confirmed it; the RFC handles this
honestly (build vs enablement split), but the risk stays until signed. The R1 pass found one
genuine spec hole — idempotency and weights-versioning referenced fields that were missing
from the DDL (evaluated_at, weights_version) — fixed in this revision along with three
smaller closures (tier-2 transcript precondition, per-metric override marker, tenancy-check
mechanics, empty-human-record GET behavior).
Quick Verdict
Why this RFC can be implemented agentically:
- Every chunk in §4.C names files + copy-pasteable commands + acceptance assertable by a test or query; ordering respects dependencies (migrations → pipeline → API → FE).
- Contracts are pinned at JSON level on both sides, with a shared-fixture cross-boundary test (§4.B) that turns contract drift into CI failure.
- Source Verification (§2.0) backs every reuse claim with file:line, including the three verified-absents that drive the design (no ingestion path, no BE handover events, no bot participant type).
Why this RFC will cause agent guessing or rework:
- The inbound payload is PROPOSED (§2.4 Inbound row 1): if DSAI's real contract diverges (field names, push trigger, PII handling), chunk 2's contract and the ingest specs rework — bounded, but real (REV-8, open).
- FE component props/emits are specified at the interface level (§2.B types), not per-component
definePropscontracts (REV-6, open — same known limitation Phase 1 carries).
Findings Ledger (carry-forward)
First cycle for this RFC — all ids minted in R1. R2 = fix pass applied to the same working-tree revision (RFC
last_updatedunchanged at 2026-07-17; changes recorded in the RFC Comment log).
| ID | Severity | Finding (one line) | RFC location | Status | First seen | Resolved in | Evidence / fix |
|---|---|---|---|---|---|---|---|
REV-1 | major | Idempotency guard ("re-score only if newer evaluated_at") and weights auditing (weights_version) referenced values not present in any table — snapshot TTL (90d) would destroy weights_version needed for 13-month agreement analysis | §2.3 DDL / §2.4 Inbound r1 / ADR-6 | fixed | R1 | R2 | Added evaluated_at (datetime) + weights_version (string) to the agent_scorecards migration, erDiagram, §1.A.5 derivation, and the GET response; ADR-6 now stores the version on the record, not the snapshot. |
REV-2 | major | Tier-2 judge input undefined when the evaluation payload omits transcript but the org has rubric'd custom params — agent would guess whether to skip, fail, or fetch | §2.4 Inbound r1 / ADR-5 / chunk 4 | fixed | R1 | R2 | §2.4 + ADR-5 now state: transcript absent ⇒ tier-2 params unscored, record partial with reason no_transcript (tier-1 unaffected); Branch & Skip Catalog row added. |
REV-3 | minor | No per-metric marker for which metric was overridden — override_reason is optional, so a reason-less override left no detail-level trace for the "edited by [SPV]" chip | §2.3 details / §2.4 r2 | fixed | R1 | R2 | scoring_state enum extended with overridden; PATCH sets it per touched metric; FE chip keys off scoring_state === 'overridden' (+ correction_by_name for the name). |
REV-4 | minor | Tenancy check mechanics ambiguous: §2.4 said "unknown org/room → 422" at API time, but the room-existence pattern cited elsewhere (hub call with a user token) is unavailable on the internal surface | §2.4 Inbound r1 / §3 Security | fixed | R1 | R2 | Clarified: ingest resolves the room from chatbot's own DB (process_async_reply context pattern, its L25-35) synchronously pre-enqueue; org mismatch → 422; no hub call on the internal surface. |
REV-5 | minor | No-param GET behavior unspecified when the room has only an AI record (no human score) — repo currently falls back to a default template; RFC didn't say which branch wins | §2.4 r1 / chunk 8 | fixed | R1 | R2 | §2.4 r1 now states: no human record ⇒ today's default template response + actors[] (so the panel shows the manual empty state and the selector reveals the AI actor). Chunk 8 acceptance extended. |
REV-6 | minor | FE component props/emits not typed per component (interfaces only, §2.B) — agent infers defineProps shapes from the parent | §2.A / §2.B | open | R1 | — | Same class of gap Phase 1 carries (its §5 #7). Low risk: single owning feature folder, typed data interfaces exist. Promoted to RFC §5 #10. |
REV-7 | minor | AI actor display_name resolution named as a note (§5 #9) but the resolving source for human names (room detail) vs AI names (chatbot AI-agent config) is not pinned to a repository call | §2.4 r1 / §5 #9 | accepted-risk | R1 | — | Entity-builder detail; §5 #9 already assigns it to BOT BE. Does not block chunks (GET spec fixes the field, not its join). |
REV-8 | blocker (enablement, not build) | Inbound evaluation contract is PROPOSED — unconfirmed by DSAI (push trigger, field names, PII hashing, segment guarantee) | §2.4 Inbound r1 / §5 #1 | open | R1 | — | Already the RFC's own top gate (§7 unchecked box 1). Not a build blocker: chunks 1–16 execute behind the flag; enablement blocked until DSAI signs. Carried as-is. |
Ledger summary: 3 open (1 enablement-blocker, 0 major, 2 minor/accepted), 5 fixed this cycle. Still-open material findings are promoted into the RFC §5 table by id (REV-6 → §5 #10; REV-7 → §5 #9; REV-8 → §5 #1).
PRD → RFC Traceability Matrix
| PRD Element | RFC Section | Coverage |
|---|---|---|
| §2 Phase context (P1 prereq, DSAI dependency) | §1 Assumptions A1/A4, Dependencies | Full |
| §3 Problem (evaluator output discarded; AI actor unscored) | §1 Overview | Full |
| §7 Constraints (60s score, 2s panel, retention table, plan scope, flag, net-new list) | §3 Performance, §2.3 + ADR-9, ADR-11, §1.A.5 | Full |
| §7.1 Data lifecycle (13mo / 90d / 7d TTLs) | §2.3 snapshots + ADR-9 + §4.A | Full |
| §8 CHG-003 (panel AI mode, actor selector, badges, override) | §2.A UI Contract + §2.C state matrix | Full |
| §9 Behavior 1 (ingest + score per actor/segment) | §2.4 Inbound r1 + ADR-1..6 | Full — HTTP method/path/schema resolved as the PRD delegated ("Claude resolves during RFC") |
| §9 Behavior 2 (override) | §2.4 r2 + ADR-7 | Full |
| §10.2 P2-S01 AC-1..6, ERR-1..2, NEG-1..2 | §1.A.4 map; chunks 2–7, 10 | Full |
| §10.2 P2-S02 AC-1..5, ERR-1, NEG-1 | §1.A.4; chunks 8, 12–14 | Full |
| §10.2 P2-S03 AC-1..4, ERR-1, NEG-1 | §1.A.4; chunks 9, 13 | Full |
| §11 Rollout + §11.1 weights rollback | §4 Rollout + §4.A (ai_qa_unified_scorecard_v2_weights) | Full |
| §12 Observability events + alerts | §3 Monitoring table (6 events, 2 alerts) | Full |
| §13 Success metrics | §1 Success Criteria + §4.D signals | Full |
| §14 Stage gates | §4 Rollout + §4.D (segment_source alpha gate) | Full — scheduling correctly deferred to delivery/ |
| §15 Dependencies (DSAI, P1, schema, G7, Design, weights) | §1 Dependencies | Full |
| §16 Decisions (7 made, 4 rejected) | §1.B + ADRs 1–11 | Full — every PRD decision has an ADR home |
| §17 Open Questions 1–7 | §5 rows 1–6 (Q7 resolved by ADR-10; Q3 recast with evidence; Q6 resolved by ADR-2) | Full |
| App. A rubric (9 metrics, veto) | metric codes pinned to Phase 1 constant (§1.A.5) | Full |
| App. B Stitch prompt | §2.A interim design spec | Full |
| Reverse check | Every ADR traces to a PRD decision or a grounding correction named in the Comment log; no scope creep found. weights_version/evaluated_at (R2 addition) trace to PRD §11.1 rollback + idempotency needs. | — |
Summary: 19 of 19 PRD elements fully covered, 0 partial, 0 missing. 0 RFC decisions without a PRD driver. Full-stack contradiction check: none found — FE consumes exactly the extended GET/PATCH the BE defines (shared fixture enforces it).
Scorecard (Full-Stack, 18 categories)
| # | Category | Source | Score | Evidence-Based Rationale |
|---|---|---|---|---|
| 1 | PRT — PRD Traceability | Merged | 9.0 | Forward: §1.A.1 walks all 17 PRD sections + appendices; §1.A.4 maps every composite AC id; §1.C per-story artifacts. Reverse: every ADR cites its PRD driver. FE: CHG-003 elements → §2.A rows · BE: §9 behaviors → §2.4 rows. |
| 2 | TDC — Technical Decisions | Merged | 8.5 | 11 ADRs, each with context/options/decision/consequences/reversibility; two honest no alternative considered entries. FE: mode-switch + selector-source decisions (ADR-2/11) · BE: ingestion/schema/judge/retention (ADR-1/3/5/9). No cross-layer contradiction. |
| 3 | CNT — Contract Specificity | FE | 7.5 | §2.B typed interfaces (AiScorecardData, ActorSummary, AiMetric + ScoringStatus union) and §2.C state matrix are pinned; per-component defineProps/emits left to implementation (REV-6, open). |
| 4 | SCB — Scope Boundaries | FE | 8.5 | §2.D two-column table is explicit (incl. "no chatbot-fe changes", "no batch-GET changes"); §1 Out of Scope lists phase deferrals with owners. |
| 5 | DEP — Dependencies | FE | 8.0 | §1 Dependencies table with blocking semantics split (build vs enablement vs GA); A4 deploy-order prereq; FE dependency on BE fields explicitly fail-safe ("fields absent ⇒ manual mode"). |
| 6 | NFS — Non-Functional Specificity | FE | 7.5 | 2s P95 panel + 60s scoring budgets with the query-shape argument (§3); a11y specifics (§3.C: veto not color-alone, aria-expanded, focus); §3.D no-new-dependency budget. No LCP/INP numbers — acceptable for a panel inside an existing lazy feature. |
| 7 | TPS — Test Plan Specificity | FE | 8.0 | Commands sourced (package.json: vitest/eslint/vue-tsc; CI rspec line cited); per-chunk acceptance is assertable; §4.B shared-fixture contract test crosses the API boundary. |
| 8 | DMS — Data Model & Schema | BE | 8.5 | Three migrations at DDL level with defaults/indexes; erDiagram; truthful actor_type: 'human' default argued from data reality; retention split justifies the snapshot table (ADR-3). R1's missing evaluated_at/weights_version (REV-1) fixed. |
| 9 | ACV — API Contract & Versioning | BE | 8.0 | Exact JSON request/response + error catalogs for all three changed surfaces; reuse/extend/new tags with evidence; additive-compat argument for existing callers. Held below 8.5: the inbound row is PROPOSED (REV-8) — precise but unconfirmed. |
| 10 | DIC — Data Integrity & Consistency | BE | 8.5 | Single-tx scorecard+details+snapshot write; idempotent upsert on the existing unique key + evaluated_at recency guard (post-REV-1); one-slot override guard re-read under lock; TTL cleanup isolated from score rows. |
| 11 | FMC — Failure Mode Coverage | Merged | 8.5 | FE: §2.C covers loading/empty/error/partial/success + veto + expired-transcript per component · BE: §3.A 11-row catalog incl. replay, org-mismatch, mid-tx failure, concurrent overrides · Cross: FE states are keyed off the BE's actual scoring_status enum, and §3.B maps each error code to its surface. |
| 12 | CSS — Concurrency & Scaling | BE | 7.0 | Queue-depth degradation argument, 240s judge ceiling with duration_ms alert, batched deletes, per-org row bound (actors × rooms). No QPS/pool numbers — advisory-only per rubric (not capacity-scoped). |
| 13 | SAS — Security & Authorization | BE | 8.5 | Tenancy: token-only org on FE surface; chatbot-DB room-org verification on internal surface (post-REV-4); prompt-injection delimiting for rubric+transcript; Rollbar scrub list; role × endpoint matrix incl. the grounded agent-on-GET nuance; DoS caps. |
| 14 | ROL — Rollout & Rollback | Merged | 8.5 | FE: flag fallback-dark · BE: forward-only migrations + 6-step ordered deploy · Cross: compatibility table covers both directions, rollback is numbered and reverse-ordered, two flags with explicit coupling (v2_weights selects version per org, no deploy). |
| 15 | OBS — Observability | Merged | 8.0 | FE: $mixpanel panel events · BE: 6 events with props incl. duration_ms/segment_source/weights_version, 2 alerts with channels, ChatGptLogWorker related_key precedent. Cross-service trace stitching explicitly descoped with rationale (caps this at 8.0 per merge rule). |
| 16 | SBC — Service Boundary & Coupling | BE | 8.5 | ADR-1/ADR-4 pin the DSAI boundary to one push contract; per-service responsibility table; hub coupling avoided deliberately (segment from evaluator, not hub events). |
| 17 | CPA — Pattern Alignment | Merged | 8.5 | FE: Pixel3-only, feature-folder co-location, $customFetch composable pattern, snake_case end-to-end rule · BE: worker/queue/tx/logging patterns each cite an opened file · Cross: naming consistent (scorecard_* events, related_key precedent). |
| 18 | CDG — Compliance & Data Governance | BE | 7.0 | Active (transcript PII). TTLs are enforced-by-design (ADR-9), scrub list named, G7 explicitly gates GA, snapshots excluded from API responses. Held down: upstream PII hashing is an expectation on an unconfirmed contract (REV-8) — the control is specified but not yet guaranteed. |
Resource & Cost Advisory (non-blocking)
- Tier-2 judge adds one OpenAI call per scored conversation for rubric'd orgs — cost scales
with conversation volume × custom-param adoption;
tier2_counton the completed event supports a cost dashboard. Snapshot storage is bounded by the 90-day TTL. No advisory concern beyond monitoring.
Decision Closure Assessment
Decision Index
| # | Decision | Status | Critical Gaps |
|---|---|---|---|
| ADR-1 | New internal push endpoint for evaluations | Resolved | contract confirmation is external (REV-8), the decision itself is closed |
| ADR-2 | AI actor = scorecard record, not participant | Resolved | — |
| ADR-3 | Extend tables + snapshot table | Resolved | — (REV-1 interface gap fixed R2) |
| ADR-4 | Segment from evaluator payload | Resolved | missing-segment fallback specified + measured |
| ADR-5 | One judge call for all tier-2 params | Resolved | — (REV-2 no-transcript branch fixed R2) |
| ADR-6 | Uniform weights constant, versioned by flag | Resolved | — (version now persisted on the record) |
| ADR-7 | Override consumes the one-edit slot | Resolved | — (REV-3 per-metric marker fixed R2) |
| ADR-8 | Actor-aware resolve guard | Resolved | — |
| ADR-9 | sidekiq-cron retention workers | Resolved | — |
| ADR-10 | Org-wide RBAC (descope team scoping) | Resolved | — |
| ADR-11 | Reuse ai_qa_unified_scorecard gate | Resolved | — |
Aggregate: 11 of 11 decisions Resolved, 0 Partial, 0 Dangling.
Spot-checks (challenge protocol, abbreviated per skill for high-scoring categories):
- ADR-1 / scale: ingest is 202-enqueue; 10x conversation volume moves the bottleneck to
the
gpt_agent_scoringqueue and OpenAI throughput — degradation is queue latency against the 60s budget, monitored viaduration_ms. Holds. - ADR-2 / reversibility: honestly rated Medium — if a bot participant type ever lands platform-wide, only the FE selector source changes; records don't migrate. Acceptable.
- ADR-5 / agent implementability: judge prompt assembly is specified to the delimiting
rule and config source (
system_preferencesrowai_agent_scoring); the exact template text is implementation detail an agent can draft from the live scorer's_build_system_promptprecedent. No question needed. - ADR-8 / consistency: cross-checked against ADR-2 and the existing webhook — the actor-scoped predicate is the only change; regression spec pinned in chunk 7. Consistent.
UI State Audit (FE deep-dive)
| Component | Loading | Empty | Error | Partial | Success | Assessment |
|---|---|---|---|---|---|---|
| ActorSelector | defined | defined (hidden single-actor / "Not scored yet") | defined (actors still listed) | defined (amber dot) | defined | 5/5 |
| AiScorecardView | defined ("scoring…" skeleton) | defined ("Not scored yet") | defined ("scoring unavailable" + retry) | defined (tier-2 "unscored" chips; transcript "expired") | defined | 5/5 |
| Override control | defined (spinner) | n/a — hidden | defined (toast + Retry, no partial state) | n/a | defined (chip + recompute) | 5/5 applicable |
Summary: 3 of 3 components have all applicable states defined (§2.C).
Data Integrity Deep-Dive (BE)
| Write Path | Transaction Scope | Partial Failure Behavior | Idempotency Key | Consistency Guarantee | Duplicate Handling |
|---|---|---|---|---|---|
| Evaluation ingest → score persist | scorecard + details + snapshot in one tx (CalculateAgentScore precedent) | full rollback, Sidekiq retry, unavailable terminal | (org, room_id, agent_id) unique + evaluated_at recency | strong within record | upsert; older evaluated_at ignored |
| Override PATCH | single tx: detail updates + parent recompute + correction_* | 500 + rollback, no partial state | one-edit slot (edit_count re-read in tx) | strong | second override 422 |
| Retention deletes | batched in_batches destroys | partial batch resumes next run | expires_at / created_at predicates | eventual (cleanup) | idempotent by predicate |
Concurrency Collision Map (BE)
| # | Shared Resource | Writers | Collision Scenario | Resolution Mechanism | Assessment |
|---|---|---|---|---|---|
| 1 | AI scorecard record | ingest worker (re-push) × override PATCH | re-score overwriting an override | evaluated_at recency guard + completed-record re-score rule; override marks scoring_state: overridden | adequate — an override followed by a newer evaluator re-push re-scores; flagged as accepted behavior in RFC §2.4 (audit preserves the override via paper_trail) |
| 2 | One-edit slot | two supervisors | double correction | slot re-read under tx lock; first wins, second 422 | adequate |
| 3 | Room's human scoring | resolve webhook × AI ingest | AI record suppressing human auto-score | actor-scoped existence predicate (ADR-8) + regression spec | adequate |
API Contract Completeness Check (BE)
| Endpoint | Request | Response | Errors | Auth | Idempotency | Examples | Assessment |
|---|---|---|---|---|---|---|---|
| POST /v1/ai_agent/evaluations | complete (PROPOSED) | complete | complete (202/422/200-skip) | service auth named | defined | yes | 6/6 (pending external confirmation) |
| GET agent_scorecards/:room_id | complete | complete (both branches + no-human-record case) | complete | roles + Usman nuance grounded | n/a — read | yes | 6/6 |
| PATCH agent_scorecards/:room_id (override) | complete | complete | complete (422×3, 403, 500) | roles pinned | one-slot semantics | yes | 6/6 |
Async Job / Event Consumer Spec (BE)
| Job | Trigger | Input Shape | Retry | DLQ | Concurrency | Idempotency | Timeout | Assessment |
|---|---|---|---|---|---|---|---|---|
| AiAgentScoringWorker | queue (ingest) | specified (§2.4 payload) | retry: 1 (precedent) | Sidekiq dead set + failed_payload 7d | queue-bound | unique key + evaluated_at | judge 240s | 7/7 |
| DeleteExpiredScorecardSnapshotsWorker | cron nightly 01:00 | expires_at predicate | cron re-run | n/a | batched | predicate | n/a | specified |
| DeleteOldAiScorecardsWorker | cron nightly | created_at < 13.months + actor_type | cron re-run | n/a | batched | predicate | n/a | specified |
Compliance Trigger Check
| Trigger | Found? | Location | Assessment |
|---|---|---|---|
| PII (transcript content) | yes | agent_scorecard_snapshots.transcript | handled: 90d TTL, scrub list, API-excluded, G7 GA gate, upstream hashing expected in contract (REV-8 dependency) |
| Auth/session data | yes | existing token flows only | unchanged |
| User content with retention | yes | snapshots + judge reasons | retention table §7.1 mirrored in DDL + workers |
CDG Status: Active — scored (7.0).
Cross-Layer Contract Verification (full-stack deep-dive)
| Endpoint | Backend Response Schema | Frontend Expected Schema | Match? | Gaps |
|---|---|---|---|---|
| GET :room_id (AI actor) | §2.4 r1 JSON (snake_case; actors[], groups[], scoring_status, veto fields) | §2.B AiScorecardData (same keys, snake_case; union matches enum) | Yes | none — shared fixture test pins it |
| GET :room_id (no param) | human record byte-compatible + additive keys | existing AgentScorecardTypes untouched | Yes | additive-only verified |
| PATCH override | recomputed record (r1 shape) | same parse path | Yes | none |
- Casing consistent (snake_case end-to-end, transformation explicitly forbidden)
- Nullability aligned (
scoring_status: nullfor manual rows typed;score_value: number \| null) - Error shape matched (§3.B catalog ↔ FE toast/retry states)
- Pagination n/a (single-record read)
- Auth flow unchanged (existing
$customFetchheaders)
Mismatches found: 0.
Cross-Layer Rollout Compatibility Matrix
| Scenario | Frontend | Backend | Works? | Notes |
|---|---|---|---|---|
| Pre-deploy | Old | Old | Yes | baseline |
| Backend first | Old | New | Yes | GET additions additive; old FE ignores new keys; records accumulate dark |
| Frontend first | New | Old | Partial-safe | AI fields absent ⇒ FE renders manual mode (explicit fail-safe); no crash — but deploy order still mandates BE first |
| Both deployed | New | New | Yes | target |
| Backend rollback | New | Old | Yes | same as frontend-first fail-safe |
| Frontend rollback | Old | New | Yes | additive keys ignored |
Deploy order: Backend first (specified, §4 table with 6 ordered steps incl. Phase 1 prereq and DSAI enablement). Incompatible scenarios: 0 unaddressed.
End-to-End Data Flow (spot-trace)
Flow: AI conversation scored and reviewed — traceable entirely from §2.1a S01+S02: evaluator push → internal Grape → Redis queue → worker (config read → tier-1 → judge → veto) → one tx write (scorecard/details/snapshot) → event; QA opens panel → GET (no param: human + actors) → select AI actor → GET ?agent_id → render graded groups → optional PATCH override → recompute → chip. Side effects (Mixpanel events, ChatGptLog rows, paper_trail versions) are named at each step. Gaps in flow: none — no step's output mismatches the next step's input.
Agentic Readiness Deep-Dive
Vague-word audit (spec sections): 0 dangling "TBD/maybe/X-or-Y" in contract or DDL sections. Deliberate open items are fenced as PROPOSED (inbound contract) or §5 rows with owners — the correct mechanism. Dangling alternatives: 0. Task decomposition: 16 chunks, all with verifiable acceptance (see Task Manifest note below).
Strengths
- Grounding discipline — §2.0 Source Verification carries 25+ file:line rows including three decisive verified-absents; five PRD premises are corrected rather than inherited (Comment log), which is exactly what prevents mid-build surprises.
- The build/enablement split on the DSAI risk (§7) — instead of blocking the RFC on an external contract, every chunk executes behind the flag and the unconfirmed contract gates only org enablement. The riskiest dependency is contained, not ignored.
- Regression protection for the live human path — ADR-8 catches a subtle interaction (AI record tripping the resolve-guard) that neither the PRD nor Phase 1 fully specified, and pins it with a dedicated regression spec (chunk 7).
Biggest Gaps
- REV-8 (open): the inbound contract is PROPOSED. If DSAI's actual evaluator emits a different shape/trigger, chunk 2 + the ingest specs rework. Bounded by the 202-enqueue design, but it is the initiative's critical path (PRD Open Q#2, due 2026-07-17 — today).
- REV-6 (open): FE per-component prop/emit contracts are deferred to implementation; an agent will infer them from §2.A/§2.B. Low risk, same posture Phase 1 shipped with.
- CDG at 7.0: the PII-hashing control lives upstream in an unconfirmed contract; until DSAI confirms, the RFC's transcript-storage safety case rests on the G7 GA gate alone.
Priority Actions
- §2.4 Inbound r1 / REV-8 — get DSAI sign-off on the PROPOSED payload (fields, push trigger, PII hashing, segment guarantee). This is the only item between "build done" and "orgs enabled". Escalate today (deadline is 2026-07-17).
- §2.A / REV-6 — when Figma frames land, add
defineProps/defineEmitstypings forActorSelectorandAiScorecardViewto §2.A (10-minute close, removes the last FE guess). - §5 #9 / REV-7 — at implementation, pin the
display_namejoins (AI: chatbot AI-agent config repo; human: room-detail response) in the entity builder and add one line to §2.4 r1.
Implementation Readiness Checklist
Unblocked: traceability ✓ · decisions resolved (11/11) ✓ · failure modes + error catalog ✓ · config contract ✓ · pattern alignment ✓ · rollout + rollback ✓ · observability ✓ · task decomposition ✓ · vague words 0 ✓ · FE contracts (interfaces ✓ / prop-level REV-6 open) · UI states ✓ · a11y ✓ · DDL precision ✓ · API contracts + examples ✓ · tx/idempotency ✓ · concurrency map ✓ · security/tenancy ✓ · migration plan ✓ · service boundary ✓ · compliance handled-with-gate ✓ · cross-layer contract ✓ · deploy order ✓ · compat matrix ✓ · E2E flow ✓ · flag coordination ✓
Blocked (before org enablement / GA, not build):
- REV-8 — DSAI contract confirmation (enablement)
- Infosec sign-off + G7 closure (GA)
- Figma frames (FE pixel-fidelity; interim Stitch spec authorized)
Verdict: Ready to implement (build); fix 1 enablement blocker before real-org rollout.
Task Manifest
The RFC specifies its own 16-chunk plan (§4.C) with files, commands, and assertable acceptance per chunk; ordering was verified against dependencies (migrations → pipeline → API → FE → contract tests → test-spec doc). No re-decomposition needed — the §4.C table is the manifest.
Dangling Decisions Log
| # | Decision | Location | Owner | Deadline |
|---|---|---|---|---|
| — | none — 11/11 resolved |
Open Questions
| # | Question | Category | Severity |
|---|---|---|---|
| 1 | Will DSAI confirm the PROPOSED evaluation payload (incl. PII hashing + segment) by the PRD deadline, or does the contract need renegotiation? | ACV / CDG | Blocking (enablement) |
| 2 | Should an evaluator re-push after a supervisor override re-score the record (current spec: yes, if evaluated_at newer — override preserved only in paper_trail)? Confirm this is the intended precedence. | DIC | Important |
| 3 | Component prop typings (REV-6) — close in §2.A when frames land? | CNT | Nice-to-have |
Evidence Notes
- §2.0 Source Verification — the review's confidence rests here: spot-checked rows against
the grounding evidence (schema line ranges,
get_scorecard.rb:16,SwitchBar.vue:156-164, worker/cron precedents) — consistent. - §2.4 — both directions of the contract carry example payloads; the inbound row's PROPOSED fencing is explicit and honest.
- §4.C — every chunk's acceptance is assertable (a spec, a validator, or a rendered state); none is "looks good".
- Mermaid validity: 9/9 blocks parse (validated with
@mermaid-js/mermaid-clithis cycle, including the R2-edited erDiagram; re-run after the fix edits).
Review History
| Cycle | Date | Reviewed RFC revision | Score | Verdict | Findings open → fixed | Notes |
|---|---|---|---|---|---|---|
| R1 | 2026-07-17 | 2026-07-17 / working tree | 8.0 | PROCEED with notes | 8 open, 0 fixed | Initial review of the rfc-starter draft; REV-1..8 minted. |
| R2 | 2026-07-17 | 2026-07-17 / working tree | 8.5 | PROCEED with notes | 3 open (1 enablement blocker, 2 minor), 5 fixed | REV-1..5 fixed in the RFC (schema idempotency/weights fields, tier-2 no-transcript branch, override marker, tenancy mechanics, empty-human GET); erDiagram re-validated. |