RFC Review: Phase 1 — Scorecard Settings & Rubric Config

Review of rfc-phase-1-settings-and-rubric-config.md against the rfc-reviewer full-stack rubric. Judgment-based, agentic-execution-readiness gate.

Task Manifest

Field	Value
RFC type	full-stack
Sub-type	FE new-feature + BE modify-feature (enhancement)
Rubric	rubric-fullstack.md (18 categories; CDG triggered → 18 scored)
Source PRD	../prds/phase-1-settings-and-rubric-config.md (v1.2)
Review cycle	R3 (R1 scored → in-session revision → R2 re-scored; R3 is a refresh/confirm cycle re-grounding every citation against current HEADs)
Confidence	High — all sections present across both layers; contracts match; flows traceable; every cited file re-opened this cycle

Overall

Overall score	8.5 / 10 (R1 8.0 → R2 8.5 → R3 8.5, held)
Rating band	Agentic-Ready (lower edge) / Strong
Agentic verdict	PROCEED — agent can execute both layers; the only true gate (FE pixel fidelity) is an external Figma dependency the RFC honestly flags, not a spec gap. R3 re-grounding found zero broken citations; two citation-drift notes are cosmetic and one (REV-9) sharpens the already-open REV-1.

---

R3 Re-Grounding Summary (the core of this cycle)

Every "existing-code" citation in the RFC was re-opened against the current HEADs (chatbot fa6dd8b79, chatbot-fe f5b80d5a, hub-service 8cfe79c61, hub-core 68586c45ee). Result: no citation is broken; all load-bearing claims hold. Three drift notes:

Citation	RFC claim	Current code	Verdict
NameUniquenessValidator#validate_create (REV-5)	exists; 422 "The name field is already exist…"	chatbot/app/core/repositories/gpt/scorecard_custom_parameter/name_uniqueness_validator.rb — validate_create (L12), invoked create.rb:74, exact message at create.rb:75	✓ VALID
scorecard_preference/patch.rb:18-22 rule(:passing_grade) 1–99	human range 1–99	if value < 1 || value > 99 → key.failure('Passing grade only between 0 - 99 are allowed')	✓ VALID (note: the human message text is literally "0 - 99"; ADR-8's "1–99" describes the guard, which is < 1 || > 99 — consistent)
entities/.../scorecard_preference.rb:7-9 defaults	DEFAULT_PASSING_GRADE=75, DEFAULT_AUTO_SCORE=true	DEFAULT_PASSING_GRADE = 75.to_f (L7), DEFAULT_AUTO_SCORE = true (L9)	✓ VALID
scorecard_preferences schema	is_auto_score bool default false + passing_grade float	chatbot_gpt_schema.rb:456-457 exact	✓ VALID
scorecard_custom_parameters.prompt is string	unused t.string "prompt"	chatbot_gpt_schema.rb:422 t.string "prompt"; migration 20241113041150 t.string :prompt	✓ VALID (still string — widen is genuinely needed)
BE mounts	gpt_api.rb:28,33; gpt_service/api.rb:21,24	gpt_api.rb:28 pref→/v1/gpt/omnichannel/scorecard_preferences; :33 custom→/v1/gpt/scorecard_custom_parameters; gpt_service/api.rb:21→/v1/scorecards/preferences; :24→/v1/scorecards/parameters/custom	✓ VALID
Feature flag (BE)	OrganizationFeatures::FindFeature + call_by_organization; used in ai_knowledge_sources/search.rb	find_feature.rb:5-19 + call_by_organization (L17); used search.rb:49	✓ VALID
Role enum (hub-core)	{owner,admin,supervisor,agent,member} user.rb:38-44	enum at L38-44; same 5 values (declaration order admin,supervisor,agent,owner,member)	✓ VALID (set identical; only listing order differs — not load-bearing)
/users/me (hub-service) surfaces role	single role from token	app/services/api/core/v1/users/resources/users.rb get :me (L371); role via UserViewProfile/AnotherUserViewProfile	✓ VALID
auto_agent_scoring.rb scores human; OpenAI client	:6,76-79; :160-179 request_timeout:240, max_attempts=2, Rollbar.error	scores agent; max_attempts=2 (L161), OpenAI::Client.new(request_timeout: 240) (L179), Rollbar.error (L57/228)	✓ VALID (line ~76-79 is the agent-fetch/auto-score region; substance holds)
room-resolve skip guard	room_resolve_interactions.rb:48-63 unless is_custom_parameter || scorecard_exists	guard at L51, gating AutoAgentScoringWorker.perform_async (L52)	✓ VALID
paper_trail on both models	:5 / :6	scorecard_preference.rb:5 + scorecard_custom_parameter.rb:6 has_paper_trail	✓ VALID
FE has no scorecard code	grep → 0 hits	grep scorecard|passing_grade|is_auto_score over common/store/pages/modules → 0 hits; pages/settings/scorecard/ absent	✓ VALID
ai-assist.ts:151-184 service pattern	$apiMain + AbortController	get_preference (151-161), update_preference (163-175) use new AbortController() + $apiMain(endpoint, {method, signal})	✓ VALID (returns {fetch, controller})
endpoint.ts:123-157	structure to add scorecard: block	ai_assist block lives at 123-157 of a nested {v1,v2,v3} map	✓ VALID anchor
botSubscriptionFeature.ts:23-40 $hasSubscription	returns boolean	hasSubscription(feature) L23-40 returns bool	✓ VALID — but file header marks it @deprecated (prefer useSubscription) → new minor finding REV-8
mixpanel-events.ts (contants dir)	[CHATBOT]-prefixed MIXPANEL_EVENTS	misspelled contants/ confirmed; MIXPANEL_EVENT_PREFIX="[CHATBOT]"	✓ VALID
ai-assist.vue referenced for component props/emits (REV-1)	infer prop shapes from ai-assist.vue	ai-assist.vue is <script setup> with NO defineProps/defineEmits — it's a page-level view, not a prop-driven component; uses defineExpose only	DRIFTED → sharpens REV-1 as REV-9 (the cited file has no prop contract to infer from)
ai-assist.vue:925 mixpanel.track(MIXPANEL_EVENTS.X, props)	single call passing props	actual call at L926 passes an inline literal ({Threshold: params.reply_limit, …}), plus L933/L939 — three calls, not props	DRIFTED (cosmetic; the pattern — mixpanel.track(MIXPANEL_EVENTS.*, {...}) — is intact and re-usable)
FE reads state.reply_limit snake_case directly (REV-6)	no transformation layer	ai-assist.vue template {{ state.reply_limit }} (L49), v-model="state.reply_limit" (L92); store seeds via ...preferencesOrigin.value, no case map	✓ VALID

Bottom line: the RFC's anti-hallucination grounding survives a full re-verification at current HEADs. The two drift notes are about how the pattern reference is described, not whether the referenced code exists. None change the score.

---

PRD → RFC Traceability

Forward (PRD → RFC):

PRD item	RFC coverage	Verdict
§7 CHG-001 (AI auto-score + threshold)	§2.3 new cols, §2.4 row 1, §2.A AutoScoreToggle	✅
§7 CHG-002 (wire prompt rubric)	§2.3 widen, §2.4 row 2, §2.A CustomParamEditor	✅
§8 New (editor + default viewer)	§2.4 row 3 (new endpoint), §2.A DefaultRubricViewer	✅
§9 Behavior 1/2	§2.4 rows 1–2 (HTTP/path/schema/errors resolved)	✅
§10 stories UASC-S01/S02/S03	§1.A.4 every AC/ERR/NEG mapped; §2.1a sequences	✅
§11 Rollout	§4 + Cross-Layer Rollout matrix	✅
§12 Observability (6 events + alert)	§3 Monitoring; all 6 events present	✅
§13 Success metrics	§1 Success Criteria + §4.D adoption signals	✅
§16 Decisions/Alternatives	§1.B + 8 ADRs	✅
§17 Open Questions	§5	✅
App. A 9 metrics	§2.4 row 3 payload (descriptions seeded)	✅

Reverse (RFC → PRD): every endpoint, column, and component in §1.A.5 / §1.C cites a PRD driver. No orphan artifacts.

Full-stack contradiction check: no FE/BE scope or behavior conflicts found (§2.G all Match? = yes).

PRT verdict: 11/11 PRD sections covered, 0 missing. The RFC corrects the PRD where the PRD contradicts code (roles, passing_grade range) rather than copying it — a strength re-confirmed this cycle (the human range message is literally "0 - 99", guard < 1 || > 99).

---

Scorecard

#	Cat	Score	Evidence (re-grounded R3)
1	PRT	9.0	Five traceability matrices (§1.A.1–1.A.5) + per-story map (§1.C), forward+reverse. All PRD-to-Schema rows verified against chatbot_gpt_schema.rb.
2	TDC	9.0	8 ADRs with options/consequences/reversibility + min-coverage checklist; cross-layer consistency §2.G. Every grounding citation re-verified.
3	CNT (FE)	7.5	§2.A component table + §2.B store/service TS shapes. Component props/emits still pattern-referenced, and R3 confirms the referenced ai-assist.vue is <script setup> with no props/emits to infer from (REV-9). Held at 7.5 — typed store/service shapes (§2.B) carry the score; missing prop contracts cap it below 8.5.
4	SCB (FE)	8.5	§2.D in/out scope + §4.C names exact FE file paths per chunk; pages/settings/scorecard/ confirmed absent (genuinely new).
5	DEP (FE)	8.0	§1 Dependencies table; new BE endpoint availability in §2.4; Figma dep flagged as blocker. botSubscriptionFeature.ts deprecation surfaced (REV-8, minor).
6	NFS (FE)	7.5	§3 Performance (500ms) + §3.C a11y + §3.D browser/perf budget.
7	TPS (FE)	7.5	§4.B FE commands + §4.C per-chunk acceptance + §4.B cross-boundary contract test; package.json scripts (lint/test/test:e2e/build) verified L10-22.
8	DMS (BE)	9.0	§2.3 ER + exact DDL; migration dialect (20241113041150) + chatbot_gpt_record connection re-verified; no-backfill.
9	ACV (BE)	8.0	§2.4 three endpoints with request/response JSON + error codes + optional-field backward compat + dual mount surfaces (mounts re-verified gpt_api.rb:28,33 / gpt_service/api.rb:21,24); casing pinned §2.B.
10	DIC (BE)	8.0	Strong consistency, org-unique upsert (upsert.rb:13-25 + model validates_uniqueness_of :organization_id with deleted_at scope), single-row tx, last-write-wins (§3.A); custom-param create dedup (NameUniquenessValidator#validate_create) re-verified.
11	FMC (merged)	8.5	§3.A/§3.B catalogs + 3 failure-path sequences; FE handles the specific BE codes/messages from §3.B.
12	CSS (BE)	7.0	Concurrency addressed; relies on platform rate-limiting. Low-volume config surface — capacity numbers advisory (rubric §RCS).
13	SAS (BE)	9.0	§3 Security: authz matrix (re-verified set_role(%w[owner admin supervisor]) on all three existing surfaces), token-only tenancy (current_user[:organization_id] / current_user.try(:[],'company_id')), input coercion/length/control-char strip, v-html ban, prompt-injection forward note, whodunnit, log scrubbing.
14	ROL (merged)	8.5	Deploy order specified (BE→FE; rollback FE→BE), Cross-Layer compat matrix, numbered rollback, flag contract.
15	OBS (merged)	7.5	6 events (FE+BE), alert, Rollbar/lograge; distributed-trace continuity scoped out for Phase 1 (REV-3, ddtrace/Aegis exist in BE).
16	SBC (BE)	8.0	§2.1 per-service responsibilities; ADR-6 sync/async; ADR-1 column boundary.
17	CPA (merged)	8.5	§2.0 Patterns-to-Follow cites real files both layers (all re-opened); naming consistent with existing snake_case FE consumption (state.reply_limit).
18	CDG (BE, cond.)	7.5	Triggered by rubric = org IP. Handled: no-log-prompt, paper_trail audit (re-verified both models), soft-delete retention; retention/DSAR scope stated.

Caps applied: none triggered — no category < 5.0; ACV ≥ 5; ROL ≥ 5 with deploy order specified; no cross-layer contract mismatch (§2.G, re-verified). 9.0+ band not reached because CNT/DIC are < 8.5 and several FE categories sit at 7.0–7.5.

---

Decision Closure (8 ADRs)

ADR	Decision	Check 1 (resolve/alts)	Check 2 (spec/failure)	Check 3 (challenge)	Verdict
1	New AI columns	✅ 3 options, chosen, files cited	✅ DDL + contract + read defaults	Reversible (drop cols); scales fine	Resolved
2	Widen prompt text	✅ vs new column	✅ migration up/down + truncation warning	Forward-only flagged	Resolved
3	Derived auto_scorable	✅ vs stored bool	✅ computed in entity	No drift	Resolved
4	New read-only rubric endpoint	✅ 3 options (endpoint absent in code — re-confirmed)	✅ payload seeded with 9 metrics	Reversible	Resolved
5	Feature gate	✅ reuse vs new	✅ BE (FindFeature) + FE ($hasSubscription) mechanism cited	Toggle off	Resolved
6	Sync writes / async analytics	✅ honest "no alternative"	✅ best-effort events (SendMixpanelEventWorker)	Holds at 10x (single-row)	Resolved
7	Reuse role enum	✅ vs new roles (enum re-verified)	✅ role matrix	Revisit if QA role lands	Resolved
8	ai_passing_grade 0–100	✅ vs 1–99 (human rule re-verified)	✅ coercion-then-range (§2.4 r1)	Divergence documented	Resolved

8 of 8 resolved, 0 dangling. This is the RFC's strongest dimension, re-confirmed at current HEAD.

---

Full-Stack Deep-Dives

Cross-Layer Contract Verification

Endpoint	BE response	FE expects	Match?
PATCH preference	{is_auto_score,passing_grade,is_ai_auto_score,ai_passing_grade} (snake_case)	same snake_case (FE consumes API shape directly, per ai-assist reply_limit — re-verified state.reply_limit direct)	Yes
POST/PATCH custom param	{id,name,prompt,auto_scorable}	same; auto_scorable boolean drives chip	Yes
GET default rubric	{status,group,metrics[{code,name,description,veto}]}	list + veto badge + PROPOSED note	Yes

No casing mismatch (FE consumes snake_case directly — verified against existing store/ai-assist). No nullability traps (AI fields optional with read defaults). No blocker.

Cross-Layer Rollout Compatibility

Scenario	FE	BE	Works?
Pre-deploy	Old	Old	Yes
Backend first	Old	New	Yes (new cols inert; AI fields optional)
Frontend first	New	Old	N/A — deploy order mandates BE first; if it occurred, FE no-ops behind flag
Both	New	New	Yes
BE rollback	New	Old	Yes (FE behind flag; AI fields optional)
FE rollback	Old	New	Yes (BE inert without FE)

Deploy order specified (§4 "Deploy BE before FE. Rollback FE before BE"). No blocker.

End-to-End Flow (traceable in one place?)

Yes — §2.1a sequences S01/S02/S03 trace User → FE → LB → API → hub-service(auth) → chatbot_gpt DB → response → FE state → Mixpanel, including failure branches. Side effects (analytics, paper_trail) shown. An agent can implement each step in order.

---

Strengths (top 3)

1. Decision closure (TDC 9.0) — 8 fully-resolved ADRs with reversibility; zero dangling. Re-confirmed at HEAD.
2. Anti-hallucination grounding (DMS/SAS/PRT) — every claim has a file:line Source-Verification row; R3 re-opened all of them and found zero broken citations. The RFC corrects the PRD where it contradicts code (roles, the 1–99 human range whose message is literally "0 - 99", the auto-scorer that does exist).
3. Security depth for a config layer (SAS 9.0) — token-only tenancy (re-verified), control-char stripping, v-html ban, log scrubbing, whodunnit, prompt-injection forward note.

Gaps (top 3, specific)

1. FE component contracts are pattern-referenced, not field-typed (CNT) — and the reference is weaker than stated. §2.A names components + backing endpoints; §2.B types store/service. Component props/emits are deferred to §5 #7. R3 finds the cited ai-assist.vue is <script setup> with no defineProps/defineEmits — so "infer prop shapes from ai-assist.vue" gives an agent nothing to copy; it would design the three components' prop contracts from scratch. (REV-1 carry-forward, sharpened by REV-9.)
2. botSubscriptionFeature.ts is @deprecated (REV-8, new minor). ADR-5 / §2.0 cite $hasSubscription from plugins/botSubscriptionFeature.ts; the file header marks it deprecated in favor of useSubscription. An agent following the RFC verbatim would extend a deprecated path.
3. Observability stops at events (OBS). No distributed trace correlating an FE error to BE root cause; explicitly scoped out for Phase 1 (REV-3). Acceptable for a config surface.

Top 3 improvement actions

1. REV-1/REV-9: add minimal props/emits TS interfaces for AutoScoreToggle, CustomParamEditor, DefaultRubricViewer directly in §2.A — don't point at ai-assist.vue, which exposes no props. Low cost; removes the only real agent-guessing surface left.
2. REV-8: in ADR-5 / §2.0, note that botSubscriptionFeature.ts is deprecated and state whether the new code uses $hasSubscription (parity) or the preferred useSubscription composable, so the agent doesn't extend a deprecated plugin by accident.
3. Keep REV-3 trace continuity scoped to Phase 2 (no action; confirmed appropriate).

---

Findings Ledger (carry-forward)

ID	Severity	Finding	RFC section to fix	first_seen	status	resolved_in
REV-1	major	FE contracts not typed: store state, service req/resp, component props pattern-referenced only	§2.A / §2.B	R1	partially-fixed (still open)	R2 (store+service typed; props deferred → RFC §5 #7); R3 confirmed still open
REV-2	major	No FE↔BE contract/integration test in the plan	§4.B / §4.C	R1	fixed	R2 (re-confirmed R3: §4.B cross-boundary test + §4.C ch.11)
REV-3	minor	No distributed-trace correlation FE→API→BE	§3 Monitoring	R1	fixed (scoped-out)	R2 (continuity noted + scoped; re-confirmed R3)
REV-4	minor	No browser-support matrix / FE performance budget	§3 / §3.C	R1	fixed	R2 (§3.D; re-confirmed R3)
REV-5	minor	Custom-param create duplicate handling / idempotency not explicit	§2.4 r2 / §3.A	R1	fixed	R2 (NameUniquenessValidator#validate_create cited; R3 re-verified the validator exists at the cited path with the exact message)
REV-6	minor	API response casing convention vs FE consumption not stated	§2.B / §2.G	R1	fixed	R2 (§2.B; R3 re-verified state.reply_limit snake_case consumption)
REV-7	minor	Rubric-text data governance (retention/DSAR scope) not stated	§3 Security	R1	fixed	R2 (re-confirmed R3)
REV-8	minor	botSubscriptionFeature.ts ($hasSubscription) cited by ADR-5/§2.0 is marked @deprecated (prefer useSubscription)	§2.0 / ADR-5	R3	open	— (promoted to RFC §5 #8)
REV-9	minor	REV-1's reference target ai-assist.vue is <script setup> with no defineProps/defineEmits, so "infer component props from ai-assist.vue" gives the agent no contract to copy — strengthens the case to type the 3 components' props in §2.A	§2.A / §2.B	R3	open	— (folded into RFC §5 #7)

Ledger summary: 3 open (0 blocker / 1 major REV-1 / 2 minor REV-8, REV-9), 0 newly fixed this cycle (R2 fixes re-confirmed against code), 0 accepted-risk. REV-1 carried forward (still open); REV-8 and REV-9 minted this cycle from re-grounding. All R2 fixes (REV-2..REV-7) re-verified as still holding in current code/RFC. Still-open material findings promoted to RFC §5.

Review History

Cycle	Date	Reviewed revision (last_updated / commit)	Score	Verdict	Findings open → fixed	Notes
R1	2026-06-20	last_updated 2026-06-20 / working tree	8.0	PROCEED	7 open, 0 fixed	Initial full-stack score; 7 findings raised.
R2	2026-06-20	last_updated 2026-06-20 (in-session revision) / working tree	8.5	PROCEED	1 open (REV-1 partial), 6 fixed	RFC revised in-session; REV-2..REV-7 fixed, REV-1 partially.
R3	2026-06-20	last_updated 2026-06-20 / chatbot fa6dd8b79 · chatbot-fe f5b80d5a · hub-service 8cfe79c61 · hub-core 68586c45ee	8.5	PROCEED	3 open (REV-1, REV-8, REV-9), 0 newly fixed	Refresh/confirm cycle. Re-grounded every citation at current HEADs: zero broken, all R2 fixes hold. Minted REV-8 (deprecated $hasSubscription plugin) + REV-9 (cited ai-assist.vue exposes no props). Score held — drift is cosmetic, not contract-breaking.

Dangling Decisions

None. All 8 ADRs resolved.

Open Questions (promoted to RFC §5)

#	Question	Category	Severity
1	REV-1 / REV-9 (carry-forward): type the three components' props/emits in §2.A rather than deferring to ai-assist.vue, which is <script setup> and exposes no prop contract.	CNT	Important
2	REV-8 (new): plugins/botSubscriptionFeature.ts is @deprecated; state whether new FE uses $hasSubscription (parity) or useSubscription (preferred).	DEP/CPA	Nice-to-have
3	Pre-existing RFC items: Figma frames (§5 #1, blocker for FE pixel fidelity), DSAI 9-metric confirmation (§5 #2), plan-tier provisioning owner (§5 #5) — none block agent execution of BE + FE logic.	—	(tracked in RFC §5)

Evidence Notes

• R3 re-grounding — every Source-Verification row + Patterns-to-Follow citation re-opened at the four current HEADs; results in the "R3 Re-Grounding Summary" table. Zero broken citations; this is what holds the score at 8.5 rather than dropping it.
• name_uniqueness_validator.rb — REV-5's fix re-verified: file exists at chatbot/app/core/repositories/gpt/scorecard_custom_parameter/name_uniqueness_validator.rb, validate_create (L12), invoked create.rb:74, exact 422 message create.rb:75.
• ai-assist.vue — re-opened: <script setup>, no defineProps/defineEmits (uses defineExpose); useVuelidate/$toast/$hasSubscription all present; mixpanel.track(MIXPANEL_EVENTS.*, {...}) at L926/933/939 (inline literals). Drove REV-9 and the cosmetic-drift note on §2.0's mixpanel pattern line.
• Role enum — hub-core user.rb:38-44: same 5 values as cited; only declaration order differs (admin,supervisor,agent,owner,member). Not load-bearing — ADR-7 maps personas onto the set, not the order.