Skip to main content

Task Breakdown — AI Agent Testing: Historical Validation

Effort Summary​

Phase / AreaFE daysBE daysQA daysTotal
Phase 1 — UI (mocked, actionable)9.5—2.512
Phase 2 — API integration (actionable)3154.522.5
Actionable subtotal12.515734.5
Blocked visibility — detail frame finalization0.5—0.51
Blocked visibility — re-run endpoint contract—1.50.52
Grand total including blocked visibility1316.5837.5

Confidence: medium-high. Key assumptions: chatbot, chatbot-fe, and qontak-designer sibling repos are the implementation targets; RFC review R3 says no open blockers/majors remain. The estimates can move if Design does not provide a dedicated comparison/detail frame before Task 1.2, or if the AI squad changes the QontakNLP production RPM/TPM ceiling from the RFC's default ai_agent_testing_nlp_rpm = 60.

Reconnaissance Notes​

SurfaceRepository nameLocal path known?Notes
Backendchatbot/Users/mekari/workspace/work/chatbotRails/Grape API, Sidekiq worker, RSpec specs.
Web FEchatbot-fe/Users/mekari/workspace/work/chatbot-feNuxt 3 + Pinia, Pixel3 @mekari/pixel3@^1.0.12, Vitest tests under tests/unit.
Design referenceqontak-designer/Users/mekari/workspace/work/qontak-designerStatic prototype only; use for list/generate flow layout references, not API/auth implementation.

Verified conventions:

  • FE components use PascalCase .vue files under modules/bot-automation/components/...; existing unit tests live under tests/unit/pages/... or tests/unit/modules/.../*.test.ts / *.spec.ts.
  • FE imports use @/ aliases; API methods already exist in common/services/main/v1/ai-agents.ts; Pinia actions/types already exist in store/ai-agent/{actions,state,getters,interface,types}.ts.
  • BE API routes live in app/api/frontend_service/v1/ai_agent/test_cases_controller.rb; use cases and repositories live under app/api/frontend_service/v1/ai_agent/{use_cases,repositories}; specs are under spec/api/frontend_service/v1/... and spec/workers/....
  • Test commands: FE pnpm test, pnpm lint, pnpm build; BE bundle exec rspec <spec path>, plus bundle exec rails db:migrate for migrations.

Phase 1 — UI (APIs mocked)​

Task 1.1: [FE] Testing Page, Generate Flow, and Navigation Shell (AITEST-S01, AITEST-S02, AITEST-S08)​

A supervisor/admin can open the AI Agent Testing page, see existing/generated test cases, and start a generate-from-inbox batch with mocked responses.

Status: ✅ Actionable

Design reference (FE tasks only): https://www.figma.com/design/LJ6ePL0PjxKbHYZZdNK4LX/%E2%9C%A8-Bot---AI?node-id=16743-298263 and https://www.figma.com/design/LJ6ePL0PjxKbHYZZdNK4LX/%E2%9C%A8-Bot---AI?node-id=16514-155786 · DS version: @mekari/pixel3@^1.0.12 · Frame: Testing page / Generate flow · Design QA: BOT Design QA

What to build​

Create the production Testing page under bot automation, porting the qontak-designer list/generate layout into chatbot-fe. Use mocked store/service responses for list, create, polling, loading, empty, failed, and completed states.

Implementation Plan​

ActionFileWhat changes
create/Users/mekari/workspace/work/chatbot-fe/pages/bot-automation/testing/index.vuePage shell, store calls, table state, generate modal/drawer orchestration, polling with mocked API responses.
create/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/TestCasesTable.vuePixel3/table wrapper modeled after AiAgentsTable.vue; columns for name, type, score, status, updated, actions.
create/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/GenerateTestCaseModal.vueSource picker; disable Phase 2/3 sources except scaffolded disabled states.
create/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/GenerateFromInboxDrawer.vueName + version selector, validation, submit event payload { name, type, version_id }.
create/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/TestCaseGeneratingModal.vueAsync progress modal and status messaging while the mocked batch is processing.
extend/Users/mekari/workspace/work/chatbot-fe/layouts/bot-automation.vueAdd feature-flag/subscription-gated Testing nav entry.
extend/Users/mekari/workspace/work/chatbot-fe/store/ai-agent/actions.tsKeep mocked action paths aligned with existing CREATE_TEST_CASE and FETCH_TEST_CASES actions.
extend/Users/mekari/workspace/work/chatbot-fe/store/ai-agent/interface.tsConfirm TestCase and CreateTestCasePayload include name, type, version_id, status, confidence_score.
create/Users/mekari/workspace/work/chatbot-fe/tests/unit/pages/bot-automation/testing.test.tsPage-level tests for role/menu visibility, list states, generate submit, polling, and ai_workspace_load_failed.
create/Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/bot-automation/components/testing/TestCasesTable.spec.tsTable rendering, empty/error states, row actions, status badges.

Implementation steps​

  1. Open /Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/ai-agents/AiAgentsTable.vue and /Users/mekari/workspace/work/qontak-designer/app/pages/bot-automation/testing/index.vue; mirror the table/empty-state conventions while keeping production imports in chatbot-fe.
  2. Write failing tests in /Users/mekari/workspace/work/chatbot-fe/tests/unit/pages/bot-automation/testing.test.ts and /Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/bot-automation/components/testing/TestCasesTable.spec.ts for menu access, blank slate, fetch error, generate submit, and polling status transitions.
  3. Scaffold /Users/mekari/workspace/work/chatbot-fe/pages/bot-automation/testing/index.vue and the four modules/bot-automation/components/testing/* files with Pixel3 components and typed emits.
  4. Wire state through store/ai-agent/actions.ts, store/ai-agent/getters.ts, and store/ai-agent/interface.ts; use mocked resolved/rejected promises until Phase 2 wiring.
  5. Add the Testing nav entry in /Users/mekari/workspace/work/chatbot-fe/layouts/bot-automation.vue, gated the same way current bot-automation routes are gated.
  6. Run pnpm test -- tests/unit/pages/bot-automation/testing.test.ts tests/unit/modules/bot-automation/components/testing/TestCasesTable.spec.ts until green.
  7. Run pnpm lint && pnpm build before marking done.

Acceptance criteria​

  • Owner/supervisor/admin can see and open the Testing page; standard agent does not see the entry.
  • List renders loading, empty, failed, processing, completed, and failed batch states from mocked store data.
  • Generate-from-inbox drawer requires name + version and emits { name, type: "inbox", version_id }.
  • Phase 2/3 generation sources are visible only as disabled/scaffolded options.
  • Fetch failure renders the blank/error slate and fires ai_workspace_load_failed.

Test strategy​

Vitest should mount the page with mocked Pinia actions, assert state-specific DOM output, verify generate payload shape, and confirm the analytics/error hook fires on rejected list fetch.

Effort estimate​

DisciplineDays
Frontend4
Backend—
QA1
Total5

Assumptions: reuses existing Pixel3 table/pagination patterns and qontak-designer layout; no new FE dependency.

Run to verify​

cd /Users/mekari/workspace/work/chatbot-fe
pnpm test -- tests/unit/pages/bot-automation/testing.test.ts tests/unit/modules/bot-automation/components/testing/TestCasesTable.spec.ts
pnpm lint
pnpm build

Depends on​

  • None.

Task 1.2: [FE] Detail Comparison, Question List, and Confidence Meter (AITEST-S04, AITEST-S05, AITEST-S06)​

A supervisor/admin can review each historical question side-by-side, compare the human answer with the AI shadow answer, rate it, and see the confidence meter update from mocked data.

Status: ⚠️ Partially blocked — comparison/detail UI is actionable from RFC props and existing ValidationDetailPanel.vue, but pixel-faithful polish waits on REV-6: dedicated Design/Figma frame for the comparison detail view.

Design reference (FE tasks only): https://www.figma.com/design/LJ6ePL0PjxKbHYZZdNK4LX/%E2%9C%A8-Bot---AI?node-id=16514-155786 · DS version: @mekari/pixel3@^1.0.12 · Frame: Comparison view (design pending dedicated detail frame; REV-6) · Design QA: BOT Design QA

What to build​

Create the detail route and comparison components using mocked TestCaseDetail data. The UI should group questions by topic, render human answer left and AI answer right, show response-time/confidence/sources, expose thumbs up/down, and show failed-generation messages.

Implementation Plan​

ActionFileWhat changes
create/Users/mekari/workspace/work/chatbot-fe/pages/bot-automation/testing/[id].vueDetail page shell, mocked fetch, loading/error/read-only states.
create/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/TestCaseComparison.vueSide-by-side human-vs-AI answer panel and rating event handling.
create/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/QuestionList.vueTopic grouping, question selection, per-question status badges.
create/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/ConfidenceMeter.vue(thumbs-up / total) * 100 display, threshold state, accessibility label.
extend/Users/mekari/workspace/work/chatbot-fe/store/ai-agent/actions.tsMock FETCH_TEST_CASE_DETAIL and optimistic UPDATE_TEST_CASE_QUESTION path for UI tests.
extend/Users/mekari/workspace/work/chatbot-fe/store/ai-agent/interface.tsConfirm TestCaseQuestion, TestCaseSource, TestCaseParameter, and TestCaseDetail cover fields used by UI.
create/Users/mekari/workspace/work/chatbot-fe/tests/unit/pages/bot-automation/testing-id.test.tsDetail route states and fetch/rating behavior.
create/Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/bot-automation/components/testing/TestCaseComparison.spec.tsHuman-left/AI-right rendering, failed question state, thumbs event.
create/Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/bot-automation/components/testing/ConfidenceMeter.spec.tsMeter math, threshold labels, aria output.

Implementation steps​

  1. Open /Users/mekari/workspace/work/chatbot-fe/modules/ai-agent/components/forms/ValidationDetailPanel.vue and /Users/mekari/workspace/work/chatbot-fe/store/ai-agent/interface.ts; reuse layout and typed fields where they fit.
  2. Write failing specs for /Users/mekari/workspace/work/chatbot-fe/pages/bot-automation/testing/[id].vue, TestCaseComparison.vue, and ConfidenceMeter.vue.
  3. Scaffold the detail page and three components with props from RFC §2.A; use mock data for questions, parameters.human_answer, answer, confidence, response_time, sources, and status.
  4. Wire optimistic rating through store/ai-agent/actions.ts; stub real HTTP with the existing store shape for now — real PATCH assertions are added in Task 2.8.
  5. Add failed and processing question presentations: failed answer says the AI answer could not be generated; processing disables rating.
  6. Run the focused Vitest files until green.
  7. Run pnpm lint && pnpm build; capture remaining Design QA deltas against REV-6.

Acceptance criteria​

  • Detail view renders question list grouped by topic.
  • Selected question renders human answer left and AI shadow answer right.
  • AI metrics render confidence, response time, and sources when present.
  • Failed question renders a non-rating failed state.
  • Rating buttons optimistically update the selected question and confidence meter; failed save rolls back.
  • Pixel-level final QA is pending REV-6 dedicated detail frame.

Test strategy​

Vitest should assert DOM output for completed/failed/processing questions, simulate thumbs up/down, verify optimistic state change and rollback, and check role="progressbar"/labeling for the meter.

Effort estimate​

DisciplineDays
Frontend3
Backend—
QA1
Total4

Assumptions: detail UX follows RFC typed props and existing ValidationDetailPanel.vue until Design provides the final frame.

Run to verify​

cd /Users/mekari/workspace/work/chatbot-fe
pnpm test -- tests/unit/pages/bot-automation/testing-id.test.ts tests/unit/modules/bot-automation/components/testing/TestCaseComparison.spec.ts tests/unit/modules/bot-automation/components/testing/ConfidenceMeter.spec.ts
pnpm lint
pnpm build

Depends on​

  • Task 1.1
  • External: REV-6 dedicated detail Figma frame for final pixel QA.

Task 1.3: [FE] Activation Gate and Manual Override UI (AITEST-S07, AITEST-S09)​

A supervisor/admin can see why activation is disabled below the threshold and can use an audited force-activate override when allowed.

Status: ✅ Actionable

Design reference (FE tasks only): https://www.figma.com/design/LJ6ePL0PjxKbHYZZdNK4LX/%E2%9C%A8-Bot---AI?node-id=16514-155786 · DS version: @mekari/pixel3@^1.0.12 · Frame: Activate button · Design QA: BOT Design QA

What to build​

Extend the existing AI Agent editor footer to reflect mocked testing confidence: disable activation below threshold, show the reason, and add a force-activate modal requiring an override reason.

Implementation Plan​

ActionFileWhat changes
extend/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/AiAgentEditor.vueDisable/enable activate CTA based on mocked confidence score and gate flag.
create/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/ForceActivateModal.vueOverride reason form, validation, confirm/cancel events.
extend/Users/mekari/workspace/work/chatbot-fe/store/ai-agent/actions.tsAdd mocked publish override payload path using existing publish action shape.
create/Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/bot-automation/components/AiAgentEditorTestingGate.spec.tsGate/override states.
create/Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/bot-automation/components/testing/ForceActivateModal.spec.tsRequires reason and emits payload.

Implementation steps​

  1. Open /Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/AiAgentEditor.vue near the footer/publish code and preserve its current save/publish flow.
  2. Write failing tests for below-threshold disabled state, threshold-met enabled state, and override reason validation.
  3. Create ForceActivateModal.vue and wire it to the editor footer using mocked score/threshold values.
  4. Add store/service payload plumbing for override_reason without changing the real endpoint yet; Task 2.6 wires the backend contract.
  5. Run focused tests, then pnpm lint && pnpm build.

Acceptance criteria​

  • Activate button is disabled when gate is on and confidence is below threshold.
  • UI explains the required threshold and current confidence.
  • Override modal requires a non-empty reason.
  • Override confirm emits/passes override_reason.

Test strategy​

Vitest should mount AiAgentEditor.vue with mocked confidence/gate values and assert button states, modal validation, and publish payload.

Effort estimate​

DisciplineDays
Frontend1.5
Backend—
QA0.5
Total2

Assumptions: uses RFC-pinned ai_agent_testing_threshold default 80; backend enforcement lands in Task 2.6.

Run to verify​

cd /Users/mekari/workspace/work/chatbot-fe
pnpm test -- tests/unit/modules/bot-automation/components/AiAgentEditorTestingGate.spec.ts tests/unit/modules/bot-automation/components/testing/ForceActivateModal.spec.ts
pnpm lint
pnpm build

Depends on​

  • Task 1.2

Task 1.4: [FE] Tree-Diagram Confidence Badge UI (AITEST-S10)​

A supervisor/admin can see average testing confidence directly on the AI Agent node in the tree diagram.

Status: ✅ Actionable

Design reference (FE tasks only): https://www.figma.com/design/LJ6ePL0PjxKbHYZZdNK4LX/%E2%9C%A8-Bot---AI?node-id=16514-155786 · DS version: @mekari/pixel3@^1.0.12 · Frame: Tree node · Design QA: BOT Design QA

What to build​

Add a confidence badge to the existing AI Agent tree node/form surface using mocked avg_confidence_score, including the "no score yet" state.

Implementation Plan​

ActionFileWhat changes
extend/Users/mekari/workspace/work/chatbot-fe/modules/conversation/views/tree-diagram/old-canvas/forms/ai-agent-form.vueRender confidence badge/no-score text from node data.
extend/Users/mekari/workspace/work/chatbot-fe/store/ai-agent/interface.tsAdd/confirm avg_confidence_score field on relevant tree/agent type if needed.
create/Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/conversation/views/tree-diagram/old-canvas/forms/ai-agent-form-confidence.spec.tsBadge and fallback states.

Implementation steps​

  1. Open /Users/mekari/workspace/work/chatbot-fe/modules/conversation/views/tree-diagram/old-canvas/forms/ai-agent-form.vue; identify the agent summary/header section that should host the badge.
  2. Write a failing spec for avg_confidence_score: 83, 0, and null.
  3. Add the Pixel3 badge/text rendering and type updates.
  4. Stub the API field in tests; Task 2.7 wires the real GET /api/v3/paths/:id/tree_diagram value.
  5. Run the focused spec, then pnpm lint && pnpm build.

Acceptance criteria​

  • Tree AI Agent node/form shows average confidence when backend data has a score.
  • It shows "no score yet" when no completed test cases exist.
  • Badge rendering does not break existing form actions.

Test strategy​

Vitest should mount the existing form with minimal node fixtures and assert badge/fallback text.

Effort estimate​

DisciplineDays
Frontend1
Backend—
QA0.5
Total1.5

Assumptions: existing tree payload can accept an additive optional field without breaking consumers.

Run to verify​

cd /Users/mekari/workspace/work/chatbot-fe
pnpm test -- tests/unit/modules/conversation/views/tree-diagram/old-canvas/forms/ai-agent-form-confidence.spec.ts
pnpm lint
pnpm build

Depends on​

  • None.

Task 1.5: [FE] Detail View Final Design Frame and Pixel QA (AITEST-S05, AITEST-S06)​

A supervisor/admin sees the final comparison/detail experience exactly as approved by Design, not only the functional fallback built from RFC props.

Status: 🚫 Blocked — REV-6 must be resolved: Design needs to provide a dedicated Figma frame for the comparison/detail view, because qontak-designer has no production-ready prototype for this surface.

Design reference (FE tasks only): n/a — design pending REV-6 · DS version: @mekari/pixel3@^1.0.12 · Frame: Comparison/detail view · Design QA: BOT Design QA

What to build​

Once the dedicated detail frame exists, reconcile TestCaseComparison.vue, QuestionList.vue, and ConfidenceMeter.vue against the approved layout, spacing, copy, empty/failed states, and accessibility expectations.

Implementation Plan​

ActionFileWhat changes
extend/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/TestCaseComparison.vueApply final layout, spacing, and copy from the approved Figma frame.
extend/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/QuestionList.vueMatch final topic grouping and selected-question states.
extend/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/ConfidenceMeter.vueMatch final threshold/meter presentation and accessibility copy.
extend/Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/bot-automation/components/testing/TestCaseComparison.spec.tsUpdate assertions for final copy/state names.
extend/Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/bot-automation/components/testing/ConfidenceMeter.spec.tsUpdate final meter label and threshold behavior tests.

Implementation steps​

  1. Wait for REV-6 to resolve with a dedicated Figma frame and Design QA owner.
  2. Open /Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/testing/TestCaseComparison.vue, QuestionList.vue, and ConfidenceMeter.vue; compare against the approved frame.
  3. Update component layout/copy/state names without changing the API contract from Task 1.2.
  4. Update focused Vitest assertions for the final copy and accessibility labels.
  5. Run the focused specs, then pnpm lint && pnpm build.

Acceptance criteria​

  • Approved comparison/detail Figma frame exists and is linked in the RFC or ticket.
  • Components match the approved frame for normal, failed, processing, and empty states.
  • Design QA signs off the comparison/detail view.
  • Existing Task 1.2 behavior and tests still pass.

Test strategy​

Vitest should keep behavioral coverage from Task 1.2 while updating final copy/state assertions; final visual acceptance is Design QA review against the dedicated frame.

Effort estimate​

DisciplineDays
Frontend0.5
Backend—
QA0.5
Total1

Assumptions: no API or data-shape change; this is a design-finalization pass over already-built functional components.

Run to verify​

cd /Users/mekari/workspace/work/chatbot-fe
pnpm test -- tests/unit/modules/bot-automation/components/testing/TestCaseComparison.spec.ts tests/unit/modules/bot-automation/components/testing/ConfidenceMeter.spec.ts
pnpm lint
pnpm build

Depends on​

  • Task 1.2
  • External: REV-6 dedicated comparison/detail Figma frame and Design QA sign-off.

Phase 2 — API Integration​

Task 2.1: [BE] Create Test Case Contract: Name, Processing Status, and Idempotency (AITEST-S01, AITEST-S08)​

A supervisor/admin can create a named historical-validation test case, receive a processing response, and avoid accidental duplicate batches.

Status: ⚠️ Partially blocked — core work is actionable; REV-8 asks the team to decide whether idempotency is a DB index, app-level dedup, or accepted duplicate risk. Implement app-level dedup unless product explicitly accepts duplicates.

What to build​

Add the required name column, accept/persist name in POST /api/v1/ai_agents/:id/test_cases, create new cases as processing, enqueue the worker, and add duplicate-submit guard behavior.

Implementation Plan​

ActionFileWhat changes
create/Users/mekari/workspace/work/chatbot/db/migrate/2026XXXXXXXXXX_add_name_to_ai_agent_test_cases.rbAdd nullable name and index.
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/test_cases_controller.rbRequire/permit name, pass to use case.
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/use_cases/create_test_cases.rbValidate name length, dedupe pending/processing duplicate submissions, enqueue worker.
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/repositories/create_test_case.rbPersist name; set status processing instead of pending.
extend/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/create_test_cases_spec.rbRequest/response/name/status/idempotency specs.
extend/Users/mekari/workspace/work/chatbot/spec/factories/ai_agent_test_case.rbAdd name factory field.

Implementation steps​

  1. Open /Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/test_cases_controller.rb, /Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/use_cases/create_test_cases.rb, and /Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/repositories/create_test_case.rb; preserve the existing Grape + ResultMatcher flow.
  2. Write failing RSpec examples in /Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/create_test_cases_spec.rb for name, status='processing', worker enqueue, version 404, and duplicate pending/processing create.
  3. Create the migration and update the factory.
  4. Update controller params and use-case contract to require name with the RFC max length.
  5. Persist name and processing in the repository; implement app-level duplicate guard unless REV-8 is resolved differently before coding.
  6. Run migration and focused RSpec until green.
  7. Run the relevant full RSpec set for AI-agent test-case create.

Acceptance criteria​

  • ai_agent_test_cases.name exists and is returned in create/list/detail responses.
  • POST /api/v1/ai_agents/:id/test_cases accepts { type, version_id, name }.
  • Created test case starts in processing and enqueues FetchRoomConversationsWorker.
  • Duplicate rapid submit does not create multiple active processing batches for the same agent/version/name, unless REV-8 is explicitly accepted as duplicate risk.

Test strategy​

RSpec should assert request contract, DB persistence, response JSON, worker enqueue args, and duplicate behavior.

Effort estimate​

DisciplineDays
Frontend—
Backend2
QA0.5
Total2.5

Assumptions: app-level dedup is acceptable for Phase 1; no backfill required for existing draft rows.

Run to verify​

cd /Users/mekari/workspace/work/chatbot
bundle exec rails db:migrate
bundle exec rspec spec/api/frontend_service/v1/ai_agent/create_test_cases_spec.rb

Depends on​

  • External: REV-8 idempotency decision, if the team wants DB-level uniqueness instead of app-level dedup.

Task 2.2: [BE] Worker Sampling, Status Lifecycle, Shadow Generation, and Persistence (AITEST-S02, AITEST-S03, AITEST-S04, AITEST-S08)​

A generated test case samples eligible historical conversations, generates AI shadow answers without sending customer messages, persists question rows, and reaches completed/failed status.

Status: ⚠️ Partially blocked — implementation is actionable with RFC defaults; production rollout still needs AI squad confirmation of the actual per-org RPM/TPM ceiling and whether REV-9 should use Redis-backed org-level throttling.

What to build​

Extend FetchRoomConversationsWorker from fetch/extract/log into the full historical-validation batch: sample rooms/pairs, call QontakNLP predict per question with throttling/backoff, persist ai_agent_test_case_questions, update statuses, and log/alert safely without PII leakage.

Implementation Plan​

ActionFileWhat changes
extend/Users/mekari/workspace/work/chatbot/app/workers/fetch_room_conversations_worker.rbSampling, status transitions, bounded retry, per-question processing/failure, non-PII logs.
create/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/repositories/generate_shadow_answer.rbWrap Repositories::QontakNlp::Predict/QontakNlp::Inference#prediction, map answer/confidence/sources/response_time.
create/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/repositories/create_test_case_question.rbPersist question, topic, AI answer, human answer in parameters, metrics, source list, status.
extend/Users/mekari/workspace/work/chatbot/app/core/repositories/qontak_nlp/predict.rbAdd any needed shadow-generation params while preserving live predict behavior.
extend/Users/mekari/workspace/work/chatbot/lib/qontak_nlp/inference.rbConfirm timeout/read_timeout behavior remains 60s and supports shadow call payload.
extend/Users/mekari/workspace/work/chatbot/spec/workers/fetch_room_conversations_worker_spec.rbSampling, status lifecycle, zero SendMessageWorker enqueue, failure branches, throttling behavior.
create/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/repositories/generate_shadow_answer_spec.rbPredict mapping and 429/backoff behavior.

Implementation steps​

  1. Open /Users/mekari/workspace/work/chatbot/app/workers/fetch_room_conversations_worker.rb, /Users/mekari/workspace/work/chatbot/app/core/repositories/qontak_nlp/predict.rb, and /Users/mekari/workspace/work/chatbot/lib/qontak_nlp/inference.rb; keep current chat-service fetch/extract behavior intact.
  2. Write failing worker specs for sample sizes: 200 rooms -> around 20, fewer than 10 -> all, large pool -> cap 50-70; assert no SendMessageWorker enqueue.
  3. Write failing specs for lifecycle: case processing, per-question completed/failed, final case completed when at least one answer succeeds, case failed on fatal setup failure.
  4. Add repository classes for shadow answer generation and question persistence.
  5. Implement throttling using RFC default SystemPreference group_code: 'engine', code: 'ai_agent_testing_nlp_rpm', default 60; on 429 use 1s -> 2s -> 4s backoff, then fail the question after retry budget.
  6. Implement worker-level retry per REV-5 with concrete intervals if not otherwise decided; recommended: 3 retries with 1s, 4s, 16s backoff and alert when a job reaches dead set.
  7. Ensure logs include IDs/counts/status only, never raw question/answer bodies.
  8. Run focused worker and repository specs until green.

Acceptance criteria​

  • Worker samples historical rooms according to 10% / 50-70 cap and all-if-small rules.
  • Worker never sends messages or enqueues send-message jobs.
  • Each persisted question includes question, topic, answer, parameters.human_answer, confidence, response_time, sources, and status.
  • Per-question NLP failure marks only that question failed and lets the batch continue.
  • Fatal setup failure marks the test case failed and reports Rollbar metadata without PII.
  • Throttle and 429 behavior are covered by specs.

Test strategy​

RSpec should mock chat-service repositories, QontakNLP predict responses, Rollbar, Sidekiq queues, and time/backoff where practical; assert DB rows and statuses, not just method calls.

Effort estimate​

DisciplineDays
Frontend—
Backend5
QA1.5
Total6.5

Assumptions: no new DB table; uses existing ai_agent_test_case_questions; true Redis org-level throttle is deferred unless REV-9 is upgraded.

Run to verify​

cd /Users/mekari/workspace/work/chatbot
bundle exec rspec spec/workers/fetch_room_conversations_worker_spec.rb spec/api/frontend_service/v1/ai_agent/repositories/generate_shadow_answer_spec.rb

Depends on​

  • Task 2.1
  • External: AI squad confirms production QontakNLP RPM/TPM ceiling before beta.

Task 2.3: [BE] Rating Aggregate Recompute and Detail Contract (AITEST-S05, AITEST-S06)​

A supervisor/admin's thumbs up/down rating updates the question and recomputes the server-authoritative confidence score for the whole test case.

Status: ✅ Actionable

What to build​

Wrap rating and aggregate recompute in a transaction, prevent rating while the case is not completed, and ensure detail/list responses return the updated confidence_score.

Implementation Plan​

ActionFileWhat changes
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/repositories/rate_test_case_question.rbTransactional question score update plus parent confidence_score recompute.
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/use_cases/rate_test_case_question.rbReject rating when test case is not completed; return updated data.
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent_test_case/use_cases/get_ai_agent_test_case_detail.rbEnsure detail includes all question status/metric/source fields.
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent_test_case/models/get_ai_agent_test_case_detail.rbExpose confidence_score, question parameters, sources, and status fields.
extend/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/rate_test_case_question_spec.rbRequest-level rating and recompute specs.
extend/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/use_cases/rate_test_case_question_spec.rbUse-case transaction/error branch specs.
extend/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/get_ai_agent_test_case_detail_spec.rbDetail response fields for FE comparison.

Implementation steps​

  1. Open /Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/repositories/rate_test_case_question.rb and the related use case/specs; preserve existing scored_by metadata behavior.
  2. Write failing specs for two thumbs-up out of three questions -> confidence 67, all down -> 0, and unrated questions excluded or included according to RFC (thumbs-up / total sample) * 100.
  3. Add a transaction around question score save and parent aggregate recompute.
  4. Add status guard so ratings on processing/failed cases return 409/422 per RFC.
  5. Extend detail response specs to include status, metrics, sources, and parameters.human_answer.
  6. Run focused RSpec until green.

Acceptance criteria​

  • PATCH rating writes question score metadata and recomputes parent confidence in one transaction.
  • Confidence score equals (thumbs-up / total sample) * 100.
  • Rating is rejected while the parent test case is not completed.
  • Detail response carries every field needed by Task 1.2.

Test strategy​

RSpec should cover happy path, rollback path, concurrent-ish recompute determinism, and response shape consumed by chatbot-fe.

Effort estimate​

DisciplineDays
Frontend—
Backend2
QA0.5
Total2.5

Assumptions: recompute reads current question rows each time; no stored delta counter.

Run to verify​

cd /Users/mekari/workspace/work/chatbot
bundle exec rspec spec/api/frontend_service/v1/ai_agent/rate_test_case_question_spec.rb spec/api/frontend_service/v1/ai_agent/use_cases/rate_test_case_question_spec.rb spec/api/frontend_service/v1/ai_agent/get_ai_agent_test_case_detail_spec.rb

Depends on​

  • Task 2.2

Task 2.4: [BE] Delete and Re-run Test Case Contracts (AITEST-S01, AITEST-S08)​

A supervisor/admin can soft-delete a test case, and the backend has a pinned re-run path that safely clears old generated questions before regeneration.

Status: ⚠️ Partially blocked — DELETE is actionable from RFC; re-run needs REV-7 product/API decision for the trigger path. Implement DELETE now and include re-run contract if resolved before coding.

What to build​

Add missing DELETE route/use case/repository for soft delete. Pin the re-run API contract and transaction boundary if REV-7 is resolved during implementation.

Implementation Plan​

ActionFileWhat changes
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/test_cases_controller.rbAdd DELETE /:id/test_cases/:test_case_id; optionally add re-run route once REV-7 resolves.
create/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/use_cases/delete_test_case.rbOrg-scoped soft delete, 404 cross-tenant, idempotent response.
create/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/repositories/delete_test_case.rbacts_as_paranoid destroy.
create/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/delete_test_case_spec.rbRequest specs for success, idempotency, cross-tenant 404.
create/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/use_cases/delete_test_case_spec.rbUse-case specs.
extend/Users/mekari/workspace/work/chatbot/app/workers/fetch_room_conversations_worker.rbIf re-run lands, clear existing questions in a transaction before regeneration.

Implementation steps​

  1. Open /Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/test_cases_controller.rb and both AiAgentTestCase models/factories to confirm acts_as_paranoid.
  2. Write failing request specs for DELETE success body 200 { data: { id } }, repeated delete, and cross-tenant 404.
  3. Implement delete use case and repository with org/company scope.
  4. If REV-7 resolves, add the re-run route/param and write specs for clear-before-regen in a single transaction; otherwise document re-run as deferred in the skipped table below.
  5. Run focused specs.

Acceptance criteria​

  • DELETE soft-deletes the test case and returns { data: { id } }.
  • Cross-tenant delete returns 404 to avoid resource enumeration.
  • Repeated delete is idempotent.
  • Re-run contract is either implemented with transactional clear-before-regen or explicitly deferred behind REV-7.

Test strategy​

RSpec should assert DB deleted_at, response body, scope behavior, idempotency, and optional re-run transaction behavior.

Effort estimate​

DisciplineDays
Frontend—
Backend2
QA0.5
Total2.5

Assumptions: restore endpoint is out of scope for Phase 1, per RFC review resolution.

Run to verify​

cd /Users/mekari/workspace/work/chatbot
bundle exec rspec spec/api/frontend_service/v1/ai_agent/delete_test_case_spec.rb spec/api/frontend_service/v1/ai_agent/use_cases/delete_test_case_spec.rb

Depends on​

  • External: REV-7 re-run trigger decision, only if re-run is included in Phase 1 implementation.

Task 2.5: [BE] Publish Gate, Threshold Preference, Override Audit, and Tree Confidence (AITEST-S07, AITEST-S09, AITEST-S10)​

An AI Agent cannot be activated below the configured confidence threshold unless an audited override is provided, and tree-diagram responses expose average completed confidence.

Status: ✅ Actionable

What to build​

Add flagged publish gate enforcement with ai_agent_testing_threshold, support override_reason, and extend tree diagram output with average confidence across completed test cases.

Implementation Plan​

ActionFileWhat changes
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/repositories/publish.rbGate publish below threshold when ai_agent_testing_gate is on; allow audited override.
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/use_cases/publish_ai_agent.rbAccept override_reason, read threshold preference, validate override.
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/test_cases_controller.rbEnsure publish contract accepts override params if publish route is mounted here; otherwise update owning controller.
extend/Users/mekari/workspace/work/chatbot/app/core/repositories/paths/get_tree_diagram_v3.rbAdd avg_confidence_score for AI Agent nodes from completed test cases.
extend/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/publish_ai_agent_spec.rbGate below/above threshold, override reason, flag off behavior.
extend/Users/mekari/workspace/work/chatbot/spec/core/repositories/paths/get_tree_diagram_v3_spec.rbAverage completed confidence and no-score state.
extend/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v3/paths/tree_diagram_spec.rbAPI response includes confidence field.

Implementation steps​

  1. Open /Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/repositories/publish.rb, /Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/use_cases/publish_ai_agent.rb, and /Users/mekari/workspace/work/chatbot/app/core/repositories/paths/get_tree_diagram_v3.rb.
  2. Write failing publish specs for gate off, gate on below 80, gate on above 80, override with reason, and override without reason.
  3. Read SystemPreference via existing feature-flag/preference pattern for ai_agent_testing_gate and ai_agent_testing_threshold default 80.
  4. Add PaperTrail/audit metadata for override reason according to the repository's existing audit style.
  5. Write failing tree specs for completed scores [80, 90] -> 85, failed/processing ignored, no completed cases -> null/no score.
  6. Implement tree response field and run focused specs.

Acceptance criteria​

  • Publish returns 422 below threshold when gate is enabled and no override is provided.
  • Publish succeeds above threshold.
  • Override requires a reason and leaves an audit trail.
  • Tree diagram returns average completed confidence for AI Agent nodes.
  • No completed test cases produce "no score yet" semantics for FE.

Test strategy​

RSpec should exercise preference defaults, flag behavior, publish response codes, PaperTrail/audit fields, and v3 tree response shape.

Effort estimate​

DisciplineDays
Frontend—
Backend3
QA1
Total4

Assumptions: threshold is org-configurable via SystemPreference group_code: 'engine', code: 'ai_agent_testing_threshold', default 80.

Run to verify​

cd /Users/mekari/workspace/work/chatbot
bundle exec rspec spec/api/frontend_service/v1/ai_agent/publish_ai_agent_spec.rb spec/core/repositories/paths/get_tree_diagram_v3_spec.rb spec/api/frontend_service/v3/paths/tree_diagram_spec.rb

Depends on​

  • Task 2.3

Task 2.6: [FE] Wire Real Testing APIs Across Page, Detail, Delete, Rating, and Publish (AITEST-S01, AITEST-S05, AITEST-S06, AITEST-S07, AITEST-S08, AITEST-S09)​

The mocked frontend testing experience uses the real chatbot APIs end-to-end with correct loading, error, optimistic update, and rollback behavior.

Status: ✅ Actionable after BE tasks land.

Design reference (FE tasks only): same FE frames as Tasks 1.1-1.3 · DS version: @mekari/pixel3@^1.0.12 · Frame: Testing page / Generate flow / Comparison view / Activate button · Design QA: BOT Design QA

What to build​

Replace mocked Testing page/detail/gate data with the existing real common/services/main/v1/ai-agents.ts client methods and updated backend contracts.

Implementation Plan​

ActionFileWhat changes
extend/Users/mekari/workspace/work/chatbot-fe/common/services/main/endpoint.tsConfirm/create endpoint paths for list, create, detail, delete, rating, publish override.
extend/Users/mekari/workspace/work/chatbot-fe/common/services/main/v1/ai-agents.tsEnsure payloads match backend contracts, including name and override_reason.
extend/Users/mekari/workspace/work/chatbot-fe/store/ai-agent/actions.tsReplace mocks with real service calls and robust error handling/rollback.
extend/Users/mekari/workspace/work/chatbot-fe/store/ai-agent/interface.tsAlign FE types with real response fields.
extend/Users/mekari/workspace/work/chatbot-fe/pages/bot-automation/testing/index.vueReal list/create/poll/delete flow.
extend/Users/mekari/workspace/work/chatbot-fe/pages/bot-automation/testing/[id].vueReal detail fetch and rating flow.
extend/Users/mekari/workspace/work/chatbot-fe/modules/bot-automation/components/AiAgentEditor.vueReal publish gate/override response handling.
extend/Users/mekari/workspace/work/chatbot-fe/tests/unit/pages/bot-automation/testing.test.tsHTTP success/failure assertions.
extend/Users/mekari/workspace/work/chatbot-fe/tests/unit/pages/bot-automation/testing-id.test.tsDetail/rating integration assertions.
extend/Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/bot-automation/components/AiAgentEditorTestingGate.spec.tsPublish 422/override behavior.

Implementation steps​

  1. Open /Users/mekari/workspace/work/chatbot-fe/common/services/main/v1/ai-agents.ts and /Users/mekari/workspace/work/chatbot-fe/common/services/main/endpoint.ts; preserve existing method names where possible.
  2. Write failing tests that assert real service methods receive the exact endpoint, method, and payload shape from the RFC.
  3. Update endpoint/service payloads for create name, delete response, rating, detail, list, and publish override.
  4. Replace mocked page/detail/gate state with real store calls and keep optimistic rating rollback.
  5. Ensure polling stops on completed/failed and cleans up on unmount.
  6. Run focused Vitest, then full pnpm test, pnpm lint, and pnpm build.

Acceptance criteria​

  • Create sends { name, type, version_id } and opens polling/generating state.
  • List/detail/delete/rating use real endpoints and handle 401/403/404/422/500 user states.
  • Rating rollback happens on rejected PATCH.
  • Publish gate error renders threshold reason; override sends override_reason.
  • Polling stops cleanly and does not leak timers.

Test strategy​

Vitest should spy on mainService.aiAgents methods and assert payloads, error states, optimistic rollback, polling cleanup, and publish override handling.

Effort estimate​

DisciplineDays
Frontend2
Backend—
QA1
Total3

Assumptions: backend response shapes match RFC §2.4 after Tasks 2.1-2.5.

Run to verify​

cd /Users/mekari/workspace/work/chatbot-fe
pnpm test -- tests/unit/pages/bot-automation/testing.test.ts tests/unit/pages/bot-automation/testing-id.test.ts tests/unit/modules/bot-automation/components/AiAgentEditorTestingGate.spec.ts
pnpm test
pnpm lint
pnpm build

Depends on​

  • Task 2.1
  • Task 2.3
  • Task 2.4
  • Task 2.5

Task 2.7: [FE] Wire Tree-Diagram Confidence From Real v3 Payload (AITEST-S10)​

The AI Agent tree node displays the backend-computed average confidence from GET /api/v3/paths/:id/tree_diagram.

Status: ✅ Actionable after Task 2.5 lands.

Design reference (FE tasks only): https://www.figma.com/design/LJ6ePL0PjxKbHYZZdNK4LX/%E2%9C%A8-Bot---AI?node-id=16514-155786 · DS version: @mekari/pixel3@^1.0.12 · Frame: Tree node · Design QA: BOT Design QA

What to build​

Replace mocked tree confidence with the real additive avg_confidence_score from the v3 tree diagram response.

Implementation Plan​

ActionFileWhat changes
extend/Users/mekari/workspace/work/chatbot-fe/modules/conversation/views/tree-diagram/old-canvas/forms/ai-agent-form.vueRead real avg_confidence_score field from node data.
extend/Users/mekari/workspace/work/chatbot-fe/common/services/main/endpoint.tsConfirm v3 tree endpoint is used where this tree surface is loaded.
extend/Users/mekari/workspace/work/chatbot-fe/tests/unit/modules/conversation/views/tree-diagram/old-canvas/forms/ai-agent-form-confidence.spec.tsReal payload fixture tests.

Implementation steps​

  1. Open the tree load path that feeds /Users/mekari/workspace/work/chatbot-fe/modules/conversation/views/tree-diagram/old-canvas/forms/ai-agent-form.vue; confirm the v3 payload field reaches the component.
  2. Update fixtures to include avg_confidence_score.
  3. Replace mocked/computed placeholder with the real field.
  4. Run the focused spec and tree-diagram related tests.

Acceptance criteria​

  • Real v3 tree payload score appears in the AI Agent node/form.
  • Missing/null score renders "no score yet".
  • Existing tree edit/publish behavior remains unchanged.

Test strategy​

Vitest should use real-shaped v3 tree fixtures and assert score/fallback rendering.

Effort estimate​

DisciplineDays
Frontend1
Backend—
QA0.5
Total1.5

Assumptions: avg_confidence_score is additive and optional in the response.

Run to verify​

cd /Users/mekari/workspace/work/chatbot-fe
pnpm test -- tests/unit/modules/conversation/views/tree-diagram/old-canvas/forms/ai-agent-form-confidence.spec.ts
pnpm lint
pnpm build

Depends on​

  • Task 2.5
  • Task 1.4

Task 2.8: [BE] Re-run Test Case Endpoint and Clear-Before-Regenerate Transaction (AITEST-S08)​

A supervisor/admin can safely regenerate a test case without leaving stale questions or duplicate in-flight batches.

Status: 🚫 Blocked — REV-7 must be resolved: the RFC/review has not pinned which endpoint or parameter triggers re-run, nor the exact idempotency guard for clear-before-regenerate.

What to build​

After the re-run trigger is decided, add the route/use case/repository path that re-enqueues FetchRoomConversationsWorker, clears prior questions for the test case in a transaction, and prevents duplicate concurrent re-runs.

Implementation Plan​

ActionFileWhat changes
extend/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/test_cases_controller.rbAdd the decided re-run route or param once REV-7 resolves.
create/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/use_cases/rerun_test_case.rbValidate ownership/status, clear existing questions transactionally, set case processing, enqueue worker.
create/Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/repositories/rerun_test_case.rbEncapsulate clear-before-regenerate and status update transaction.
extend/Users/mekari/workspace/work/chatbot/app/workers/fetch_room_conversations_worker.rbAccept re-run context if needed and avoid duplicating stale question rows.
create/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/rerun_test_case_spec.rbRequest specs for trigger, status, duplicate guard, and response body.
create/Users/mekari/workspace/work/chatbot/spec/api/frontend_service/v1/ai_agent/use_cases/rerun_test_case_spec.rbTransaction rollback and enqueue specs.

Implementation steps​

  1. Wait for REV-7 to resolve the API shape, for example POST /api/v1/ai_agents/:id/test_cases/:test_case_id/rerun or a rerun=true action on an existing endpoint.
  2. Open /Users/mekari/workspace/work/chatbot/app/api/frontend_service/v1/ai_agent/test_cases_controller.rb and /Users/mekari/workspace/work/chatbot/app/workers/fetch_room_conversations_worker.rb; preserve existing create/delete/rating contracts.
  3. Write failing specs for the decided route, including duplicate re-run while processing, cross-tenant 404, and rollback if question clearing fails.
  4. Implement rerun_test_case.rb use case and repository so clearing old questions and setting the parent case to processing happen in one transaction before worker enqueue.
  5. Run focused specs and the worker spec to ensure regenerated questions do not duplicate stale rows.

Acceptance criteria​

  • Re-run trigger path is explicitly documented and implemented.
  • Existing questions are cleared in the same transaction that marks the parent test case processing.
  • Duplicate re-run while the case is already processing is rejected or idempotently returns the in-flight case, per REV-7 decision.
  • Worker regeneration creates a fresh question set without stale rows.
  • Cross-tenant re-run returns 404.

Test strategy​

RSpec should cover request contract, transaction rollback, duplicate/in-flight behavior, worker enqueue args, and stale-row prevention.

Effort estimate​

DisciplineDays
Frontend—
Backend1.5
QA0.5
Total2

Assumptions: re-run reuses FetchRoomConversationsWorker; no new worker class or table is needed.

Run to verify​

cd /Users/mekari/workspace/work/chatbot
bundle exec rspec spec/api/frontend_service/v1/ai_agent/rerun_test_case_spec.rb spec/api/frontend_service/v1/ai_agent/use_cases/rerun_test_case_spec.rb spec/workers/fetch_room_conversations_worker_spec.rb

Depends on​

  • Task 2.2
  • External: REV-7 re-run endpoint/param and idempotency decision.

Ordering rationale​

  • Phase 1 builds all user-visible screens with mocked contracts first, so Design/Product can review the complete workflow before backend behavior is fully wired.
  • Backend create/worker work starts Phase 2 because every real FE flow depends on a persisted, processing test case and generated question rows.
  • Rating aggregate and publish gate follow worker persistence because they depend on completed question rows and confidence scores.
  • DELETE can run in parallel with rating/publish once create exists; the fully blocked re-run task is visible as Task 2.8 and should wait for REV-7.
  • The fully blocked detail-frame task is visible as Task 1.5; keep Task 1.2 moving with the functional fallback, then do Task 1.5 once Design unblocks it.
  • Final FE API wiring is last so the frontend integrates against stable request/response contracts instead of chasing backend churn.

Skipped stories​

StoryReason
AITEST-S09 manual override auditNot skipped; included in Tasks 1.3 and 2.5 as Could-Have scope from the RFC.
REV-6 detail Figma frameNot skipped; included as fully blocked Task 1.5 for visibility.
REV-7 re-run triggerNot skipped; included as fully blocked Task 2.8 for visibility.