AI Agent Simplified Config — Grilling Session Decisions (2026-07-22)
Outcome of a structured decision-grilling session (PM: Dimas Fauzi Hidayat) over the
Simplified Config ADJUSTMENT PRD draft (ai_agent_simplified_config_adjustment_v1.0_07Jul.md,
Downloads drafts), the qontak-designer feat/ai-agents-ut-updates branch (the
UX-research-validated Steps/SOP redesign — Milestones deleted, Completion card deleted,
per-step WHEN/THEN Exceptions added), and the approved PIH research
(chatbot/ai-agent-configuration/ut-2026-06-23/ — 16 participants; Milestones = #1 blocker;
"logic is invisible" = P1). Each decision is committed; the v1.1 PRD inherits them
directly. Code anchors verified against
chatbot/app/api/frontend_service/v2/ai_agent/repositories/sync_to_ai_service.rb and
process_async_reply.rb on 2026-07-22.
Decision register
| # | Decision area | Committed answer |
|---|---|---|
| D1 | Gate derivation substrate | The shipped config model has no cross-action data-flow structure (args are {name,type,description}, LLM-filled) — R2's "derive gates from data flow" was unimplementable as written. Gates derive from SOP step order instead: each actioned step is gated on the previous actioned step's success milestone. The design branch's ordered, drag-reorderable Steps make execution order tenant-authored, not inferred. |
| D2 | Existing V2 agents | Derive-on-save. Published agents untouched until next human edit; first save under the new editor derives + lints, with honest "checkpoints recalculated" messaging. Pre-launch batch lint dry-run over all live V2 agents; would-fail list handed to support before rollout. Old capabilities open in the new drawer with instructions preserved and steps empty. |
| D3 | "Resolve conversation" THEN | Staging probe first: author a routing rule with a non-enum exit reason and observe whether DSAI relays rule-exit objects verbatim. Verbatim → BE mints resolve_conversation, handled in the same new enum-exit handler as user_request_human_handoff (zero DSAI work, one code path for all THEN outcomes). Enum-enforced → Resolve is cut from the MVP dropdown. The same probe verifies DSAI tolerates an omitted completion (legal Rails-side: build_completion → nil, .compact). |
| D4 | Completion | Derived, not authored — the design deleted the Completion card; the PRD follows. Completion = the last action of the last actioned step in SOP order; required_slots derivation anchors to it. Knowledge-only capabilities (no actions) omit completion pending the D3 probe. Research nuance recorded: participants asked for a comprehensible dropdown, not removal — derivation goes further than the evidence demanded, so the D6 task test is also the check on this bet. |
| D5 | Pattern A home | Per-step Exceptions are the only action_error surface. R4's message-contains + status-code matcher fields move onto the exception WHEN (step context already names the action). The global Routing tab is reserved for conversation-level rules (user-intent, catch-all) and never carries action_error. Matches the research 5.3 mental model (global = catch-all, capability-level = skill-specific). New linter check #6: exceptions compile to pack-level rules keyed by action tool name, which are not skill-scoped — warn on conflicting exceptions for one action across capabilities. |
| D6 | Visible logic | Steps satisfy the research P1 "make the logic visible" demand for MVP, conditional on the task test. AISC-S06 is rewritten around Steps (the Checkpoints-tags AC is dropped — milestones no longer exist to display); the AC becomes the research's own pass criterion: a non-technical participant completes a 2-step agent setup unassisted. No flow-map in MVP; a failed task test is the evidence that buys one. |
| D7 | Refine-with-AI rail | Non-goal holds despite heavy design-branch investment (option cards, inline diffs, scroll-to-field). Zero endpoint consumers exist and adoption would put a DSAI dependency inside the MVP. In validation: concept-only, labeled "later phase", shown outside the task test so a pass is attributable to the Steps model alone. The design change-request brief states this explicitly. Participant pull during tasks is recorded as Phase-2 demand evidence. |
| D8 | THEN dropdown cuts (PRD Q1/Q6) | "Switch action" and "Ask clarification" cut from both THEN surfaces (step exceptions + global routing) before validation — no wire representation exists. switch_to_action filed as future DSAI contract work only on independent participant demand. PRD Q6 (handover_to_human completion type) is moot: there is no completion control. |
| D9 | Description/goal char limit (PRD Q7) | Keep the shipped 10000. Tightening adds a save-failure mode for existing agents mid-migration and delivers no user value. Revisit only on DSAI prompt-budget pressure. |
Confirmed fixes (no decision required)
sources:wiring — the PRD's "no research yet" evidence-gap note is stale: approved findings exist at PIHchatbot/ai-agent-configuration/ut-2026-06-23/(Milestones #1 blocker; Guided Form preferred; "Completes When" dropdown requested; renames incl. Milestone→Checkpoint; 8× credit-cost bill-shock). Wiresources:in the v1.1 PRD and the reverseinforms:in the research doc.- Terminology drift — design renames: triggers→keywords, description→goal; Instructions+Steps merged into one card; "Checkpoints" language in the PRD is stale (superseded by Steps).
- Milestone derivation stays one-per-action (
<action>_succeeded); steps without actions emit no milestone.
Post-register evidence — Iteration-II UT (added 2026-07-23)
PIH chatbot/ai-agent-configuration/ut-2026-07/ (merged by Devina, 6 participants — 5
internal + 1 external technical) landed after this register was committed. It bears on
the decisions as follows — no D1–D9 decision changes:
- Validates D5 + D6's core bet (Strong): "Steps" fixed the Milestone blocker; merging Routing+Milestones+Actions removed the "same work three times" duplication. Verdict: "move toward a limited pilot."
- D6's pass criterion is still unmet: the round sampled no non-technical externals — the unassisted 2-step task test (AISC-S06/AC-3) waits for the external round.
- New top risk the register did not cover: prompt-hierarchy overlap (Goals/Guidance/Instructions/Steps — "which wins?", Strong) and Instructions-vs-Steps redundancy (High). Handled in PRD v1.6: hierarchy helper copy (CHG-001) + AISC-S06/AC-4 predictability probe + §16 rows 9–10; layer-labeling added to the Wulan brief.
- D7/CHG-007 evidence note: the Refiner chips were liked (soft, 2 participants) — demand evidence for the Refine-reconciliation follow-up phase, not against the MVP disable.
Consequence for the PRD framing
The "4 tenant decisions per capability" headline is no longer accurate — under the Steps model tenants author title, keywords, goal, instructions, ordered steps (name/description/ actions), and exceptions. Each is SOP-shaped and comprehensible; everything tenants cannot reason about (slots, milestones, gates, completion) stays derived. Reframe as "you write your SOP; we compile it." The drift from v1.0 is a v1.1 rewrite before the validation session, not a patch.