Skip to main content

Data/AI/ML Alignment — Meeting Notes

  • Purpose: Align on the open items blocking the Phase 2 RFC (Refine proxy + Refine rail) before the Chunk-9 real-upstream integration pass.
  • Source: RFC — Autonomous AI Agent Phase 2: AI-Assisted Refinement, §5 Concerns/Questions/Known Limitations (OQ-1, OQ-4, OQ-11, OQ-12, OQ-13).
  • Chatbot squad: Dimas Fauzi Hidayat (PM), Eko Aprianto (Eng Lead), Hilmi (TPM)
  • Data/AI/ML Platform: noncore-mrag / mekari-agent owners (companion doc author: Grasia Meliolla)
  • Date: TBD — target before 2026-07-15 (Chunk 9 real-upstream pass)
  • Status: Draft agenda — pending scheduling

Context​

The chatbot squad is building a stateless proxy (POST /v2/ai_agents/:id/refine) in front of the documented "as-built" refine-skill-pack endpoint, plus a "Refine" chat rail in the agent editor. The RFC has been grounded against the companion doc (QON 51226214880) and RFC §10.3b, but several contract details remain prose-only or ambiguous. A few items — deployment status, multi-turn semantics, and the revert story — need joint confirmation before the chatbot side builds against them.

Blocking — must resolve before Chunk 9 (target: 2026-07-15)​

#TopicThe question (plain)Why it matters / backgroundRFC ref
1aDeployment statusIs refine-skill-pack deployed and callable in staging and prod today?The endpoint is currently known only from the companion doc. The Chunk-9 pass builds the real integration, not a mock — so confirmation it is live at the gateway path /qontak-ai-noncore-mrag/api/ai-agent/refine-skill-pack is needed first.OQ-1a
1bOption label/descriptionCan the endpoint return a short title + description for each option, so the card can display them?The option cards in the design (prototype /bot-automation/ai-agents/customer-support) each need a real title. The proxy currently stamps a hardcoded label because the response carries none.OQ-1b
1cTools registry freshnessDoes the company_tools registry stay in sync with chatbot's AiAgentTool table — i.e. is the empty-list auto-fill authoritative and current?Companion doc §5 already resolves the "must the list be sent?" half: sending an empty available_tools triggers a noncore auto-fill from company_tools. What remains open is whether company_tools is a fresh, complete mirror of AiAgentTool — if it lags, relying on auto-fill would hydrate stale/partial tools, so the proxy may still need to send the list explicitly.OQ-1c
1dInvented / non-existent refsWhen a request references an action or knowledge-base that doesn't exist, does the model include that non-existent reference in the proposed change, or is it cleaned out before the response?The two upstream docs conflict: companion doc §3 says the refine path runs no post-processing / no orphan cleanup (refs survive), while RFC §10.3b says updated_skill_pack runs "the same defensive pipeline as the drafter (gate validation, tone coercion, orphan cleanup)". Which one is as-built decides whether refs reach the tenant. The only chatbot-side guard is a 400 at Save time (CapabilityRefPresence) — its role as sole safety net hinges on this answer.OQ-1d
1eOne option vs severalWill the endpoint return 1–3 alternative options per turn, and if so, on what timeline?The design shows up to 3 options per turn; the endpoint returns exactly one today. At launch the single result is wrapped in a one-item list, but multi-option is the target — a timeline is needed.OQ-1e
1fData retention / PIIHow long does the endpoint retain the user_message / chat_history it receives, and are there PII-handling rules?This is tenant-pasted customer content passing through the endpoint. The companion doc is silent on retention, and an answer is needed to state the data-processing basis under UU PDP.OQ-1f
1gProactive config auditDoes the model flag config problems on its own (not only when asked), and have "audit my config" prompts been tuned/tested?"Audit my configuration" is a named feature (story REFINE-S04) with a planned suggestion chip. The documented model heuristics are all complaint-triggered — if the model never audits proactively, that feature will not behave as designed.OQ-1g
2Multi-turn contextchat_history carries only conversational text ({role, content}), not the prior proposed/accepted patches — is that flat text enough for a follow-up that builds on an earlier change ("now also handle the timeout case"), or does the prior patch state need to be re-supplied?Companion doc §5 + RFC §10.3b already fix the shape: a flat [{role, content}] list, "last ~10 turns for conversational continuity." What is unconfirmed is whether that text-only history lets the model reason about a change it proposed a turn earlier (the patches themselves aren't in chat_history) — this is the actual risk for the multi-turn Refine rail.OQ-11
3API spec / repo accessIs there an OpenAPI/Swagger spec for refine-skill-pack (and related endpoints), or read access to the owning repo?Implementation currently runs against prose in a Confluence companion doc. A real spec or repo access would reduce the risk of the contract drifting during the integration.OQ-12

Needs decision before Beta / GA​

#TopicThe question (plain)Why it matters / backgroundRFC ref
4aKB scope — confirmJSON Patch ops target any pointer, so a sources/vector-store path is technically patchable — is the model actually instructed to propose KB-affecting patches, or does it steer clear of sources in practice?Companion doc §3 (opaque JSON-pointer ops on any path) implies KB paths can be patched; what is unconfirmed is whether the model treats sources as in-scope. This is the prerequisite for 4b — the allow/guard decision only matters if the model will actually emit such a patch.OQ-4
4bKB scope — allow or guard?If yes (4a): should a KB-affecting patch be allowed through, or guarded against?On Save, a KB-affecting change makes the stateful resolver create/purge vector stores (the same behavior a manual edit triggers). A decision is needed before Beta on whether that is acceptable on the refine path or should be blocked in the FE differ + BE validation.OQ-4
5Revert after a bad applyDoes user-facing "undo a bad refine" ship this phase, or does server-side-only revert ship with the UI deferred?PaperTrail is the baseline revert. An older Draft-versioning feature also exists — but it was built for the old AI Engine, never shipped, and would need rework to fit the new Autonomous AI Engine's config model. The decision turns on whether that rework is realistic before GA; if not, the fallback is server-side revert with the UI deferred.OQ-13

Already resolved (no need to re-litigate, FYI only)​

  • trace (workflow_state/recent_turns) — decided to omit entirely from v1; workflow_state lives in the mekari-agent DB and chatbot cannot supply it (OQ-2, closed 2026-07-07).
  • chat_history cap — 10 turns, matching the documented "last ~10 turns" guidance (OQ-3, closed 2026-07-05).

Notes / decisions from the meeting​

(fill in during/after the session — capture the outcome of each topic above, then update RFC §5 and the Comment log accordingly)

#OutcomeFollow-up owner