Skip to main content

WhatsApp audio/voice note — attachment type field to AI Service

Context​

Dimas Fauzi Hidayat asked whether a voice note from a chat room (http://chatbot-log.qontak.net/?room=0d666821-d1a9-4d74-ac0c-06e3edde81c3) had been delivered to us. This led into a broader discussion about whether attachments sent to the AI Service carry an explicit type (audio/image/video) field, and where WhatsApp voice-note speech-to-text transcription should happen.

Notes​

  • Agus Suparman confirmed the voice note webhook was received; media file was https://cdn.qontak.com/uploads/message/file/09e59929-4fe9-46a0-aedf-b79ba9e89506/1402153128454104.ogg, file type .ogg. Agus noted this is "cuma sebatas webhook aja" (webhook receipt only).
  • Dimas raised whether transcription (speech-to-text) should happen in the AI Service (after we forward the raw audio file) or on our side (transcribe first, then send text). Referenced Meta's WhatsApp audio message docs (https://developers.facebook.com/documentation/business-messaging/whatsapp/messages/audio-messages), which shows Meta can provide a transcript directly.
  • Puji Triwibowo: for consistency, transcription should happen on the AI Service side, since image-to-text (image translation) is already done there.
  • Puji confirmed the .ogg file is already sent to the AI Service as an attachment (screenshot shared in Bitbucket chat, not reproduced here).
  • Open question Dimas raised: as an attachment, does the payload to AI Service include an explicit type field (e.g. "type": "audio", per Meta's outbound message payload shape) so AI Service can distinguish audio/image/video attachments, or does AI Service have to auto-detect from file extension?
  • Puji: currently no explicit type is sent; type could be auto-detected from extension.
  • Dimas: auto-detect risks hurting accuracy; prefers we add an explicit type field so the source of truth is on our side and AI Service just consumes it.
  • Puji asked what the source of truth for that type would be if we added it.
  • Dimas asked whether the payload sent from chat panel to CAA already includes a type — Puji confirmed chat-sent messages currently do include a type (example: https://augflow.triwibowo.com/s/f716342af9eeefc3082c48a70b2635af33bd040847a6d75cf9b4258a43b6b10a), but said other media types still need checking before relying on that as a pattern.
  • Puji added a note about the audio request body directly on chatbot-log.qontak.net for future reference (cc Eko Aprianto, Farras Herdicho, Agus Suparman).
  • Dimas is separately having an AI agent (Opus) analyze the relevant code and the Confluence payload catalogue: "Qontak Chatbot AI (CAA) Webhook Payload Catalogue — Message Types (Chat Bot)" (https://jurnal.atlassian.net/wiki/spaces/QON/pages/51273531691/Qontak+Chatbot+AI+CAA+Webhook+Payload+Catalogue+Message+Types+Chat+Bot), and asked teammates to review it when free.

Decisions / changes​

  • None yet — discussion is still open. Direction leaning toward: (a) transcription stays on the AI Service side (consistent with image handling), (b) explicit attachment type field to be added on our side rather than relying on AI Service auto-detection, but source of truth for that type is still unresolved (Unconfirmed).

Follow-ups​

  • Dimas — confirm with team whether chat-panel-to-CAA payload's type field pattern can be relied on for other media types (per Puji's caveat), or if a new explicit type field needs to be introduced across attachment paths.
  • Team — review the Confluence "Qontak Chatbot AI (CAA) Webhook Payload Catalogue — Message Types" page once Dimas's AI analysis is ready.
  • TBD — decide source of truth for attachment type sent to AI Service (chat panel vs webhook vs derived) and record the decision, likely in an RFC/ADR once settled.