# WhatsApp ↔ Ticket Media Audit

**Project:** `wa_support_node_v1` (Node API + React + Baileys worker)  
**Audit started:** 2026-06-04  
**Last updated:** 2026-06-04  
**Scope:** Map images / files / voice between WhatsApp (Baileys) and the ticket thread; document inbound + outbound gaps; track Trello checklist delivery.

**Related:** `docs/whatsapp_media_milestone1.md`, `docs/TestSprite_PRD_WA_Ticketing.md` (Flow 5, Flow 7, § messages API)

> **Note:** Trello prompt referenced `docs/to-dos.md` — that file does not exist. Use this document and `whatsapp_media_milestone1.md`.

---

## Ticket closure status (Trello)

| Trello checklist item | Status |
|------------------------|--------|
| Inbound media shows on ticket thread | **Done** (image, voice, document when download + MIME OK) |
| Outbound image send from ticket UI | **Done** (images, voice, documents via multipart) |
| Non-text AI path documented | **Done** — [§ AI / talkback path](#ai--talkback-path) |
| Failed sends show `message.send_failed` in UI | **Done** — delivery error on bubble + `message:sent` |
| Channel disconnected blocks send with clear message | **Done** — banner + disabled composer |

**Card deliverable (audit doc + gaps + build order):** **Complete.**

**Before marking Done in Trello:** run migration `020` on existing DBs; smoke-test per [§ Verification](#verification); rebuild `frontend`, `api`, `whatsapp` if testing via Docker.

**Follow-up backlog (not required to close card):** video/sticker, inbound download-failure rows, Office inline preview, AI multimodal, `delivery_status` column, TestSprite regression pack.

---

## Executive summary

**Shipped (Milestone 1 + checklist fixes):**

- **Inbound:** `imageMessage`, `audioMessage` (voice), `documentMessage` → `POST /internal/messages` → Supabase Storage → ticket thread (signed URLs).
- **Outbound:** Staff/portal multipart → `sendMediaMessage` / worker `POST /send` for `image`, `voice_note`, `document`.
- **UI:** Inline images (lightbox), voice `<audio>`, documents (preview modal for PDF/text; download for Office/zip), failed-send styling, disconnected-channel guard.
- **Docs / AI:** Non-text talkback uses placeholders (`[image]`, `[voice_note]`, `[document]`); no vision/audio/file bytes to the LLM.

**Still open (edge cases / out of scope):**

- Video, stickers, unsupported WA types → text only.
- Baileys download failure → placeholder text, **no** `message_type` media row.
- Outbound row saved before WA send (bubble can show delivery failure via `!wa_message_id` + `message:sent`).
- No `delivery_status` DB column (heuristic + socket only).

---

## Checklist (Trello) — detail

| Item | Status | Notes |
|------|--------|-------|
| Inbound media on ticket thread | **Done** | Image, voice, document when `downloadMediaMessage` + MIME validation succeed. Edge: download fail / unsupported type / rejected MIME → see [Inbound failure paths](#inbound-failure-paths-no-attachment-in-db). |
| Outbound image send from ticket UI | **Done** | `TicketDetail.tsx` multipart; worker kinds `image`, `voice_note`, `document`. |
| Non-text AI path documented | **Done** | This doc § AI; `talkbackService.ts` `runAiReply()`. |
| Failed sends in UI | **Done** | `TicketDetail.tsx` + `ticketMessageDelivery.ts`; audit still logs `message.send_failed`. |
| Channel disconnected | **Done** | Banner, disabled composer, `channel:connected` / `channel:disconnected` on ticket cache. |

---

## Supported types (today)

| Direction | Image | Voice note (PTT / audio) | Document / file | Video / sticker |
|-----------|-------|--------------------------|-----------------|-----------------|
| Inbound WA | Yes | Yes (`audioMessage`) | Yes (`documentMessage`) | No |
| Outbound WA (staff) | Yes | Yes | Yes (multipart) | No |
| Portal | Yes | Yes | Yes (multipart) | No |
| Email | No (400 on multipart) | No | No | No |

**Preview in browser (ticket UI):** PDF, `text/plain`, `text/csv`, and images use `TicketMediaPreviewDialog` / lightbox. Word, Excel, PPT, zip → download only.

MIME allowlists: `ticketMediaService.ts` — `IMAGE_MIMES`, `VOICE_MIMES`, `DOCUMENT_MIMES`; `inferMediaKindFromMime()`, `validateInboundMime()`, `validateStaffMime()`.

---

## Flow 1 — Inbound (WhatsApp → ticket thread)

```mermaid
sequenceDiagram
  participant WA as WhatsApp
  participant B as Baileys session.ts
  participant API as API internal.ts
  participant S as Supabase Storage
  participant UI as TicketDetail.tsx

  WA->>B: messages.upsert
  B->>B: buildInboundApiPayload()
  alt imageMessage audioMessage documentMessage
    B->>B: downloadMediaMessage()
    B->>API: POST /internal/messages (base64)
    API->>S: uploadTicketObject()
    API->>API: messagingService.appendMessage()
    API->>UI: message:new + media_signed_url
  else other types
    B->>API: POST text only
  end
```

### Step-by-step (exact symbols)

| # | What | File | Function / route |
|---|------|------|------------------|
| 1 | Receive WA message | `services/whatsapp/src/session.ts` | `sock.ev.on('messages.upsert')` |
| 2 | Skip fromMe / resolve phone | `session.ts` | `resolvePhoneWithSocket()`, `extractInboundCustomerPhone()` |
| 3 | Build payload | `session.ts` | `buildInboundApiPayload()` — `imageMessage`, `documentMessage`, `audioMessage`; else `extractInboundText()` |
| 4 | Download media | `session.ts` | `downloadMediaMessage(msg, 'buffer', {}, ctx)` |
| 5 | POST to API | `session.ts` | `postToApi(apiUrl, '/internal/messages', body)` |
| 6 | Dedup, ticket, contact | `apps/api/src/routes/internal.ts` | `POST /internal/messages` |
| 7 | Store file | `internal.ts` | `decodeBase64ToBuffer`, `validateInboundMime`, `uploadTicketObject` |
| 8 | Persist row | `internal.ts` | `messagingService.appendMessage(..., { messageType, mediaPath, ... })` |
| 9 | Realtime | `apps/api/src/services/messageEmitService.ts` | `emitMessageNew()` → `enrichMessageWithMediaUrl()` |
| 10 | Alerts + AI | `internal.ts` | `notifyInboundAlerts()`, `talkbackService.handleInbound()` / `waSignupTalkbackService` |
| 11 | Staff loads thread | `apps/api/src/routes/tickets.ts` | `GET /:id` — enriched messages |
| 12 | Render | `apps/frontend/src/pages/TicketDetail.tsx` | Image lightbox, audio, document preview/download |

### Inbound failure paths (no attachment in DB)

- Download error → `session.ts` returns `[Image|Document|Voice note — download failed]` as **text only** (no `messageType`).
- Unsupported WA type → `extractInboundText()` → often `[unsupported message type]`.
- MIME not in allowlist → `validateInboundMime()` → 400; message not stored.

---

## Flow 2 — Outbound (ticket UI → WhatsApp)

| # | What | File | Function / route |
|---|------|------|------------------|
| 1 | Pick files / record voice | `TicketDetail.tsx` | `sendMediaFiles()`, image/audio/document inputs, `startRecording()` |
| 2 | Multipart POST | `TicketDetail.tsx` | `POST /tickets/:id/messages` or `/portal/messages` |
| 3 | Parse parts | `apps/api/src/routes/tickets.ts` | `parseStaffMultipartFiles()` — `content`/`caption`, `file` + `filename` |
| 4 | Kind + upload | `tickets.ts` | `requireMediaKindFromMime()`, `uploadTicketObject()` |
| 5 | Insert message (before WA send) | `tickets.ts` | `messagingService.appendMessage()` |
| 6 | Channel gate | `tickets.ts` | `channel.status !== 'connected'` → error + `message:sent` error |
| 7 | Send to worker | `apps/api/src/services/channelService.ts` | `sendMediaMessage()` |
| 8 | Baileys send | `services/whatsapp/src/worker.ts` | `kind`: `text` \| `image` \| `voice_note` \| `document` |
| 9 | Store `wa_message_id` | `tickets.ts` | SQL update on success |
| 10 | Socket | `tickets.ts` | `message:sent` ok/error; `message.send_failed` audit on error |
| 11 | UI | `TicketDetail.tsx` | `message:sent` listener, failed bubble, invalidate on send |

**Portal / email:** Portal skips worker. Email rejects multipart.

---

## Flow 3 — Portal media (reference)

| Direction | File | Entry |
|-----------|------|--------|
| Inbound | `apps/api/src/services/portalService.ts` | `postInboundFromCompanyUserWithFiles()` |
| Route | `apps/api/src/routes/portal.ts` | Multipart `POST /api/v1/portal/messages` |
| Staff on portal ticket | `apps/api/src/routes/tickets.ts` | Multipart, `source: portal` |

---

## AI / talkback path

| Step | File | Behaviour |
|------|------|-----------|
| Trigger | `internal.ts` | `talkbackService.handleInbound({ inboundText: displayText })` |
| `displayText` for media | `internal.ts` | Image: caption or `[Image]`; voice: `[Voice note]`; document: `[Document: filename]` or caption |
| LLM context | `talkbackService.ts` | `runAiReply()` — non-text → `` `[${message_type}] ${content}` `` |
| Outbound AI | `automatedOutboundService.ts` | Text only via `sendMessage()` |
| Onboarding on media | `talkbackService.ts` | `nextOnboardingStep(..., inboundText)` may advance on placeholders like `[Image]` |

**By design:** No file URLs or binary content in prompts. Multimodal AI is a separate feature.

---

## Open gaps (backlog)

| Category | Gap | Severity |
|----------|-----|----------|
| **Inbound** | Video / sticker → unsupported text | Medium |
| **Inbound** | WA download failure → no media row | Medium |
| **Inbound** | Rejected MIME → 400, no message | Low |
| **Outbound** | Message persisted before WA send (failed bubble mitigates) | Low (mitigated in UI) |
| **Outbound** | Same caption on every image in a batch | Low |
| **AI** | Placeholder-only; onboarding on `[Image]` etc. | Medium |
| **Preview** | Word/Excel/PPT not inline in browser | Low (expected) |
| **Errors** | No `delivery_status` column | Medium |
| **Errors** | `media_signed_url` null → empty media UI | Medium |
| **Errors** | Audit `message.send_failed` not linked from bubble | Low |
| **Docs** | PRD path `/internal/inbound` vs actual `/internal/messages` | Low |

**Closed since initial audit (no longer gaps):** documents in/out, failed-send UI, disconnected composer, PDF/text preview.

---

## Recommended build order

1. **Verification** — [§ Verification](#verification) (required before Trello Done).
2. ~~Failed-send UX~~ — 2026-06-04.
3. ~~Disconnected channel UX~~ — 2026-06-04.
4. ~~Inbound/outbound documents~~ — 2026-06-04 (`020`, `session.ts`, worker, UI).
5. ~~PDF/text preview~~ — 2026-06-04 (`TicketMediaPreviewDialog.tsx`).
6. **Inbound download failure** — Optional: persist failed-media indicator.
7. **Video / sticker** — Only if product requires.
8. **TestSprite / automated regression** — Media flows in E2E plan.

---

## Verification

Manual smoke (connected WhatsApp channel):

1. Inbound: image, voice note, PDF document → thread shows media; PDF opens in preview.
2. Outbound: image, voice, PDF from staff composer → customer receives on WA.
3. Disconnect channel → banner, composer disabled; reconnect → send works.
4. Send while disconnected (or stop worker) → outbound bubble shows delivery error.
5. Portal ticket: upload image + PDF (no WA worker).

**Docker:** `docker compose up -d --build frontend api whatsapp` after code changes.

**DB:** `020_message_document_type.sql` on existing databases (see `whatsapp_media_milestone1.md`).

---

## File index

| Area | Path | Primary symbols |
|------|------|-----------------|
| Baileys inbound | `services/whatsapp/src/session.ts` | `buildInboundApiPayload`, `documentMessage` |
| Baileys outbound | `services/whatsapp/src/worker.ts` | `POST /send` (`document` + `fileName`) |
| Internal API | `apps/api/src/routes/internal.ts` | `POST /messages` |
| Staff messages | `apps/api/src/routes/tickets.ts` | `parseStaffMultipartFiles`, `POST /:id/messages` |
| Portal | `apps/api/src/routes/portal.ts`, `portalService.ts` | multipart + filenames |
| WA adapter | `apps/api/src/services/channelService.ts` | `sendMediaMessage` |
| Storage | `apps/api/src/services/ticketMediaService.ts` | `DOCUMENT_MIMES`, `uploadTicketObject` |
| Socket | `apps/api/src/services/messageEmitService.ts` | `emitMessageNew` |
| AI | `apps/api/src/services/talkbackService.ts` | `runAiReply` |
| Schema | `018_message_media.sql`, `020_message_document_type.sql` | `message_content_type` |
| Types | `packages/shared/src/types.ts` | `MessageContentType` |
| Thread UI | `apps/frontend/src/pages/TicketDetail.tsx` | media send/render, delivery, channel status |
| Delivery UX | `apps/frontend/src/lib/ticketMessageDelivery.ts` | failed detection |
| Document label | `apps/frontend/src/lib/documentMessageLabel.ts` | filename display |
| Preview mode | `apps/frontend/src/lib/mediaPreview.ts` | `getMediaPreviewMode` |
| Image lightbox | `apps/frontend/src/components/TicketImageLightbox.tsx` | zoom |
| File preview | `apps/frontend/src/components/TicketMediaPreviewDialog.tsx` | PDF/text iframe |
| Milestone | `docs/whatsapp_media_milestone1.md` | config, limits, migrations |

---

## Environment & limits

See `docs/whatsapp_media_milestone1.md` — `MEDIA_MAX_BYTES`, `MAX_MEDIA_FILES_PER_REQUEST` (10), `MEDIA_SIGNED_URL_TTL_SEC` (default 3600). Inbound JSON `bodyLimit`: ~58MB in `internal.ts`.

---

## Audit change log

| Date | Change |
|------|--------|
| 2026-06-04 | Initial audit (Trello: map WA media ↔ ticket thread). |
| 2026-06-04 | Implemented: failed-send UI, disconnected composer. |
| 2026-06-04 | Implemented: documents in/out + migration `020`. |
| 2026-06-04 | Implemented: PDF/text preview modal. |
| 2026-06-04 | **Doc refresh:** closure status, executive summary, gaps/backlog split, verification section, flows aligned with code. |