Skip to main content

Runbook — Available-balance gate & Available-only display (BIF-8742 / T3)

For orgs on the wa_hold_settlement toggle, both send-gates and the displayed WA balance enforce Available = Pooled − Reserved, where Reserved = Σ wa_balance_holds.estimated_amount in held|delivered (settled/refunded/expired no longer reserve). Toggle off ⇒ byte-identical legacy behavior everywhere.

Single source of truth​

Repositories::Billings::Helpers#available_wa_balance(wa_package, organization_package) = balance_initial + balance (+ postpaid_limit, billing-v3 postpaid orgs only) − reserved_wa_balance(package).

The gate and the display both call this helper (via reserved_wa_balance), so the number a client sees is exactly the number the gate enforces — they can never disagree.

What changed where​

SurfaceFileBehavior (toggle ON)
Chat send-gateabstract_iteractor.rb#validate_wa_balanceFailure 'cannot send message, insufficient balance' when available − cost < 0
Broadcast send-gatewa_cloud/repositories/broadcast/send.rb#validate_balancebilling_error('insufficient_balance') when available − cost < 0
Balance display (get usage balance)repositories/billings/report.rbReservation netted out of the displayed pools in deduction-ladder order (balance_initial → balance → postpaid); no separate "Held" line item (PRD AC-2)
Balance display (V2 summary)billings/repositories/v2/reports/summary.rbSame netting on wa.remaining_initial/remaining_additional/remaining_total via the shared Helpers#net_wa_hold_reservation ladder

Display note: an over-commit beyond every pool (in-flight broadcasts racing the gate, PRD AC-3) surfaces as a negative wa_balance rather than being hidden.

Gate semantics note (intended change, RFC "diverges from PRD Non-Goal #4"): the legacy gate passed if ANY single pool covered the cost; the new gate sums the pools and subtracts reservations.

Verify on a pilot org​

# Console — the three numbers that must line up:
include Repositories::Billings::Helpers
org  = Models::Organization.find('<organization_id>')
pkg  = org.package
wapk = find_whatsapp_package(org, pkg)

reserved_wa_balance(pkg)                  # Σ held|delivered estimated_amount
available_wa_balance(wapk, pkg)           # pooled − reserved (what the gate enforces)
Repositories::Billings::Report.new(moderator_account_id: org.moderator_account_id).call
  .success.remaining_balance              # displayed pools must sum to the same available number

Functional check: push a pilot org's available below one message cost (or seed a large hold in staging) → chat send returns cannot send message, insufficient balance, broadcast log gets billing_error: insufficient_balance; the balance page shows the reduced (possibly negative) figure with no Held line.

Performance hardening (IMPLEMENTED with T3)​

reserved_wa_balance runs per send-gate check and delivered holds accumulate until EOD settlement, so the raw SUM degrades linearly through the day for high-volume orgs. Two layers shipped with T3:

  1. Partial composite index (migration 20260707000002):

    add_index :wa_balance_holds, %i[organization_package_id estimated_amount],
              name:  :idx_wbh_active_reservation,
              where: "state IN ('held', 'delivered')"

    Composite key instead of INCLUDE — identical index-only plan for this SUM, and Rails 6.1 can both write and schema-dump it. Settled/refunded/expired rows drop OUT of the partial index, so it only ever contains the active set. Verified:

    Aggregate
      -> Index Only Scan using idx_wbh_active_reservation on wa_balance_holds
           Index Cond: (organization_package_id = '...'::uuid)

  2. Short-TTL Redis cache of the sum — Helpers#reserved_wa_balance reads "#{package_id}::wa_reserved" (REDIS_R), falls back to the DB sum on miss and writes it back with RESERVED_WA_BALANCE_CACHE_TTL (10s). CreateHold calls clear_reserved_wa_balance_cache after every insert, so a fresh hold is visible to the very next gate check; a broadcast burst costs one SUM per package per 10s window instead of one per recipient. Staleness from transitions (T4 refund/settle) is bounded by the TTL and covered by the RFC's accepted gate↔hold race (A-5). T4–T6 may optionally also call clear_reserved_wa_balance_cache on transitions for tighter freshness — not required.

Escalation path if metrics still demand it: maintained Redis counter (INCRBYFLOAT on hold create, decrement on transitions) with a nightly reconcile from the DB sum — O(1) reads but owns drift.

Rollback​

Toggle the org off (extras key removed / global flag off) — both gates and the display instantly revert to the legacy pooled-only reads. No data changes needed.

Sources (grounded against hub_core)​

  • Helper: app/core/domains/repositories/billings/helpers.rb (reserved_wa_balance, available_wa_balance); truth-table + math specs in co-located helpers_spec.rb.
  • Gates: app/core/domains/interactors/abstract_iteractor.rb#validate_wa_balance (spec: abstract_iteractor_spec.rb "Deduction V2 available-balance gate"); app/apps/wa_cloud/repositories/broadcast/send.rb#validate_balance (spec: co-located send_spec.rb "wa_hold_settlement" contexts).
  • Display: shared ladder Repositories::Billings::Helpers#net_wa_hold_reservation (initial → balance → postpaid; over-commit lands as negative balance), applied by app/core/domains/repositories/billings/report.rb#apply_wa_hold_reservation (spec: co-located report_spec.rb) and app/apps/billings/repositories/v2/reports/summary.rb#apply_wa_hold_reservation (spec: co-located summary_spec.rb "Available-only display" context).