Task Breakdown — Postpaid Usage Monthly Scheduler (Backend)
Source RFC:
postpaid-usage-monthly-scheduler.md· Mode: vertical (one task per story/component end-to-end) · Blocked tasks: included (full picture) · Scope: full (bothmoderator-be+hub_core). Repos (paths verified locally):moderator-be(../../../../moderator-be),hub_core(../../../../hub_core). Theqontak_chat_servicehost-app route in Task 1 lands in a third repo[unverified — host repo owner].
Effort Summary
| Task | Story IDs | FE days | BE days | QA days | Total |
|---|---|---|---|---|---|
| T1 — [BE] hub_core bulk summary + chat_service route | PUMS-S01 (data contract) | — | 3 | 0.5 | 3.5 |
| T2 — [BE] Snapshot table + monthly generation cron | PUMS-S01 | — | 3 | 1 | 4 |
| T3 — [BE] Dashboard read/search/filter endpoint | PUMS-S02, S04 | — | 1.5 | 0.5 | 2 |
| T4 — [FE] Modpanel dashboard view + type labels | PUMS-S02, S03 | 2 | — | 0.5 | 2.5 |
| T5 — [FE] Bulk selection + action bar + status toast | PUMS-S05, S06 (UI) | 2 | — | 0.5 | 2.5 |
| T6 — [BE] Export trigger + poll endpoints + size guard | PUMS-S06 (trigger) | — | 2 | 0.5 | 2.5 |
| T7 — [BE] 🚫 Async ZIP worker + TTL cleanup | PUMS-S06 (delivery) | — | 3 | 1 | 4 |
| Grand total | 4 | 12.5 | 4.5 | 21 |
Confidence: medium. Biggest unknowns that could move this: OQ-1 (object storage) fully blocks T7 — if Infra must provision a bucket + client, T7 grows ~1 day; OQ-2-bis/OQ-3 (V3 quota codes,
billing_version "2.0.0") could expand T1's hub_core query; and the chat_service route (T1) lands in theqontak_chat_servicehost app, which is not in the two named repos — if that repo's owners differ, T1 splits across squads. FE estimates assume no Figma (OQ-5) so views are built functionally.
Task 1: [BE] hub_core bulk postpaid-usage summary + chat_service route (PUMS-S01 data contract)
The snapshot cron can fetch, in one paginated call, every postpaid CID's prior-month usage per billing type — including
waba_id, V3 Call Balance/quotas, and the decrypted company name — so it never holds the billing encryption key.
Status: ✅ Actionable (pending OQ-2-bis / OQ-3 confirmation of V3 quota codes + "2.0.0" handling — can build the V1/V3-core path now)
Design reference: n/a — backend only
What to build
A new bulk summary repository/interactor/builder in hub_core that extends the existing DownloadPostpaidMonthlyUsage read (reusing read_from_replica_db + the org join + lockbox decryption), returning paginated JSON of {cid, organization_id, waba_id, company_name, billing_type, usage_value} for all postpaid CIDs in a given year_month. Plus the qontak_chat_service host-app route that mounts it.
Implementation Plan
| Action | File | What changes |
|---|---|---|
| create | hub_core: app/apps/billings/repositories/v1/reports/summary_postpaid_usage.rb | Bulk read: MonthlyUsage (mcc_total→WA, muv_total→MUV) + V3 Call Balance/quotas from organization_package_component_postpaid_limits; org+waba join; pagination |
| create | hub_core: app/apps/billings/interactors/v1/reports/summary_postpaid_usage.rb | Dry contract (year_month, page, per_page); orchestrates repo → builder |
| create | hub_core: app/apps/billings/builders/v1/reports/summary_postpaid_usage.rb | JSON shape incl. waba_id + decrypted company_name + billing_type |
| extend | hub_core: app/core/domains/models/billing/organization_package.rb | (read-only) confirm billing_version/payment_type scoping helpers |
| create | qontak_chat_service: config/routes.rb (host app) [unverified — host repo] | GET /api/core/v1/reports/billing/summary_postpaid_usage → interactor |
| create | hub_core: app/apps/billings/repositories/v1/reports/summary_postpaid_usage_spec.rb | Returns waba_id, V1+V3 rows, decrypted name; pagination; excludes prepaid |
Implementation steps
- Explore — Open
hub_core: app/apps/billings/repositories/v1/reports/download_postpaid_monthly_usage.rband its interactor/builder. Noteread_from_replica_db { Models::Organization.select(:id,:name,:company_id,:moderator_account_id)… }andModels::Billing::MonthlyUsage.where(timestamp: @start_date..@end_date). Openapp/core/domains/models/organization.rb(store_accessor :settings, :waba_id;has_many :waba_accounts) andorganization_package.rb(has_encrypted :organization_name,billing_version,payment_type). - Write failing tests (red) — Create
summary_postpaid_usage_spec.rb; fixtures for a V1 org (WA+MUV) and a V3 org (WA+MUV+Call+quota). Assert eachbilling_typerow,waba_idpresent,company_namedecrypted, prepaid excluded. RunRAILS_ENV=test bundle exec rspec app/apps/billings/repositories/v1/reports/summary_postpaid_usage_spec.rb→ fails. - Scaffold repo — Copy the structure of
DownloadPostpaidMonthlyUsage; filterOrganizationPackage.where(payment_type: 'postpaid'); branch onbilling_version("1.0.0"=V1 WA+MUV; "3.0.0"=V3 +Call+quotas). Gate"2.0.0"behind OQ-3 — exclude explicitly until confirmed. - Add WABA join — Resolve
waba_idfromorganization.settings['waba_id']elsewaba_accounts(OQ-4: pick per-CID single WABA for now). - Add V3 quotas — Read
organization_package_component_postpaid_limitsfor Call Balance + whitelisted quota codes (confirm code set — OQ-2-bis). - Builder + interactor + route — JSON page shape; wire the host-app route with chat-panel auth.
- Go green —
RAILS_ENV=test bundle exec rspec app/apps/billings/repositories/v1/reports/summary_postpaid_usage_spec.rb. - Quality gate —
bundle exec rubocop && bundle exec brakeman.
Acceptance criteria
- Returns one row per postpaid CID per billing type for the requested
year_month(PUMS-S01/AC-3, AC-4). - V1 orgs yield only
WA_BALANCE_V1,MUV_V1; V3 orgs addCALL_BALANCE_V3+ quota codes. - Each row carries a resolved
waba_id(or null when none) and a decryptedcompany_name. - Prepaid orgs are excluded;
"2.0.0"explicitly excluded pending OQ-3. - Response is paginated; endpoint requires the chat-panel auth token (401 without).
Test strategy
RSpec on the repo with V1/V3 org fixtures; key mock = none (real AR against test billing DB); key assertions = exact billing_type set per billing_version, waba_id resolution, decrypted name, prepaid exclusion.
Effort estimate
| Discipline | Days |
|---|---|
| Frontend | — |
| Backend | 3 |
| QA | 0.5 |
| Total | 3.5 |
Assumptions: reuses the existing report's replica-read + org-join; net-new work is the V3 quota query + WABA join + JSON builder + route. Excludes
"2.0.0"(OQ-3).
Run to verify
RAILS_ENV=test bundle exec rspec app/apps/billings/repositories/v1/reports/summary_postpaid_usage_spec.rb && bundle exec rubocop
Depends on
- External confirmation: OQ-2-bis (V3 quota codes), OQ-3 (
"2.0.0"), OQ-4 (WABA cardinality) — core V1/V3 path buildable now. - chat_service route lands in
qontak_chat_servicehost app[unverified — host repo owner].
Task 2: [BE] Snapshot table + monthly generation cron (PUMS-S01)
On the 1st of each month at 02:00 Asia/Jakarta, the system auto-generates and freezes one usage snapshot per CID per billing type for the prior month — so Finance always has static, ready data with no live querying.
Status: ✅ Actionable (consumes Task 1's summary; can develop against a stubbed summary repo until T1 lands)
Design reference: n/a — backend only
What to build
The postpaid_usage_snapshots migration + model in moderator-be's chat_billing DB, a pigeon_get summary-read repository, and the PostpaidUsageSnapshotWorker (per-CID isolation, snapshot_failed logging + >5% alert metric), wired to a sidekiq-cron entry.
Implementation Plan
| Action | File | What changes |
|---|---|---|
| create | moderator-be: db/billing/migrate/<ts>_create_postpaid_usage_snapshots.rb | Table + unique idx (cid,year_month,billing_type) + year_month / search indexes |
| create | moderator-be: app/models/billings/postpaid_usage_snapshot.rb | < Billings::ApplicationRecord; billing_type enum; validations |
| create | moderator-be: app/domains/core/repositories/app_integrations/chat_panel/reports/summary_postpaid_usage.rb | pigeon_get to summary_postpaid_usage + chat-panel auth; paginate |
| create | moderator-be: app/domains/core/workers/sidekiq/postpaid_usage_snapshot_worker.rb | < AbstractWorker, queue: :postpaid_usage_snapshot, retry: 0; per-CID rescue; INSERT … ON CONFLICT DO NOTHING; Datadog metrics |
| extend | moderator-be: config/schedule.yml | postpaid_usage_snapshot_task: cron "0 2 1 * * Asia/Jakarta" |
| create | moderator-be: app/domains/core/workers/sidekiq/postpaid_usage_snapshot_worker_spec.rb | Inserts N rows; per-CID raise → logs + continues; idempotent re-run; flag-off no-op |
Implementation steps
- Explore — Open
app/domains/core/workers/sidekiq/chatbot_ai_reset_monthly_worker.rb+abstract_worker.rb(< Core::Workers::Sidekiq::AbstractWorker,sidekiq_options queue:, retry: 0). Openconfig/schedule.yml(chatbot_ai_reset_monthly_task: cron "0 0 1 * * Asia/Jakarta"). Openapp/domains/core/repositories/app_integrations/chat_panel/reports/download_monthly_postpaid_usage.rb(thepigeon_get+Core::Services::Redis::ChatPanel::Authpattern to copy). Opendb/billing_schema.rb(UUID PKgen_random_uuid(), decimal 15,2) andapp/models/billings/application_record.rb. - Write failing tests (red) — Create the worker spec; stub the summary repo to return 3 CIDs (one raising). Assert: 2 succeed → rows inserted; the raise logs
snapshot_failed(cid,year_month,reason)and the run completes; second run inserts no duplicates; flag OFF → no-op. Runbundle exec rspec app/domains/core/workers/sidekiq/postpaid_usage_snapshot_worker_spec.rb→ fails. - Migration + model — Create the migration in
db/billing/migrate/; runbin/rails db:migrate:billing; add the model underapp/models/billings/. - Summary read repo — Copy
DownloadMonthlyPostpaidUsage; swap path tosummary_postpaid_usage; paginate;parse_response. - Worker — Compute prior month; flag check (
Core::Services::Preference.new.enabled?(:postpaid_usage_scheduler_enabled)); loop pages; per-CIDbegin/rescue→snapshot_failed;ON CONFLICT DO NOTHING; emitmoderator_panel_postpaid_snapshot_generated/_failed; post-run >5% alert metric. - Cron — Add
postpaid_usage_snapshot_tasktoschedule.yml. - Go green — rerun the spec.
- Quality gate —
bundle exec rubocop && bundle exec brakeman.
Acceptance criteria
- Running the worker for a fixture month inserts rows = eligible CIDs × types (PUMS-S01/AC-1).
- A per-CID failure logs
snapshot_failedand the run does not abort (PUMS-S01/ERR-1). - Re-running the same month inserts no duplicates (unique index +
ON CONFLICT). -
snapshot_generatedmetric per CID; >5% failure run emits the alert metric (PUMS-S01/ERR-2). - Flag OFF → worker no-ops.
- Snapshot rows are never updated after insert (frozen — PUMS-S01/AC-2).
Test strategy
RSpec on the worker with a stubbed summary repo; key mock = summary repo returning mixed success/raise; key assertion = run completes with partial inserts + snapshot_failed logged + idempotent re-run.
Effort estimate
| Discipline | Days |
|---|---|
| Frontend | — |
| Backend | 3 |
| QA | 1 |
| Total | 4 |
Assumptions: reuses the monthly-cron + pigeon patterns; money-critical (QA Lane B) so QA weighted higher. No backfill (first run = next 1st-of-month).
Run to verify
bin/rails db:migrate:billing && bundle exec rspec app/domains/core/workers/sidekiq/postpaid_usage_snapshot_worker_spec.rb && bundle exec rubocop
Depends on
- Task 1 (summary endpoint contract) — develop against a stub, integrate when T1 lands.
Task 3: [BE] Dashboard read / search / filter endpoint (PUMS-S02, PUMS-S04)
Finance can load a paginated table of all clients' postpaid usage for a chosen month and search by exact CID or WABA ID — reading from the local frozen snapshots (no cross-service call on page load).
Status: ✅ Actionable (depends on the snapshot table from Task 2)
Design reference: n/a — backend (JSON endpoint; UI in Task 4)
What to build
A GET /api/v1/qontak/chat_panel/reports/postpaid_usage_snapshots action (Pagy 50/page, year_month default-latest, exact-match search_query on CID/WABA), gated by a new Finance permission key.
Implementation Plan
| Action | File | What changes |
|---|---|---|
| extend | moderator-be: app/controllers/api/v1/qontak/chat_panel/reports/reports_controller.rb | Add postpaid_usage_snapshots action + can? + Pagy + Dry::Matcher::ResultMatcher |
| extend | moderator-be: config/routes/api/v1.rb | Add get '/postpaid_usage_snapshots' route |
| create | moderator-be: app/domains/core/use_cases/app_integrations/chat_panel/reports/list_postpaid_usage_snapshots.rb | Query snapshots; filters; default latest month |
| create | DB permission seed billing_postpaid_usage_index | Grant to Finance role |
| create | moderator-be: app/domains/core/use_cases/app_integrations/chat_panel/reports/list_postpaid_usage_snapshots_spec.rb + request spec | 50/page; exact filter; combined filter; empty-state; 403 non-Finance |
Implementation steps
- Explore — Open
reports_controller.rb(download_monthly_postpaid_usage+Dry::Matcher::ResultMatcher,before_action :authenticate_user!). Openapplication_controller.rbcan?→Core::UseCases::Permissions::Check. Find how Pagy is used (include Pagy::Backend). Open the existing reports use case for the contract pattern. - Write failing tests (red) — Request spec: seed 60 snapshots for a month → 50/page +
pagy; exact CID returns subset; CID+year_monthANDs; no-match → empty; missing permission → 403.bundle exec rspec app/.../reports/list_postpaid_usage_snapshots_spec.rb→ fails. - Use case — Query
Billings::PostpaidUsageSnapshot;year_month(default = latestMAX(year_month));search_queryexact match oncidORwaba_id(parameterized). - Controller + route — Add action with
before_action :can?; Pagy paginate; matcher success/failure. - Permission seed — Add
billing_postpaid_usage_index; grant to Finance role. - Go green — rerun specs.
- Quality gate —
bundle exec rubocop && bundle exec brakeman.
Acceptance criteria
- Returns ≤ 50 rows/page with WABA ID, CID, Company Name, Postpaid Type, Year-Month, Report Date (PUMS-S02/AC-1, AC-3).
- No filter → defaults to most recently generated month (PUMS-S02/AC-2).
- Exact CID and exact WABA filters work and combine with
year_month(PUMS-S04/AC-1, AC-2, AC-4). - No-match → empty result; DB error → 422 error state (PUMS-S04/ERR-1, PUMS-S02/ERR-1).
- Non-Finance user → 403 (
can? billing_postpaid_usage_index).
Test strategy
Request spec hitting the endpoint with seeded snapshots; key mock = none (real AR); key assertions = pagination count, exact-match filtering, 403 without permission.
Effort estimate
| Discipline | Days |
|---|---|
| Frontend | — |
| Backend | 1.5 |
| QA | 0.5 |
| Total | 2 |
Assumptions: reuses controller + Pagy + permission patterns; read-only, no new external calls.
Run to verify
bundle exec rspec app/domains/core/use_cases/app_integrations/chat_panel/reports/list_postpaid_usage_snapshots_spec.rb && bundle exec rubocop
Depends on
- Task 2 (snapshot table + model).
Task 4: [FE] Modpanel dashboard view + postpaid-type labels (PUMS-S02, PUMS-S03)
Finance sees the usage data as a clear table at
/modpanel/postpaid-usage, with each row labelled by postpaid type (WA Balance / MUV / Call Balance), plus a month picker and search box.
Status: ✅ Actionable (consumes Task 3's endpoint; can build against a mocked JSON response first)
Design reference: n/a — design pending (OQ-5). Build functionally; visual polish is a follow-up.
What to build
Server-rendered Rails + Hotwire view(s) under app/views/billing/postpaid_usage/: the table with the six columns, a billing_type → label helper (unknown → "Unknown"), the YearMonthPicker + SearchInput (Turbo-driven), and loading/empty/error states.
Implementation Plan
| Action | File | What changes |
|---|---|---|
| create | moderator-be: app/views/billing/postpaid_usage/index.html.erb | Page shell, table, columns, states |
| create | moderator-be: app/views/billing/postpaid_usage/_usage_row.html.erb | Row partial w/ type label |
| create | moderator-be: app/controllers/billing/postpaid_usage_controller.rb [verify namespace] | Web controller rendering the view (reads via the Task 3 endpoint / use case) |
| extend | moderator-be: config/routes.rb (namespace :billing) | /modpanel/postpaid-usage route |
| create | moderator-be: app/helpers/billing/postpaid_usage_helper.rb | postpaid_type_label(billing_type) map + "Unknown" fallback |
| create | moderator-be: app/javascript/controllers/postpaid_usage_table_controller.js [unverified — confirm stimulus dir] | Filter/search → Turbo frame reload |
| create | view/helper specs | Type-label map; states render |
Implementation steps
- Explore — Open an existing
app/views/billing/page + its controller underapp/controllers/billing/to copy the Modpanel layout,before_action :can?, and Hotwire/Turbo frame patterns. Confirm the Stimulus controllers directory (likelyapp/javascript/controllers/). - Write failing tests (red) — Helper spec: each
billing_type→ correct label; unmapped → "Unknown". View/controller spec: 403 without permission; empty month → empty-state copy. Run the relevantbundle exec rspecpath → fails. - Helper —
postpaid_type_labelmap (WA_BALANCE_* → "WA Balance",MUV_* → "MUV",CALL_BALANCE_V3 → "Call Balance", else "Unknown"). - Controller + route —
Billing::PostpaidUsageController#indexbehindcan? billing_postpaid_usage_index; route undernamespace :billing→/modpanel/postpaid-usage. - Views — Table + row partial + the 4 states (loading skeleton, empty, error+retry, success). Mock the data source until Task 3 integrates.
- Stimulus — Wire YearMonthPicker + SearchInput to reload the Turbo frame.
- Go green — rerun specs.
- Quality gate —
make lint(rubocop + JS lint).
Acceptance criteria
-
/modpanel/postpaid-usagerenders the six columns (PUMS-S02/AC-1). - Postpaid type column shows WA Balance / MUV / Call Balance correctly; unmapped → "Unknown" (PUMS-S03/AC-1..AC-3, ERR-1).
- Default view = most recent month; >50 → pagination control visible (PUMS-S02/AC-2, AC-3).
- Empty month → "No usage data available for this period."; load error → retry (PUMS-S02/ERR-1, ERR-2).
- Page hidden / 403 for non-Finance.
Test strategy
Helper unit spec for the label map; controller/view spec for states + authz. Key assertion = label mapping incl. Unknown fallback; empty/error copy.
Effort estimate
| Discipline | Days |
|---|---|
| Frontend | 2 |
| Backend | — |
| QA | 0.5 |
| Total | 2.5 |
Assumptions: server-rendered ERB+Hotwire (no SPA build); no Figma so functional layout only (OQ-5); reuses existing Modpanel layout +
can?.
Run to verify
bundle exec rspec app/helpers/billing/postpaid_usage_helper_spec.rb && make lint
Depends on
- Task 3 (read endpoint) — mock JSON until integrated.
Task 5: [FE] Bulk selection + action bar + status toast (PUMS-S05, PUMS-S06 UI)
Finance can select individual rows or all filtered records across pages, then click "Download All" and watch a status toast until the ZIP link is ready.
Status: ⚠️ Partially blocked — the selection UI + bulk-action bar + toast polling are fully buildable now; the trigger POST wires to Task 6 and the real ZIP link to Task 7 (stub both until then).
Design reference: n/a — design pending (OQ-5).
What to build
A postpaid_usage_selection Stimulus controller: per-row checkboxes, select-all-across-pages (with count), selection persistence across pagination, auto-clear on filter change and on refresh; the BulkActionBar (shows count + DownloadAllButton, disabled at 0); and the ZIPStatusToast that polls the export status endpoint.
Implementation Plan
| Action | File | What changes |
|---|---|---|
| create | moderator-be: app/javascript/controllers/postpaid_usage_selection_controller.js [unverified — confirm stimulus dir] | Selection state, select-all-across-pages, clear-on-filter, count |
| create | moderator-be: app/views/billing/postpaid_usage/_bulk_action_bar.html.erb | Count + DownloadAllButton (disabled when 0) |
| create | moderator-be: app/views/billing/postpaid_usage/_zip_status_toast.html.erb | Generating / link / error states; Stimulus poll |
| extend | moderator-be: app/views/billing/postpaid_usage/_usage_row.html.erb | Row checkbox + select-all header checkbox |
| create | JS/system spec | Select-all count; clear-on-filter; refresh clears; 0 → disabled |
Implementation steps
- Explore — Open the Stimulus controller created in Task 4 and any existing selection/bulk pattern in
app/javascript/controllers/. Confirm how Turbo frames pass current-filter params. - Write failing tests (red) — JS/system spec: clicking select-all sets "N records selected"; changing the month/search clears selection; refresh clears; 0 selected → DownloadAllButton disabled. Run the JS test runner → fails.
- Selection controller — Track selected ids client-side; select-all-across-pages records the active filter (select-all = "all matching filter", not just current page); persist across Turbo navigations; clear on filter-change + unload.
- Bulk action bar + toast partials — Render count; DownloadAllButton; toast with generating/link/error.
- Wire trigger (stub) — On click, POST to the Task 6 endpoint; stub the response until Task 6 lands; poll status (real link arrives via Task 7).
- Go green — rerun JS/system specs.
- Quality gate —
make lint.
Acceptance criteria
- Row checkbox adds to selection; BulkActionBar appears with "1 record selected" (PUMS-S05/AC-1).
- Select-all selects all filter-matching records across pages with total count; toggling again clears (PUMS-S05/AC-2, AC-4).
- Selection persists across pagination; clears on filter change and on refresh (PUMS-S05/AC-3, AC-5, ERR-1).
- DownloadAllButton disabled at 0 selected (PUMS-S06/ERR-3).
- Toast shows "File is generating…" on trigger; updates to link on completion (PUMS-S06/AC-1, AC-2) — pending Task 6/7.
Test strategy
Stimulus/system spec; key mock = stubbed export POST + status poll; key assertion = select-all count, clear-on-filter, disabled-at-zero.
Effort estimate
| Discipline | Days |
|---|---|
| Frontend | 2 |
| Backend | — |
| QA | 0.5 |
| Total | 2.5 |
Assumptions: client-side selection only (no server persistence); reuses the Task 4 Stimulus/Turbo setup.
Run to verify
make lint # + JS test runner once confirmed (e.g. yarn test path)
Depends on
- Task 4 (view/rows). Trigger/poll: Task 6 (endpoints), Task 7 (real ZIP) — stub until then.
Task 6: [BE] Bulk export trigger + poll endpoints + 50 MB size guard (PUMS-S06 trigger)
Clicking "Download All" enqueues a background export job (rejecting selections over 50 MB up front) and returns a
job_idFinance can poll for status.
Status: ✅ Actionable (the trigger/poll/guard layer is independent of where the ZIP is stored)
Design reference: n/a — backend only
What to build
The postpaid_usage_exports migration + model + status enum, the POST (validate ≤ 50 MB, insert pending, enqueue, return job_id) and GET /:job_id (status + link/expiry) endpoints, behind billing_postpaid_usage_download.
Implementation Plan
| Action | File | What changes |
|---|---|---|
| create | moderator-be: db/billing/migrate/<ts>_create_postpaid_usage_exports.rb | Table + unique job_id idx + (status,expires_at) idx |
| create | moderator-be: app/models/billings/postpaid_usage_export.rb | < Billings::ApplicationRecord; status enum |
| extend | moderator-be: app/controllers/api/v1/qontak/chat_panel/reports/reports_controller.rb | postpaid_usage_exports (POST) + :job_id (GET) actions + can? |
| extend | moderator-be: config/routes/api/v1.rb | post '/postpaid_usage_exports', get '/postpaid_usage_exports/:job_id' |
| create | moderator-be: app/domains/core/use_cases/app_integrations/chat_panel/reports/create_postpaid_usage_export.rb, show_postpaid_usage_export.rb | Size estimate + enqueue; status read |
| create | DB permission seed billing_postpaid_usage_download | Grant to Finance role |
| create | request spec | >50 MB → 422; 0 → 422; returns job_id; poll returns status/expiry |
Implementation steps
- Explore — Reopen
reports_controller.rbmatcher pattern; opendb/billing_schema.rbfor dialect; open a worker-enqueue call site (Core::Workers::Sidekiq::…Worker.perform_async(params)). - Write failing tests (red) — Request spec: selection est. >50 MB → 422 +
zip_size_limit_exceededlogged; 0 selected → 422; valid → 202 +job_id+pendingrow; poll returns status; pastexpires_at→expired. Run spec → fails. - Migration + model — Create exports table in
db/billing/migrate/;bin/rails db:migrate:billing; model + status enum +expires_at. - Create use case — Estimate size from selected snapshot count/bytes; if >50 MB →
Failure(:zip_size_limit_exceeded); else insertpending,perform_async(job_id), returnjob_id. - Show use case + endpoints —
GET /:job_idreturns status,download_url(when completed + not expired), orexpired. - Permission seed —
billing_postpaid_usage_download. - Go green — rerun specs.
- Quality gate —
bundle exec rubocop && bundle exec brakeman.
Acceptance criteria
- Selection est. > 50 MB → reject +
zip_size_limit_exceededlogged (PUMS-S06/ERR-1). - 0 selected → 422 (PUMS-S06/ERR-3 server guard).
- Valid trigger →
pendingexport row +bulk_download_triggeredevent +job_id(PUMS-S06/AC-1). - Poll returns current status;
completedreturns a download URL; pastexpires_at→ "expired" (PUMS-S06/AC-4). - Both endpoints require
can? billing_postpaid_usage_download(403 otherwise).
Test strategy
Request spec with Sidekiq in fake mode; key mock = enqueue assertion; key assertions = size-guard rejection, job_id issuance, status/expiry transitions.
Effort estimate
| Discipline | Days |
|---|---|
| Frontend | — |
| Backend | 2 |
| QA | 0.5 |
| Total | 2.5 |
Assumptions: reuses controller/use-case/permission patterns; storage-agnostic (
file_refindirection) so independent of OQ-1.
Run to verify
bin/rails db:migrate:billing && bundle exec rspec app/domains/core/use_cases/app_integrations/chat_panel/reports/create_postpaid_usage_export_spec.rb && bundle exec rubocop
Depends on
- Task 2 (snapshots exist to select). Enqueues Task 7's worker.
Task 7: [BE] 🚫 Async ZIP export worker + TTL cleanup (PUMS-S06 delivery)
The background job packages each selected snapshot into one
.xlsxper CID per type inside a single ZIP, stores it, and serves Finance a download link that expires in 24 hours.
Status: 🚫 Blocked on OQ-1 (object storage). No S3/OSS/GCS client or storage gem exists in moderator-be. Unblock: Infra confirms an object-storage bucket + client (Decision 6 Option A), OR the team accepts the documented fallback (DB/volume blob served via authenticated send_data). The worker logic + caxlsx/rubyzip build are buildable; only the store + presigned-URL delivery step is blocked.
Design reference: n/a — backend only
What to build
PostpaidUsageExportWorker (read selected snapshots → one caxlsx .xlsx per CID per type → rubyzip archive → store → set completed + expires_at 24 h; retry w/ exponential backoff → failed) and a daily PostpaidUsageExportCleanupWorker (mark past-TTL expired, delete object).
Implementation Plan
| Action | File | What changes |
|---|---|---|
| extend | moderator-be: Gemfile | Add gem 'rubyzip' (no zip gem present) |
| create | moderator-be: app/domains/core/workers/sidekiq/postpaid_usage_export_worker.rb | Build XLSX (caxlsx) + ZIP (rubyzip); store; status + file_ref + expires_at; backoff |
| create | moderator-be: app/domains/core/workers/sidekiq/postpaid_usage_export_cleanup_worker.rb | Daily TTL sweep → expired + object delete |
| create | storage client wrapper [blocked — OQ-1] | put_object + presigned_url (or fallback) |
| extend | moderator-be: config/schedule.yml | postpaid_usage_export_cleanup_task daily cron |
| create | worker specs | Exact file naming; status→completed; backoff→failed; cleanup→expired |
Implementation steps
- Explore — Open
chatbot_ai_reset_monthly_worker.rb(worker shape) and confirmcaxlsxusage in the repo. Check howdownload_mcc_v2's existing'zip'path produces an archive (precedent). - Write failing tests (red) — Worker spec: build produces one
.xlsxper CID per type named{CID} {Company Name} {Postpaid Usage YYYY-MM} {Type}.xlsx; success →completed+file_ref+expires_at = +24h+zip_job_completed; storage failure → backoff thenfailed+error_details; cleanup →expired+ delete. → fails. - Add
rubyzip—bundle install. - Builder —
caxlsxper-CID/type XLSX with the exact filename format. - Storage [BLOCKED] — Implement
file_refstore + presigned URL once OQ-1 resolves (Option A bucket/client, or fallback DB/volume +send_data). - Worker + cleanup cron — Status transitions + backoff; daily TTL sweep.
- Go green — rerun specs.
- Quality gate —
bundle exec rubocop && bundle exec brakeman.
Acceptance criteria
- ZIP contains one
.xlsxper CID per type named exactly{CID} {Company Name} {Postpaid Usage YYYY-MM} {Type}.xlsx(PUMS-S06/AC-3). - Success →
completed+zip_job_completed(job_id,user_id,file_size_mb,duration_seconds) (PUMS-S06/AC-2). - Failure → exponential backoff up to MaxFails →
failed+zip_job_failed(PUMS-S06/ERR-2). - Link valid 24 h; past TTL →
expired+ object deleted (PUMS-S06/AC-4). - (blocked) ZIP stored + presigned URL served — pending OQ-1.
Test strategy
Worker spec with snapshot fixtures + stubbed storage; key mock = storage put_object/presigned; key assertion = file naming, status transitions, backoff→failed, cleanup→expired.
Effort estimate
| Discipline | Days |
|---|---|
| Frontend | — |
| Backend | 3 |
| QA | 1 |
| Total | 4 |
Assumptions:
caxlsxalready bundled; +rubyzip; storage client is net-new (OQ-1) — estimate assumes Option A (object storage); fallback (DB/volume) is similar effort.
Run to verify
bundle install && bundle exec rspec app/domains/core/workers/sidekiq/postpaid_usage_export_worker_spec.rb && bundle exec rubocop
Depends on
- Task 6 (export row + enqueue). External: OQ-1 object storage decision (hard blocker).
Ordering rationale
- T1 → T2 is the critical path. Everything reads from snapshots, and snapshots need the hub_core summary contract. Start T1 immediately and push the
qontak_chat_servicehost-app route + OQ-2-bis/OQ-3 confirmations in parallel, since T1 spans a repo outside the two named ones. - T2 unblocks the widest surface (T3 dashboard read, then T4 view, then T5 selection). Land the snapshot table early so FE work (T4/T5) can develop against real local data instead of mocks.
- T3 and T4/T5 can parallelize across BE/FE devs once T2's table exists — T3 (endpoint) and T4 (view) only converge at integration.
- T6 is independent of storage and can land alongside T3 — it's the trigger/poll/guard layer; ship it so the FE (T5) has real endpoints to wire.
- T7 is the only hard-blocked task — escalate OQ-1 (object storage) now; it gates the entire bulk-download payoff (PUMS-S06 delivery). Phase 1a (T1–T6) ships and delivers the dashboard value without it.
Skipped / blocked stories
| Story / Task | Reason |
|---|---|
| T7 — PUMS-S06 ZIP delivery | 🚫 Blocked on OQ-1 — no object-storage client/bucket in moderator-be; needs Infra decision (Decision 6) or the DB/volume fallback. Worker logic buildable; store+presigned step blocked. |
| T1 chat_service route | ⚠️ Lands in qontak_chat_service host app [unverified — outside the two named repos]; confirm owner before scheduling. |
PUMS-S01 V3 quota codes / "2.0.0" | Needs OQ-2-bis + OQ-3 confirmation before the hub_core summary's V3/V2 branches are final (core V1/V3 path buildable now). |