[PRD] Handling Duplicate Contacts Before Migration — Duplicate Contact Cleansing

HEADER BLOCK

Field	Value
PM	Zhelia Alifa
PRD Version	1.3
Status	DRAFT
PRD Type	TECH (Data / Backend)
Epic	TBD
Squad	CDP Squad (+ BI / Data as the mapping owner)
RFC Link	TBD — cleansing script/job design
Datamart	mekari_datamart.cdp_duplicate_mapping (BI-owned)
Parent	PRD CDP Q3 2026
Labels	epic:qontak-cdp | module:customers | feature:duplicate-contact-cleansing
Last Updated	2026-07-24

---

0. Technical Context (TECH Conditional Block)

Technical problem: companies have duplicate CDP contacts (multiple records sharing one phone within a company_sso_id). Migrating as-is propagates the duplicates and corrupts identity resolution + associations. BI already produces the retain/delete mapping (mekari_datamart.cdp_duplicate_mapping); what is missing is an engineering cleansing step — a safe, per-company script that keeps the PRIMARY and deletes the ARCHIVED, run before migration.

Expected outcome: after cleansing a company, exactly one contact per phone per company remains for every READY duplicate group, with zero orphaned associations and zero cross-company deletes — measured in §11.

User-facing UI changes: None built here. This is a backend/data script/job. It emits the per-company cleansing report (DCC-S05); the existing Qontak One migration dashboard (http://qontak-one-migration.qontak.net/?cid={company_id}) renders that report — the dashboard is a separate, already-built surface (a consuming dependency, §13), not UI created by this PRD.

Scope	Owner	Items
PM-owned	Zhelia Alifa (CDP PM)	The keep/delete source of truth (Migration Status); the safety bar (dry-run → backup → soft-then-hard delete, exactly-one-PRIMARY invariant); per-company sequencing before migration; success + rollout gates.
Eng-owned	CDP BE / Data	Where the script runs (contact-service job vs ops script), API-delete vs direct DB (OQ-5), the MergeDataService re-point implementation (DCC-S06), backup mechanism, idempotency.
BI-owned	BI / Data	Duplicate detection + the retain/delete decision + READY recommendation (the datamart — not in this PRD's scope).

---

1. One-liner + Problem

One-liner: Cleanse duplicate CDP contacts (same phone number within a company) before migration — keep one PRIMARY contact per duplicate group and delete the ARCHIVED ones — driven by the BI datamart mekari_datamart.cdp_duplicate_mapping.

Problem: Companies have duplicate contacts in CDP — multiple contact records that share the same phone number within one company (company_sso_id). If the data is migrated as-is, the duplicates propagate into the migrated dataset and corrupt identity resolution and associations (a phone maps to several contact ids). BI already runs duplicate detection and, per duplicate group, decides which contact to retain and which to delete. What is missing is an engineering cleansing step: a script that, per company, deletes the flagged duplicates and keeps the retained one — executed before the migration runs.

Cleansing is not enough on its own. The CDP create path keeps minting new duplicates (root cause: a non-atomic find-then-insert with no unique index; ephemeral dedup key for web_chat). This PRD removes today's duplicates; the prevention of new ones is the sibling PRD Prevent Duplicate Contacts in CDP — Root-Cause Fix. Sequencing: cleanse (this PRD) → build the partial unique index → enable the atomic upsert (prevention PRD) — the unique index cannot build while duplicates still exist.

---

2. Target Users + Persona Context

Persona	Role	Goal	Pain	Workaround
Primary — CDP Data / Platform Engineer	Runs the cleansing before migration	Delete the ARCHIVED duplicates and keep the PRIMARY per company, safely and repeatably	No script exists; must translate the BI mapping into safe, scoped deletes	Manual, error-prone deletes / no cleansing at all
Secondary — BI / Data team	Owns duplicate detection + the retain/delete decision	Provide a reliable duplicate mapping	Already done — the datamart is produced	—
Secondary — Migration owner / PM	Sequences the migration	Migrate clean data so identity resolution and associations are correct	Duplicates would corrupt the migrated dataset	Migrate dirty data, clean up after (costly)

---

3. Background — the BI duplicate mapping

The BI / Data team already has logic to look up duplicate contacts based on phone number and identifier, and to decide, per duplicate group, which contact can be retained vs deleted — factoring in CRM / chat associations, recency, and a duplication score. The output is the datamart:

mekari_datamart.cdp_duplicate_mapping

Engineering owns only the cleansing — a script that reads this mapping and, per company, deletes the ARCHIVED contacts and keeps the PRIMARY. Engineering does not own the duplicate detection or the retain/delete decision (that is BI's).

3.1 Datamart schema — mekari_datamart.cdp_duplicate_mapping

Column	Meaning
Company Sso ID	Tenant key (company_sso_id) the contact belongs to.
Company ID	Numeric company id (the per company id the cleansing is scoped by).
Duplicate Group	The dedupe key {company_sso_id}|{phone_number} — all contacts in a group share this.
Contact ID	The CDP contact _id (Mongo ObjectID) — the record to keep or delete.
Name	Contact name (may differ across records in the same group — see OQ-3).
Last Updated At	Contact last-update timestamp.
Has Crm Association	0/1 — whether the contact has a CRM association.
Has Chatpanel Association	0/1 — whether the contact is linked to chat.
Migration Status	PRIMARY = keep · ARCHIVED = delete (the cleansing decision).
Cluster Size	Number of contacts in the duplicate group.
N Records With Association	Number of contacts in the group that have an association.
Duplication Score	0–100 confidence/similarity (100 = exact duplicate).
Migration Recommendation	READY = safe to cleanse (only act on READY).
Phone Number	Normalized phone — the dedupe identifier.

3.2 Migration Status semantics (the cleansing rule)

• PRIMARY → the contact to KEEP (retained in the database).
• ARCHIVED → the contact to DELETE from the database.

3.3 Example (from the sample export)

Each duplicate group has exactly one PRIMARY and the rest ARCHIVED (all READY, Duplication Score 100):

Company ID	Phone (Duplicate Group)	Contact ID	Name	Migration Status
391113	6285850341642	68bbe5b913c1aa887b260d9f	etak	PRIMARY (keep)
391113	6285850341642	68bbe5b913c1aa887b260da2	etak	ARCHIVED (delete)
391113	6285850341642	68bbe5b913c1aa887b260da0	etak	ARCHIVED (delete)
391113	6285850341642	68bbe5b913c1aa887b260da1	etak	ARCHIVED (delete)
749865	6563372511	67eded2418939bd1b55e23ae	Ayuni Aziz	PRIMARY (keep)
749865	6563372511	67d7ef7b74df57dc9aa4542e	Chris Godfrey	ARCHIVED (delete)
749865	6563372511	67d7ef7974df57dc9aa45411	Iffah Fathin	ARCHIVED (delete)
749865	6563372511	67d7ef79d9f45db31d1be152	Jane Tsai	ARCHIVED (delete)
327385	6282125155055	6865e91b3aeacc710dd665d2	Christopher Harsamto	PRIMARY (keep)
327385	6282125155055	6865e91b5336c0391c4598c8	Bapak Christopher Harsamto	ARCHIVED (delete)

Note (edge case): in the 749865 / 6563372511 group the ARCHIVED records carry different names (Chris Godfrey / Iffah Fathin / Jane Tsai) yet share one phone — phone-based dedupe can group different-named contacts. See OQ-3.

---

4. Non-Goals

1. Not the duplicate detection / retain-delete decision — that logic is owned by BI (the datamart is the input).
2. Not a field-value merge in v1 — the script does not copy custom-field values from ARCHIVED into PRIMARY (PRIMARY's field values stay as-is; see OQ-2). (This is distinct from association re-point — moving conversations / deal-ticket-task-company links / chat_data / crm_data to the PRIMARY — which is in scope and required per the grounded gap §6.4 / DCC-S06, to avoid delete failure and orphaned associations.)
3. Not the migration itself — the migration is a separate initiative; this is the cleansing step that runs before it.
4. No UI — this is a backend script/job, not a user-facing feature.
5. No change to live product dedupe (IdentifierResolver, real-time create) — this is a one-off pre-migration cleanse driven by the BI mapping.

---

Scope Changes

Engineering surfaces this PRD touches (controlled vocab: Backend · Frontend · Mobile · Infra · Data · Design · Docs · None).

• Backend — contact-service: a per-company cleansing script/job that deletes the ARCHIVED duplicate contacts and keeps the PRIMARY. Per the grounded gap (§6.4), for ARCHIVED contacts that carry associations it must re-point / merge the ARCHIVED contact's conversations + deal/ticket/task/company activity-log links + chat_data + crm_data into the PRIMARY before delete (reuse MergeDataService, internal/app/service/merge_data.go) — because the standard delete (DeleteContact, delete_contact.go:32) refuses a contact with an unresolved conversation and does not re-point associations.
• Data — consume the BI-owned mekari_datamart.cdp_duplicate_mapping (duplicate detection + PRIMARY/ARCHIVED decision + Has Crm/Chatpanel Association flags). No change to BI's detection logic.

---

5. Constraints

Constraint	Value
Scope	Runs per company (company_sso_id / Company ID) — never a cross-company delete.
Eligibility	Only act on rows where Migration Recommendation = READY; skip others.
Grouping invariant	Each duplicate group must have exactly one PRIMARY; abort/skip the group if 0 or >1 (safety).
Safety	Dry-run first, backup the ARCHIVED contacts before delete, idempotent re-run.
Delete semantics	Hard-delete vs soft-delete (is_deleted = true) — OQ-1. contact-service has a soft-delete flag (is_deleted, contact/base.go:59).
Sequencing	Executes before the migration for that company (gate).
Auth	Uses the contact-delete permission (CustomersCustomersDeleteKey) if going through the API path.

---

6. Proposed Solution

6.1 Cleansing script (per company)

For each company (company_sso_id), read the mapping and process each duplicate group:

1. Load the datamart rows for the company, filtered to Migration Recommendation = READY.
2. Validate the group: exactly one PRIMARY; if 0 or >1, skip + log (do not delete).
3. Keep the PRIMARY Contact ID (no change).
4. Delete each ARCHIVED Contact ID, scoped to the company.
5. Report counts per company: groups processed, kept, deleted, skipped.

6.2 Delete path (grounded — contact-service)

• DELETE /contacts/{id} — ContactHandler.Delete, guarded by CustomersCustomersDeleteKey (internal/server/rest_router.go:146).
• Bulk: POST /contacts/.../bulk_delete — ContactHandler.BulkDelete (rest_router.go:142).
• Soft-delete flag: IsDeleted *bool bson:"is_deleted,omitempty" (contact/base.go:59) — the segmentation/read layer already filters is_deleted = false, so a soft-delete hides the ARCHIVED contact from all read surfaces even without a hard delete (OQ-1).

6.3 Safety guards

• Dry-run mode — report what would be deleted without deleting.
• Backup — export the ARCHIVED contacts (full docs) before deletion, so a bad group is recoverable.
• Idempotent — re-running skips already-deleted contacts; safe to resume.
• One-PRIMARY validation — never delete a group unless exactly one PRIMARY remains.
• Association guard — if an ARCHIVED contact has Has Crm Association = 1 or Has Chatpanel Association = 1, do not silently delete; re-point the association to the PRIMARY first, or skip + flag (OQ-4). (In the sample all associations are 0.)

6.4 Grounded gap — a plain delete fails or orphans associations (codebase assessment)

The standard delete path — ContactHandler.Delete → ContactService.DeleteContact (internal/app/service/delete_contact.go:32) — has two behaviors that break naive cleansing of ARCHIVED contacts that carry associations (the datamart's Has Crm Association / Has Chatpanel Association / N Records With Association columns exist precisely for these):

1. It refuses to delete a contact with an unresolved conversation. Before deleting, it calls chatClient.CheckUnresolvedConversation(...); if a conversation exists it returns ErrConversationExist and aborts (delete_contact.go:45-53). So an ARCHIVED duplicate with Has Chatpanel Association = 1 (an open chat) cannot be deleted via this path.
2. It does not re-point / merge associations to the PRIMARY. DeleteContact deletes the doc (DeleteByID), logs a destroy activity, and emits a delete event + webhook — but it does not move the ARCHIVED contact's deal/ticket/task/company activity-log links, chat_data (chatrooms / BSUID), or crm_data to the PRIMARY. Those associations are orphaned.

Consequence: for ARCHIVED contacts with associations, the cleansing cannot just call delete — it either fails (open conversation) or silently loses the associations. The cleanse must first re-point / merge the ARCHIVED contact into the PRIMARY — a MergeDataService (internal/app/service/merge_data.go: MergeChatData, MergeAccounts, mergeCustomFields, crm/chat data merge) already merges chat_data / crm_data / custom fields / accounts — and resolve/close its conversations, then delete. This turns OQ-4 into a grounded requirement (story DCC-S06), not just an open question. (In the sample export all rows are Has Association = 0, so those delete cleanly; the full dataset will contain association rows.)

---

7. Behavior

#	Behavior	Entity	Triggered by	Expected behavior	Failure behavior
1	Load mapping for a company	cdp_duplicate_mapping (read)	Cleansing run for company_sso_id	Read rows for the company where Migration Recommendation = READY; group by Duplicate Group.	Datamart unavailable → abort run, alert.
2	Validate group	in-memory	per group	Require exactly one PRIMARY; else skip + log the group.	0 or >1 PRIMARY → skip group, record in report.
3	Delete ARCHIVED	contact	per ARCHIVED Contact ID	Delete the contact scoped to company_sso_id (hard or soft per OQ-1). Keep PRIMARY untouched.	Delete fails → retry; on persistent failure log + continue; contact stays.
4	Backup before delete	export store	before delete	Persist the ARCHIVED contact doc for recovery.	Backup fails → do not delete that contact.
5	Report	run report	end of run	Emit per-company counts (groups, kept, deleted, skipped, errors).	—

---

8. System Flow + User Stories + ACs

8.1 System Flow

1. BI publishes / refreshes mekari_datamart.cdp_duplicate_mapping.
2. For a target company, the cleansing script loads the READY rows and groups them.
3. Each group is validated (exactly one PRIMARY) → PRIMARY kept, ARCHIVED backed up then deleted.
4. A per-company report is produced.
5. Migration for that company proceeds only after cleansing is confirmed.

8.2 User Stories

User Story	Importance	Mockup	Technical Notes	Acceptance Criteria
[DCC-S01] — Consume the duplicate-mapping datamart As a CDP engineer, I want to read the BI cdp_duplicate_mapping per company so that I have the authoritative keep/delete decision.	Must Have	N/A — backend/data job (no UI)	Data fields: • company_sso_id, Duplicate Group, Contact ID, Migration Status, Migration Recommendation (source: mekari_datamart.cdp_duplicate_mapping, BI-owned) Before → After: Before — nothing consumes the mapping. After — the script reads READY rows per company, grouped by Duplicate Group.	• AC-1: Given a company_sso_id, when the script runs, then it loads that company's rows from mekari_datamart.cdp_duplicate_mapping. • AC-2: Given the rows, when filtered, then only Migration Recommendation = READY rows are processed; non-READY are skipped and logged. • AC-3: Given the rows, then they are grouped by Duplicate Group ({company_sso_id}|{phone_number}).
[DCC-S02] — Cleanse: keep PRIMARY, delete ARCHIVED (per company) As a CDP engineer, I want the script to keep the PRIMARY and delete the ARCHIVED contacts per company so that only one contact per phone remains before migration.	Must Have	N/A — backend/data job (no UI)	Data fields: • Contact ID (Mongo _id), Migration Status (PRIMARY/ARCHIVED), company_sso_id Delete path (grounded): DELETE /contacts/{id} → DeleteContact scoped by company (delete_contact.go:39 checks contact.CompanySsoID == companySsoID); bulk via bulk_delete (rest_router.go:142). Before → After: Before — N duplicates per phone. After — exactly one (PRIMARY) remains.	• AC-1: Given a duplicate group with exactly one PRIMARY, when the script runs, then the PRIMARY Contact ID is kept unchanged. • AC-2: Given the same group, then every ARCHIVED Contact ID is deleted, scoped to the company. • AC-3: Given the run completes, then no contact from a different company is affected. • AC-4: Given a phone that had N duplicates, then exactly one contact remains for that phone in the company after the run.
[DCC-S03] — Safety: validation, dry-run, backup, idempotent As a CDP engineer, I want safety guards so that a bad mapping never causes data loss.	Must Have	N/A — backend/data job (no UI)	Notes: dry-run mode; export ARCHIVED doc before delete; idempotent re-run; exactly-one-PRIMARY invariant per group. Soft-delete option: is_deleted (contact/base.go:59) — read layer already filters it (OQ-1).	• AC-1: Given a group with 0 or >1 PRIMARY, when processed, then no delete happens for that group; it is skipped and reported. • AC-2: Given dry-run mode, when the script runs, then it reports what would be deleted without deleting. • AC-3: Given a delete, then the ARCHIVED contact is backed up first; if backup fails, the delete is not performed. • AC-4: Given the script is re-run, then already-deleted contacts are skipped (idempotent) and the result is unchanged.
[DCC-S04] — Sequencing: cleanse before migration As a migration owner, I want cleansing to run before migration so that the migrated data has no duplicates.	Must Have	N/A — backend/data job (no UI)	Notes: cleansing is a gate that must complete (per company) before the migration reads that company's contacts.	• AC-1: Given a company, when migration is triggered, then it proceeds only after cleansing for that company is confirmed complete. • AC-2: Given cleansing has not run (or failed), then migration for that company is blocked / flagged.
[DCC-S05] — Report cleansing results per company As a CDP engineer / PM, I want a per-company report so that the outcome is auditable.	Should Have	Report surfaced on the Qontak One migration dashboard, per company via the cid query param — http://qontak-one-migration.qontak.net/?cid={company_id} (e.g. ?cid=555126).	Report fields: per company — groups processed, kept, deleted, skipped (+reason), errors. Surface: the migration dashboard reads the per-company run report and renders it, keyed by cid (= Company ID).	• AC-1: Given a completed run, then a report shows per company: groups processed, contacts kept, contacts deleted, groups skipped, and errors. • AC-2: Given a skipped group, then the reason (non-READY / 0 or >1 PRIMARY / association guard / backup fail) is recorded. • AC-3: Given a completed run for a company, when the user opens http://qontak-one-migration.qontak.net/?cid={company_id} (e.g. ?cid=555126), then the per-company cleansing report (groups processed, kept, deleted, skipped + reason, errors) is displayed for that cid. • AC-4: Given a cid with no cleansing run yet (or an unknown/empty cid), when the dashboard is opened, then it shows a clear empty / no-data state (not an error).
[DCC-S06] — Re-point / merge associations of ARCHIVED contacts before delete (grounded gap — §6.4) As a CDP engineer, I want an ARCHIVED contact's conversations and associations moved to the PRIMARY before deleting it, so that no association is orphaned and the delete does not fail.	Must Have	N/A — backend/data job (no UI)	Grounded gap: DeleteContact (delete_contact.go:32) (1) rejects a contact with an unresolved conversation — CheckUnresolvedConversation → ErrConversationExist (:45-53), and (2) does not re-point deal/ticket/task/company activity-log links, chat_data, or crm_data to the PRIMARY. Approach: reuse MergeDataService (merge_data.go: MergeChatData, MergeAccounts, mergeCustomFields, crm/chat merge) to merge the ARCHIVED contact into the PRIMARY + resolve/re-point conversations, then delete. Gated on the datamart Has Crm/Chatpanel Association flags. Before → After: Before — a plain delete fails (open chat) or orphans associations. After — associations are merged to PRIMARY, then the ARCHIVED contact is deleted.	— Happy Path — • AC-1: Given an ARCHIVED contact with Has Chatpanel Association = 1 (unresolved conversation), when a plain delete is attempted, then it is rejected (ErrConversationExist); the cleanse must first resolve / re-point the conversation to the PRIMARY. • AC-2: Given an ARCHIVED contact with Has Crm/Chatpanel Association = 1, when cleansed, then its deal/ticket/task/company links, chat_data, and crm_data are merged/re-pointed into the PRIMARY before the ARCHIVED contact is deleted (no orphaned associations). • AC-3: Given the merge + delete completes, then the PRIMARY retains all associations and the ARCHIVED contact is removed. • AC-4: Given an ARCHIVED contact with Has Association = 0, then it is deleted directly (no merge needed). — Error / Unhappy Path — • ERR-1: Given the merge or conversation-resolution fails, then the ARCHIVED contact is not deleted; the group is skipped and reported.

Dependencies: DCC-S06 depends on DCC-S01 (mapping) + DCC-S02 (delete); it gates DCC-S02 for any ARCHIVED contact with associations.

---

9. Rollout

Aspect	Detail
Mode toggle	cleanse_mode: dry-run | soft | hard — the run mode that gates how far a cleanse goes (report-only → soft-delete → hard-delete). Default: dry-run.
Rollout sequence	Per company: (1) dry-run (report only, no deletes) → (2) soft-delete the ARCHIVED (is_deleted = true; the read/segmentation layer already hides them, contact/base.go:59) → (3) hard-delete once verified. DCC-S06 merge/re-point runs before any delete for association rows.
Backward compatibility	Yes. Soft-delete is reversible; PRIMARY is untouched; read surfaces already filter is_deleted. Hard-delete only after the soft-delete window is verified clean.
Backout / rollback	ARCHIVED contacts are backed up before delete (§6.3); a bad group is restored from the backup. Soft-deleted contacts are restored by clearing is_deleted. Per-company, no global cutover.
Sequencing gate	Cleansing for a company must complete (and be confirmed) before the migration reads that company's contacts (DCC-S04).

---

10. Observability

Events / run metrics

Event	Trigger	Properties
cleanse_group_processed	Each duplicate group handled	company_sso_id, duplicate_group, kept (1), deleted (n), outcome
cleanse_association_merged	DCC-S06 merge/re-point of an ARCHIVED contact into PRIMARY	company_sso_id, archived_contact_id, primary_contact_id, has_crm, has_chat
cleanse_backup_written	ARCHIVED contact backed up before delete	company_sso_id, contact_id
cleanse_group_skipped	Group not acted on	company_sso_id, duplicate_group, reason (non-READY / ≠1 PRIMARY / merge-fail / backup-fail)

Alerts

Condition	Threshold	Routing
Group with ≠1 PRIMARY encountered	any (≥1)	Halt the run + page CDP Data/BE — safety invariant breach
Backup failure before a delete	any (≥1)	Halt that delete; page — never delete unbacked-up
Merge / conversation-resolution failure (DCC-S06)	rate > 0 on association rows	Page CDP BE + CRM/Chat — ARCHIVED left intact, group skipped
Cross-company delete attempt	any (≥1)	Hard stop — scope invariant breach

Dashboard owner: CDP Data / BE squad. Post-run monitoring: review the per-company run report (§6.5 / DCC-S05) immediately after each run; keep soft-deleted contacts observable for a verification window before hard-delete.

---

11. Success Metrics

Category	Metric	Definition	Baseline	Target
Quality ⭐	⭐ READY groups reduced to 1 contact / phone / company	% of READY duplicate groups left with exactly one (PRIMARY) contact after the run	Datamart Cluster Size per group (today N > 1)	100% of READY groups per cleansed company
Safety	Orphaned associations	ARCHIVED-with-association rows deleted without a DCC-S06 merge/re-point	0 target	0
Safety	Cross-company deletes	Contacts deleted outside the target company_sso_id	0 target	0
Safety	Backup coverage	% of deleted ARCHIVED contacts backed up before delete	—	100%
Safety	Wrong-group deletes	Groups deleted with ≠1 PRIMARY	0 target	0 (invariant halts the run)
Efficiency	Cleanse completeness before migration	% of target companies cleansed before their migration runs	—	100% (sequencing gate, DCC-S04)

⭐ Primary KPI = 100% of READY groups reduced to one contact per phone per company, with 0 orphaned associations and 0 cross-company deletes. This is the Gate-2 substitute for this UI-less TECH PRD.

---

12. Launch Plan & Stage Gates

Stage	Audience	Duration	Success gate
1 — Dry-run	1 company, report-only (cleanse_mode: dry-run)	1 run	Report matches the datamart (groups/kept/deleted counts); 0 ≠1-PRIMARY anomalies; association rows correctly flagged for DCC-S06
2 — Soft-delete	small cohort of companies (cleanse_mode: soft)	1 week verify window	0 orphaned associations; ARCHIVED hidden from all read surfaces; PRIMARY intact; fully reversible
3 — Hard-delete / GA	widen per company (cleanse_mode: hard)	until all target companies cleansed	Sustained 0 safety-KPI violations (§11); each company cleansed before its migration (DCC-S04)

Each gate references the §11 ⭐ + safety metrics and the §10 alerts. Failing a gate → hold; a bad group is restored from backup (§9) before continuing.

---

13. Dependencies

Dependency	Owner	Deliverable	Blocking?
mekari_datamart.cdp_duplicate_mapping	BI / Data	Duplicate detection + PRIMARY/ARCHIVED decision + READY recommendation	YES (input)
Contact delete	CDP Backend (contact-service)	Delete-by-id / bulk-delete (or a direct DB op), scoped by company; soft vs hard (OQ-1)	YES
Backup / export of ARCHIVED contacts	CDP Backend	Recoverable snapshot before delete	YES (safety)
Migration sequencing	Migration owner	Cleansing runs before migration per company	YES
Association re-point / merge before delete (grounded gap, §6.4 / DCC-S06)	CDP Backend + CRM/Chat	Merge the ARCHIVED contact's conversations + deal/ticket/task/company links + chat_data + crm_data into the PRIMARY (reuse MergeDataService) and resolve conversations before delete — the standard DeleteContact rejects open conversations and doesn't re-point associations.	YES for any ARCHIVED row with Has Crm/Chatpanel Association = 1
Qontak One migration dashboard renders the per-company report (DCC-S05)	Qontak One / Migration dashboard owner	Read the per-company cleansing report and display it at http://qontak-one-migration.qontak.net/?cid={company_id} (keyed by cid = Company ID), incl. an empty/no-data state	YES (for DCC-S05 display; the cleanse run itself is unaffected)

---

14. Key Decisions + Alternatives Rejected

14a — Decisions Made

ID	Decision	Rationale
D-1	BI owns detection; engineering owns cleansing.	BI already produces the mapping; engineering only executes keep/delete safely.
D-2	Drive keep/delete from Migration Status (PRIMARY keep / ARCHIVED delete).	Single source of truth from the datamart.
D-3	Per-company scoping, READY-only, exactly-one-PRIMARY invariant.	Bounds blast radius and prevents accidental over-deletion.

14b — Alternatives Rejected

Alternative	Why rejected
Delete duplicates live via product dedupe	This is a one-off, BI-decided pre-migration cleanse; the live resolver is a different mechanism.
Migrate dirty, dedupe after	Duplicates corrupt identity resolution/associations post-migration; far costlier to fix.

---

15. Open Questions

#	Type	Question	Default / Mitigation	Owner	Deadline
OQ-1	Decision	Hard-delete or soft-delete (is_deleted = true)?	Default: soft-delete first (reversible; read layer already hides is_deleted), hard-delete later once verified.	CDP BE + Data	2026-08-07 (RFC)
OQ-2	Decision	Field merge — should the PRIMARY inherit any values the ARCHIVED had before delete?	Default v1: no merge (keep PRIMARY as-is). Revisit if data loss is a concern.	PM + Data	2026-08-07 (RFC)
OQ-3	Assumption	Different names, same phone (e.g. Chris Godfrey / Iffah Fathin / Jane Tsai on one phone) — true duplicates or a shared/misused phone?	BI to confirm the confidence; the script trusts Migration Status + READY.	BI / Data	2026-08-07 (before cleanse)
OQ-4	Risk	ARCHIVED with associations (Has Crm/Chatpanel Association = 1) — delete, or re-point to PRIMARY first?	Grounded (§6.4 / DCC-S06): a plain delete fails on an open conversation (ErrConversationExist) and orphans associations — so the cleanse must merge/re-point into PRIMARY (reuse MergeDataService) then delete. No longer just an open question — it is a build requirement.	CDP BE + CRM/Chat	2026-08-07 (build req)
OQ-5	Decision	Where the script runs — contact-service job vs an ops/data script; via API delete vs direct DB.	Decide in the RFC; API path gives permissioned, auditable deletes.	CDP BE	2026-08-07 (RFC)

---

PRD CHANGELOG

Version	Date	By	Section	Type	Summary
1.3	2026-07-25	Zhelia Alifa	§0, §8.2 (DCC-S05), §13	UPDATED	DCC-S05 report surface. Added AC-3 (the per-company cleansing report is displayed on the Qontak One migration dashboard at http://qontak-one-migration.qontak.net/?cid={company_id}, e.g. ?cid=555126) and AC-4 (empty/no-data state for a cid with no run). Updated the DCC-S05 Mockup/Technical-Notes to name that surface; clarified §0 "UI: none built here — the report is rendered by the existing migration dashboard (a consuming dependency)"; added the dashboard as a §13 dependency. No change to the cleanse logic.
1.2	2026-07-24	Zhelia Alifa (drafted w/ AI, grounded)	§0 (new), §9-§12 (new), §15	UPDATED	Added the mandatory TECH operational sections flagged by score-prd v3.3 (Layer 1) — bringing this PRD to parity with the sibling prevention PRD. New: §0 Technical Context (PM/Eng/BI-owned split + "UI changes: None"); §9 Rollout (cleanse_mode: dry-run|soft|hard, per-company sequence, backup/backout, sequencing gate); §10 Observability (4 run events + 4 safety alerts incl. ≠1-PRIMARY halt + cross-company hard-stop + owner + post-run cadence); §11 Success Metrics (⭐ 100% READY groups → 1 contact/phone/company + safety KPIs: 0 orphaned associations / 0 cross-company deletes / 100% backup coverage); §12 Launch Plan (Dry-run → Soft-delete → Hard-delete/GA stage-gates). Added Type + Deadline columns to §15. Fixed the v1.1 changelog row's unescaped pipes (broke the table on Confluence). No change to §6.4 grounded gap, stories, or decisions.
1.1	2026-07-24	write-prd (targeted)	Scope Changes, §6.4, §8.2, §13, §15	UPDATED	Latest write-prd template pass: added a ## Scope Changes body section; split the §8.2 story table into 5 columns (User Story | Importance | Mockup | Technical Notes | Acceptance Criteria); added a grounded codebase gap (§6.4) — the standard delete (DeleteContact, delete_contact.go:32) rejects contacts with an unresolved conversation (CheckUnresolvedConversation → ErrConversationExist, :45-53) and does not re-point/merge associations (deal/ticket/task/company activity-log links, chat_data, crm_data) to the PRIMARY — so ARCHIVED contacts with associations must be merged into PRIMARY (reuse MergeDataService, merge_data.go) then deleted. Added story DCC-S06, upgraded §13 dependency + OQ-4 from open-question to grounded requirement.
1.0	2026-07-24	Zhelia Alifa (drafted w/ AI, grounded)	All	CREATED	First PRD for pre-migration duplicate contact cleansing, driven by the BI datamart mekari_datamart.cdp_duplicate_mapping. Defines the datamart schema (14 columns) + the keep/delete rule (Migration Status: PRIMARY keep, ARCHIVED delete), the per-company cleansing script (READY-only, exactly-one-PRIMARY invariant, dry-run + backup + idempotent), grounded delete path in contact-service (DELETE /contacts/{id} rest_router.go:146; soft-delete is_deleted contact/base.go:59), user stories DCC-S01..S05 with ACs, dependencies, decisions, and open questions (soft-vs-hard delete, field merge, same-phone-different-name, ARCHIVED-with-associations).