Skip to main content

CDP Duplicate Contact Handling (Cleansing + Prevention)

Handle duplicate CDP contacts (same phone / email / chat account within a company) end to end:

  1. Cleanse before migration — remove today's duplicates, keeping one PRIMARY per group and deleting the ARCHIVED ones, driven by the BI datamart mekari_datamart.cdp_duplicate_mapping.
  2. Prevent new duplicates after migration — fix the contact-service create path (root cause: a non-atomic find-then-insert with no unique index, affecting all channels/entrypoints, plus an unwired phone normalizer and weaker dedup gates on the direct-API/bulk paths) so migration lands on data that stays clean.

Cleansing alone is not enough — the create path re-creates duplicates, so the initiative covers both.

Scope Changes​

  • Backend — contact-service: (1) a cleansing script/job that deletes ARCHIVED duplicate contacts (keeps PRIMARY) per company, using the datamart mapping — including merging associations of an ARCHIVED contact into the PRIMARY before delete; (2) an atomic upsert contact-create path
    • duplicate-key→merge + RequestID idempotency to prevent new duplicates.
  • Data — consume the BI-owned mekari_datamart.cdp_duplicate_mapping (duplicate detection + retain/delete decision).
  • Infra — partial unique indexes on the Contact collection (company_sso_id + email / phone / account) as the DB-level safety net (built after cleanse).

PRDs​