Skip to main content

GDPR & Privacy

Audience: Engineering · Support · Management · Where in app: Leads drawer (erase / export), Chats & Calls (delete), Settings → Organization → Recording Privacy, Admin → Settings (Deleted Conversations, Recording Retention), Admin → Companies (GDPR Protection, masking v2, established region) · Plan availability: All plans

Shipped 2026-11-09 → 11-16 across backend, frontend, widget, ms-sessions, ms-communication and ms-ai. Customer page: docs-site settings/privacy-gdpr.

1. GDPR safe mode (all companies)​

Builds on the earlier person-level block (see Identity Graph › GDPR gate); backend 99b94dcc6.

Who is restricted. app/lib/GdprGeo.js RESTRICTED_COUNTRIES (EU27 + IS/LI/NO + GB + CH); an empty/unknown country is restricted. lib/visitorCountry.js is the one shared check: it looks up the first x-forwarded-for hop and request.ip (restricted if either is), one retry, LOOKUP_TIMEOUT_MS 3000, private IPs ignored. Used by GET /company/:id (returns identification_restricted to the loader) and POST /user-session/create, which stores user_sessions.gdpr_restricted (server-set; a client value is deleted). Pre-feature sessions have no flag = not restricted.

Switch. companies.gdpr_protection_enabled (default true; only a literal false opens it). PUT /company/:_id/gdpr-protection — super admin only; turning it off needs {confirmed: true, confirmation_note} (note ≥ 20 chars, else 422) and writes companies.gdpr_override {note ≤1000, by, by_email, at} (cleared when turned back on; returned only to super admins via withoutGdprOverride). Admin company detail: "GDPR Protection" switch, off-form checkbox "The customer has confirmed this in writing." + "Where the confirmation is recorded (at least 20 characters)". GDPR/recording flags are in DEDICATED_ROUTE_COMPANY_FIELDS — the generic company PUT can't write them.

What waits for an explicit grant (backend/public/widget.js ConsentGate):

GateRestricted visitor needsEveryone else
recordingAllowed()explicit behaviour grantbehaviourAllowed(); a site/CMP "no" blocks
formCaptureAllowed()explicit identification grantallowed unless a site/CMP "no"
fingerprintAllowed()explicit identification grantidentificationAllowed()
Device storagekkStoreMode pending → allowed / anonymous; anonymous writes nothing new to sessionStorage (kk_tab_id, _kk_visit_history, kk_consent_<companyId>) — except a visitor who really used chat/call (kkEngaged) keeps the session across pagesunchanged

This applies whether or not consent_gating is on — with gating off and no CMP, restricted visitors are never asked and stay unrecorded (hence the dashboard notice below). Snitcher (company-level) stays on; anonymous visits reuse a lookup for 1 h via snitcher_reuse_cache (sha256(company_id:ip), no raw IP, TTL 3600 s).

Explicit = kkExplicitGrant(cat): kkConsentExplicit && !kkConsentWeak && knockKnockConsent[cat] === true. Sources: banner, stored, site (preset window.knockKnockConsent or kkSetConsent), cmp_cookiebot (Cookiebot.hasResponse; marketing → identification, statistics → behaviour), cmp_onetrust (OnetrustActiveGroups + OptanonAlertBoxClosed cookie; C0004 → identification, C0002 → behaviour; a missing group is a "no" only when C0001 is present), cmp_gcm (dataLayer consent update; ad_storage → identification, analytics_storage → behaviour; an update without a prior denied default is weak — counts as no, never as a grant). Re-read on load and on CookiebotOnAccept/Decline/ConsentReady, OneTrustGroupsUpdated and dataLayer.push. The non-blocking banner's default-accept (source: 'default') is never explicit. Our banner waits ~1.5 s + window load and stays hidden if a CMP appears (_showBannerUnlessCmp).

A "no" for everyone. _siteAnswered() && kkConsentGiven.<cat> === false blocks recording/forms for any visitor even with our banner off; applyCaptureGates() calls stopRecording() (window.__kkRecordingOff) and sets window.__kkFormCaptureOff mid-page. A restricted visitor's explicit identification "no" also wipes device storage (kkWipeDeviceStore).

Server backstops. ms-sessions SocketService.canRecord refuses gdpr_restricted sessions unless consent.explicit && consent.behaviour (allow cached 5 min RECORD_ALLOW_TTL_MS, refusal 30 s). FormSubmissionService drops form_intercept posts when consentRecord.formCaptureBlocked (masking v1 path); FormEventController stores nothing when recordingBlocked(session).

Anonymised storage (anonymizeSessionBody, restricted visit without an explicit identification grant): gdpr_anonymous: true, user_ip/ip_hash cleared; user_session_id, widget_user_session_id, fp_hash, soft_signature_hash, every *person_id/shadow id, email, name, phone_number, image removed; identity_trust: 'anonymous', user_meta: {}; ip_info cut to country/countryCode, knox.geo deleted; page_url loses query + fragment; traffic_source.gclid/.referrer deleted. Left: country, coarse device, page path, UTMs. gdpr_anonymous sessions are never claimed by device/IP recognition.

  • POST /user-session/:id/consent (public) {company_id, consent: {identification, behaviour, explicit, source, cmp}}; lib/consentRecord.sanitize whitelists source (banner|stored|site|default|none|cmp_cookiebot|cmp_onetrust|cmp_gcm) and cmp (cookiebot|onetrust|gcm) — an unknown source becomes none and voids every grant.
  • recordConsent (UserSessionController) dedupes against user_sessions.consent (sameDecision on identification/behaviour/explicit/source), writes user_sessions.consent {…, at}, and appends a consent_events row only when explicit. At session create a row is logged only for a restricted, explicit, non-stored decision.
  • consent_events (ConsentEventModel, append-only): company_id, user_session_id, identification, behaviour, explicit, source, cmp, banner_version (first 12 hex of sha256 of the resolved banner texts), gdpr_restricted, country_code, ip_hash, user_agent (≤300), created_at; index {user_session_id: 1, created_at: -1}.
  • Withdrawal (withdrawalUpdate: restricted, not yet anonymous, explicit identification: false): $set {gdpr_anonymous: true, user_ip: '', ip_hash: ''}, $unset {fp_hash, soft_signature_hash, widget_user_session_id}; person links from a form/chat are kept.
  • A late grant calls grant-tracking, which uploads fp_hash + soft signature and forces retryRecognition.

Dashboard notice​

companies.consent_signals {restricted_seen_at, cmp, cmp_seen_at} (lib/consentSignals.js, written from session create and the consent endpoint, ≤ once per company per field per hour). GdprPausedNotice.vue (utils/gdprNotice.js) shows when protection is on, restricted_seen_at is within 30 days, consent_gating.enabled !== true, and no CMP was seen in 30 days. Title "EU visitors: recording and form capture paused"; link "Turn on the consent banner" (→ settings-widget?tab=install) for users with settings.widget and agency client access, else "Ask your account owner or agency to turn on the consent banner". On DashboardHome.vue (hidden while paused) and the Install tab (button "Show consent gating settings" instead of the link).

2. Recording masking v2 (per company, off by default)​

companies.recording_masking_v2 — PUT /company/:_id/recording-masking-v2 {enabled}, super admin (AdminRecordingPrivacyToggles.vue: "Recording masking v2").

  • Shared rrweb config backend/public/recording-config.js (loaded only when the flag is on; globals kkRecordingOptions, kkSetRecordingConfig): every typed value masked per character (maskAllInputs: false + maskInputFn; date/time/hidden/select masked, checkbox/radio not), shown only when its field key is allowlisted or inside data-kk-unmask; data-kk-mask always masks (text too); contenteditable masked unless under data-kk-unmask; the widget hosts are blocked (#kka-chat-widget, #kk-consent-root, #kk-permission-arrow, [data-knockknock-widget]). Console text is no longer recorded — errors become kk-error custom events {level, type, stack frames, source} with no message, ≤ 50 per page (recoded_session.js).
  • Never-unmask (app/lib/neverUnmask.js, mirrored in the browser): type=password, autocomplete cc-*, names matching ssn/passport/tax id/national id/driver's licence/iban/sort code/routing/account number/cvv/cvc/pin, and card-like values (13–19 digits, browser only).
  • Field discovery — inputs/textareas with a name or id are reported to POST /recording/fields (public, rate-limited, top window only, ≤ 200 per page, ≤ every 5 s, stops after a consent stop). recording_fields (RecordingFieldModel, one row per company_id + field_key; path, form_id, field_name, field_id, label, type, autocomplete, never_unmask (sticky), first_seen_at, last_seen_at, seen_count; never values; ≤ 1000 per company). Key (lib/recordingFieldKey.js): lower(path)|lower(formId||'-')|lower(name||id). Backfill: scripts/backfill-recording-fields.js (dry-run default; local-dev only — prod needs a migration endpoint).
  • Allowlist (lib/recordingAllowlist.js): company recording.unmasked_fields[] + recording.use_agency_default (default true); agency recording_default_unmasked_fields[] + recording_default_unmasked_fields_locked. Resolution: locked agency list → company list (when use_agency_default === false) → agency default → company list (no agency). Max 500.
  • Routes (routes/recordingRoutes.js): GET /company/:_id/recording/fields; PUT /company/:_id/recording/unmasked-fields {unmasked_fields, use_agency_default} (409 "Your agency manages the visible fields for this account" when locked; 400 with rejected for never-unmask keys); PUT /agencies/:id/recording/default-unmasked-fields {fields, locked}. Company writes: super admin, owner, members with settings.organization, agency owners, agency staff with agency.manage_settings or agency.edit_clients.
  • UI: Settings → Organization → Recording Privacy (/settings/organization/recording, settings.organization; menu entry hidden unless the flag is on, page says "Contact support to turn it on" otherwise); agency Recording Privacy (/agency/settings/recording, agency.manage_settings, "Lock for sub-accounts", "Pick from a sub-account").
  • Form capture on real submit only — form_intercept.js captures values only on a native submit or a provider "form sent" message; submit-like clicks/Enter only mark the form sent. Payload carries gpc (navigator.globalPrivacyControl) and form_key.
  • Form events — POST /api/intake/events (alias /form-events), public, rate-limited, never accepts values, ≤ 1000 per session. form_events (FormEventModel, 30-day TTL): company_id, user_session_id, form_key, type (form_viewed | field_focused | field_completed | last_field_before_exit | form_submitted), field_key, completed, page_path, at. FormEventAbandonmentJob (5 min, flagged companies only; session idle > 15 min, 24 h lookback; once per form per session via form_abandon_states.fired_forms; Redis lock job:form-event-abandonment) fires form_abandoned through FormAbandonmentService; ms-sessions skips its recording-based trigger for these companies.
  • Pooling (lib/canPoolIdentity.js, only for flagged companies): restricted & not consented → 'none' (unlinked form_leads row — no session/person/IP/full URL, deduped; Leads id f:<id>); restricted & consented → 'company'; company or agency established_in_restricted_region → 'company'; Sec-GPC/gpc → 'company'; else 'global'. 'company' = linked to this session + this company's profile only, never the shared people graph. established_in_restricted_region (companies + agencies, default false): PUT /company/:_id/established-region, PUT /agencies/:id/established-region {enabled}, super admin.
  • Widget iframe (widget ff382f6): the chat app's own rrweb recorder was removed — the widget is never recorded (Nox reads chats from the DB). setUserInteraction({notifyParent}) tells the host page only about visitor-initiated opens (that signal keeps an anonymous visitor's session across pages). Backend public/widget-iframe.js (customer cross-origin iframes) still records; it asks the top page for the masking config and under v2 masks every typed value, ignoring the allowlist — it has no consent check of its own.
  • No typed values in analysis (ms-sessions e060401, dom-resolver.js): attributes.value dropped except as a button / submit/button/reset/image input caption; textarea/contenteditable text never becomes textContent, a label or accessible name (falls back to name); extractors/form.js keeps length buckets only; extractors/error.js reads rrweb console events and kk-error.

3. Deleting chats and calls​

Backend 91f104fe3, ms-comm 002f914, ms-ai a753f6805, frontend fd4cfc77, widget 4b2753c.

  • Who (lib/conversationDeletion.js canDeleteConversations, mirrored in ms-comm canDeleteCalls and frontend useConversationDelete.js): admin, web_owner, agency; agency_staff only with agency.edit_clients or agency.add_client_subscription. Never team members — there is no member permission for it.
  • Endpoints: DELETE /chats/:id (ChatController@destroy, also canManageCompanyId); widget call DELETE /remove-record/:id (ServiceRoomController@deleteRecord — video_deleted, deleted_by, $min deleted_at); phone call ms-comm DELETE /inbound-outbound-calls/:_id (deleted_at, first date kept).
  • Chat delete is a detach: $rename of company_id, user_session_id, agent_id, messages, previous_chats, last_message_at under deleted_from.* + deleted_at/deleted_by, so every reader, count and sweep misses it with no filter. Then: removed from chat_queue; Firestore copy deleted (retried by the job); ms-comm POST /internal/chats/deleted re-broadcasts queue positions, clears chat-<session>-agent-connected, and emits chat-deleted to the visitor and company rooms (dashboard refreshes silently, App.vue). Widget dropDeletedConversation() (on chat-deleted or HTTP 410) clears chat state, previous messages, voice links and the unread badge so deleted turns are never resent as AI context; the next message opens a fresh chat.
  • Hidden from: Leads rows + lead transcript API (a deleted call no longer makes a lead), DailyRecapEmailJob, Nox (KnoxV2InternalController), AI call transcripts per session, call lists, identity-graph source trust, recording proxies (Ultravox proxy 404s; ms-comm _isDeletedRecording for ElevenLabs links). Super admins still see deleted rooms in one admin list.
  • UI: Chats header Delete (title "Delete chat"); Calls CallThread More menu whose only item is Delete call (menu hidden for non-deleters). Dialog text from GET /settings/conversation-retention/days (any logged-in user): purge on → "…removed from your account now and permanently deleted, with its recording and transcript, after N days. This can't be undone."; off → "…removed from your account. This can't be undone."

Permanent purge — ConversationPurgeJob (every 15 min, schedular)​

  • Setting conversation_retention (settings collection): enabled (default false), retention_days (default 30, 8–365 — the floor protects ms-ai's 7-day ElevenLabs missed-call billing catch-up), erasure_days (default 8, 8–21); the job writes totals, last_run, last_error. GET/PUT /settings/conversation-retention super admin; card Admin → Settings → Deleted Conversations (tiles Waiting / Past the window now / Removed for good / Failing (retrying) / Status; switch "Remove deleted conversations for good"; "Keep deleted conversations for (days)"; "Erase person (GDPR): delete at vendors after (days)").
  • Per run: 50 items per collection, 5-min budget, backoff 1 h → 24 h cap. Order: erasures → copies (always, even with the switch off: Firestore + ms-ai transcript copies via POST /knox-signals/transcripts/purge, ≤ 500 ids) → chats / widget calls / phone calls past the window (only when enabled).
  • Purge: Firestore chat/call copies + live transcripts; voice-engine audio on S3 then ms-ai POST /voice-engine/sessions/:id/recording/purged (nulls recording_*, row kept for billing); vendor media via ms-comm POST /internal/purge/call-media — widget calls: Twilio Video composition + room recordings, Ultravox call, ElevenLabs conversation; phone calls: Twilio call recording (main account, then the company's own token), ElevenLabs conversation, Twilio call log (CA…, best effort). The Mongo doc is deleted only after every step succeeds. Telnyx and Vapi media are not purged.
  • Never touched: a conversation id still used by a live (non-deleted) call (_liveConversationIds, ms-comm _vendorConversationPurgeable), chat attachments, billing collections (wallet activity, AI credit logs, provider usage).
  • Indexes to create by hand before deploy on chats, service_rooms, inbound_outbound_calls: deleted_purge_v1 {deleted_at: 1} and deleted_copies_v1 {'purge.copies_at': 1, deleted_at: 1}, both partial {deleted_at: {$type: 'date'}} (the job hints them by name).

4. Erasing a person (Art. 17)​

Backend 38ebee775, ms-comm ee4d093, frontend 3eea37e1.

  • POST /leads/:company_id/erase {id, delete_ghl_contact?} — id matches ^(p|s|chat|call|f):<24hex>$; passport only, deliberately outside companyPaused; canManageCompany + canDeleteConversations (403 "Only the account owner can erase a person"). 409 when the match looks like more than one person (sessions > 1000, chats/rooms/calls > 500, sms > 2000). Returns {message, not_covered, erasure_id, counts}.
  • lib/personErasure.js collects only that person's records in that company (person must exist there; a shared phone/email never pulls in someone else; customer-side call matching; NA-only phone widening; one indexed query per branch — no new indexes on big collections), stores the ids on person_erasures, then applyErasure: chats detached as above; widget calls, phone calls and SMS soft-deleted (SMS to_phone, body, ghl_contact_id, crm_links moved under deleted_from), all tagged purge.erasure_id; queued Nox/Agent K sends cancelled (error: 'gdpr_erasure') and dependents blocked; campaign rows skipped and blanked (erased:<id>); kk_tokens deleted; visits anonymised (SESSION_PII_FIELDS + booking contact fields unset, erased_at, gdpr_anonymous: true, identity_trust: 'anonymous'); company profile, IP signals and funnel/watch rows deleted; the global Person deleted only when no other company refers to it; the person's lead rows tombstoned (ids + erased_at/deleted_at, never rebuilt); form_leads deleted; RB2B cache rows for their IPs lose data; their exports cancelled and files deleted.
  • Vendor deletes run in ConversationPurgeJob._finishErasures erasure_days later, regardless of the retention switch: chat/room/phone-call media as above plus SMS (ms-comm kind: 'sms' → Twilio message delete) and Twilio call logs. GHL contact delete (when ticked) runs on the next run with backoff, up to 10 attempts (done_with_errors after). A running erasure untouched > 10 min is resumed (failed after 10 resumes); an item failing 10 times gets one best-effort pass, then the local doc is deleted and vendor_failures incremented. Stored ids are unset when done.
  • No re-identification: PersonResolverService checks person_erasures (isErasedPerson, cached yes 5 min / no 30 s) before any claim; a session with erased_at never resolves, never fires new_lead, and gets no IP history back.
  • person_erasures (PersonErasureModel): company_id, lead_key, person_id, person_ids[], requested_by, requested_by_type, delete_ghl_contact, ghl {pending[], done[], last_error, attempts, next_at}, ids (Mixed), counts {chats, calls, phone_calls, sms, scheduled_cancelled, sessions_scrubbed, leads, form_leads}, vendor_failures, status (running → pending_vendors → done | done_with_errors | failed), next_check_at, completed_at; undeclared resume_attempts, last_error written by the job. Index {company_id: 1, person_id: 1}.
  • Not covered (NOT_COVERED): LinkedIn (Unipile), HubSpot, Salesforce, Vapi, calendar events, session recordings, Agent K history, AI-extracted facts. The drawer dialog lists only the first five; the success toast lists all.
  • UI: Leads drawer icon Erase person (GDPR) (only for deleters, on p:/s:/chat:/call:/f: rows) → ConfirmDialog (danger) with the optional "Also delete the contact in GoHighLevel" checkbox, shown only when GHL is connected.

5. Per-person data export (Art. 15)​

Backend a526873d2, frontend 42da41e6.

  • POST /leads/:company_id/export {id} → 202 (same role rule as erase, outside companyPaused); 409 when one is queued/building or the person was erased; 429 over 20 per company per 24 h; 400 when the requester has no email. GET /leads/:company_id/export?id= latest status ({export: null} for non-deleters); GET …/export/download?id= mints a 10-minute signed link on click (410 if erased).
  • PersonExportJob (every 2 min, schedular): one export per run across replicas, oldest first, atomic claim; stale build retried after 1 h; ≤ 3 attempts (retry after attempts × 15 min); files deleted after 7 days (expired), request rows kept 365 days.
  • ZIP (lib/personExport.js, personExportMedia.js, lib/zipFile.js; built on disk, streamed to private data-exports storage): person.json, visits.json, chats.json, calls.json, messages.json, forms.json, ai.json, crm.json, consent.json, enrichment.json, summary.html, README.txt, recordings/ (call audio/video + session replay data, newest first, 500 MB cap) and recordings.json listing what didn't fit or is vendor-only. JSON capped at ~100 MB. NOT_INCLUDED: over-cap / phone-provider-only recordings, email delivery logs, bookings outside meetings, notifications, automation/webhook logs, records only in connected tools, chat attachments.
  • Email views/templates/person-export/ready.html — subject "A personal data export is ready", no link and no personal data; download happens from the drawer.
  • Removal: cancelForErasure on erasure; ArchivedAccountDeletionJob deletes export files before the rows.
  • person_exports (PersonExportModel, no secondary index): company_id, lead_key, requested_by, requested_by_type, requester_email, status (queued → building → ready → expired | failed), attempts, started_at, ready_at, file_id, pending_file_id, retry_at, counts, error.
  • Env: FILE_LINK_SECRET (falls back to RECORDING_LINK_SECRET; must match ms-communication; set in backend, schedular and ms-comm secrets) — the export refuses to build without it; MS_SESSIONS_URL + BACKEND_API_KEY for replay data.

6. Recordings access​

  • Call recordings play through logged-in, company-checked backend proxies with short-lived stream URLs (c83d4cde3, df983134f; frontend dba706f8/6f0bdc04). CallThread.vue: ALLOW_RECORDING_SHARE = false (no "Copy link to this call"), ALLOW_RECORDING_DOWNLOAD = true ("Download MP3"/"Download video", "Download transcript") — not role-gated beyond seeing the call.
  • ms-sessions GET /video-recordings/:id requires login + company access (9f4d058).
  • Chat / WhatsApp / certificate uploads are private with signed links (85852ef4f).
  • Per-visitor recording retention: see Recorded Sessions.

7. Telemetry and AI disclosure​

  • Sentry (backend otel.js, app/lib/sentryScrub.js): sendDefaultPii: false; scrubEvent drops request.data, request.env.REMOTE_ADDR and event.user.
  • PostHog (frontend store/app.js): identify gets only the user id + {user_type, company_id} (was the whole /user/me record incl. card details); posthog.reset() on logout.
  • ms-ai: OpenAI Agents SDK tracing off (setTracingDisabled(true)); enhancedDatabaseReporting: false keeps Postgres bind values off traces.
  • ms-sessions: no default OTLP endpoint — OpenTelemetry starts only when OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_TRACES_ENDPOINT is set.
  • Honest AI disclosure: default inbound/outbound/playground call prompts (backend AiOutboundCallController, lib/inboundAgentSync.js; frontend usePlaygroundConfig.js, both Behavior.vue, useInboundAgentSync.js) say "If asked whether you are an AI, say yes honestly." instead of "Never reveal that you are an AI.", and drop "human" from "professional representative". Existing tenant prompts are unchanged. Axiom was removed from ApiSubscriptionSeeder (the existing DB row must be deleted in admin).