GDPR & Privacy
Audience: Engineering · Support · Management · Where in app: Leads drawer (erase / export), Chats & Calls (delete), Settings → Organization → Recording Privacy, Admin → Settings (Deleted Conversations, Recording Retention), Admin → Companies (GDPR Protection, masking v2, established region) · Plan availability: All plans
Shipped 2026-11-09 → 11-16 across backend, frontend, widget, ms-sessions, ms-communication and ms-ai. Customer page: docs-site settings/privacy-gdpr.
1. GDPR safe mode (all companies)
Builds on the earlier person-level block (see Identity Graph › GDPR gate); backend 99b94dcc6.
Who is restricted. app/lib/GdprGeo.js RESTRICTED_COUNTRIES (EU27 + IS/LI/NO + GB + CH); an empty/unknown country is restricted. lib/visitorCountry.js is the one shared check: it looks up the first x-forwarded-for hop and request.ip (restricted if either is), one retry, LOOKUP_TIMEOUT_MS 3000, private IPs ignored. Used by GET /company/:id (returns identification_restricted to the loader) and POST /user-session/create, which stores user_sessions.gdpr_restricted (server-set; a client value is deleted). Pre-feature sessions have no flag = not restricted.
Switch. companies.gdpr_protection_enabled (default true; only a literal false opens it). PUT /company/:_id/gdpr-protection — super admin only; turning it off needs {confirmed: true, confirmation_note} (note ≥ 20 chars, else 422) and writes companies.gdpr_override {note ≤1000, by, by_email, at} (cleared when turned back on; returned only to super admins via withoutGdprOverride). Admin company detail: "GDPR Protection" switch, off-form checkbox "The customer has confirmed this in writing." + "Where the confirmation is recorded (at least 20 characters)". GDPR/recording flags are in DEDICATED_ROUTE_COMPANY_FIELDS — the generic company PUT can't write them.
What waits for an explicit grant (backend/public/widget.js ConsentGate):
| Gate | Restricted visitor needs | Everyone else |
|---|---|---|
recordingAllowed() | explicit behaviour grant | behaviourAllowed(); a site/CMP "no" blocks |
formCaptureAllowed() | explicit identification grant | allowed unless a site/CMP "no" |
fingerprintAllowed() | explicit identification grant | identificationAllowed() |
| Device storage | kkStoreMode pending → allowed / anonymous; anonymous writes nothing new to sessionStorage (kk_tab_id, _kk_visit_history, kk_consent_<companyId>) — except a visitor who really used chat/call (kkEngaged) keeps the session across pages | unchanged |
This applies whether or not consent_gating is on — with gating off and no CMP, restricted visitors are never asked and stay unrecorded (hence the dashboard notice below). Snitcher (company-level) stays on; anonymous visits reuse a lookup for 1 h via snitcher_reuse_cache (sha256(company_id:ip), no raw IP, TTL 3600 s).
Explicit = kkExplicitGrant(cat): kkConsentExplicit && !kkConsentWeak && knockKnockConsent[cat] === true. Sources: banner, stored, site (preset window.knockKnockConsent or kkSetConsent), cmp_cookiebot (Cookiebot.hasResponse; marketing → identification, statistics → behaviour), cmp_onetrust (OnetrustActiveGroups + OptanonAlertBoxClosed cookie; C0004 → identification, C0002 → behaviour; a missing group is a "no" only when C0001 is present), cmp_gcm (dataLayer consent update; ad_storage → identification, analytics_storage → behaviour; an update without a prior denied default is weak — counts as no, never as a grant). Re-read on load and on CookiebotOnAccept/Decline/ConsentReady, OneTrustGroupsUpdated and dataLayer.push. The non-blocking banner's default-accept (source: 'default') is never explicit. Our banner waits ~1.5 s + window load and stays hidden if a CMP appears (_showBannerUnlessCmp).
A "no" for everyone. _siteAnswered() && kkConsentGiven.<cat> === false blocks recording/forms for any visitor even with our banner off; applyCaptureGates() calls stopRecording() (window.__kkRecordingOff) and sets window.__kkFormCaptureOff mid-page. A restricted visitor's explicit identification "no" also wipes device storage (kkWipeDeviceStore).
Server backstops. ms-sessions SocketService.canRecord refuses gdpr_restricted sessions unless consent.explicit && consent.behaviour (allow cached 5 min RECORD_ALLOW_TTL_MS, refusal 30 s). FormSubmissionService drops form_intercept posts when consentRecord.formCaptureBlocked (masking v1 path); FormEventController stores nothing when recordingBlocked(session).
Anonymised storage (anonymizeSessionBody, restricted visit without an explicit identification grant): gdpr_anonymous: true, user_ip/ip_hash cleared; user_session_id, widget_user_session_id, fp_hash, soft_signature_hash, every *person_id/shadow id, email, name, phone_number, image removed; identity_trust: 'anonymous', user_meta: {}; ip_info cut to country/countryCode, knox.geo deleted; page_url loses query + fragment; traffic_source.gclid/.referrer deleted. Left: country, coarse device, page path, UTMs. gdpr_anonymous sessions are never claimed by device/IP recognition.
Consent proof
POST /user-session/:id/consent(public){company_id, consent: {identification, behaviour, explicit, source, cmp}};lib/consentRecord.sanitizewhitelistssource(banner|stored|site|default|none|cmp_cookiebot|cmp_onetrust|cmp_gcm) andcmp(cookiebot|onetrust|gcm) — an unknown source becomesnoneand voids every grant.recordConsent(UserSessionController) dedupes againstuser_sessions.consent(sameDecisionon identification/behaviour/explicit/source), writesuser_sessions.consent {…, at}, and appends aconsent_eventsrow only whenexplicit. At session create a row is logged only for a restricted, explicit, non-storeddecision.consent_events(ConsentEventModel, append-only):company_id,user_session_id,identification,behaviour,explicit,source,cmp,banner_version(first 12 hex of sha256 of the resolved banner texts),gdpr_restricted,country_code,ip_hash,user_agent(≤300),created_at; index{user_session_id: 1, created_at: -1}.- Withdrawal (
withdrawalUpdate: restricted, not yet anonymous, explicitidentification: false):$set {gdpr_anonymous: true, user_ip: '', ip_hash: ''},$unset {fp_hash, soft_signature_hash, widget_user_session_id}; person links from a form/chat are kept. - A late grant calls
grant-tracking, which uploadsfp_hash+ soft signature and forcesretryRecognition.
Dashboard notice
companies.consent_signals {restricted_seen_at, cmp, cmp_seen_at} (lib/consentSignals.js, written from session create and the consent endpoint, ≤ once per company per field per hour). GdprPausedNotice.vue (utils/gdprNotice.js) shows when protection is on, restricted_seen_at is within 30 days, consent_gating.enabled !== true, and no CMP was seen in 30 days. Title "EU visitors: recording and form capture paused"; link "Turn on the consent banner" (→ settings-widget?tab=install) for users with settings.widget and agency client access, else "Ask your account owner or agency to turn on the consent banner". On DashboardHome.vue (hidden while paused) and the Install tab (button "Show consent gating settings" instead of the link).
2. Recording masking v2 (per company, off by default)
companies.recording_masking_v2 — PUT /company/:_id/recording-masking-v2 {enabled}, super admin (AdminRecordingPrivacyToggles.vue: "Recording masking v2").
- Shared rrweb config
backend/public/recording-config.js(loaded only when the flag is on; globalskkRecordingOptions,kkSetRecordingConfig): every typed value masked per character (maskAllInputs: false+maskInputFn; date/time/hidden/select masked, checkbox/radio not), shown only when its field key is allowlisted or insidedata-kk-unmask;data-kk-maskalways masks (text too);contenteditablemasked unless underdata-kk-unmask; the widget hosts are blocked (#kka-chat-widget, #kk-consent-root, #kk-permission-arrow, [data-knockknock-widget]). Console text is no longer recorded — errors becomekk-errorcustom events{level, type, stack frames, source}with no message, ≤ 50 per page (recoded_session.js). - Never-unmask (
app/lib/neverUnmask.js, mirrored in the browser):type=password,autocompletecc-*, names matching ssn/passport/tax id/national id/driver's licence/iban/sort code/routing/account number/cvv/cvc/pin, and card-like values (13–19 digits, browser only). - Field discovery — inputs/textareas with a name or id are reported to
POST /recording/fields(public, rate-limited, top window only, ≤ 200 per page, ≤ every 5 s, stops after a consent stop).recording_fields(RecordingFieldModel, one row percompany_id + field_key;path,form_id,field_name,field_id,label,type,autocomplete,never_unmask(sticky),first_seen_at,last_seen_at,seen_count; never values; ≤ 1000 per company). Key (lib/recordingFieldKey.js):lower(path)|lower(formId||'-')|lower(name||id). Backfill:scripts/backfill-recording-fields.js(dry-run default; local-dev only — prod needs a migration endpoint). - Allowlist (
lib/recordingAllowlist.js): companyrecording.unmasked_fields[]+recording.use_agency_default(default true); agencyrecording_default_unmasked_fields[]+recording_default_unmasked_fields_locked. Resolution: locked agency list → company list (whenuse_agency_default === false) → agency default → company list (no agency). Max 500. - Routes (
routes/recordingRoutes.js):GET /company/:_id/recording/fields;PUT /company/:_id/recording/unmasked-fields {unmasked_fields, use_agency_default}(409 "Your agency manages the visible fields for this account" when locked; 400 withrejectedfor never-unmask keys);PUT /agencies/:id/recording/default-unmasked-fields {fields, locked}. Company writes: super admin, owner, members withsettings.organization, agency owners, agency staff withagency.manage_settingsoragency.edit_clients. - UI: Settings → Organization → Recording Privacy (
/settings/organization/recording,settings.organization; menu entry hidden unless the flag is on, page says "Contact support to turn it on" otherwise); agency Recording Privacy (/agency/settings/recording,agency.manage_settings, "Lock for sub-accounts", "Pick from a sub-account"). - Form capture on real submit only —
form_intercept.jscaptures values only on a nativesubmitor a provider "form sent" message; submit-like clicks/Enter only mark the form sent. Payload carriesgpc(navigator.globalPrivacyControl) andform_key. - Form events —
POST /api/intake/events(alias/form-events), public, rate-limited, never accepts values, ≤ 1000 per session.form_events(FormEventModel, 30-day TTL):company_id,user_session_id,form_key,type(form_viewed | field_focused | field_completed | last_field_before_exit | form_submitted),field_key,completed,page_path,at.FormEventAbandonmentJob(5 min, flagged companies only; session idle > 15 min, 24 h lookback; once per form per session viaform_abandon_states.fired_forms; Redis lockjob:form-event-abandonment) firesform_abandonedthroughFormAbandonmentService; ms-sessions skips its recording-based trigger for these companies. - Pooling (
lib/canPoolIdentity.js, only for flagged companies): restricted & not consented →'none'(unlinkedform_leadsrow — no session/person/IP/full URL, deduped; Leads idf:<id>); restricted & consented →'company'; company or agencyestablished_in_restricted_region→'company';Sec-GPC/gpc→'company'; else'global'.'company'= linked to this session + this company's profile only, never the sharedpeoplegraph.established_in_restricted_region(companies + agencies, default false):PUT /company/:_id/established-region,PUT /agencies/:id/established-region{enabled}, super admin. - Widget iframe (widget
ff382f6): the chat app's own rrweb recorder was removed — the widget is never recorded (Nox reads chats from the DB).setUserInteraction({notifyParent})tells the host page only about visitor-initiated opens (that signal keeps an anonymous visitor's session across pages). Backendpublic/widget-iframe.js(customer cross-origin iframes) still records; it asks the top page for the masking config and under v2 masks every typed value, ignoring the allowlist — it has no consent check of its own. - No typed values in analysis (ms-sessions
e060401,dom-resolver.js):attributes.valuedropped except as a button / submit/button/reset/image input caption; textarea/contenteditable text never becomestextContent, a label or accessible name (falls back toname);extractors/form.jskeeps length buckets only;extractors/error.jsreads rrweb console events andkk-error.
3. Deleting chats and calls
Backend 91f104fe3, ms-comm 002f914, ms-ai a753f6805, frontend fd4cfc77, widget 4b2753c.
- Who (
lib/conversationDeletion.jscanDeleteConversations, mirrored in ms-commcanDeleteCallsand frontenduseConversationDelete.js):admin,web_owner,agency;agency_staffonly withagency.edit_clientsoragency.add_client_subscription. Never team members — there is no member permission for it. - Endpoints:
DELETE /chats/:id(ChatController@destroy, alsocanManageCompanyId); widget callDELETE /remove-record/:id(ServiceRoomController@deleteRecord—video_deleted,deleted_by,$min deleted_at); phone call ms-commDELETE /inbound-outbound-calls/:_id(deleted_at, first date kept). - Chat delete is a detach:
$renameofcompany_id, user_session_id, agent_id, messages, previous_chats, last_message_atunderdeleted_from.*+deleted_at/deleted_by, so every reader, count and sweep misses it with no filter. Then: removed fromchat_queue; Firestore copy deleted (retried by the job); ms-commPOST /internal/chats/deletedre-broadcasts queue positions, clearschat-<session>-agent-connected, and emitschat-deletedto the visitor and company rooms (dashboard refreshes silently,App.vue). WidgetdropDeletedConversation()(onchat-deletedor HTTP 410) clears chat state, previous messages, voice links and the unread badge so deleted turns are never resent as AI context; the next message opens a fresh chat. - Hidden from: Leads rows + lead transcript API (a deleted call no longer makes a lead),
DailyRecapEmailJob, Nox (KnoxV2InternalController), AI call transcripts per session, call lists, identity-graph source trust, recording proxies (Ultravox proxy 404s; ms-comm_isDeletedRecordingfor ElevenLabs links). Super admins still see deleted rooms in one admin list. - UI: Chats header Delete (title "Delete chat"); Calls
CallThreadMore menu whose only item is Delete call (menu hidden for non-deleters). Dialog text fromGET /settings/conversation-retention/days(any logged-in user): purge on → "…removed from your account now and permanently deleted, with its recording and transcript, after N days. This can't be undone."; off → "…removed from your account. This can't be undone."
Permanent purge — ConversationPurgeJob (every 15 min, schedular)
- Setting
conversation_retention(settingscollection):enabled(default false),retention_days(default 30, 8–365 — the floor protects ms-ai's 7-day ElevenLabs missed-call billing catch-up),erasure_days(default 8, 8–21); the job writestotals,last_run,last_error.GET/PUT /settings/conversation-retentionsuper admin; card Admin → Settings → Deleted Conversations (tiles Waiting / Past the window now / Removed for good / Failing (retrying) / Status; switch "Remove deleted conversations for good"; "Keep deleted conversations for (days)"; "Erase person (GDPR): delete at vendors after (days)"). - Per run: 50 items per collection, 5-min budget, backoff 1 h → 24 h cap. Order: erasures → copies (always, even with the switch off: Firestore + ms-ai transcript copies via
POST /knox-signals/transcripts/purge, ≤ 500 ids) → chats / widget calls / phone calls past the window (only when enabled). - Purge: Firestore chat/call copies + live transcripts; voice-engine audio on S3 then ms-ai
POST /voice-engine/sessions/:id/recording/purged(nullsrecording_*, row kept for billing); vendor media via ms-commPOST /internal/purge/call-media— widget calls: Twilio Video composition + room recordings, Ultravox call, ElevenLabs conversation; phone calls: Twilio call recording (main account, then the company's own token), ElevenLabs conversation, Twilio call log (CA…, best effort). The Mongo doc is deleted only after every step succeeds. Telnyx and Vapi media are not purged. - Never touched: a conversation id still used by a live (non-deleted) call (
_liveConversationIds, ms-comm_vendorConversationPurgeable), chat attachments, billing collections (wallet activity, AI credit logs, provider usage). - Indexes to create by hand before deploy on
chats,service_rooms,inbound_outbound_calls:deleted_purge_v1{deleted_at: 1}anddeleted_copies_v1{'purge.copies_at': 1, deleted_at: 1}, both partial{deleted_at: {$type: 'date'}}(the job hints them by name).
4. Erasing a person (Art. 17)
Backend 38ebee775, ms-comm ee4d093, frontend 3eea37e1.
POST /leads/:company_id/erase {id, delete_ghl_contact?}—idmatches^(p|s|chat|call|f):<24hex>$; passport only, deliberately outsidecompanyPaused;canManageCompany+canDeleteConversations(403 "Only the account owner can erase a person"). 409 when the match looks like more than one person (sessions > 1000, chats/rooms/calls > 500, sms > 2000). Returns{message, not_covered, erasure_id, counts}.lib/personErasure.jscollects only that person's records in that company (person must exist there; a shared phone/email never pulls in someone else; customer-side call matching; NA-only phone widening; one indexed query per branch — no new indexes on big collections), stores the ids onperson_erasures, thenapplyErasure: chats detached as above; widget calls, phone calls and SMS soft-deleted (SMSto_phone,body,ghl_contact_id,crm_linksmoved underdeleted_from), all taggedpurge.erasure_id; queued Nox/Agent K sends cancelled (error: 'gdpr_erasure') and dependents blocked; campaign rows skipped and blanked (erased:<id>);kk_tokensdeleted; visits anonymised (SESSION_PII_FIELDS+ booking contact fields unset,erased_at,gdpr_anonymous: true,identity_trust: 'anonymous'); company profile, IP signals and funnel/watch rows deleted; the global Person deleted only when no other company refers to it; the person's lead rows tombstoned (ids +erased_at/deleted_at, never rebuilt);form_leadsdeleted; RB2B cache rows for their IPs losedata; their exports cancelled and files deleted.- Vendor deletes run in
ConversationPurgeJob._finishErasureserasure_dayslater, regardless of the retention switch: chat/room/phone-call media as above plus SMS (ms-commkind: 'sms'→ Twilio message delete) and Twilio call logs. GHL contact delete (when ticked) runs on the next run with backoff, up to 10 attempts (done_with_errorsafter). Arunningerasure untouched > 10 min is resumed (failed after 10 resumes); an item failing 10 times gets one best-effort pass, then the local doc is deleted andvendor_failuresincremented. Storedidsare unset when done. - No re-identification:
PersonResolverServicechecksperson_erasures(isErasedPerson, cached yes 5 min / no 30 s) before any claim; a session witherased_atnever resolves, never firesnew_lead, and gets no IP history back. person_erasures(PersonErasureModel):company_id,lead_key,person_id,person_ids[],requested_by,requested_by_type,delete_ghl_contact,ghl {pending[], done[], last_error, attempts, next_at},ids(Mixed),counts {chats, calls, phone_calls, sms, scheduled_cancelled, sessions_scrubbed, leads, form_leads},vendor_failures,status(running → pending_vendors → done | done_with_errors | failed),next_check_at,completed_at; undeclaredresume_attempts,last_errorwritten by the job. Index{company_id: 1, person_id: 1}.- Not covered (
NOT_COVERED): LinkedIn (Unipile), HubSpot, Salesforce, Vapi, calendar events, session recordings, Agent K history, AI-extracted facts. The drawer dialog lists only the first five; the success toast lists all. - UI: Leads drawer icon Erase person (GDPR) (only for deleters, on
p:/s:/chat:/call:/f:rows) →ConfirmDialog(danger) with the optional "Also delete the contact in GoHighLevel" checkbox, shown only when GHL is connected.
5. Per-person data export (Art. 15)
Backend a526873d2, frontend 42da41e6.
POST /leads/:company_id/export {id}→ 202 (same role rule as erase, outsidecompanyPaused); 409 when one is queued/building or the person was erased; 429 over 20 per company per 24 h; 400 when the requester has no email.GET /leads/:company_id/export?id=latest status ({export: null}for non-deleters);GET …/export/download?id=mints a 10-minute signed link on click (410 if erased).PersonExportJob(every 2 min, schedular): one export per run across replicas, oldest first, atomic claim; stale build retried after 1 h; ≤ 3 attempts (retry afterattempts × 15 min); files deleted after 7 days (expired), request rows kept 365 days.- ZIP (
lib/personExport.js,personExportMedia.js,lib/zipFile.js; built on disk, streamed to privatedata-exportsstorage):person.json, visits.json, chats.json, calls.json, messages.json, forms.json, ai.json, crm.json, consent.json, enrichment.json,summary.html,README.txt,recordings/(call audio/video + session replay data, newest first, 500 MB cap) andrecordings.jsonlisting what didn't fit or is vendor-only. JSON capped at ~100 MB.NOT_INCLUDED: over-cap / phone-provider-only recordings, email delivery logs, bookings outside meetings, notifications, automation/webhook logs, records only in connected tools, chat attachments. - Email
views/templates/person-export/ready.html— subject "A personal data export is ready", no link and no personal data; download happens from the drawer. - Removal:
cancelForErasureon erasure;ArchivedAccountDeletionJobdeletes export files before the rows. person_exports(PersonExportModel, no secondary index):company_id,lead_key,requested_by,requested_by_type,requester_email,status(queued → building → ready → expired | failed),attempts,started_at,ready_at,file_id,pending_file_id,retry_at,counts,error.- Env:
FILE_LINK_SECRET(falls back toRECORDING_LINK_SECRET; must match ms-communication; set in backend, schedular and ms-comm secrets) — the export refuses to build without it;MS_SESSIONS_URL+BACKEND_API_KEYfor replay data.
6. Recordings access
- Call recordings play through logged-in, company-checked backend proxies with short-lived stream URLs (
c83d4cde3,df983134f; frontenddba706f8/6f0bdc04).CallThread.vue:ALLOW_RECORDING_SHARE = false(no "Copy link to this call"),ALLOW_RECORDING_DOWNLOAD = true("Download MP3"/"Download video", "Download transcript") — not role-gated beyond seeing the call. - ms-sessions
GET /video-recordings/:idrequires login + company access (9f4d058). - Chat / WhatsApp / certificate uploads are private with signed links (
85852ef4f). - Per-visitor recording retention: see Recorded Sessions.
7. Telemetry and AI disclosure
- Sentry (backend
otel.js,app/lib/sentryScrub.js):sendDefaultPii: false;scrubEventdropsrequest.data,request.env.REMOTE_ADDRandevent.user. - PostHog (frontend
store/app.js):identifygets only the user id +{user_type, company_id}(was the whole/user/merecord incl. card details);posthog.reset()on logout. - ms-ai: OpenAI Agents SDK tracing off (
setTracingDisabled(true));enhancedDatabaseReporting: falsekeeps Postgres bind values off traces. - ms-sessions: no default OTLP endpoint — OpenTelemetry starts only when
OTEL_EXPORTER_OTLP_ENDPOINTorOTEL_EXPORTER_OTLP_TRACES_ENDPOINTis set. - Honest AI disclosure: default inbound/outbound/playground call prompts (backend
AiOutboundCallController,lib/inboundAgentSync.js; frontendusePlaygroundConfig.js, bothBehavior.vue,useInboundAgentSync.js) say "If asked whether you are an AI, say yes honestly." instead of "Never reveal that you are an AI.", and drop "human" from "professional representative". Existing tenant prompts are unchanged. Axiom was removed fromApiSubscriptionSeeder(the existing DB row must be deleted in admin).