Skip to main content

Identity Graph

The identity graph turns anonymous visitor sessions into resolved, enriched people — the backbone of leads, cross-session context, and CRM sync. It lives primarily in backend, with ms-ai consuming it (via a gRPC client) for AI context.

The model​

EntityMeaning
UserSessionModelone raw visitor session (device, fingerprint, IPs, URLs)
PersonModela resolved distinct visitor (may span many sessions/devices)
PersonProfileModelenriched identity for a Person (name, email, company, B2B data)
PersonIpSignalModelIP→person signals used for IP-assisted resolution

Person is the identity; PersonProfile is what we know about them. A session resolves to a Person; multiple sessions/devices can collapse into one Person as evidence accumulates.

The resolution chain​

Wired in backend/app/hooks.js boot() as an ordered subsystem:

IdentityGraphRenderer → IdentityGraphConfig → SubscriptionGate
  → IdentityGraph → Verifier → PersonResolver → PersonWatch
  • IdentityGraphConfig — per-company verification policy (permissive/moderate/strict/regulated) controlling how aggressively sessions are merged.
  • SubscriptionGate — gates identity features by plan.
  • Verifier — applies the policy to candidate matches.
  • PersonResolver — the core resolution: match a session to an existing Person or create a new one, using fingerprint, IP signals, and form captures.
  • PersonWatch — drives the Nox "visitor watch" / auto-engage-on-return behavior.

Trust model (post-overhaul, 2026-07)​

user_sessions.identity_trust is anonymous | claimed | verified. Provenance is enforced by source-set helpers in PersonResolverService.js, not a constants file:

  • _isHighTrustSource() — form, crm, inbound_call, booking, chat_form. Only these may overwrite canonical name/email/phone (_mayOverwriteCanon); everything else fills empty fields only.
  • GENUINE_USER_SOURCES (in _decideTrust) — high-trust ∪ chat, call. Only genuine sources can verify; a first profile minted by a non-genuine source stays unverified (first_profile_untrusted_source). Enrichment can never mint verified or overwrite a self-provided name.
  • Direct-trigger reasons: first_party_form, first_party_booking (bookings auto-verify), crm_tracked_landing, caller_id_match, and chat/call self-identification → verify_chat / reason chat_identification.
  • Name moderation — app/lib/NameModeration.js screen(name) (obscenity + denylists) rejects junk/profane names before persisting; the frontend falls back to "Unknown Visitor" (getSessionName, frontend/src/common/helpers/session.js).
  • AI prompts get verified names only: IdentityGraphRendererService emits a name only when name_addressable (verified). The Apollo colleague list (company_contacts) is excluded from the default section set — opt-in via identity_graph_config.enabled_sections.

Resolution order on session create​

UserSessionController.create resolves internally before spending enrichment quota: _applyWidgetUserSessionId() → _backfillPersonFromPriorVisit() (widget-id/cookie link at claimed) → personResolver.wouldClaimDevice() (device-signature match at claimed). Snitcher/Apollo only fire when !internally_resolved && user_ip && suspected_bot !== true — returning identified visitors don't burn quota. When the client's own IP lookup is ad-blocked, the server fills user_ip from x-forwarded-for / x-real-ip.

Device claims are cross-tenant (d8a283a1b, 2026-09-09): PersonResolverService.evaluateDeviceCandidate({ fpHash, softSig }) dropped companyId from the signature and the query, and the PersonProfileModel device indexes lost their company_id prefix. A hash matching a single verified PersonProfile on any tenant, seen within IP_CANDIDATE_FRESHNESS_DAYS, yields would_claim at trust claimed — never verified. Since c5484a81e, a claim requires an fp_hash match; a soft signature alone can corroborate but never claim (ad visitors were being device-suspected as the wrong person). More than one person on a hash is ambiguous and rejected. _isAllowedForCompany still gates every caller, and _isDeviceSuspectedClaim (the anchor guard) makes a first-party identifier on a device-suspected session mint its own Person rather than attach to the suspect. PersonWatchService._recognizeByDeviceSignals now delegates to the resolver instead of its weaker soft-signature copy.

Suspected identity — cross-device guesses are shadow markers, not a trust tier: ip_shadow_person_id / device_shadow_person_id on UserSessionModel. IdentityGraphService._buildSuspectedIdentity renders an owner-facing "⚠ Suspected Identity (UNCONFIRMED)" block; the frontend shows an amber SuspectedIdentityBadge on live/recorded session rows. Session display naming (getSessionName) tiers: person → self-provided → suspected → Apollo person → Snitcher company (business type) → email → Snitcher domain (non-business, ⚠).

Signals feeding resolution​

  • Fingerprint (FingerprintJS) + device/browser attributes.
  • IP signals (PersonIpSignalModel, IpShadowOutcomeModel, app/lib/ipSignals.js) — IP-assisted resolution, shadow scoring, and backfill.
  • Form captures — emails/names auto-captured from forms (HubSpot/GHL/Calendly/ Typeform interception) attach identity to a session; the widget pre-chat form posts POST /api/intake with source: 'form' (direct verification trigger).
  • B2B enrichment — Snitcher (reverse IP → company) then Apollo (domain → person) fill PersonProfile. Clearbit's live API call is disabled (cache-read only); RB2B remains a webhook path. HappierLeads is gone (d8a283a1b): service, config, HappierLeadSessionModel, inbound webhook, provisioning route, and the ms-communication mirror fields were deleted (the pixel left the widget 2026-07-01); only readers of the historic happier_leads_data session blob remain (StatsV2, KnoxCrm, Notification, Chat, ServiceRoom controllers, VisitorIdentityHelper). The enrichment-guess CRM push job (LeadContactSyncJob) is disabled globally (commented out in SchedulerService). Enrichment is consent-gated when the tenant enables consent gating (identification category).
  • Per-page-view arrival source (7a5930202) — the classifier moved to app/lib/trafficSource.js (+ formatSourceLabel) and is stamped on the live socket path in SocketService.saveSessionStart (source_category). URL markers (?kk=, UTMs, gclid) win, else referrer; same-host referrer is internal navigation; widget.js sends document.referrer on the first page-load emit only. app/lib/visitGroups.js splits page-views into visits on a 30-minute gap; a visit's source is the page-view that opened it. The frontend Repeat badge (2+ visits) and Session Timeline grouping use the same rule; StatsV2Controller exposes return_visitors + return_visitor_ids (capped 50).

Who gets identified: the one rule, and the GDPR gate​

Identification is snitcher_enabled && (snitcher_allowed ?? 20) > 0 (KnoxStatsController ~3245, mirrored by the leads-screen notice). Everything else is gone: companies.lead_intelligence / lead_intelligence_allowed, the admin toggle and cap, /happier-leads/quota, and the package-sync writes to them were deleted in d8a283a1b — they only ever gated the HappierLeads pixel. Two consequences:

  • snitcher_allowed: 0 now blocks (67534d306); before, a zero cap skipped the cap block and meant unlimited. The ?? 20 is deliberate — callers use .lean(), which applies no schema defaults.
  • Identification is package-driven for everyone (d2e1dce98): SubscriptionHelper sets snitcher_enabled = !!features.lead_intelligence_allowed and snitcher_allowed = features.lead_intelligence ?? 20 at all five sync sites (trial, Stripe subscribe, subscription update, package change, platform create). A super-admin override is reset by the next sync. Partner-invite redemption clears snitcher_enabled (AuthController ~895); archiving a company or agency switches identification off.

GDPR person-level block (655140ec5, plan GDPR_IDENTIFICATION_PLAN.md):

  • app/lib/GdprGeo.js — RESTRICTED_COUNTRIES is 32 codes: EU27 + IS/LI/NO + GB (UK GDPR/PECR) + CH (revFADP). personLevelBlocked() returns true for empty/unknown — a failed geo lookup fails closed. blockedForCompany(company, codes) accepts one code or an array and takes the restrictive answer; only company.gdpr_protection_enabled === false opens it — never test truthiness, because .lean() docs have no default.
  • companies.gdpr_protection_enabled defaults true — the inverse of every other gate in that schema. Toggle: PUT /company/:_id/gdpr-protection (admin-only, logs the actor); admin company detail shows an amber note while off.
  • Gate points: Apollo people-match in UserSessionController.getApolloData (new 4th arg countryCode, the single choke for session-create and grantTracking); widget fingerprint — GET /company/:id returns identification_restricted, and public/widget.js ConsentGate.fingerprintAllowed() skips computeSoftSignature() / fetchFpHash() so the FingerprintJS bundle is never fetched (ePrivacy Art. 5(3): the hash must not be computed, not merely discarded). It is deliberately separate from identificationAllowed(), which drives Snitcher.
  • Not gated: Snitcher — company-level reverse IP on legitimate interest (comment at the call site). A visit from Acme still shows as Acme.
  • Geo: app/lib/IpGeo.js + app/lib/DbipCountry.js with the vendored data/dbip-country-lite.csv.gz (DB-IP Lite, CC-BY 4.0, no network call) and IpGeoModel (30-day TTL). ip-api is display-only fallback; IP_API_KEY is read only in IpGeo.js and is not set in any configmap yet — the plan's "both configmaps" line is an instruction, not current state.

Enrichment quota​

UserSessionController.getSnitcherData: daily cap company.snitcher_allowed, counted in company-local timezone; 00:00–06:00 local spends only yesterday's leftover; bots never spend quota. Apollo has no separate quota — it only runs after Snitcher resolves a domain. Apollo error responses are never cached (error-cache poisoning fix); POST /migrations/cleanup-apollo-records (and the admin wrapper POST /admin/identity/maintenance/cleanup-apollo-records) purges poisoned ApolloRecordModel rows so domains retry.

Writing identity back to the CRM​

  • app/lib/ghlContactFill.js (014f45457) — STATED_SOURCES = {chat_form, new_lead, transcript_ai, session_create} may write onto a matched GHL contact, not just a created one: name only over a placeholder (unknown/guest/user/not provided, or name == email local part), email replaced only when the match was by phone (else only when the contact has none), phone only when empty. Enrichment sources (apollo, lead_sync) never write. Applied at three CrmSyncHelper._fillGhlContact sites (already-linked, email match, phone match).
  • _syncGhlLinkedin (2d5310b08) writes visitor + employer LinkedIn (Apollo → RB2B → HappierLeads blob) on matched contacts, never over an existing value. It also fixed the custom-field write shape to {id, field_value} — the old {key, value} never landed. _ensureField lives in app/lib/ghlCustomFields.js.
  • Partial-write wipe (798d6337b): ChatController.updateContactInformation merges into chat.contact_information (new value still wins) instead of $setting the whole subdoc, and ChatTranscriptEmailJob is fill-only — an email-only extraction was nulling a pre-chat-form name.
  • Agent K → graph (ec5b2e36f): PersonResolver.recordContact() is a session-less find-or-create called from AgentKPushService._finalize on every delivery path; fill-only on an existing Person; no match key → no Person. New profiles are outbound_only: true so _decideTrust treats them as no history; any real resolve clears it. person_id now rides into enterFunnel.
  • GHL visit activity — GhlVisitActivityJob (every 15 min, opt-in go_high_level.log_visit_activity) posts one internal conversation comment per ended visit (GHL has no Activity write API): sessions with a GHL contact, non-bot/non-owner, updated in the last 7 days but quiet for VISIT_GAP_MS; 200 sessions per company, max 10 comments per run. Dedupe is the watermark user_sessions.ghl_visit_activity_synced_at, written per comment — internal comments cannot be deleted, so a missed watermark means a duplicate forever.
  • kk_token coverage (33a9bbea6): company.kk_tokens.app_contacts (stamp every contact the app creates, fire-and-forget in saveContact) and kk_tokens.auto_sync; KkTokenSyncJob ticks hourly with RESYNC_MS = 22h, reusing AgentKPushService.startTokenBackfill(id, {createField: true}) and ai_agent_k.kk_backfill.status/finished_at as the day-claim; tokens mint with source ghl_backfill so clicks never attribute to Agent K. Picker: GET /company/:id/kk-tokens/untokenized, POST …/stamp (25 per request). The kk-click stitch (65908d2da, d1dc0f893) now checks three homes for the click's contact — session CRM stamp, PersonProfile by (person_id, company_id), and the LeadConnector email-click recipient — because the resolver writes the contact onto the profile and never stamps the session back; a background backfill repairs click rows written without one.

Blocked visitors​

BlockedModel (collection blocked, {type: 'ip'|'url'|'company', value, company_id}, index {type, value, company_id}): company_id set = tenant block; null = platform-wide "don't track at all" (super admin). PUT /user-session/:_id/block (setBlocked) scopes to every session for that company sharing user_ip in the last 30 days (BLOCK_LOOKBACK_MS — bounded so a NAT'd office isn't a six-figure updateMany), sets user_sessions.is_blocked, upserts/deletes the IP row, and broadcasts block-user to each session room and the company room. Tracking and recording continue; is_active is untouched. is_blocked is off-limits to the public PUT, and the spoofable client-emitted relay in SocketService was deleted.

ms-communication enforces it in the existing socket.use middleware via BlockedVisitorsHelper.createBlockedVisitorCache — first lookup awaited (a scripted socket got one message through otherwise), 30 s TTL, 5,000-entry cap, DB failure keeps the last known set — so one check covers chat, calls, and AI; auto-trigger delivery receipts stay exempt. The widget's applyBlocked() ends any call in flight and asks the host page to drop the iframe, listening for block-user on all three sockets.

Admin surface​

Super-admins manage identity via frontend/src/repositories/admin/identity.js against backend's adminIdentityRoutes.js: list persons/profiles/form- submissions, session detail (fingerprint/verified-signal display), the identity graph config (moved admin-side: GET/PUT /admin/identity/config/defaults, PUT /admin/identity/config/company/:id), maintenance panels (trim session events — NDJSON progress, retry poisoned Apollo domains, unlink enrichment identities, backfill chat human-message flag, IP-signal backfills, the crm_links backfill and the CRM smoke test), and the GHL reconcile-conflicts queue.

Person repair (5675bd36f, f4a1c2dca, 33ea5733d, all in AdminIdentityController, dry-run unless apply):

  • POST …/persons/:id/detach-match-key — removes the key, unlinks sessions, drops matching form submissions, forgets those sessions as device proof, recomputes canonical email/phone.
  • POST …/persons/:id/split-match-keys — moves selected keys to a new Person; genuinely-identified sessions re-link at claimed, device-suspected sessions detach to anonymous (a device stamp is a guess); form submissions follow to the new per-company profile.
  • PUT …/persons/:id/identity — edit canonical name/email/phone. Keys are rebuilt by the resolver's own _buildMatchKeys; phone must be E.164; a fully-unchanged save is a no-op (protects the 50-cap on identity_events); blanking a field that still has a match key is refused — Detach is the removal path.
  • POST …/persons/:id/reset-trust-signals, POST …/sessions/:id/unlink.

Every write emits an admin_repair identity event plus session identity_audit entries; getPerson returns identity_audit and the shadow markers for the frontend Copy JSON diagnostic export.

Consumers​

  • AI agents — cross-session context is injected into chat/voice prompts so the AI knows a returning visitor.
  • Nox — visitor lookups, presence, and watch directives.
  • CRM sync — resolved identities + outcomes flow to GHL/HubSpot. See CRM Integration.

Reliability​

Identity resolution has had a dedicated reliability effort (IP-assisted resolution, form-intercept fixes, calls/read-gate/reconcile work). Treat resolution changes as high-blast-radius: they affect lead counts, dedup, and CRM records. Validate against real session samples before shipping.