Skip to main content

Identity Graph (Admin)

Audience: Ops · Super-admins · Engineering · Support escalations · Where in app: /admin/identity/* · Access: Super-admin only — every endpoint enforces request.user.user_type === 'admin' (AdminIdentityController._requireAdmin, returns 403 Unauthorized otherwise).

The Identity Graph is the platform's record of who a visitor is across every session, device, and company. A Person is the global identity; a PersonProfile is that person's per-company facet (their CRM links, extracted facts, bookings, form answers for one tenant). These admin screens let a super-admin browse those records cross-tenant, tune how aggressively identity is trusted, resolve CRM write conflicts, and run one-shot data-maintenance sweeps. For the conceptual model and runtime behavior see the subsystem and module docs linked under Related.

What it does​

  • Browse the graph — list/search Persons, per-company Profiles, and every captured form submission across all tenants.
  • Inspect one Person — see their profiles, recent sessions (with trust state), bookings, match keys, identity audit trail, CRM enrichment, and IP signals.
  • Configure verification posture — set global Identity Graph defaults and per-company overrides (the super-admin replacement for the old customer-side settings UI).
  • Reconcile CRM conflicts — review GoHighLevel contacts where first-party identity should supersede enrichment but looked like a real pre-existing contact, so the system flagged instead of overwrote.
  • Repair and delete — edit a Person's identity, detach/split match keys, reset trust signals, unlink a session, delete a Person or one company's Profile.
  • Review abandoned-form detections — a verdict queue over forms the recording analyser flagged as abandoned, with the recording and stored submissions as evidence.
  • Run maintenance jobs — dry-runnable backfills/cleanups (cleanup, normalize, moderate names, trim bloated session-event arrays, IP-signal backfill and analyses, session-name backfill, GHL reconcile backfill, Apollo-contact unlink, device-claim backfill, duplicate-Person merge, IP-signal hash keying).

How it works​

The frontend repository frontend/src/repositories/admin/identity.js wraps the backend routes declared in backend/app/routes/adminIdentityRoutes.js (all under /admin/identity/*, all passport-authed). The controller is backend/app/controllers/AdminIdentityController.js.

Reads (persons / profiles / form-submissions / reconcile-conflicts) are paginated via a shared buildQuery that converts {page, pageSize} into skip/limit. List queries cap MongoDB execution at 15s (maxTimeMS) and return 504 on timeout rather than hanging until the load balancer cuts the connection.

Maintenance writes are super-admin-gated wrappers around the api-key-gated /migrations/* sweeps, so they can be triggered from the admin UI. Every write action defaults to a dry run (apply:true must be sent to actually mutate). Short jobs return a {mode, company_id, elapsed_ms, stats} envelope; long ones start a background job and return its id, which MigrationPanel polls at …/jobs/:job_id (so a page refresh or ingress timeout doesn't lose the run). Since 2026-10-13 the panel keeps polling through transient poll failures — the job is still running server-side — and only gives up after 20 consecutive failures or a 404 (ef7e3129); the raw result has a copy button and JSON highlighting (8a99ec23). The older Trim Session Events job streams NDJSON progress instead (see below).

Screens & fields​

Cross-entity navigation​

Every Identity screen shows a row of nav pills (Persons, Profiles, Form Submissions, CRM Reconcile, Maintenance, Config) so an admin can move between facets of the same data.

Persons list — /admin/identity/persons​

Source: identity/Persons.vue → getPersons. The global directory of unique people.

| Filter | Values | Notes | | ----------- | -------------------------------------- | ----------------------------------------------------------------------------- | ------ | | Search | free text | matches name, email, or phone (case-insensitive regex), debounced 500ms | | Has Profile | All / With profiles / Without profiles | sends has_profile=true | false | | Sort | Last active / First seen / Name | sort field; order is -1 (desc) |

ColumnSource field
Nameperson.name
Emailperson.email
Phoneperson.phone
Match keyscount of person.match_keys[]
Profilesperson.profile_count
Sessionsperson.session_count
Last activeperson.last_active_at
ActionsView → person detail

Rows are click-through to the detail page. Pagination total comes from pagination.total.

Person detail — /admin/identity/persons/:id​

Source: identity/PersonDetail.vue → getPerson (controller aggregates counts server-side to avoid shipping the heavy form_submissions / extracted_facts / crm_enrichment arrays over the wire). Header shows name, first-seen date, and match-key count, plus the repair actions (see Person repair) and Delete person (ConfirmDialog → DELETE /admin/identity/persons/:id). The edit dialog seeds its phone country picker from the most recent session's ip_info (returned by getPerson); before, vue-tel-input's failed auto-detect landed on Afghanistan and re-saved numbers with the wrong code. Tabs:

TabContents
Company Profilesone card per PersonProfile: company, first-seen/last-active, form-submission count, extracted-fact count, CRM-presence flag, recent identity events, a "conflicting claims" flag (any event whose source starts with conflict_with_verified:)
Sessionslast 50 sessions (Company, Identified as, Trust, Current URL, Created); trust derives from identity_trust / verified_person_id / claimed_person_id; passive links (device / IP / widget id) carry the amber badge until verified
Bookingsbookings flattened across all sessions + per-company profiles
Match KeysType, Normalized value, Hash — the deterministic keys (email/phone/etc.) used to link records

Profile detail — /admin/identity/persons/:id/profiles/:company_id​

Source: identity/PersonProfileDetail.vue → getPersonProfile. The full per-company facet, with Delete profile (DELETE /admin/identity/persons/:id/profiles/:company_id — this company's profile, IP signals, session stamps and funnel links only; the Person stays). Tabs:

TabContents
Form Submissionsevery captured form answer for this person+company
Extracted Factsfacts the AI extracted from conversations
Bookingsthis profile's bookings
CRM EnrichmentHubSpot and Go High Level enrichment blocks
Identity Auditthe audit log: Event, Source, Score, Session, When; flags Conflict and Needs review rows
IP Addressesreverse-IP signals: IP ("hashed only" when the row has only an ip_hash — Snitcher-enriched sessions have their raw IP wiped), Subnet, Ver (verified), Hits, First seen, Last seen

Profiles list — /admin/identity/profiles​

Source: identity/PersonProfiles.vue → getPersonProfiles. A flat cross-tenant list of PersonProfiles (filterable by company, sort/order via buildQuery).

Form submissions — /admin/identity/form-submissions​

Source: identity/FormSubmissions.vue → getFormSubmissions. Every form answer captured across the graph, rendered as cards with Person/Company/profile crumbs.

FilterValues
Searchfield label or value (debounced 500ms)
Form typeAll · contact_info · pre_call · booking · website_form · custom
SourceAll · chat · call · booking_flow · auto_trigger · form_intercept

Each card shows form_type, source, the relationship crumbs, submitted_at, and a key/value grid of fields[] (label/key → value).

CRM Reconcile — /admin/identity/reconcile-conflicts​

Source: identity/ReconcileConflicts.vue → getReconcileConflicts / resolveReconcileConflict. The review queue of GoHighLevel contacts where a visitor declared real first-party details (form/booking/call) but the linked contact looked like a real, pre-existing person, so the system flagged instead of overwrote.

Each conflict card shows: Person crumb, Company crumb, First-party email, First-party phone, GHL contact id, Flagged via (latest_conflict_source, with a ×N repeat count), and latest_conflict_at. Actions per card:

ActionEffectAPI payload
Overwrite anywayPush first-party details to the GHL contact (re-links to another contact if that email/phone already belongs to one). Use only when the contact IS this person. Opens a confirm dialog.{person_id, company_id, action:'overwrite'}
DismissKeep the existing GHL contact, drop from queue. Use when enrichment matched a different person. Optional Also unlink clears the wrong person↔contact association.{person_id, company_id, action:'dismiss', unlink?}

The reconcile backfill itself is run from the Maintenance screen (see "GHL Reconcile Backfill" below), not here.

Form Abandonment — /admin/form-abandonment​

Source: views/admin/form-abandonment/Index.vue + Detail.vue (repository repositories/admin/formAbandonment.js) → AdminFormAbandonmentController (routes/adminFormAbandonmentRoutes.js). Sidebar: Identity Graph → Form Abandonment. Shipped 2026-10-08 (backend b8e4cd5b7, frontend 2a6a515c) to measure the recording analyser's "abandoned form" detector before a form_abandoned Agent K trigger is built on it.

  • Queue — GET /admin/form-abandonment: forms from ms-sessions session_analyses.events.forms.forms, filtered by results (detector outcome: abandoned default, unknown, likely_submitted, submitted), from/to (default last 7 days, max 92), company_id, search (page URL), review_status (unreviewed default, reviewed, or a verdict). Returns verdict stats alongside the page.
  • Detail — GET /admin/form-abandonment/:file_id/:form_index: the session recording (jumps to each field, last activity and stored submissions — RecordingPlayer gained seekTo(absoluteTs) + a loaded event), the detector data, the page visit, form submissions stored on the visitor's profile during the recording (counter-evidence from the form interceptor; the window is the recording −5 min / +10 min, absorbing visitor-clock skew), company setup, and diagnostic warnings (capture_disabled, booking, partial, visitor_unknown, focused_on_exit, short_tail, little_typing, no_fields, only_script_fields). Newer analyses carry exact page_entered_at / last_activity_at; older ones fall back to time/label matching.
  • Script-filled inputs (2026-10-09, backend ade59a83c, ms-sessions 0c83a9b) — page scripts (pixels, CRM snippets) write hidden/tracking inputs on every click and rrweb records them as input events. The ms-sessions form extractor now skips hidden/non-form inputs, marks each field focused: true|false, and only lets focused fields start a form or move its last activity. The detail view shows only fields the visitor worked (focused, or for older analyses any focus time), reports script_field_count, re-derives "stopped at" from the last real field, and warns only_script_fields ("probably Not a real form") when nobody typed.
  • Review — PUT …/:file_id/:form_index/review with a verdict (confirmed_abandoned, submitted, not_a_form, recording_issue, unclear) + notes; Save & next walks the queue. Stored in form_abandonment_reviews (unique on file_id, form_index), with a submission snapshot and a time-ordered history[] of earlier verdicts. (file_id, form_index) is an array position a re-analysis or the phantom-form scrub can reshuffle, so a review only counts while the form there still has the reviewed page + fields_touched_order; otherwise the old verdict stays in history instead of landing on the wrong form.

Maintenance — /admin/identity/maintenance​

Source: identity/IdentityMaintenance.vue, each job rendered via components/admin/MigrationPanel.vue. Every job defaults to a dry run (reads/counts only); review the stats, then Apply. Most accept an optional Company ID to scope to one tenant.

JobEndpointWrites?Key fieldsWhat it does
GHL Reconcile BackfillPOST …/maintenance (runReconcileBackfill → /admin/identity/reconcile-backfill)yesCompany ID, Limit (1–5000, default 200)Sweeps already-verified sessions where first-party identity should supersede Happier-Leads enrichment and corrects their GHL contacts; real pre-existing contacts are flagged into CRM Reconcile, not overwritten.
Cleanup IdentitiesPOST /admin/identity/maintenance/cleanupyesCompany ID, Clean sessions, Drop foreign submissions, Prune empty Persons (destructive — requires Apply)Removes invalid identity values (bad emails/phones/names) from Person, profiles, sessions, form submissions; optionally prunes empty Persons and drops foreign submissions.
Normalize Identity Values…/cleanup with normalize:trueyesCompany ID (sessions only)Lowercases emails, promotes phones to E.164 where a country code is known (explicit + or session region — never a default), collapses name whitespace, adds match keys for the normalized values (never drops existing keys; rows whose key another Person owns are skipped). Rewrites values; deletes nothing.
Moderate Names…/cleanup with moderateNames:trueyesCompany ID (sessions only)Blanks stored Person + session names the NameModeration gate now rejects (junk / non-name phrases / profanity); display falls back to "Unknown Visitor".
Backfill IP SignalsPOST /admin/identity/maintenance/backfill-ip-signalsyesCompany ID, Days (1–365, default 90)Seeds the reverse-IP index (person_ip_signals) from recent sessions so IP-assisted resolution and the per-person IP tab have data.
IP Uniqueness AnalysisPOST /admin/identity/maintenance/ip-uniquenessread-onlyCompany ID, Days (default 90)Measures how many people share an IP / subnet — informs how aggressively IP can be trusted.
IP Shadow StatsPOST /admin/identity/maintenance/ip-shadow-statsread-onlyCompany ID, Days (default 30)How often a live IP claim matched the person the visitor later resolved to (confirmed) vs a different one (contradicted). IP claims are live; a contradicted claim contests its IP for 90 days.
Recognition CoveragePOST /admin/identity/maintenance/recognition-coverageread-onlyCompany ID, Days (1–90, default 7)Of new sessions by people we already knew, how many were recognized at session start (carried / device / IP) or by link (kk, CRM, email click) vs had to identify again — with the refusing gate per route, device/browser breakdown, and tenants whose graph is unavailable. Counts only sessions created after 2026-10-15. See Recognition coverage.
Backfill Session Display NamePOST /admin/identity/maintenance/backfill-session-nameyesCompany ID, Sample rows (0–100, default 20)Populates user_session.name from legacy first_name+last_name (treats Guest/User/empty as no real data).
Trim Session EventsPOST /admin/identity/maintenance/trim-session-eventsyesCompany ID, Keep last N (default 500), Page size (default 100), Max pages/call (default 500), Resume after _idCaps tab_focus_events / video_unmutes arrays bloated by the (now $slice-capped) uncapped $push. Pages by _id so it never loads the bloated docs; resumable.
Unlink Apollo CRM ContactsPOST /admin/identity/maintenance/unlink-apollo-contacts (job)yesCompany ID, include anonymousCuts sessions loose from a GHL contact an Apollo guess produced (link, crm_links row, cached ghl_context); the contact itself is never touched. Two passes: the ghl_appolo_sync audit trail, then a tenant-by-tenant scan for the pre-2026-08-13 backlog. Samples show found_by and the person we think the visitor is.
Backfill Device ClaimsPOST /admin/identity/maintenance/backfill-device-claims (job)yesCompany ID, Days (default 30)Re-runs the cross-tenant device claim over anonymous sessions created before 2026-09-09. Starts from fresh verified fingerprints, scans in one-hour slices.
Merge Duplicate PersonsPOST /admin/identity/maintenance/merge-duplicate-persons (job)yesMax cluster size (default 5), Rebuild unique index, Detach shared keysMerges Persons sharing a match key (oldest wins) when every member agrees on email + phone; others are reported as conflicting_clusters. Dry run reports indexes_missing — Apply refuses until those indexes finish building after a deploy. Rebuild is refused while conflicting/oversized clusters remain.
Key IP Signals by HashPOST /admin/identity/maintenance/ip-hash-index (job)yes—Fills ip_hash on legacy person_ip_signals rows, folds twins, swaps the unique index. Also runs automatically on backend boot under a Redis lock; the card is for visibility and re-runs.

Trim Session Events — NDJSON streaming​

This job can run long, so it streams NDJSON (backend/app/lib/progressStream.js) instead of buffering one JSON body — a heartbeat keeps the ingress idle-timeout from killing the request. The repository (runTrimSessionEvents) reads responseType:'text', splits on newlines, and parses each line:

{"type":"progress", "...":"..."}
{"type":"ping"}
{"type":"result","mode":"APPLY","stats":{"cap":500,"scanned":1200,"trimmed":340,"done":false,"next_after":"<objectId>"}}
{"type":"error","message":"..."}

Errors arrive in-band with HTTP 200, so the body must be inspected. Because the job is batched and resumable, when done:false paste the returned next_after into Resume after _id and Apply again until done:true.

Config — /admin/identity/config​

Source: identity/Config.vue + identity/_ConfigForm.vue → getIdentityGraphDefaults / updateIdentityGraphDefaults / updateCompanyIdentityGraphOverride. Sets the global defaults that apply to every company, plus optional per-company overrides.

The config form fields (shared between global and override):

FieldTypeDefaultMeaning
Enable Identity GraphtoggleonWhen off, AI agents receive no cross-session identity context.
Capture Website FormstoggleonInjects the form-intercept script alongside the widget so customer-site form submissions flow into the graph.
Verification PolicyradiostrictHow aggressively a session that claims an identity is promoted to "verified" — see below.
Trust Inbound Caller-IDtoggleonPromote an inbound voice call's session to verified when caller-ID matches a known phone.
AI Agent Context SectionscheckboxesallWhich sections to include in visitor context (Identity, Company Insights, Engagement, Current Session, Recent Pages, Past Conversations, Behavioral Signals, CRM State, Bookings, Form Responses, Extracted Facts). Leave all unchecked = include everything.
Custom Introtextarea—Paragraph prepended to every AI prompt's visitor-context block.
Max Recent Pagesnumber 1–205Limits on context size.
Max Past Conversationsnumber 1–205
Max Extracted Factsnumber 1–5015

Verification Policy options:

PolicyBehavior
PermissiveTrust every claim — any session typing a known email/phone gets full cross-session access. Highest impersonation risk.
ModerateScore-based — auto-verifies when corroborating signals (FingerprintJS hash, browser signature, IP/subnet, known identifier) reach a moderate threshold.
StrictSame scoring engine, higher threshold — requires multiple corroborating signals or a direct trigger.
RegulatedScore-based path disabled — only form / CRM / inbound-call / first-profile / returning-device triggers verify.

Per-company override flow: search a company (debounced lookup against /admin/companies), tick Use a custom Identity Graph config for this company, then save (PUT /admin/identity/config/company/:id with {use_custom_config:true, config:{...}}). Override fields left blank inherit from the global defaults. Unticking and saving sends {use_custom_config:false} to revert the company to global defaults.

Behaviors & edge cases​

  • All endpoints are super-admin only. Non-admin requests get 403 Unauthorized.
  • Dry-run by default. Maintenance writes require apply:true. "Prune empty Persons" additionally rejects a non-apply request (400 — run a dry-run count first).
  • Query timeouts. Person/profile reads cap at 15s and return 504 — Query timed out, try again or narrow the request; the company_id-less variant of the has_profile lookup is rejected as unbounded.
  • Trim Session Events errors are in-band (HTTP 200, {type:'error'} line) — the body must be checked; a thrown UI error means the stream reported a failure.
  • Conflict provenance. A profile's "conflicting claims" flag is true when any identity event's source matches ^conflict_with_verified:.
  • Delete is a hard cascade. Delete person/profile has no dry run — the ConfirmDialog is the only guard, and the sessions fall back to anonymous.
  • Why didn't this session get claimed? The markdown-preview diagnostic lists the device-claim reason for unlinked sessions: no signature (fingerprinting blocked, e.g. GDPR region), no verified profile on any tenant carries it, shared by N persons, soft-signature-only, last verification older than 90 days — and the IP-claim reason (no raw IP, GDPR-blocked, private/CGNAT, bot, shared network, unknown, ambiguous, unverified, stale).
  • Detach / split / unlink retract signals. The repaired sessions' fingerprint, IP and widget id are pulled from the Person they left, so they stop recognizing future visits as that Person.
  • Reconcile "overwrite anyway" may re-link the GHL contact to a different contact if the email/phone already belongs to one — confirm the contact really is this person first.

Technical implementation​

  • Routes: backend/app/routes/adminIdentityRoutes.js (all /admin/identity/*, passport-authed).
  • Controller: backend/app/controllers/AdminIdentityController.js — also exposes a diagnostic GET /admin/identity/sessions/:user_session_id/markdown-preview returning the exact markdown an AI agent would receive plus the gating flags (handy when context is unexpectedly empty).
  • Frontend repo: frontend/src/repositories/admin/identity.js; views under frontend/src/views/admin/identity/.
  • Services that back these screens: IdentityGraphConfigService, IdentityGhlReconcileService, IdentityCleanupService (prunePersons / deleteProfile shared with the delete endpoints), IdentityNormalize, IpSignalSeedService (wired as global.identityCleanup, global.ipSignalSeed), plus lib/userSessionNameBackfill.js, lib/sessionEventsTrim.js, lib/progressStream.js, lib/apolloContactUnlink.js, lib/deviceClaimBackfill.js, lib/personMerge.js, lib/ipHashIndexMigration.js.
  • Config defaults model: backend/app/models/IdentityGraphDefaultsModel.js.

Person repair (2026-08-17)​

Person detail gained Edit identity, Detach match key, Split selected match keys into a new Person, Reset trust signals, Unlink session, and Copy JSON (diagnostic export incl. identity_audit + shadow markers). All dry-run unless apply; every write emits an admin_repair identity event. Blanking a canonical field that still has a match key is refused — Detach is the removal path; phones must be E.164. Details and endpoints in Identity Graph › Admin surface. Identity Maintenance also hosts the CRM Port Smoke Test and the CRM Links backfill cards (both async job + poll). The CRM Links backfill retries a company's scan and each bulk write up to 4 times with backoff on transient Mongo errors (timed out / network / ECONNRESET / socket) instead of failing the whole run on one dropped connection; seen-sets stop retries double counting (a9301ed8b).