Identity Graph (Admin)
Audience: Ops · Super-admins · Engineering · Support escalations · Where in app: /admin/identity/* · Access: Super-admin only — every endpoint enforces request.user.user_type === 'admin' (AdminIdentityController._requireAdmin, returns 403 Unauthorized otherwise).
The Identity Graph is the platform's record of who a visitor is across every session, device, and company. A Person is the global identity; a PersonProfile is that person's per-company facet (their CRM links, extracted facts, bookings, form answers for one tenant). These admin screens let a super-admin browse those records cross-tenant, tune how aggressively identity is trusted, resolve CRM write conflicts, and run one-shot data-maintenance sweeps. For the conceptual model and runtime behavior see the subsystem and module docs linked under Related.
What it does
- Browse the graph — list/search Persons, per-company Profiles, and every captured form submission across all tenants.
- Inspect one Person — see their profiles, recent sessions (with trust state), bookings, match keys, identity audit trail, CRM enrichment, and IP signals.
- Configure verification posture — set global Identity Graph defaults and per-company overrides (the super-admin replacement for the old customer-side settings UI).
- Reconcile CRM conflicts — review GoHighLevel contacts where first-party identity should supersede enrichment but looked like a real pre-existing contact, so the system flagged instead of overwrote.
- Repair and delete — edit a Person's identity, detach/split match keys, reset trust signals, unlink a session, delete a Person or one company's Profile.
- Review abandoned-form detections — a verdict queue over forms the recording analyser flagged as abandoned, with the recording and stored submissions as evidence.
- Run maintenance jobs — dry-runnable backfills/cleanups (cleanup, normalize, moderate names, trim bloated session-event arrays, IP-signal backfill and analyses, session-name backfill, GHL reconcile backfill, Apollo-contact unlink, device-claim backfill, duplicate-Person merge, IP-signal hash keying).
How it works
The frontend repository frontend/src/repositories/admin/identity.js wraps the backend routes declared in backend/app/routes/adminIdentityRoutes.js (all under /admin/identity/*, all passport-authed). The controller is backend/app/controllers/AdminIdentityController.js.
Reads (persons / profiles / form-submissions / reconcile-conflicts) are paginated via a shared buildQuery that converts {page, pageSize} into skip/limit. List queries cap MongoDB execution at 15s (maxTimeMS) and return 504 on timeout rather than hanging until the load balancer cuts the connection.
Maintenance writes are super-admin-gated wrappers around the api-key-gated /migrations/* sweeps, so they can be triggered from the admin UI. Every write action defaults to a dry run (apply:true must be sent to actually mutate). Short jobs return a {mode, company_id, elapsed_ms, stats} envelope; long ones start a background job and return its id, which MigrationPanel polls at …/jobs/:job_id (so a page refresh or ingress timeout doesn't lose the run). Since 2026-10-13 the panel keeps polling through transient poll failures — the job is still running server-side — and only gives up after 20 consecutive failures or a 404 (ef7e3129); the raw result has a copy button and JSON highlighting (8a99ec23). The older Trim Session Events job streams NDJSON progress instead (see below).
Screens & fields
Cross-entity navigation
Every Identity screen shows a row of nav pills (Persons, Profiles, Form Submissions, CRM Reconcile, Maintenance, Config) so an admin can move between facets of the same data.
Persons list — /admin/identity/persons
Source: identity/Persons.vue → getPersons. The global directory of unique people.
| Filter | Values | Notes |
| ----------- | -------------------------------------- | ----------------------------------------------------------------------------- | ------ |
| Search | free text | matches name, email, or phone (case-insensitive regex), debounced 500ms |
| Has Profile | All / With profiles / Without profiles | sends has_profile=true | false |
| Sort | Last active / First seen / Name | sort field; order is -1 (desc) |
| Column | Source field |
|---|---|
| Name | person.name |
person.email | |
| Phone | person.phone |
| Match keys | count of person.match_keys[] |
| Profiles | person.profile_count |
| Sessions | person.session_count |
| Last active | person.last_active_at |
| Actions | View → person detail |
Rows are click-through to the detail page. Pagination total comes from pagination.total.
Person detail — /admin/identity/persons/:id
Source: identity/PersonDetail.vue → getPerson (controller aggregates counts server-side to avoid shipping the heavy form_submissions / extracted_facts / crm_enrichment arrays over the wire). Header shows name, first-seen date, and match-key count, plus the repair actions (see Person repair) and Delete person (ConfirmDialog → DELETE /admin/identity/persons/:id). The edit dialog seeds its phone country picker from the most recent session's ip_info (returned by getPerson); before, vue-tel-input's failed auto-detect landed on Afghanistan and re-saved numbers with the wrong code. Tabs:
| Tab | Contents |
|---|---|
| Company Profiles | one card per PersonProfile: company, first-seen/last-active, form-submission count, extracted-fact count, CRM-presence flag, recent identity events, a "conflicting claims" flag (any event whose source starts with conflict_with_verified:) |
| Sessions | last 50 sessions (Company, Identified as, Trust, Current URL, Created); trust derives from identity_trust / verified_person_id / claimed_person_id; passive links (device / IP / widget id) carry the amber badge until verified |
| Bookings | bookings flattened across all sessions + per-company profiles |
| Match Keys | Type, Normalized value, Hash — the deterministic keys (email/phone/etc.) used to link records |
Profile detail — /admin/identity/persons/:id/profiles/:company_id
Source: identity/PersonProfileDetail.vue → getPersonProfile. The full per-company facet, with Delete profile (DELETE /admin/identity/persons/:id/profiles/:company_id — this company's profile, IP signals, session stamps and funnel links only; the Person stays). Tabs:
| Tab | Contents |
|---|---|
| Form Submissions | every captured form answer for this person+company |
| Extracted Facts | facts the AI extracted from conversations |
| Bookings | this profile's bookings |
| CRM Enrichment | HubSpot and Go High Level enrichment blocks |
| Identity Audit | the audit log: Event, Source, Score, Session, When; flags Conflict and Needs review rows |
| IP Addresses | reverse-IP signals: IP ("hashed only" when the row has only an ip_hash — Snitcher-enriched sessions have their raw IP wiped), Subnet, Ver (verified), Hits, First seen, Last seen |
Profiles list — /admin/identity/profiles
Source: identity/PersonProfiles.vue → getPersonProfiles. A flat cross-tenant list of PersonProfiles (filterable by company, sort/order via buildQuery).
Form submissions — /admin/identity/form-submissions
Source: identity/FormSubmissions.vue → getFormSubmissions. Every form answer captured across the graph, rendered as cards with Person/Company/profile crumbs.
| Filter | Values |
|---|---|
| Search | field label or value (debounced 500ms) |
| Form type | All · contact_info · pre_call · booking · website_form · custom |
| Source | All · chat · call · booking_flow · auto_trigger · form_intercept |
Each card shows form_type, source, the relationship crumbs, submitted_at, and a key/value grid of fields[] (label/key → value).
CRM Reconcile — /admin/identity/reconcile-conflicts
Source: identity/ReconcileConflicts.vue → getReconcileConflicts / resolveReconcileConflict. The review queue of GoHighLevel contacts where a visitor declared real first-party details (form/booking/call) but the linked contact looked like a real, pre-existing person, so the system flagged instead of overwrote.
Each conflict card shows: Person crumb, Company crumb, First-party email, First-party phone, GHL contact id, Flagged via (latest_conflict_source, with a ×N repeat count), and latest_conflict_at. Actions per card:
| Action | Effect | API payload |
|---|---|---|
| Overwrite anyway | Push first-party details to the GHL contact (re-links to another contact if that email/phone already belongs to one). Use only when the contact IS this person. Opens a confirm dialog. | {person_id, company_id, action:'overwrite'} |
| Dismiss | Keep the existing GHL contact, drop from queue. Use when enrichment matched a different person. Optional Also unlink clears the wrong person↔contact association. | {person_id, company_id, action:'dismiss', unlink?} |
The reconcile backfill itself is run from the Maintenance screen (see "GHL Reconcile Backfill" below), not here.
Form Abandonment — /admin/form-abandonment
Source: views/admin/form-abandonment/Index.vue + Detail.vue (repository repositories/admin/formAbandonment.js) → AdminFormAbandonmentController (routes/adminFormAbandonmentRoutes.js). Sidebar: Identity Graph → Form Abandonment. Shipped 2026-10-08 (backend b8e4cd5b7, frontend 2a6a515c) to measure the recording analyser's "abandoned form" detector before a form_abandoned Agent K trigger is built on it.
- Queue —
GET /admin/form-abandonment: forms from ms-sessionssession_analyses.events.forms.forms, filtered byresults(detector outcome:abandoneddefault,unknown,likely_submitted,submitted),from/to(default last 7 days, max 92),company_id,search(page URL),review_status(unrevieweddefault,reviewed, or a verdict). Returns verdict stats alongside the page. - Detail —
GET /admin/form-abandonment/:file_id/:form_index: the session recording (jumps to each field, last activity and stored submissions —RecordingPlayergainedseekTo(absoluteTs)+ aloadedevent), the detector data, the page visit, form submissions stored on the visitor's profile during the recording (counter-evidence from the form interceptor; the window is the recording −5 min / +10 min, absorbing visitor-clock skew), company setup, and diagnostic warnings (capture_disabled,booking,partial,visitor_unknown,focused_on_exit,short_tail,little_typing,no_fields,only_script_fields). Newer analyses carry exactpage_entered_at/last_activity_at; older ones fall back to time/label matching. - Script-filled inputs (2026-10-09, backend
ade59a83c, ms-sessions0c83a9b) — page scripts (pixels, CRM snippets) write hidden/tracking inputs on every click and rrweb records them as input events. The ms-sessions form extractor now skips hidden/non-form inputs, marks each fieldfocused: true|false, and only lets focused fields start a form or move its last activity. The detail view shows only fields the visitor worked (focused, or for older analyses any focus time), reportsscript_field_count, re-derives "stopped at" from the last real field, and warnsonly_script_fields("probably Not a real form") when nobody typed. - Review —
PUT …/:file_id/:form_index/reviewwith a verdict (confirmed_abandoned,submitted,not_a_form,recording_issue,unclear) + notes; Save & next walks the queue. Stored inform_abandonment_reviews(unique onfile_id, form_index), with a submission snapshot and a time-orderedhistory[]of earlier verdicts.(file_id, form_index)is an array position a re-analysis or the phantom-form scrub can reshuffle, so a review only counts while the form there still has the reviewed page +fields_touched_order; otherwise the old verdict stays in history instead of landing on the wrong form.
Maintenance — /admin/identity/maintenance
Source: identity/IdentityMaintenance.vue, each job rendered via components/admin/MigrationPanel.vue. Every job defaults to a dry run (reads/counts only); review the stats, then Apply. Most accept an optional Company ID to scope to one tenant.
| Job | Endpoint | Writes? | Key fields | What it does |
|---|---|---|---|---|
| GHL Reconcile Backfill | POST …/maintenance (runReconcileBackfill → /admin/identity/reconcile-backfill) | yes | Company ID, Limit (1–5000, default 200) | Sweeps already-verified sessions where first-party identity should supersede Happier-Leads enrichment and corrects their GHL contacts; real pre-existing contacts are flagged into CRM Reconcile, not overwritten. |
| Cleanup Identities | POST /admin/identity/maintenance/cleanup | yes | Company ID, Clean sessions, Drop foreign submissions, Prune empty Persons (destructive — requires Apply) | Removes invalid identity values (bad emails/phones/names) from Person, profiles, sessions, form submissions; optionally prunes empty Persons and drops foreign submissions. |
| Normalize Identity Values | …/cleanup with normalize:true | yes | Company ID (sessions only) | Lowercases emails, promotes phones to E.164 where a country code is known (explicit + or session region — never a default), collapses name whitespace, adds match keys for the normalized values (never drops existing keys; rows whose key another Person owns are skipped). Rewrites values; deletes nothing. |
| Moderate Names | …/cleanup with moderateNames:true | yes | Company ID (sessions only) | Blanks stored Person + session names the NameModeration gate now rejects (junk / non-name phrases / profanity); display falls back to "Unknown Visitor". |
| Backfill IP Signals | POST /admin/identity/maintenance/backfill-ip-signals | yes | Company ID, Days (1–365, default 90) | Seeds the reverse-IP index (person_ip_signals) from recent sessions so IP-assisted resolution and the per-person IP tab have data. |
| IP Uniqueness Analysis | POST /admin/identity/maintenance/ip-uniqueness | read-only | Company ID, Days (default 90) | Measures how many people share an IP / subnet — informs how aggressively IP can be trusted. |
| IP Shadow Stats | POST /admin/identity/maintenance/ip-shadow-stats | read-only | Company ID, Days (default 30) | How often a live IP claim matched the person the visitor later resolved to (confirmed) vs a different one (contradicted). IP claims are live; a contradicted claim contests its IP for 90 days. |
| Recognition Coverage | POST /admin/identity/maintenance/recognition-coverage | read-only | Company ID, Days (1–90, default 7) | Of new sessions by people we already knew, how many were recognized at session start (carried / device / IP) or by link (kk, CRM, email click) vs had to identify again — with the refusing gate per route, device/browser breakdown, and tenants whose graph is unavailable. Counts only sessions created after 2026-10-15. See Recognition coverage. |
| Backfill Session Display Name | POST /admin/identity/maintenance/backfill-session-name | yes | Company ID, Sample rows (0–100, default 20) | Populates user_session.name from legacy first_name+last_name (treats Guest/User/empty as no real data). |
| Trim Session Events | POST /admin/identity/maintenance/trim-session-events | yes | Company ID, Keep last N (default 500), Page size (default 100), Max pages/call (default 500), Resume after _id | Caps tab_focus_events / video_unmutes arrays bloated by the (now $slice-capped) uncapped $push. Pages by _id so it never loads the bloated docs; resumable. |
| Unlink Apollo CRM Contacts | POST /admin/identity/maintenance/unlink-apollo-contacts (job) | yes | Company ID, include anonymous | Cuts sessions loose from a GHL contact an Apollo guess produced (link, crm_links row, cached ghl_context); the contact itself is never touched. Two passes: the ghl_appolo_sync audit trail, then a tenant-by-tenant scan for the pre-2026-08-13 backlog. Samples show found_by and the person we think the visitor is. |
| Backfill Device Claims | POST /admin/identity/maintenance/backfill-device-claims (job) | yes | Company ID, Days (default 30) | Re-runs the cross-tenant device claim over anonymous sessions created before 2026-09-09. Starts from fresh verified fingerprints, scans in one-hour slices. |
| Merge Duplicate Persons | POST /admin/identity/maintenance/merge-duplicate-persons (job) | yes | Max cluster size (default 5), Rebuild unique index, Detach shared keys | Merges Persons sharing a match key (oldest wins) when every member agrees on email + phone; others are reported as conflicting_clusters. Dry run reports indexes_missing — Apply refuses until those indexes finish building after a deploy. Rebuild is refused while conflicting/oversized clusters remain. |
| Key IP Signals by Hash | POST /admin/identity/maintenance/ip-hash-index (job) | yes | — | Fills ip_hash on legacy person_ip_signals rows, folds twins, swaps the unique index. Also runs automatically on backend boot under a Redis lock; the card is for visibility and re-runs. |
Trim Session Events — NDJSON streaming
This job can run long, so it streams NDJSON (backend/app/lib/progressStream.js) instead of buffering one JSON body — a heartbeat keeps the ingress idle-timeout from killing the request. The repository (runTrimSessionEvents) reads responseType:'text', splits on newlines, and parses each line:
{"type":"progress", "...":"..."}
{"type":"ping"}
{"type":"result","mode":"APPLY","stats":{"cap":500,"scanned":1200,"trimmed":340,"done":false,"next_after":"<objectId>"}}
{"type":"error","message":"..."}
Errors arrive in-band with HTTP 200, so the body must be inspected. Because the job is batched and resumable, when done:false paste the returned next_after into Resume after _id and Apply again until done:true.
Config — /admin/identity/config
Source: identity/Config.vue + identity/_ConfigForm.vue → getIdentityGraphDefaults / updateIdentityGraphDefaults / updateCompanyIdentityGraphOverride. Sets the global defaults that apply to every company, plus optional per-company overrides.
The config form fields (shared between global and override):
| Field | Type | Default | Meaning |
|---|---|---|---|
| Enable Identity Graph | toggle | on | When off, AI agents receive no cross-session identity context. |
| Capture Website Forms | toggle | on | Injects the form-intercept script alongside the widget so customer-site form submissions flow into the graph. |
| Verification Policy | radio | strict | How aggressively a session that claims an identity is promoted to "verified" — see below. |
| Trust Inbound Caller-ID | toggle | on | Promote an inbound voice call's session to verified when caller-ID matches a known phone. |
| AI Agent Context Sections | checkboxes | all | Which sections to include in visitor context (Identity, Company Insights, Engagement, Current Session, Recent Pages, Past Conversations, Behavioral Signals, CRM State, Bookings, Form Responses, Extracted Facts). Leave all unchecked = include everything. |
| Custom Intro | textarea | — | Paragraph prepended to every AI prompt's visitor-context block. |
| Max Recent Pages | number 1–20 | 5 | Limits on context size. |
| Max Past Conversations | number 1–20 | 5 | |
| Max Extracted Facts | number 1–50 | 15 |
Verification Policy options:
| Policy | Behavior |
|---|---|
| Permissive | Trust every claim — any session typing a known email/phone gets full cross-session access. Highest impersonation risk. |
| Moderate | Score-based — auto-verifies when corroborating signals (FingerprintJS hash, browser signature, IP/subnet, known identifier) reach a moderate threshold. |
| Strict | Same scoring engine, higher threshold — requires multiple corroborating signals or a direct trigger. |
| Regulated | Score-based path disabled — only form / CRM / inbound-call / first-profile / returning-device triggers verify. |
Per-company override flow: search a company (debounced lookup against /admin/companies), tick Use a custom Identity Graph config for this company, then save (PUT /admin/identity/config/company/:id with {use_custom_config:true, config:{...}}). Override fields left blank inherit from the global defaults. Unticking and saving sends {use_custom_config:false} to revert the company to global defaults.
Behaviors & edge cases
- All endpoints are super-admin only. Non-admin requests get
403 Unauthorized. - Dry-run by default. Maintenance writes require
apply:true. "Prune empty Persons" additionally rejects a non-apply request (400— run a dry-run count first). - Query timeouts. Person/profile reads cap at 15s and return
504 — Query timed out, try again or narrow the request; thecompany_id-less variant of thehas_profilelookup is rejected as unbounded. - Trim Session Events errors are in-band (HTTP 200,
{type:'error'}line) — the body must be checked; a thrown UI error means the stream reported a failure. - Conflict provenance. A profile's "conflicting claims" flag is true when any identity event's
sourcematches^conflict_with_verified:. - Delete is a hard cascade. Delete person/profile has no dry run — the ConfirmDialog is the only guard, and the sessions fall back to anonymous.
- Why didn't this session get claimed? The markdown-preview diagnostic lists the device-claim reason for unlinked sessions: no signature (fingerprinting blocked, e.g. GDPR region), no verified profile on any tenant carries it, shared by N persons, soft-signature-only, last verification older than 90 days — and the IP-claim reason (no raw IP, GDPR-blocked, private/CGNAT, bot, shared network, unknown, ambiguous, unverified, stale).
- Detach / split / unlink retract signals. The repaired sessions' fingerprint, IP and widget id are pulled from the Person they left, so they stop recognizing future visits as that Person.
- Reconcile "overwrite anyway" may re-link the GHL contact to a different contact if the email/phone already belongs to one — confirm the contact really is this person first.
Technical implementation
- Routes:
backend/app/routes/adminIdentityRoutes.js(all/admin/identity/*, passport-authed). - Controller:
backend/app/controllers/AdminIdentityController.js— also exposes a diagnosticGET /admin/identity/sessions/:user_session_id/markdown-previewreturning the exact markdown an AI agent would receive plus the gating flags (handy when context is unexpectedly empty). - Frontend repo:
frontend/src/repositories/admin/identity.js; views underfrontend/src/views/admin/identity/. - Services that back these screens:
IdentityGraphConfigService,IdentityGhlReconcileService,IdentityCleanupService(prunePersons/deleteProfileshared with the delete endpoints),IdentityNormalize,IpSignalSeedService(wired asglobal.identityCleanup,global.ipSignalSeed), pluslib/userSessionNameBackfill.js,lib/sessionEventsTrim.js,lib/progressStream.js,lib/apolloContactUnlink.js,lib/deviceClaimBackfill.js,lib/personMerge.js,lib/ipHashIndexMigration.js. - Config defaults model:
backend/app/models/IdentityGraphDefaultsModel.js.
Related
- Identity Graph subsystem
- Identity module — overview
- Person vs Profile
- Verification policy
- Cross-session context
- Form intercept
- Identity admin controls
- CRM integration subsystem
- Backend service
Person repair (2026-08-17)
Person detail gained Edit identity, Detach match key, Split selected
match keys into a new Person, Reset trust signals, Unlink session, and
Copy JSON (diagnostic export incl. identity_audit + shadow markers). All
dry-run unless apply; every write emits an admin_repair identity event.
Blanking a canonical field that still has a match key is refused — Detach is
the removal path; phones must be E.164. Details and endpoints in
Identity Graph › Admin surface.
Identity Maintenance also hosts the CRM Port Smoke Test and the
CRM Links backfill cards (both async job + poll). The CRM Links backfill retries a company's scan and each bulk write up to 4 times with backoff on transient Mongo errors (timed out / network / ECONNRESET / socket) instead of failing the whole run on one dropped connection; seen-sets stop retries double counting (a9301ed8b).