Recorded Sessions
Audience: Sales · Support · QA · Engineering · Management · Where in app: Recorded Sessions (/recorded-sessions) · Plan availability: All plans
Every visit is recorded with rrweb. After the visitor leaves, the recording finalizes and is analyzed into structured signals (clicks, scrolls, forms, intent, friction). Admins can browse the list, filter and semantically search it ("people who scrolled past the testimonials and got stuck on pricing"), and replay any session like looking over the visitor's shoulder. This is where teams reconstruct what actually happened before a chat, a booking, or a bounce.
What it does
- Captures and stores every session, then analyzes it into a structured
session_analysesartifact. - Lets admins filter by search (visitor/company/IP), Identified / Verified, IP address, location, page URL, traffic source, Recording (With replay / All visitors), device, new vs returning, duration, and date range (
RecordedSessionsFilter.vue). - Supports free-text semantic search across per-visit narrative prose (pgvector
knox_search_index). - Replays the session with full playback controls, an event timeline, page-trip navigator, friction markers, and a conversation overlay.
- Rolls each session up into a per-visitor profile so the AI and Nox see cumulative history.
How it works
Capture
The widget initializes rrweb on page load and captures DOM mutations, mouse moves, scrolls, clicks, inputs, console errors and logs. Events stream to ms-sessions as NDJSON chunks (each chunk carries a first_event_at/last_event_at window), buffered in Redis and drained to S3 in 100-event chunks. Chunk metadata is a counter (session_files.chunk_count, objects named <n>.ndjson) — playback reconstructs names via resolveChunkNames(); legacy files[] arrays are still honored. Session HTTP endpoints are rate-limited (~30/min).
Tenants with consent gating enabled don't capture at all (behaviour category) until the visitor consents; a mid-session grant records forward-only.
Finalization
A session moves active → finalized (a.k.a. stale) when the visitor closes the tab (best-effort), goes idle, or navigates away from every domain the widget is on. Finalize grace is 90s (FINALIZE_GRACE_TTL_SECONDS, was 5min) with a 15s sweep (FINALIZE_SWEEP_INTERVAL_MS, was 30s) — a recording is typically analyzed ~2 minutes after the visitor leaves. The sweep only looks back 2 days (FINALIZE_SWEEP_MAX_AGE_MS). Finalization enqueues a BullMQ recording-analysis job. A backend sweeper (RecordingBackfillSweeperJob, ~every 15 min) re-enqueues any session whose handoff failed.
Analysis pipeline
A separate worker deployment consumes the recording-analysis queue (concurrency ~4, ~120s job timeout; autoscaled on queue depth, ~1–10 pods). It:
- Fetches NDJSON chunks from S3.
- Stitches them into a coherent timeline split by page trip (
stream-stitcher). - Rebuilds the DOM (
dom-resolver) for click/scroll/form context. - Runs extractors — page-trip, click, scroll, form, hover, idle, viewport, error, selection (see Per-Session Signals).
- Synthesizes intent (
intent-synth) — primary intent + confidence + evidence + friction points + high-intent signals. - Geo-backfills the visitor IP → country/city.
- Stitches video frames → MP4 and uploads to S3.
- Rolls up the visitor profile atomically.
- Hands off to ms-ai (
ms-ai-handoffqueue) for embedding/index + selective tier-2 narration.
Result: a session_analyses row keyed by file_id, plus an updated visitor_profiles row.
Naming note: the nox-KB describes extractors under
ms-sessions/app/workers/recording-analysis/processor.js+/extractors/. The deployed worker entrypoint isbin/recording-analysis-worker.js; the analysis stages (dom-resolver,intent-synth,stream-stitcher,geo-backfill) live inapp/workers/recording-analysis/. (verify exact extractor file paths if citing them directly)
Tier-2 narration (selective LLM pass)
For high-value or stuck sessions (visited a high-value page, long dwell, high intent) a second LLM pass writes narrative prose (human summary), a refined intent, and action recommendations. It is capped per company by RECORDING_LLM_DAILY_CAP_PER_COMPANY and policy in companies.knox.high_value_pages[].
Replay
The dashboard plays recordings with rrweb-player (frontend/src/components/SessionPlayer/). Controls: play/pause, playback speed 1x–8x (KB notes 0.5x–4x in one place; UI exposes up to 8x — verify exact range), skip inactive, timeline scrubbing, jump-to-event (clicks/inputs/scroll/navigation), a page-trip navigator, friction markers, and a conversation overlay if a chat/call happened during the session.
Multi-tab playback: Follow Activity mode auto-switches between tabs as the visitor did; Individual Tab view watches one tab in isolation.
Session details panel (redesigned 2026-07/08)
frontend/src/components/sessions/SessionMoreInfoPanel.vue:
- Meeting Booked callout when the session carries a booking.
- Session Summary — the tier-2 narration prose, which lives only in ms-ai
Postgres; fetched via the backend proxy
GET /user-session/:_id/narration. - Engagement score tooltip explaining the score's inputs.
- Identity "possible match" is now an icon-only badge (no unverified name shown).
- Agent K outreach panel (
SessionAgentKOutreach.vue) when the session came from tracked outreach. - Arrival sources on the timeline, including labels Agent K Outreach, ManyReach Campaign, and CRM Email Link.
- Ask Nox about this visitor (2026-10-20, frontend
6af9995d) — an input at the top of the panel (live and recorded; hidden when the company is paused). Submits viaknoxStore.ask(q, { entity_context: { type: 'visitor', id: <session id>, name } })and opens the Nox panel.nameis the verifiedsession.person.nameonly — a suspected match is never passed as who the visitor is. Asking about a different session than last time callsknoxStore.clearConversation()first, because Nox follows earlier turns'entity_context; the old thread stays in history.
Configuration & options
| Setting / field | Where | Effect |
|---|---|---|
companies.knox.high_value_pages[] | Company Nox config | Pages that make a session eligible for tier-2 narration and feed high-intent signals |
RECORDING_LLM_DAILY_CAP_PER_COMPANY | Env | Daily cap on tier-2 LLM narration passes per company |
RECORDING_DOC_SIZE_LIMIT_BYTES | Env | Max recording-doc size before a session is force-finalized/split |
RECORDING_WORKER_CONCURRENCY / RECORDING_WORKER_JOB_TIMEOUT_MS | Worker env | Analysis worker concurrency (~4) and job timeout (~120s) |
recording_retention setting (enabled, retention_days, speed) | Admin → Settings → Recording Retention | Per-visitor retention (below). Off by default; retention_days default 60, floor 30; speed 1–64, default 32 |
with_playable_data (query param) | Recorded Sessions UI | true/absent = only sessions with a replay (default); false = All visitors, incl. those who left before the recorder started |
For QA/support: the Recording filter's default (With replay) hides visitors who left before the recorder's ~6 s start delay — the dashboard counts them, so switch to All visitors (pill "Incl. no replay"; the dashboard ?playable=any deep link sets it) when numbers don't match (backend 4226ab500, frontend 8f23cbbd). Use search to pull all of one visitor's recordings.
Behaviors & edge cases
- Cross-origin iframes are not captured unless the iframe companion script is installed — replay of those regions will be blank.
- Heavy canvas/WebGL pages produce large chunks and may exceed
RECORDING_DOC_SIZE_LIMIT_BYTES, forcing early finalization/splitting. - Tier-2 narration only runs on eligible sessions within the daily cap — most ordinary sessions get structured signals but no prose summary.
- Form input may contain sensitive data. Recordings can capture what visitors typed (except masked/
data-rrweb-blockfields and password inputs). Teams should follow their privacy policy when viewing. - Per-visitor retention (2026-11-04) — backend
RecordingRetentionJob(every 15 min, ≤10 min per run so runs never overlap; schedular) calls ms-sessionsPOST /internal/recordings/expire(internal api key;retention_days,apply,time_budget_ms,concurrency1–64) repeatedly whilehas_more. It deletes the recordings (session_filesdoc + S3 prefix / local dir, storage first) of visitors with no recorded visit inside the window — a visitor who came back keeps all of their recordings. "Came back" = latestuser_sessions.sessions.start_time(alsouser_sessions_archive), or a newuser_sessionsrow with the samecompany_id+widget_user_session_idsince the cutoff; a visitor on the site now always counts. Visitors left with none gethas_recordings: false. Scan walks_idascending with the cursor in Redis (a full pass at most once a day, 250 ms between batches); each query capped at 15 s and each storage delete at 30 s; a failed step returnserror+has_morewithout advancing the cursor, and the job records it on the admin card and ends the run.session_analysesis retained. The admin card (super-admin only) shows lifetime counters, last run/error, per-steptimings_ms, andstored(estimatedDocumentCountofsession_files). Ad-hoc cleanup still runs through Storage Maintenance (POST /admin/storage/session-files-cleanup, dry-run by default). - Privacy — what the recorder captures depends on GDPR safe mode and recording masking v2; see GDPR & Privacy. Call/room video recordings (
GET /video-recordings/:id, ms-sessions) now require a logged-in user with access to the room's company (9f4d058); a non-ObjectId id is a 400. - Global kill switch —
ms-sessionssession_recordings_enabled(hardcoded ON at HEAD) can drop all ingest/playback; a per-session chunk cap was trialled and reverted after a prod incident. - Semantic search matches narrative prose, which only exists for tier-2-narrated sessions — searches may miss sessions that never got a prose pass.
- Opens on Identified (frontend
56e5c49b, 2026-10-09) —RecordedSessionsView.applyDefaultIdentity()setsrecordedFilters.identified = 'true'on entry unless a link scopes the list (identified,verified,sessions,date,visitor_type,playablequery params — insight cards / dashboard tiles count ALL sessions) or Verified is on. Filters are applied duringsetup, notonMounted, so the slow Identified query isn't fired twice.useRecordedSessionsQuerykeeps pages for 30 min (gcTime, default 5) — revisits render cached rows and refetch in the background. - View Chat with no chat — old empty per-visit chats are archived, so a session may have none: Recorded Sessions shows the info toast "This visitor didn't chat" instead of an error (
2072f9b2). - Deep-linked sessions list all sessions. Opening a session by deep link (e.g. from a kk-click or notification) now resolves against all sessions, not just recorded ones; partial indexes support the kk-click/deep-link listing (e.g.
kk_click_sessionson{ from_email_click: true }in backendUserSessionModel).
Plan & limits
- Recorded Sessions is on all plans. Retention is platform-wide (the
recording_retentionsetting), not per plan. - Semantic search quality scales with how many sessions received tier-2 narration, which is capped per company per day.
- B2B company resolution and identity on the visitor info panel depend on enrichment/identity features — see Lead Intelligence.
Technical implementation
- Service:
ms-sessions—RecordedSessionController(list/search/replay),VideoRecordingController+ jobs (RoomVideoMerging,RoomVideoProcessing,DeleteExpiredVideos,DeleteRawVideoFiles) for video synthesis/retention. Worker:bin/recording-analysis-worker.jsconsuming therecording-analysisBullMQ queue, thenms-ai-handoff. - Collections:
session_files(recording metadata +recording_status),session_analyses(per-file structured artifact: page_trips, clicks, scroll metrics, forms, hovers, idles, selections, errors, intent, friction_points, high_intent_signals, optional narrative_prose/action_recommendations),visitor_profiles(one per(company_id, user_session_id): visit_count, recent_sessions cap ~20, cumulative active_ms, unique_pages_visited, recurring_topics/friction, intent_trajectory, engagement_score, conversion_signals, narrative_summary). - Search index:
knox_search_index(pgvector) in ms-ai for semantic search. - Nox tools:
knox-session-analysis(file_id),knox-session-search(query, filters),knox-aggregate(bounce_rate, top_exit_pages, funnel_path…),knox-visitor-profile(user_session_id),knox-history-lookup(person_id). - Subsystems: Session Recording, Identity Graph, Knowledge Base / RAG.
Related
- Live Sessions — watch the same stream in real time
- Per-Session Signals — every field the extractors produce
- Lead Scoring — how session behavior feeds the score
- Booking Sources — attributing a booking back to its session journey
- ms-sessions · Session Recording