ms-sessions — Recording & Presence

Captures visitor sessions (rrweb), turns them into analyzed recordings, and is the source of truth for visitor presence. A Nodevel service on MongoDB + Redis + S3.

• Port: 3003. A separate recording-analysis worker process runs the heavy pipeline.

Responsibilities

• Capture — receive rrweb DOM events from the widget over Socket.IO, buffer in Redis, flush to disk, enqueue for processing.
• Video synthesis — stitch event streams into MP4 (ffmpeg / webm-to-mp4 / video-stitch / handbrake), tag for AI analysis.
• Session state — user_sessions, session_files, lead scoring, visitor profiles.
• Presence — a cross-pod tab registry queried by ms-ai/Knox and ms-communication to know if a visitor is live.
• Lead qualification — engagement metrics (pages, time on high-intent pages, chat/call acceptance, bookings).

Boot flow

Standard Nodevel; app/hooks.js boot() loads Redis, Auth, Mongo, Cron, Logger, Mail, Permissions, Socket, S3, and a 30 req/min rate limiter on session endpoints. The HTTP server binds port 3003; the worker boots the same app with __skip_server = true.

Directory map

app/
├── models/       UserSessionModel, SessionFileModel, CompanyModel, UserModel
├── services/     S3Service, SocketService (rrweb buffering), AuthService, MailService
├── controllers/  RecordedSessionController, LiveSessionController, KnoxPresenceController
├── jobs/         DeleteExpiredVideos, RoomVideoMerging, RoomVideoProcessing
├── queues/       recording-analysis.queue.js, ms-ai-handoff.queue.js
├── workers/      recording-analysis/ (dom-resolver, intent-synth, stream-stitcher, geo-backfill)
└── routes/       api.js, internalKnox.js
bin/
├── www                          HTTP server
└── recording-analysis-worker.js BullMQ consumer (separate k8s Deployment)

Data models

Model	Purpose
user_sessions	session lifetime, current URL, nested lead_score
session_files	per-page rrweb recording; chunk_count counter (S3 objects named <n>.ndjson; legacy files[] arrays still honored — app/lib/chunkNames.js resolveChunkNames() reconstructs); recording_status (active/stale/finalized/analyzed); first/last event timestamps. Index: compound user_session_id + file_id

Recording pipeline

Global gate: app/configs/app.js session_recordings_enabled (hardcoded true at HEAD) short-circuits ingest and playback.

1. Browser streams rrweb events → widget:event (Socket.IO) → buffered in Redis, drained to S3 in 100-event chunks (FLUSH_CHUNK_EVENTS, drainListInChunks — OOM fix); each chunk $incs chunk_count.
2. When a session goes stale, a recording-analysis BullMQ job is enqueued. Finalize grace is FINALIZE_GRACE_TTL_SECONDS (default 90s), sweep every FINALIZE_SWEEP_INTERVAL_MS (default 15s) → ~2min end-to-end. The sweep is bounded to a 2-day window (FINALIZE_SWEEP_MAX_AGE_MS const), batches capped at 200, adopts status-less backend-created rows, and the crash sweep addresses docs by _id (STALE_FINALIZE_THRESHOLD_MS 15min safety net).
3. The worker (bin/recording-analysis-worker.js, concurrency 4, 120s timeout; ms-ai-handoff also runs at concurrency 4):
   • rebuilds the DOM (dom-resolver),
   • extracts intent signals (intent-synth — high-intent pages, form submits),
   • geo-backfills from visitor IP,
   • stitches video frames → MP4 (handbrake),
   • uploads to S3,
   • enqueues ms-ai-handoff to send the analyzed session + transcripts to ms-ai for enrichment.

Cron sweeps prune expired videos and raw .webm chunks. See Session Recording for the end-to-end view.

Presence

Per-pod sockets register tabs in a Redis hash session_tabs:{user_session_id} (90s TTL, renewed every 30s) so any pod can answer "is this visitor live, and on which tabs?". Exposed via GET /internal/knox/presence/snapshot and /internal/knox/presence/visitor. The snapshot carries identity per visitor: person_id (verified_person_id preferred) + display_name (KnoxPresenceController).

Storage maintenance

POST /migrations/session-files-cleanup (internalKnox.js → MigrationController@sessionFilesCleanup) — dry-run by default, requires older_than_days / min_files to match anything, apply: true to execute; job-poll endpoint alongside. Called by backend's POST /admin/storage/session-files-cleanup proxy + Storage Maintenance admin screen.

Realtime

Socket.IO 4.5 with @socket.io/redis-streams-adapter on a dedicated redis.duplicate() connection (the adapter's blocking XREAD on the shared client serialized all Redis traffic — prod outage root cause, fixed 2026-07-20). Admin UI gated behind SOCKET_ADMIN_UI (was unauthenticated in prod).

Cross-service

Reads company/auth config from backend; emits session-finalized + lead scores back to backend; hands analyzed recordings to ms-ai; serves presence to ms-communication and Knox.

Deployment

Docker (Node 22 Alpine, 6 GB heap, port 3003) → ECR → EKS. Two deployments: a stateless HTTP pod and an autoscaling recording worker (HPA on queue depth, 1–10 pods). Worker env: RECORDING_WORKER_CONCURRENCY, RECORDING_WORKER_JOB_TIMEOUT_MS. Finalization env: FINALIZE_GRACE_TTL_SECONDS (90), FINALIZE_SWEEP_INTERVAL_MS (15000). SOCKET_ADMIN_UI gates the Socket.IO admin UI.