Data Stores
Each service owns its own persistence. There is no cross-service shared schema —
data crosses boundaries via API calls and IDs (company_id, user_session_id,
person_id), not joins.
MongoDB (backend, ms-communication, ms-sessions)
The three Nodevel services use MongoDB via Mongoose. They run against
mongo:7.0 locally (MongoDB 8.0 segfaults on some kernels).
backend — the largest schema (~70 models)
| Group | Key models |
|---|---|
| Auth / RBAC | UserModel, TeamModel, RoleModel, PermissionModel |
| Tenancy / billing | CompanyModel, AgencyModel, SubscriptionModel, AgencySubscriptionModel, PackageModel, AddonModel, InvoiceModel |
| Conversations | UserSessionModel, ChatModel, CallModel (inbound/outbound) |
| Identity graph | PersonModel (unique match_keys_hash_unique), PersonProfileModel, PersonIpSignalModel (keyed by HMAC ip_hash), SessionIpsModel (ipdetective class + device churn) |
| AI / Knox | AIConversationModel, AICreditLogModel, KnoxVisitorDirectiveModel, KnoxVisitorWatchEventModel, NoxActionModel |
| CRM / enrichment | OutboundCallGhlSyncModel, HubspotRequestModel, ApolloRecordModel, ClearbitSessionModel, WebhookModel |
| Platform | ChatBotModel, CompanyFaqModel, FileModel, SessionFileModel, LogModel, NotificationModel |
| Audit / review | AuditLogModel (audit_logs, 180-day TTL, written by the lib/auditLog plugin on 39 models), FormAbandonmentReviewModel, ApiRequestModel (credentials redacted) |
| Secrets / security | CredentialModel (credentials — every tenant OAuth token/API key as one SecretCrypto blob per owner + provider, only via CredentialVault), CredentialKeyModel (credential_keys — per-version key fingerprints), AdminAllowedIpModel (admin_allowed_ips). See Security & Secrets |
| Settings | SettingModel (settings — key/value: admin settings, Feature Flags platform switches read with a one-minute cache, job settings such as recording_retention, archived_company_deletion, partner_hub) |
| Ops / support | ops_cards, ops_stages, ops_scrum_notes, ops_tickets, ops_ticket_messages, ops_agent_actions, ops_announcements, ops_announcement_seen, ops_roadmap_votes / _guests / _comments, ops_feature_requests, ops_video_jobs, ops_video_workers, ops_state, ops_counters; product updates product_update_sends / _suppressions / _videos / _agency_decisions / _rollouts. See Ops workspace |
| Partner Hub | partner_hub_tasks, _claims, _taskers, _creators, _creator_messages, _outreach, _topics, _subreddits, _payouts, _agent_runs, _api_usage. See Partner Hub |
| Admin misc | admin_notifications + popup_receipts (alert popups), voice_previews, api_subscriptions, chats_empty_archive (empty chats moved by ArchiveEmptyChatsJob, TTL) |
Index notes (2026-09): person_id is indexed on every collection the Person
merge repoints (and the session claimed/verified/device_shadow person ids) —
without them each repoint updateMany was a full scan past the 45 s socket
timeout. service_rooms carries the partial fact_extraction_sweep_v1; chats
moved (2026-10-14) to fact_extraction_sweep_v2 and transcript_email_sweep_v4,
which hold only chats with a message (every visit creates an empty chat);
ai_conversations, chat_queues, users (online_status) and
companies.go_high_level.location_id gained indexes they had lacked.
user_sessions deliberately has no fp_hash index (storage) — sweeps that
need it slice by time instead. The strict backend SessionFileModel declares
the sweeper fields (recording_status, skipped_reason, finalized_at,
handoff_attempt_count, …) without indexes; ms-sessions owns that index list.
ms-sessions
user_sessions (lifetime + nested lead_score), session_files (per-page rrweb
recording + video metadata + recording_status), plus mirrored companies /
users.
ms-communication
chats (with embedded messages[]), inbound_outbound_calls (Twilio SID,
recording URL, transcript), user_devices (APN/FCM tokens), plus companies /
agencies. service_rooms.call_logs (per-call step log, last 1,000 entries,
select:false) is written here by CallLogHelper.
Postgres + pgvector (ms-ai)
ms-ai uses Postgres 17 with the pgvector extension via TypeORM. Schema is
migration-managed (ms-ai/migrations/, npm run migration:run).
| Group | Key entities |
|---|---|
| Knowledge / RAG | knox_knowledge_chunks (dual 1536-dim embeddings, HNSW index), knox_knowledge_documents, knox_knowledge_collections |
| Knox conversations | knox_agent_conversations, knox_admin_memory (pgvector) |
| Visitor insight | knox_visitor_profile, knox_session_analysis |
| Action audit | knox_action_events, knox_action_audit, knox_ghl_automation_events |
| Integrations / cost | Company, Integration (integrations.credentials secret keys encrypted at rest with ms-ai's own keyring), elabs_phone_number (carrier keys in meta encrypted), AgentMemory, PdfContent, cost-tracking tables |
Schema features: pgvector columns with HNSW indexes for sub-100ms similarity
search, JSONB for structured payloads, is_active soft-delete flags for
re-ingestion.
Redis (shared infra, not shared data)
Used by every realtime service, but with isolated instances/keys per service
(redis-backend, redis-communication, redis-sessions in the cluster):
- Socket.IO adapters (pub/sub or streams) for multi-pod fan-out.
- BullMQ queues (ms-ai and ms-sessions).
- Visitor presence registry (ms-sessions).
- Rate limiting and short-lived caches.
- Backend cron firing claims (
cron:claim:<job>:<minute>, 120 s) so one schedular replica runs each firing.
Firestore (ms-communication)
Real-time chat message store — each socket subscribes to
/rooms/{room_id}/messages; SocketService keeps Firestore and Socket.IO state in
sync.
Object storage (S3)
Session recordings (stitched MP4 + raw rrweb chunks), uploaded files, and call
recordings live in AWS S3, fronted by CloudFront / Azure CDN. Managed by
backend's S3Service and ms-sessions' upload pipeline.