Skip to main content

Data Stores

Each service owns its own persistence. There is no cross-service shared schema — data crosses boundaries via API calls and IDs (company_id, user_session_id, person_id), not joins.

MongoDB (backend, ms-communication, ms-sessions)

The three Nodevel services use MongoDB via Mongoose. They run against mongo:7.0 locally (MongoDB 8.0 segfaults on some kernels).

backend — the largest schema (~70 models)

GroupKey models
Auth / RBACUserModel, TeamModel, RoleModel, PermissionModel
Tenancy / billingCompanyModel, AgencyModel, SubscriptionModel, AgencySubscriptionModel, PackageModel, AddonModel, InvoiceModel
ConversationsUserSessionModel, ChatModel, CallModel (inbound/outbound)
Identity graphPersonModel, PersonProfileModel, PersonIpSignalModel
AI / KnoxAIConversationModel, AICreditLogModel, KnoxVisitorDirectiveModel, KnoxVisitorWatchEventModel, NoxActionModel
CRM / enrichmentOutboundCallGhlSyncModel, HubspotRequestModel, ApolloRecordModel, ClearbitSessionModel, WebhookModel
PlatformChatBotModel, CompanyFaqModel, FileModel, SessionFileModel, LogModel, NotificationModel

ms-sessions

user_sessions (lifetime + nested lead_score), session_files (per-page rrweb recording + video metadata + recording_status), plus mirrored companies / users.

ms-communication

chats (with embedded messages[]), inbound_outbound_calls (Twilio SID, recording URL, transcript), user_devices (APN/FCM tokens), plus companies / agencies.

Postgres + pgvector (ms-ai)

ms-ai uses Postgres 17 with the pgvector extension via TypeORM. Schema is migration-managed (ms-ai/migrations/, npm run migration:run).

GroupKey entities
Knowledge / RAGknox_knowledge_chunks (dual 1536-dim embeddings, HNSW index), knox_knowledge_documents, knox_knowledge_collections
Knox conversationsknox_agent_conversations, knox_admin_memory (pgvector)
Visitor insightknox_visitor_profile, knox_session_analysis
Action auditknox_action_events, knox_action_audit, knox_ghl_automation_events
Integrations / costCompany, Integration, AgentMemory, PdfContent, cost-tracking tables

Schema features: pgvector columns with HNSW indexes for sub-100ms similarity search, JSONB for structured payloads, is_active soft-delete flags for re-ingestion.

Redis (shared infra, not shared data)

Used by every realtime service, but with isolated instances/keys per service (redis-backend, redis-communication, redis-sessions in the cluster):

  • Socket.IO adapters (pub/sub or streams) for multi-pod fan-out.
  • BullMQ queues (ms-ai and ms-sessions).
  • Visitor presence registry (ms-sessions).
  • Rate limiting and short-lived caches.

Firestore (ms-communication)

Real-time chat message store — each socket subscribes to /rooms/{room_id}/messages; SocketService keeps Firestore and Socket.IO state in sync.

Object storage (S3)

Session recordings (stitched MP4 + raw rrweb chunks), uploaded files, and call recordings live in AWS S3, fronted by CloudFront / Azure CDN. Managed by backend's S3Service and ms-sessions' upload pipeline.