Collections & Tables Reference
Audience: Engineering · QA · Support · Where in app: Internal · Plan availability: All plans
A high-density map of where data lives. Each service owns its own persistence — there is no cross-service shared schema; data crosses boundaries via API calls and shared IDs (company_id, user_session_id, person_id, file_id), never database joins. Use this page when you need to know which store holds a given record, or how to follow a record from one store to another.
Engineering deep-dive: Data Stores covers which service owns which database and the schema features (pgvector HNSW, JSONB, soft-deletes). This page is the collection/table catalogue. For the visitor-specific records (UserSession / VisitorProfile / SessionAnalysis / Person) see Visitor data; for companies.* config see Company configuration.
MongoDB — backend, ms-communication, ms-sessions
The three Nodevel services use MongoDB via Mongoose (locally mongo:7.0; backend is the largest schema, ~70 models).
Core tenant + identity
| Collection | What |
|---|
companies | Tenants. Owns all per-tenant config (see Company configuration). |
users | Team members (admins + agents). A self-serve signup holds its business details in pending_company until onboarding answers "my own business" (POST /user/company creates the company); an agency signup never gets a company (ee0dff872). |
people | Global Person records (name/email/phone/match_keys). |
person_profiles | Per-tenant Person view (extracted_facts, form_submissions, bookings, crm_enrichment). |
user_sessions | One row per visit. widget_user_session_id, lead_score, page visits, contact info, traffic source. |
user_devices | Device fingerprints for identity verification (also APN/FCM push tokens in ms-communication). |
person_ip_signals / session_ips | IP-assisted identity-resolution signals. |
playground_companies | Sandbox tenants. |
Communication
| Collection | What |
|---|
chats | Final chat records (transcript messages[], knox enrichment). Every website visit creates an empty one, so the 5-min sweeps run on partial indexes that only hold chats with a message: transcript_email_sweep_v4 ({last_message_at, created_at}, also declared in ms-communication so its autoIndex matches) and fact_extraction_sweep_v2 (abba5cf9e; the jobs repeat the last_message_at > HAS_MESSAGE_FLOOR term and hint the index). |
service_rooms | Active session containers (chat + call). call_logs (select:false, last 1000 entries) = per-call step log written by ms-communication CallLogHelper — server lifecycle, socket events, widget + dashboard call-log steps; read via /admin/calls/logs. |
chat_queue | Pending chats waiting for an agent. |
chatbots | Flow-builder definitions. |
inbound_outbound_calls | Call records (Twilio SID, recording URL, transcript) — in ms-communication. |
call_events | Real-time call event log. |
Sessions & recording (ms-sessions)
| Collection | What |
|---|
session_files | rrweb recordings — chunk metadata, S3 paths, recording_status. |
session_analyses | Per-visit analysis artifact. |
visitor_profiles | Cross-visit rollup keyed on (company_id, user_session_id). |
Privacy / GDPR
See GDPR.
| Collection | What |
|---|
consent_events | Append-only proof of each visitor consent decision (categories, source, banner version); latest per session mirrored on user_sessions.consent. |
person_erasures | Proof of each erasure (ids + counts only); ConversationPurgeJob moves it pending_vendors → done. Also the never-re-identify stop list. |
person_exports | Access requests built by PersonExportJob; file removed after 7 days (expired), row kept. |
recording_fields | Form fields the recorder has seen (never values) — feeds Settings → Recording → Visible fields. |
form_events | Structured form behaviour (viewed / focused / completed / exit / submitted, never values), read by FormEventAbandonmentJob; 30 days. |
snitcher_reuse_cache | One-hour reuse of a company-level lookup for anonymous GDPR-mode visits, keyed by sha256(company + IP). |
Bookings & leads
| Collection | What |
|---|
bookings | All meetings (Google / Calendly / GHL / manual). |
form_submissions | Captured forms (audit). |
form_abandonment_reviews | Super-admin verdicts on forms the recording analyser flagged as abandoned, keyed (file_id, form_index) with a history[] — see Identity Graph admin. |
leads | One Leads-page row per lead, rebuilt from chats, calls, forms and Agent K (LeadsController); only edits / deleted_at are tenant-written — see Leads. |
tags / contact_tags | A company's lead tags (slug = lower-cased name) · one person's tags keyed by contact keys, shared by Leads, campaigns and automations — see Lead tags. |
campaigns / campaign_leads | Outbound campaigns (steps compiled to an automation graph) · one lead per campaign, queued → active → done/skipped — see Campaigns. |
form_leads | Unlinked website-form leads from GDPR-restricted visitors (not tied to session, device or identity graph). |
Agent K & automations
| Collection | What |
|---|
agent_k_prospects | The Agent K queue — every feeder's leads (source outbound / reactivation / knockid / import) through trickle → push. |
agent_k_imports | A tenant-uploaded lead list; its rows wait in agent_k_prospects (source: 'import') and AgentKImportJob releases them. |
do_not_contacts | Per-company do-not-contact / unsubscribe list, honoured by every agent. |
lead_lifecycle_events | Append-only Agent K lead stages (pushed → sent → clicked → replied → booked). |
funnel_contacts | Nox funnel stage state, one per (company, contact), keyed by ghl_contact_id or person_id. |
automation_rules / automation_events / automation_claims / automation_contacts | Automation workflows · run log · exactly-once claims and stop markers · non-Agent-K people an automation runs on — see Automations. Re-engage has no collection of its own (company.ai_agent_k.re_engage — see Re-engage). |
CRM / integrations / enrichment
| Collection | What |
|---|
webhooks | Outbound webhook subscriptions. |
webhook_logs | Webhook delivery audit (the old webhook_events catalog model was removed — the catalogue lives in code, app/configs/webhookEvents.js). |
credentials | Credential vault — every tenant/agency OAuth token and API key, one SecretCrypto blob per owner + provider (data select:false); credential_keys holds key fingerprints. Only via CredentialVault — see Security & Secrets. |
hubspot_requests | HubSpot sync audit. |
hubspot_oauth_handshakes | Single-use HubSpot install state (1h). |
outbound_call_ghl_sync | GHL outbound-call sync state. |
clearbit_sessions, rb2b_sessions, snitcher_sessions, apollo_records | B2B enrichment caches (HappierLeads was removed 2026-09). |
AI & automation
| Collection | What |
|---|
ai_conversations | Q+A training pairs (FAQ). |
ai_credit_logs | Every credit consume/refill (180-day expiry). |
ai_auto_trigger_messages | Auto-trigger message templates. |
nox_actions | Nox-initiated action records (NoxActionModel). |
nox_* (Nox engine v2) | nox_days, nox_plan_items, nox_rapport, nox_email_accounts / nox_email_messages, nox_tasks, nox_group_followups, nox_admin_calls, nox_contact_notes, nox_stalls, nox_event_outbox, nox_invoke_keys, nox_suggestion_decisions, nox_agentk_changes, nox_agent_lessons / nox_reply_lessons / nox_sales_lessons, plus sales_reply_drafts — see Nox engine v2. |
knox_visitor_directives | Auto-engage-on-return directives (KnoxVisitorDirectiveModel). |
knox_visitor_watch_events | Return-visit watch events (KnoxVisitorWatchEventModel). |
identity_graph_defaults | Singleton global IG config. |
Billing & subscription
| Collection | What |
|---|
subscriptions | Active subscriptions (credits, expiry, trial). |
packages | Product tier SKUs. Agency client packages carry archived_at (archived = kept by existing clients, no longer offered; /agencies/client-packages/:id/{usage,archive,unarchive}, delete only when unused — c0a749e47). |
addons | Add-on SKUs. |
invoices | Stripe invoice mirror. |
agencies, agency_packages, agency_addons, agency_subscriptions | Reseller objects. |
appsumo_packages / appsumo_licenses | AppSumo tiers (display price, no Stripe product) · one row per license key → company — see AppSumo. |
Operational
| Collection | What |
|---|
notifications | In-app notifications. |
admin_notifications | Admin broadcasts (optionally an alert popup — popup subdoc); popup_receipts tracks each user's views/snoozes/clicks. |
email_logs | Email send audit. |
api_requests | API call audit (credential headers dropped, secret-looking body keys [redacted] — lib/requestRedaction.js). |
audit_logs | Record-change audit trail on 39 models (180-day TTL) — see Audit Trail. |
logs | General backend activity log. |
files | Uploaded assets (videos, logos, PDFs). Private folders (KB docs, certification proof, chat/WhatsApp attachments, certificates, product-update videos) have a private ACL and no url — served via signed links. |
permissions, roles | RBAC. |
teams | Team groupings. |
settings | Global key-value config: admin settings, Feature Flags platform switches (agency_client_tickets, nox_quality_check_enabled, nox_voice_server_reports, …; one-minute cache), job settings (recording_retention, archived_company_deletion, partner_hub, …). |
admin_allowed_ips | Super-admin IP allowlist. |
ops_*, product_update_* | Ops workspace: Zij cards/stages/scrum notes, support tickets + messages, AI-draft approvals, roadmap votes/requests, announcements, local-worker jobs — see Ops workspace. |
partner_hub_* | Partner Hub tasks, claims, taskers, creators, outreach, payouts, agent runs — see Partner Hub. |
voice_previews, api_subscriptions | Admin: S2S voice preview clips; third-party subscription register. |
chats_empty_archive | Empty per-visit chats moved out of chats by ArchiveEmptyChatsJob (TTL). |
blocked | Block-list for phone/email. |
PostgreSQL + pgvector — ms-ai
ms-ai uses Postgres 17 with the pgvector extension via TypeORM; schema is migration-managed (ms-ai/migrations/).
Knox knowledge base (RAG)
| Table | What |
|---|
knox_knowledge_chunks | Embeddings + metadata (this docs corpus's chunks; dual 1536-dim, HNSW index). |
knox_knowledge_documents | Source markdown (pre-chunk). |
knox_knowledge_collections | Folders/groups of docs. |
knox_knowledge_sources | External sources (URLs) ingested. |
knox_page_registry | Internal app URL map for navigation. |
knox_search_index | Cross-tenant semantic-search embeddings (visitor narratives, session intents) — results filtered by company_id. |
Knox analytics & memory
| Table | What |
|---|
knox_session_analyses | Per-visit analysis (PK: file_id). |
knox_visitor_profiles | Per-visitor cumulative profile (unique (company_id, user_session_id)). |
knox_call_transcripts | Voice call transcripts + quality scores. |
knox_agent_conversations | Every Nox Q+A turn (audit + feedback). |
knox_admin_memory | Durable per-admin facts (pgvector). |
Knox tenant config + actions
| Table | What |
|---|
knox_ai_models | LLM config per tenant. |
knox_tenant_config | Custom Nox behaviors. |
knox_usage_events | Token / cost usage tracking per tool call. |
knox_engagement_rules | Automation conditions. |
knox_action_events | Unified write-side action stream (GHL / Nox / ElevenLabs / calendar / email). |
knox_action_audit | Audit of Knox-initiated actions. |
knox_ghl_automation_events | Raw GHL webhook events. |
Nox engine v2
src/knox-v2/entities — purposes in Nox engine v2.
| Table | What |
|---|
knox_v2_company_flags, knox_v2_runs, knox_v2_tasks, knox_v2_events, knox_v2_run_notes | Per-company switch · runs · planned work · event ledger (SSE replay) · per-run worker notes |
knox_v2_run_effects, knox_v2_undo_records, knox_v2_park, knox_v2_confirm_prefs, knox_v2_screen_consent, knox_v2_escalations | Costly/outward effects · undo versions · parked cards · confirm prefs · screen consent · hand-offs |
nox_trigger, nox_schedule, nox_event, nox_email_queue | Proactive runs · per-admin wake time · judgeable events · delayed group emails |
nox_contact_log, nox_guard_rules, nox_guard_log | Reach-the-admin decisions · admin limits · guard decisions |
nox_rule (+ nox_rule_event), nox_case (+ nox_case_event), nox_goal | Rulebook · followed matters · goals |
nox_standup_item, nox_thread_summary, nox_business_profile, nox_conversation_outcome, nox_product_notes | Plan items · thread summaries · business profile · conversation outcomes · learned product notes |
nox_jev_shadow, nox_quality_runs | Jev shadow judgements · daily quality checks |
Identity & integrations
| Table | What |
|---|
agent_memories | Persistent conversation facts per visitor. |
elabs_agents | ElevenLabs Conversational AI agent configs. |
elabs_phone_number | Mapped phone numbers for inbound AI calls (Twilio/Telnyx keys in meta encrypted at rest). |
integrations | Tenant platform credentials (Shopify / WooCommerce); secret keys in credentials encrypted with ms-ai's own keyring. Google tokens ms-ai no longer uses are dropped by the encrypt-at-rest job. |
rag_products | Synced ecommerce products with embeddings. |
Redis — cross-service (shared infra, not shared data)
Isolated instances/keys per service (redis-backend, redis-communication, redis-sessions).
| Use | Where |
|---|
| Socket.IO adapter (multi-pod fan-out) | All sockets |
| Cache (subscription gate, CRM enrichment) | Backend, ms-ai |
| Rate limiting | ms-communication call/message limiters |
| Session storage | Backend auth |
| Visitor presence registry | ms-sessions (powers knox_presence) |
| BullMQ queues | ms-sessions analysis worker, ms-ai fact-extraction + embedding workers |
Firestore — ms-communication
| Use | What |
|---|
| Real-time chat/call state | Each socket subscribes to /rooms/{room_id}/messages; SocketService keeps Firestore + Socket.IO in sync. Twilio WebRTC call tracking. |
S3 — object storage
| Use | Where |
|---|
| Recording NDJSON chunks (raw rrweb) | ms-sessions |
| Stitched recordings / video files (call recordings, default videos) | backend |
| Uploaded files (logos, PDFs, agent knowledge) | backend |
Join-key cheat sheet
The IDs that stitch records across stores:
| Want | Join |
|---|
| Sessions for a person | people._id → person_profiles.person_id (per tenant) → user_sessions.person_id |
| Chats for a person | people._id → person_profiles.person_id → chats.person_id |
| Recording for a session | user_sessions.widget_user_session_id ← session_files.user_session_id (then session_files._id = file_id) |
| Analysis for a recording | session_files._id = session_analyses.file_id = knox_session_analyses.file_id |
| Visitor profile from session | (user_sessions.company_id, user_sessions.widget_user_session_id) = visitor_profiles.(company_id, user_session_id) |
| Cross-tenant Person check | people.match_keys contains sha256(normalized_email) |
| Booking attribution | bookings.user_session_id → user_sessions → full journey |