Knowledge-Base RAG
Nox's product knowledge comes from the nox-knowledge-base repo — markdown
files ingested by ms-ai's KnoxDocumentService into a
pgvector store and retrieved with reranking.
Source: nox-knowledge-base
A markdown repo of ~60+ files across 15 category dirs (platform, widget,
communication, ai-agents, sessions, identity-graph, leads-and-scoring, bookings,
integrations, billing, settings, agency, desktop, data-model, nox-self), indexed by
INDEX.md.
Each file carries frontmatter that drives retrieval:
title: <human-readable>
content_type: feature_guide | how_to | troubleshooting | policy | api_reference |
faq | concept | screen_guide | setting | workflow | ...
product_area: dashboard | chat | voice | visitors | settings | integrations |
widget | analytics | billing | knox | identity-graph | ...
plan_scope: all | starter | growth | pro | enterprise
tags: [keywords]
related_urls: [/settings/integrations/*, ...] # enables URL-filtered retrieval
Ingestion pipeline (KnoxDocumentService)
markdown file
→ save as KnoxKnowledgeDocument
→ semantic chunking (split on H2/H3 headings)
→ per chunk: contextual prefix (section hierarchy) + AI summary
→ dual embeddings (OpenAI text-embedding-3-small, 1536-dim):
embedding = contextual_prefix + content_text
summary_embedding = intent-level matching
→ store in knox_knowledge_chunks (Postgres + pgvector, HNSW index)
→ organized under KnoxKnowledgeCollection
Re-ingestion (POST /knowledge/ingest, knoxDocumentQueue) soft-replaces
prior chunks for the same source (is_active=false) rather than deleting.
Retrieval (3-prong + rerank)
For a query, retrieval combines:
- URL-filtered search — chunks whose
related_urlsmatch the admin's current page rank higher. - Global vector search over
embedding(HNSW). - Summary embedding search over
summary_embedding.
The prongs are merged with reciprocal rank fusion (1/(60+rank) summed per
chunk, KnoxKnowledgeService merge step) — without Cohere this is the final
ranking. Prong priority order used to let any chunk tagged with the admin's page
outrank a better global match, and raw distances aren't comparable across the two
embeddings. Candidates are then reranked with Cohere rerank and the top-K
(topK: 6) injected into the Nox prompt. If COHERE_API_KEY is unset or the call
fails, CohereService returns the caller's order cut to topN (it used to return
every candidate).
knox_knowledge_search takes only query — the filter_feature_area /
filter_content_type parameters were removed (2026-10, ms-ai 37644d6a0): the
model set them on unscoped questions and the AI-derived labels hid the right
chunks. product_area / content_type frontmatter is still stored, just not
used as a model-set filter.
Authoring workflow
- Write markdown with the required frontmatter.
- Use H2/H3 headings (the chunk split boundary); lead each section with the canonical feature/object name; prefer fact lists and tables over prose (~200–800 words/file).
- Cross-reference related files; add an entry to
INDEX.md. POST /knowledge/ingestto re-ingest — the soft-replace handles stale chunks.
Or sync the whole repo from the super-admin Knowledge screen (/admin/knox, see
Admin → Knox): Bulk Import opens Sync Knowledge Base (KnoxBulkImportModal.vue). Drop the
nox-knowledge-base folder (subfolders = collections, files = documents) and it
diffs against what's stored before writing anything:
- Documents match on collection slug + title (
utils/knoxKbSync.jsmatchCollection). Same markdown + samerelated_urls= Unchanged (skipped); otherwise Updated (saved with the existingid) or New. A stored doc infailedstatus is always re-sent. - Stored documents with no matching file (deleted files, renamed titles, duplicates from older imports) and whole collections missing from the folder are listed as Removed; the "Remove N item(s) that aren't in this folder" checkbox (on by default) deletes them after the uploads.
- An unreadable file aborts the scan — syncing without it would delete its stored doc. The button reads Apply N changes / Already up to date.
- Gotcha: the stored-docs list is one
getKnoxDocumentscall per collection, capped server-side at 100. A collection past 100 docs gets unseen docs re-created as duplicates — page the call before that happens.
New product routes Nox can deep-link to must also be added to the page registry
(ms-ai/src/knox-agent/page-registry/route-map.ts) — the 2026-10 KB sync added 25.
Related
- The same pgvector approach backs Knox admin memory (operator-specific context) — see Nox Assistant.
- Company website/FAQ/PDF content is separately embedded for the AI chat agent
(scraping/embedding/PDF queues in ms-ai), distinct from the Nox product KB. Its
search returns
KB_TOP_RESULTSpassages (env, default 4) and product searchPRODUCT_TOP_RESULTS(default 4), with the same original-order fallback when Cohere is missing; website crawls are capped byCRAWL_MAX_LINKS(default 5000 — a hardcoded 500 inscrapping-queue.processor.tsused to override it).