01
Uploads never block chat — BullMQ workers index documents in the background
featured projectAug 2026 - Present
Nexus
Upload the documents. Ask the questions. Get answers with sources you can verify.
A self-hosted AI RAG workspace — queue-based indexing, hybrid search and real-time streaming, all on your infrastructure.
Hybrid
pgvector + full-text
Streaming
Token-by-token WebSocket
Isolated
Per-workspace documents
Self-hosted
Docker Compose + GHCR
Why it stands out
The decisions, trade-offs, and details that make Nexus worth your time.
02
Hybrid pgvector + PostgreSQL full-text retrieval, streamed token-by-token over WebSockets
03
Grounded answers with inline citations and a strict refuse-to-guess system prompt
04
Fully self-hosted with Docker — bring a frontier API key or a local model
What it does
Nexus turns a pile of documents into an answerable knowledge base — ingesting asynchronously in the background, retrieving with hybrid search, and answering with sources you can verify.
Workspaces
Documents live in isolated workspaces — each with its own uploads, documents, and conversation history. One instance, many independent knowledge bases.
Queue-based indexing
Uploads never block the API. A BullMQ worker on Redis extracts pages from PDFs, recursively chunks documents (1000-token chunks, 200-token overlap), and embeds them in batches of 100 with exponential-backoff retries — one bad document never blocks the queue.
Hybrid search
Retrieval fuses semantic similarity (pgvector cosine distance) with PostgreSQL full-text ranking using configurable weights (default 0.6 vector / 0.4 keyword, top-6), with an automatic fallback to pure vector search.
Streaming RAG chat
Answers stream token-by-token over a real-time WebSocket (tRPC subscription): sources first, then tokens, then a done event with final citations. A strict system prompt enforces grounded answers with inline [1], [2] references and explicit refusal to guess.
Optional authentication
Set AUTH_PASSWORD and Nexus shows a login screen with httpOnly session cookies (30-day lifetime); leave it unset for a single-user dev mode. Ownership checks scope every document and query to its workspace.
Settings in the UI
Chunk size, top-K, model, temperature, and even the OpenAI key are managed from a Settings panel — secrets encrypted at rest with AES-256-GCM, with UI → env → default precedence. No .env edits or restarts needed.
From upload to answer
A fault-tolerant pipeline — each stage is decoupled so indexing never blocks chatting, and one failing document never stalls the queue.
Upload
Drag PDF, TXT, Markdown, CSV, or JSON files (up to 25 MB each) into a workspace. Uploads are staged safely, validated, and enqueued.
Index
A BullMQ embedding worker extracts text, splits it into overlapping chunks, and generates embeddings (1536-dim, batches of 100) with retries and per-document failure isolation.
Search
At query time, hybrid retrieval scores chunks by weighted vector + full-text similarity and returns the most relevant context with per-workspace ownership enforcement.
Answer
The LLM streams a grounded answer with inline citations over WebSocket. Every conversation turn is persisted with its sources for later review.
Architecture
Frontend (React + tRPC) ── HTTP /trpc + WS /ws ──▶ Express API ──▶ BullMQ (Redis)
│ │ │
│ tRPC router embedding.worker
│ + uploads │
│ │ chunk + embed
│ ▼ ▼
└────────────────────────────── PostgreSQL 16 + pgvector ◀────────┘
(documents, chunks, chat)Tech stack
End-to-end TypeScript, type-safe across every boundary.
ReactTypeScripttRPCExpressBullMQpgvectorRedisDocker
Licensed under MIT.
Repositorygithub.com/Pranesh-Selvaraj/Nexus
LicenseMIT
FrontendReact 18 + Vite + TypeScript + Tailwind 4 (tRPC v11, TanStack Query v5)
BackendNode.js + Express, tRPC router, Drizzle ORM
QueueBullMQ on Redis 7 — embedding worker
SearchPostgreSQL 16 + pgvector (cosine) + full-text ts_rank
ChunkingLangChain recursive splitter (1000-token chunks, 200 overlap)
EmbeddingsOpenAI text-embedding-3-small, batches of 100, 3 retries
ChatOpenAI-compatible streaming (SSE over tRPC subscription)
AuthOptional password → httpOnly session cookies (30-day TTL)
SecretsAES-256-GCM at rest (SETTINGS_SECRET), gitleaks-scanned CI
DeploymentGHCR images (nexus-backend, nexus-frontend) + docker-compose.prod.yml
Monorepopnpm workspaces + Turborepo (backend, frontend, shared-types)
CI/CDTypecheck, build, lint, Vitest, gitleaks, CodeQL, pnpm audit
Built in the open. Star it, try it, break it.
Nexus is MIT-licensed and developed in public. Feedback, issues, and contributions are genuinely welcome.