Workspaces
Documents live in isolated workspaces — each with its own uploads, documents, and conversation history. One instance, many independent knowledge bases.
featured project
Nexus
A self-hosted AI RAG workspace — upload documents, and chat with them over a real-time stream with cited answers.
Queue-based indexing. Hybrid search. Fully self-hosted.
What it does
Nexus turns a pile of documents into an answerable knowledge base — ingesting asynchronously in the background, retrieving with hybrid search, and answering with sources you can verify.
Queue-Based Indexing
Uploads never block the API. A BullMQ worker on Redis extracts pages from PDFs, recursively chunks documents (1000-token chunks, 200-token overlap), and embeds them in batches of 100 with exponential-backoff retries — one bad document never blocks the queue.
Hybrid Search
Retrieval fuses semantic similarity (pgvector cosine distance) with PostgreSQL full-text ranking using configurable weights (default 0.6 vector / 0.4 keyword, top-6), with an automatic fallback to pure vector search.
Streaming RAG Chat
Answers stream token-by-token over a real-time WebSocket (tRPC subscription): sources first, then tokens, then a done event with final citations. A strict system prompt enforces grounded answers with inline [1], [2] references and explicit refusal to guess.
Optional Authentication
Set AUTH_PASSWORD and Nexus shows a login screen with httpOnly session cookies (30-day lifetime); leave it unset for a single-user dev mode. Ownership checks scope every document and query to its workspace.
Settings in the UI
Chunk size, top-K, model, temperature, and even the OpenAI key are managed from a Settings panel — secrets encrypted at rest with AES-256-GCM, with UI → env → default precedence. No .env edits or restarts needed.
From upload to answer
A fault-tolerant pipeline — each stage is decoupled so indexing never blocks chatting, and one failing document never stalls the queue.
Upload
Drag PDF, TXT, Markdown, CSV, or JSON files (up to 25 MB each) into a workspace. Uploads are staged safely, validated, and enqueued.
Index
A BullMQ embedding worker extracts text, splits it into overlapping chunks, and generates embeddings (1536-dim, batches of 100) with retries and per-document failure isolation.
Search
At query time, hybrid retrieval scores chunks by weighted vector + full-text similarity and returns the most relevant context with per-workspace ownership enforcement.
Answer
The LLM streams a grounded answer with inline citations over WebSocket. Every conversation turn is persisted with its sources for later review.
Architecture
Frontend (React + tRPC) ── HTTP /trpc + WS /ws ──▶ Express API ──▶ BullMQ (Redis)
│ │ │
│ tRPC router embedding.worker
│ + uploads │
│ │ chunk + embed
│ ▼ ▼
└────────────────────────────── PostgreSQL 16 + pgvector ◀────────┘
(documents, chunks, chat)Tech stack
End-to-end TypeScript, type-safe across every boundary.
ReactTypeScripttRPCExpressBullMQpgvectorRedisDocker
Repositorygithub.com/Pranesh-Selvaraj/Nexus
LicenseMIT
FrontendReact 18 + Vite + TypeScript + Tailwind 4 (tRPC v11, TanStack Query v5)
BackendNode.js + Express, tRPC router, Drizzle ORM
QueueBullMQ on Redis 7 — embedding worker
SearchPostgreSQL 16 + pgvector (cosine) + full-text ts_rank
ChunkingLangChain recursive splitter (1000-token chunks, 200 overlap)
EmbeddingsOpenAI text-embedding-3-small, batches of 100, 3 retries
ChatOpenAI-compatible streaming (SSE over tRPC subscription)
AuthOptional password → httpOnly session cookies (30-day TTL)
SecretsAES-256-GCM at rest (SETTINGS_SECRET), gitleaks-scanned CI
DeploymentGHCR images (nexus-backend, nexus-frontend) + docker-compose.prod.yml
Monorepopnpm workspaces + Turborepo (backend, frontend, shared-types)
CI/CDTypecheck, build, lint, Vitest, gitleaks, CodeQL, pnpm audit