Knowledge Base (RAG)
chakra-convai features an embedded Retrieval-Augmented Generation (RAG) system optimized for fast voice interactions without heavy external vector database infrastructure.
Document Storage & Formats
Every file under the data/ directory is indexed (subfolders included):
- Supported Formats: Markdown (
.md), Plain Text (.txt), PDF (.pdf), Word (.docx), CSV (.csv), Excel (.xlsx), JSON (.json). - Tabular & Structured Data:
- A CSV or Excel file produces one searchable passage per row.
- A JSON file holding a list of objects produces one passage per object (using
category+contentkeys). - Drafts Convention: Any file or folder whose name starts with an underscore (
_) is treated as a draft and is never indexed. Rename it without the leading underscore when you are ready to publish.
Building the Index
The vector index is compiled using Google text embeddings and indexed into a local Spotify Annoy index (data/rag.ann) with metadata store (data/rag.json).
Generate or update the vector index locally before deployment:
To clear existing indices and rebuild completely from scratch:
Test retrieval directly from your terminal:
Tool Configuration in Prompts
The built-in tool query_knowledge_base is automatically enabled when declared in chakra.toml:
The phrasing the model receives with retrieved results is configured in config/instructions.json under:
- tools.query_knowledge_base.found_directive
- tools.query_knowledge_base.not_found_directive
Admin API & S3 Mirroring
chakra-convai admin-api starts a lightweight HTTP document management service on port 8080:
| Method and Path | Purpose |
|---|---|
POST /upload |
Upload a new document into data/ |
GET /files |
List all current documents in data/ |
DELETE /files/{filename} |
Delete a document |
POST /trigger_rebuild |
Trigger a background vector index rebuild |
GET /status, GET /status/{job_id} |
Check progress and status of a rebuild job |
Distributed Deployments (S3)
If RAG_S3_BUCKET is configured in .env: