Persistent Storage Capacity
SQLiteVectorStore is a local persistent store for small-to-medium RAG indexes. It keeps vectors in a SQLite database and calculates exact cosine similarity for every question.
What determines capacity
Capacity is measured in chunks, not source files. One large document can become many chunks after TextChunker processes it.
Each search compares the question embedding with every stored chunk. As the chunk count grows, both search time and memory use grow linearly.
Practical guidance
| Index size | Expected fit |
|---|---|
| A few thousand chunks | Comfortable for local development and personal knowledge bases. |
| 10,000 to 50,000 chunks | Usable on capable local hardware; measure query latency and memory before production use. |
| Hundreds of thousands or more | Not recommended for this exact-search SQLite store; use an indexed vector database adapter when available. |
These are planning guidelines, not fixed limits. Embedding dimension, document size, available memory, and local CPU performance all affect the result.
Check the current index size
from pyaistack import RAG
from pyaistack.vectorstores import SQLiteVectorStore
rag = RAG(vector_store=SQLiteVectorStore("knowledge.db"))
print(rag.document_count)
The returned count is the number of stored chunks.
When to move beyond SQLite
Keep SQLiteVectorStore for local and modest knowledge bases. Plan a dedicated indexed vector database when latency becomes unacceptable, multiple processes need to write concurrently, or the index grows beyond tens of thousands of chunks.