Skip to content

RAGConfig

RAGConfig controls retrieval and answer-context behavior for a RAG instance. It is an immutable configuration object: create a new instance when settings need to change.

from pyaistack import RAG, RAGConfig

rag = RAG(
    config=RAGConfig(
        top_k=5,
        min_score=0.25,
        max_context_chars=20_000,
        include_sources=True,
    )
)

Fields

Field Type Default Purpose
top_k int 3 Maximum number of chunks returned by retrieval.
min_score float \| None None Minimum cosine similarity score a result must meet.
max_context_chars int 12_000 Maximum retrieval-context characters sent to the chat provider.
include_sources bool False Appends verified source details to answer.text.
include_citations bool False Appends verified context citations to answer.text.
insufficient_context_response str Grounded response Returned without an LLM call when context is insufficient.
min_context_results int 1 Minimum excerpts that must fit before answer generation.
retrieval_mode str "vector" "vector" or "hybrid"; hybrid requires a capable store.
candidate_k int 20 Candidates per hybrid branch, raised to the effective top_k when needed.
rerank_candidate_k int 20 Candidates retrieved before a configured reranker selects top_k.
observability_metadata_keys tuple[str, ...] () Explicit allowlist of metadata keys included in indexing events.

See hybrid retrieval for usage, scores and migration. Integer fields reject booleans and fractional values; score thresholds must be finite. Context limits include headers and separators but not the surrounding question or system prompt. answer.context_sources gives the actual excerpts; displayed Sources entries use these rather than all retrieved candidates. See observability and evaluation for privacy guidance on observability_metadata_keys.

top_k

Use top_k to control how many relevant chunks are returned and considered as LLM context.

rag = RAG(config=RAGConfig(top_k=5))
answer = rag.ask("How should a monthly budget be reviewed?")

Pass top_k directly to search() or ask() to override the configured value for one call:

results = rag.search("monthly budget", top_k=10)

top_k must be greater than zero.

min_score

Use min_score to exclude weak cosine-similarity matches before they reach the LLM. Scores range from -1.0 to 1.0; choose the threshold after testing with your documents and embedding model.

rag = RAG(config=RAGConfig(min_score=0.25))

Set min_score=None to disable score filtering; top_k and metadata filters still apply. The value must be between -1.0 and 1.0 when set.

max_context_chars

Use max_context_chars to bound the retrieved context passed to the chat provider. PyAIStack adds retrieved chunks in rank order until this character limit is reached.

rag = RAG(config=RAGConfig(max_context_chars=8_000))

This is a character limit, not a token limit. Reduce it when using a model with a smaller context window; increase it only after measuring answer quality and latency. The value must be greater than zero.

include_sources

Set include_sources=True to append a verified Sources block to answer.text. Each entry is derived from the actual retrieval result, not generated by the LLM.

rag = RAG(config=RAGConfig(include_sources=True))
answer = rag.ask("What information is available?")
print(answer.text)

answer.sources is always available programmatically, regardless of this setting.

Validation errors

Invalid values raise ConfigurationError when RAGConfig is created:

from pyaistack import RAGConfig
from pyaistack.exceptions import ConfigurationError

try:
    config = RAGConfig(top_k=0)
except ConfigurationError as error:
    print(error)

Small local knowledge base

RAGConfig(top_k=3, min_score=0.20, max_context_chars=12_000)

More selective answers

RAGConfig(top_k=5, min_score=0.35, max_context_chars=16_000)

Debugging source selection

RAGConfig(top_k=5, min_score=None, include_sources=True)

These are starting points, not universal thresholds. Tune them with realistic questions from your own knowledge base.