Skip to content

Reranking and verified citations

Reranking adds a second relevance-sorting step after vector or hybrid retrieval. PyAIStack first retrieves a wider candidate set, passes it to your reranker, and keeps the highest-scoring results for context and answer generation.

The runtime provides the reranking contract. You choose the implementation: a local cross-encoder, a hosted reranking service, or an application-specific model. PyAIStack does not add a reranking-model dependency.

How it works

Pass an object with a rerank(query, candidates) method to RAG. The method must return one finite numeric score for every candidate, in the same order as the candidates it received. Higher scores rank first.

from collections.abc import Sequence

from pyaistack import RAG, RAGConfig, SearchResult


class KeywordReranker:
    def rerank(self, query: str, candidates: Sequence[SearchResult]) -> Sequence[float]:
        query_terms = set(query.lower().split())
        return [
            float(len(query_terms.intersection(item.document.text.lower().split())))
            for item in candidates
        ]


rag = RAG(
    reranker=KeywordReranker(),
    config=RAGConfig(
        top_k=3,
        rerank_candidate_k=10,
    ),
)

With this configuration, PyAIStack retrieves up to 10 candidates, reranks them, and returns the best 3. Without a reranker, rerank_candidate_k has no effect and normal retrieval behavior is unchanged.

SearchResult.score remains the original cosine similarity. When a reranker is used, SearchResult.rerank_score contains the score returned by your reranker. In hybrid mode, SearchResult.ranking_score remains the separate reciprocal-rank-fusion score.

Production reranker

Use a model or service that scores the query and each document together. Keep the provider client inside your reranker so the rest of the application stays provider-neutral.

from collections.abc import Sequence

from pyaistack import SearchResult


class CrossEncoderReranker:
    def __init__(self, client) -> None:
        self.client = client

    def rerank(self, query: str, candidates: Sequence[SearchResult]) -> Sequence[float]:
        pairs = [(query, item.document.text) for item in candidates]
        scores = self.client.score_pairs(pairs)
        if len(scores) != len(candidates):
            raise ValueError("reranker returned an unexpected score count")
        return scores

client.score_pairs() is an example application adapter, not a PyAIStack API. It can call your chosen local cross-encoder or hosted reranking provider. Set rerank_candidate_k to a bounded value appropriate for its cost and latency.

If the reranker raises an error, returns a different score count, or returns a non-finite score, PyAIStack raises RerankerError. It does not silently fall back to a different ranking.

Verified citations

Set include_citations=True to append a Citations block to the generated answer. Each citation is created from the exact context excerpt supplied to the chat model. It includes the stored source path, optional chunk_index, and retrieval scores.

rag = RAG(
    reranker=CrossEncoderReranker(client),
    config=RAGConfig(
        top_k=3,
        rerank_candidate_k=10,
        include_citations=True,
    ),
)

answer = rag.ask("What does policy FIN-042 require?")
print(answer.text)

for citation in answer.citations:
    print(citation.source, citation.chunk_index, citation.rerank_score)

answer.citations contains structured Citation objects. answer.context_sources contains the exact text excerpts used to make those citations. This verifies context provenance; it does not determine which individual generated sentence is supported by which source.

Relevance and insufficient context

Use min_score to remove low cosine-similarity candidates before reranking. Use min_context_results when an answer needs more than one relevant excerpt. If either condition leaves insufficient context, ask() returns insufficient_context_response without calling the chat provider.

config = RAGConfig(
    min_score=0.25,
    min_context_results=2,
    insufficient_context_response="I could not find enough relevant policy context.",
)

Runnable examples

The following repository examples use the implemented API:

  • examples/reranking/in_memory_evaluation.py — in-memory documents, custom reranking, verified citations, and Recall@k/MRR evaluation.
  • examples/reranking/hybrid_directory.py — indexes the bundled examples/hybrid_knowledge/ directory into SQLite, then uses hybrid retrieval, reranking, and verified citations.

Run them from the repository root after starting Ollama and pulling the default models:

ollama pull embeddinggemma
ollama pull gemma3:4b
python examples/reranking/in_memory_evaluation.py
python examples/reranking/hybrid_directory.py