Hybrid Vector Search & Reranking Optimization: Eliminating Hallucinations

Technical Challenge & Context

Dense vector embeddings excel at capturing conceptual semantics but frequently struggle with exact product SKUs, medical codes, or contract clause numbers. Pure keyword search captures exact hits but misses semantic context. Relying on either alone leads to retrieval gaps and downstream hallucinated LLM responses.

System Architecture & Engineering Pattern

Our production RAG pattern uses a two-stage hybrid retrieval mechanism. First, parallel retrieval queries both a Qdrant HNSW dense vector index and a BM25 sparse index. Second, candidate chunks (Top-50) pass into a Cohere Rerank v3 cross-encoder model to return the top 5 most relevant context passages.

Production Code Blueprint

from qdrant_client import QdrantClient
from cohere import Client as CohereClient

qdrant = QdrantClient(host="localhost", port=6333)
cohere = CohereClient(api_key="COHERE_API_KEY")

def hybrid_rerank_retrieval(query: str, top_k: int = 5):
    # Stage 1: Dense Vector Query
    dense_hits = qdrant.search(collection_name="clinical_docs", query_vector=embed(query), limit=25)
    
    # Stage 2: Cohere Cross-Encoder Reranking
    documents = [hit.payload["text"] for hit in dense_hits]
    rerank_results = cohere.rerank(model="rerank-english-v3.0", query=query, documents=documents, top_n=top_k)
    return [documents[r.index] for r in rerank_results.results]

Key Operational Takeaways

• Hybrid search improves retrieval recall by 34% over pure dense vector search.
• Cross-encoder reranking reduces prompt context window bloating, saving up to 40% in token costs.
• Chunking strategy matters: Use semantic parent-child chunking (512 token child, 2048 parent).

Author Overview

Rohit

Senior AI & Data Science Consultant

2+ Decades AI Experience

First built neural networks in C language in 2004 at IIT Roorkee under the mentorship of Dr. Sunil Padhi (HOD, Electrical Department) to predict annual sunspots. Today designing enterprise Agentic AI workflows, vLLM GPU clusters, and Custom RAG.