Financial RAG: Hybrid+Rerank Achieves Recall@5 of 0.816; BM25 Outperforms Dense Retrieval on 23k Financial Queries

Akarso, Karaman & Mierbach (Technische Hochschule Ingolstadt / TU Eindhoven / Raduate). “From BM25 to Corrective RAG: Benchmarking Retrieval Strategies for Text-and-Table Documents.” arXiv:2604.01733. Benchmark: T²-RAGBench (EACL 2026).

This original State of AI redraw explains why the architecture choice belongs in the test plan. It is based on Akshay Pachaar’s RAG vs. Graph RAG vs. Agentic RAG diagram and Tech with Mak’s seven-architecture diagram, credited to Robbert van Vlijmen. In this finance note, the practical control is still retrieval quality over text-and-table evidence; architecture labels do not replace source-hit and numerical-correctness checks.

Problem

RAG systems for financial applications must retrieve from documents containing both text and tables — earnings reports, SEC filings, annual reports where text provides context and tables carry precise numerical values. This creates a fundamental tension: semantic embedding models capture paraphrased context but may dilute precise numerical signals; lexical models match exact terminology but miss conceptual synonymy. No prior study systematically compared more than six retrieval strategies on a heterogeneous text-and-table financial corpus at this scale.

Dataset

T²-RAGBench — 23,088 question-context-answer triplets over 7,318 unique financial documents, each averaging approximately 920 tokens with mixed text and markdown-formatted tables. Three source datasets unified:

Documents are drawn from real SEC filings and annual reports. Questions require locating specific figures (e.g., year-over-year revenue growth) that may appear in tables, in narrative text, or require combining both.

Methods Evaluated (10 total)

All dense methods use OpenAI text-embedding-3-large (3,072 dimensions) with FAISS IndexFlatIP. Hybrid RRF uses k=60 smoothing constant. Reranking retrieves 50 candidates, returns top 10.

Key Results

Main retrieval results (T²-RAGBench, 23,088 queries):

All pairwise differences between BM25, dense, and Hybrid RRF are statistically significant (p < 0.001, paired bootstrap, B=10,000 iterations, Bonferroni-corrected).

Headline Findings

BM25 outperforms state-of-the-art dense retrieval on every metric except Recall@20, where they tie (0.797 vs. 0.798). This is the paper’s most counterintuitive result: financial documents have precise terminology (company names, metric labels, fiscal period identifiers) that appear verbatim in both queries and documents, making lexical matching a competitive baseline that semantic embeddings do not consistently beat.

Hybrid + Cohere Rerank dominates all single-stage methods by wide margins. Recall@5 of 0.816 vs. 0.695 for Hybrid alone (+11.4 pp), vs. 0.644 for BM25 (+26.7 pp), vs. 0.587 for dense (+39.0 pp). The cross-encoder architecture processes queries and candidates jointly, producing query-aware relevance scores that improve MRR@3 from 0.433 (Hybrid alone) to 0.685 — a 58.2% relative improvement in ranking precision.

HyDE underperforms vanilla dense retrieval (Recall@5: 0.544 vs. 0.587). Financial questions require precise numerical answers; LLM-generated hypothetical documents hallucinate plausible but incorrect figures, pulling embeddings away from the true relevant context. This is an important warning for finance RAG deployments that use query expansion.

CRAG triggers correction on 63% of queries (14,569/23,088), indicating that initial hybrid retrieval frequently returns suboptimal results. Despite this correction mechanism, CRAG (0.658) falls below simple Hybrid RRF (0.695) — query rewriting alone cannot match the complementary strengths of sparse and dense retrieval.

Contextual Retrieval provides consistent gains. Prepending LLM-generated context summaries (company name, reporting period, key metrics) at indexing time improves Recall@5 by +2.8 pp for dense and +2.2 pp for hybrid. This suggests financial documents benefit from explicit metadata enrichment that helps retrieval systems identify the right entity and time period.

TAT-DQA is the hardest subset (best Recall@5 = 0.647 vs. 0.755 for ConvFiNQA), likely due to complex table layouts and diverse numerical operations. Hybrid fusion provides the largest absolute improvement on TAT-DQA (+8.1 pp Recall@5 over BM25) — the subset where combining lexical and semantic signals is most valuable.

Practical Recommendations

1. Start with BM25 for financial document retrieval — it outperforms dense retrieval and is cheaper to operate.
2. Add hybrid fusion for meaningful gains at low additional cost.
3. Add neural reranking (cross-encoder) if retrieval quality is the bottleneck and latency is acceptable.
4. Avoid HyDE and multi-query expansion for precise numerical financial queries — minimal or negative impact.
5. Apply contextual retrieval (LLM-generated document summaries at indexing time) for consistent marginal gains.

Relevance

This is the most comprehensive RAG retrieval benchmark on financial text-and-table documents available. The BM25 > dense finding challenges a common assumption in enterprise RAG deployments and is directly actionable for teams deploying RAG over financial filings.

August 2026 cross-benchmark update

The result remains a required finance baseline, but the newer corporate-brain benchmark cluster narrows the claim. WorkSurface-Bench and Workspace-Bench add multi-surface routing and cross-file dependencies; OfficeQA Pro and Office Comprehension Bench add native office-file fidelity; finance LEDGER and ARQA add long-context, OCR, table/cell evidence, and recomputable derivations. The updated deployment hypothesis is BM25 plus dense retrieval plus structural routing and reranking, validated separately on filings, wikis, office files, graphs, and private research repositories. BM25 superiority should not be generalized across those surfaces without a paired run.