# RAG End to End: A Practical Tutorial

Key idea: The value of RAG is not in “plugging in a vector store”: it is in reading documents properly, searching in a hybrid way, reranking and, above all, measuring retrieval and answers separately.

In Anthropic’s tests (2024), adding context to chunks, BM25 and a reranker cut retrieval failures by 67%. Commercial legal RAG tools hallucinate 17–33% of the time (Stanford, 2024), and on Meta’s CRAG benchmark (2024) the best industrial RAG systems answer without hallucinating only 63% of the time. For measuring all this, an LLM judge agrees with humans more than 80% of the time (Zheng et al., 2023).

I checked almost everything here against the official documentation and the cited papers on 23 September 2026. A few tool details marked † come from general experience and I did not re-check them. Features marked (preview) exist, but the vendor does not recommend them for production yet. Acronyms are spelled out the first time they appear and collected in the glossary at the end; papers are cited by name in the text (SPLADE, ColBERT, RAGAS), with authors, year and link in the references.

Part I · Concepts

1. What RAG is, explained with a library

RAG (Retrieval-Augmented Generation) means that, before answering, a language model searches your documents and answers from what it found, citing the source.

Why do you need it? A language model does not know your company’s internal documents, its knowledge stops at a cutoff date, and it can make things up (“hallucinate”). With RAG the answer rests on specific documents that anyone can check, and updating the knowledge does not require retraining the model.

The easiest way to see how the pieces fit together is a library, and someone who walks up to the desk with a question:

1. The library is your documents: everything the system is allowed to answer from, such as PDFs, wikis or internal policies.
2. The index cards are the chunks. Nobody rereads a whole book for every question, so each document is cut into small cards, each holding a piece of information that makes sense on its own.
3. The catalog is the index. Every card is filed two ways: by the words it contains and by what it means. That is what later lets you search both by keyword and by meaning.
4. The librarian is the search. When a question arrives, the librarian quickly pulls about 50 cards that look relevant, without reading them closely.
5. The expert is the reranker. The expert reads the question next to each of those cards and keeps the best 5.
6. The writer is the language model. Only now does anyone write the answer, using those 5 cards and citing the card behind each sentence. If the cards do not contain the answer, the honest reply is “I don’t know”.
7. The teacher is the evaluation. The teacher grades two things separately: did the librarian bring the right cards, and is the answer faithful to them?

The rest of the tutorial follows the same order: preparing the documents (stages 1 to 3), search (4), reranking (5), generation (6) and evaluation (7).

Not every RAG system has all seven stages. Gao et al. (2023/24) describe three generations:

In library terms, naive RAG is a librarian who hands the first k cards straight to the writer, with no expert in between. This tutorial describes an advanced RAG, with modular pieces where they pay off (section 10).

2. The three circuits of a production RAG system

A RAG system in production runs three circuits. The first prepares the documents once, offline: sources, parsing and chunking, embeddings and the index (stages 1 to 3 of the library). The second runs for every question: hybrid search, the reranker and the model that answers with citations (stages 4 to 6). The third evaluates, before deployment with a golden dataset and in production by sampling real traffic, and feeds improvements back to the other two (stage 7). Parts II to VI cover them in that order.

Part II · Preparing the documents

3. Reading the document properly (parsing)

Parsing has more impact than any other step, and it is the one teams neglect most. Convert a PDF to “plain text” and you lose the headings, the tables get scrambled, and each chunk loses its context.

The studies agree on this:

• The ColPali authors (ICLR 2025) “typically find that optimizing the ingestion pipeline yields much greater improvements than optimizing the embedding model”. ColPali itself skips text extraction and chunking and indexes page images directly; on the ViDoRe benchmark it scored 81.3 nDCG@5 against ~65–67 for parsing pipelines.
• In Unstructured’s FinanceBench study (2024), chunking by document elements (headings, tables) reached 53.2% accuracy versus 48.2% with fixed 512-token chunks, and used half as many chunks.
• A Turkish-language study (2026) found that layout-aware chunking helps much more on documents with tables than on text-only documents.

The usual tools are the Document Layout skill (Azure), Document AI Layout Parser (Google) and Docling (IBM, open source). Section 22 compares them.

4. Chunking

Chunking splits each document into small pieces (chunks) that are indexed separately.

4.1 The strategies

4.2 What the evidence says: semantic chunking is overrated

• Vectara (2024) asked “Is Semantic Chunking Worth the Computational Cost?” and concluded that “the computational costs associated with semantic chunking are not justified by consistent performance gains”. Fixed-size chunking won on all 4 real-document datasets (on HotpotQA, for example, F1@5 was 90.59 fixed vs 87.37 semantic). The embedding model mattered more than the chunking.
• Chroma (2024) found that the chunking strategy moves recall by up to 9%. Its default semantic chunker came in slightly below average (83.6% recall), while a recursive 200-token chunker with no overlap reached 88.1%. The best result (91.9%) used a language model and cost much more.
• NVIDIA (June 2025) got the best average accuracy (0.648) and the lowest variance across datasets with page-level chunking.
• In biomedicine (2026), semantic chunking gained +8.4 F1 points on one dataset, but on the others fixed-size chunking “remains competitive or better”. It depends on the domain.

4.3 What does work: adding context to each chunk

Anthropic’s Contextual Retrieval (Sep 2024) has a model write 50–100 tokens of context and prepends them to each chunk. Measured as the retrieval failure rate within the top 20 results, starting from a 5.7% baseline:

• context in the vectors only: 3.7% (−35%)
• context in the vectors and in BM25: 2.9% (−49%)
• all of the above plus a reranker: 1.9% (−67%)

The one-time cost is ~$1.02 per million document tokens with prompt caching. Be careful when you quote these numbers: all three percentages are reductions relative to the 5.7% baseline, so the reranker’s own contribution is the step from 2.9% to 1.9%.

Late chunking (Jina, 2024) computes the embedding of the whole document first and only then splits it, so each vector “knows” which document it comes from. It improves nDCG@10 by +2.7% to +3.6% without retraining.

4.4 Starting sizes

Bhat et al. (2025) suggest 64–128 tokens for short factual questions and 512–1024 tokens for questions that need broad context. Azure recommends starting with 512 tokens and 25% overlap, and adding the document title to the middle chunks.

A reasonable default: start with recursive or section/page chunking of 256–512 tokens, don’t split tables, add context, and tune by measuring (Part VI).

5. Turning text into vectors (embeddings)

An embedding is a list of numbers that represents the meaning of a text. Texts with similar meanings end up close together:

Measure on your own data. Public leaderboards (such as MTEB) change every month and don’t always reflect your domain. And pin the model version: if you mix vectors from different models, searches stop making sense.

6. Index and permissions

The index stores each chunk with its text, its vector and its metadata:

Without allowed_groups and a security filter on every search, an intern could receive chunks from executive committee documents.

Part III · Retrieval

7. BM25: keyword search

BM25 is the classic keyword (lexical) search algorithm and the default in Elasticsearch, OpenSearch and Azure AI Search. It runs on an inverted index, which works like the alphabetical index at the back of a book:

The score has three parts:

BM25 is fast, needs no model, can be explained, and is excellent for exact terms such as codes, acronyms and proper names. Its weakness is synonyms: “cheap laptop” won’t find “budget notebook”.

8. Semantic search: by meaning

8.1 Dense vectors (nearest-neighbor search)

Here the question becomes a vector and you retrieve the closest chunks. Doing that quickly over millions of vectors takes an approximate index, usually HNSW (a graph of neighbors).

Dense search understands synonyms, paraphrases and different languages. On the other hand, it is a black box (you can’t explain why something matched), it can confuse nearly identical codes (ORA-00942 vs ORA-00943), and it uses more memory. Quantization reduces the memory by compressing the numbers, for example to 8 bits or to binary.

8.2 Learned sparse (SPLADE, ELSER)

Learned sparse sits between the two. A model expands the text with related terms and their weights, and the search then runs on an inverted index, as with BM25:

"cheap laptop" → { laptop: 2.1, notebook: 1.8, computer: 1.2, cheap: 1.9, budget: 1.5, price: 0.9 }

It is easier to explain than dense vectors and still finds synonyms. The catch is that it depends on the model’s language (ELSER is recommended for English only) and has a token limit (ELSER encodes the first 512 tokens of each field).

8.3 Which one wins

None of them wins every time, which is why you combine them.

9. Hybrid search and RRF

9.1 The problem

The two searches return scores on incompatible scales:

Adding 12.4 + 0.81 makes about as much sense as adding euros and kilos.

9.2 The solution: RRF (Reciprocal Rank Fusion)

RRF ignores the scores and uses only the position of each document in each list:

RRF(doc) = Σ  1 / (k + position of doc in that list)      with k = 60 typically
       over each list

Take the same rankings as above:

doc_A wins because it ranks high in both lists. RRF rewards agreement between the two methods.

9.3 Why k = 60?

The value comes from the original paper (Cormack, Clarke and Büttcher, SIGIR 2009), where it was chosen empirically, and Azure AI Search documents that it works best with small values such as 60.

9.4 Advantages, limits and variants

RRF needs no scale calibration and nothing to train, and it accepts N lists (BM25, several vectors, several rewordings of the question). It has two weaknesses. It ignores magnitude, so being first “by a mile” is worth the same as being first “by a hair”. And a bad list counts just the same: if vector search returns junk, that junk also gets points.

There are two common variants:

• Weighted RRF gives one of the lists more weight (for example, the vector list ×2). It is available in Azure (vector weighting), in LangChain’s EnsembleRetriever †, and in Google Vector Search with rrf_ranking_alpha.
• Linear combination normalizes the scores and adds them with weights. It uses the magnitude, but you have to calibrate it with data. Examples are Elasticsearch’s linear retriever and DBSF in Qdrant †.

Keep in mind that RRF is not a reranker. It only fuses lists, and the reranker comes afterwards.

10. Advanced retrieval strategies

11. Case study: Elasticsearch

Elasticsearch has four different “semantic” pieces, and they are easy to mix up:

• A. Dense: dense_vector + knn query, with int8, int4 and BBQ quantization. You can use E5 (multilingual), Jina (via the Elastic Inference Service) or external models (OpenAI, Azure OpenAI, Cohere, Bedrock, Vertex AI, Hugging Face).
• B. ELSER expands terms. What it adds are learned associations, not synonyms. On Elastic’s own BEIR benchmark it improves nDCG@10 over BM25 by 18% on average (10 wins, 1 tie, 1 loss). It is recommended for English, reads 512 tokens per field and requires a paid subscription.
• C. Reranker: the text_similarity_reranker retriever or the RERANK command in ES|QL.
• D. semantic_text (GA since version 9.0). If you don’t pin inference_id, new indices may use a different model after a version upgrade, so always pin the model in production.

Here is hybrid search with a reranker in a single call:

{
  "retriever": {
    "text_similarity_reranker": {
      "retriever": {
        "rrf": {
          "retrievers": [
            { "standard": { "query": { "match":    { "content": "remote work days manager Spain" } } } },
            { "standard": { "query": { "semantic": { "field": "content_semantic",
                                                     "query": "remote work days manager Spain" } } } }
          ],
          "rank_window_size": 50,
          "rank_constant": 60
        }
      },
      "field": "content",
      "inference_id": "my-rerank-endpoint",
      "inference_text": "remote work days manager Spain",
      "rank_window_size": 50
    }
  }
}

You can also fuse with the linear retriever (minmax or l2_norm normalizers) or, in ES|QL, with FORK + FUSE (RRF or LINEAR) + RERANK. In the multi-field format, Elastic normalizes the lexical and semantic fields so that each group contributes 50%.

Watch the vocabulary. In Elastic, “semantic search” means searching with embeddings; in Azure, the “semantic ranker” is a reranker.

Part IV · Reranking

12. Reranking

12.1 What it is

A reranker takes the ~50–150 candidates from the search and reorders them by reading the question and each chunk together. That is more accurate than comparing vectors, but slower, so you only apply it to a few candidates.

The typical architecture has two stages: hybrid search (which prioritizes coverage), then a reranker over 100–150 candidates, and finally between 10 and 20 chunks passed to the model that writes the answer.

12.2 Types

12.3 Notable models (verified)

12.4 Practical rules

A reranker is the cheapest and most proven improvement you can make. In Anthropic’s numbers, adding one on its own takes the failure rate from 2.9% to 1.9%.

But the reranker only reorders what the search found. If the correct document isn’t among the candidates, it can’t save you, which is why you measure retrieval coverage (recall) first.

What currently sets rerankers apart is instruction following: you can give them business rules such as “prioritize recent content”. Every vendor claims to be the best, so measure with your own data.

Some add-ons are worth having. MMR removes redundant chunks. Compression with LongLLMLingua gives +21.4% quality with ~4× fewer tokens. And placement in the prompt matters: put the most relevant content at the beginning or the end (“Lost in the Middle”).

Part V · Generation and guardrails

13. Generation with citations and guardrails

The prompt looks like this:

System:   Answer ONLY from the context. Cite every claim as [n].
          If the context does not contain the answer, say "I don't know".
Context:  [1] Remote Work Policy §3.2 Spain, p. 4: "Manager: 2 days/week..."
          [2] 2026 Annex, p. 1: "...starting January 2026, 3 days for..."
Question: remote work days for a manager in Spain

Guardrails sit before, during and after generation:

1. Before, detect attempts to manipulate the model (“jailbreak” or prompt injection).
2. During, if the reranker leaves no chunk above the threshold, answer “I couldn’t find that information” instead of making something up.
3. After, check that every sentence of the answer is supported by the chunks. If the check fails, regenerate or answer cautiously.

Part VI · Evaluation

14. Two separate tests

If you only look at the final answer, you don’t know what to fix. Microsoft calls evaluating the retrieval step process evaluation and evaluating the answer system evaluation.

15. The golden dataset (the “answer key”)

The golden dataset is a set of 100 to 300 questions, each with its correct answer and the documents that should come up:

{"query": "Remote work days, manager, Spain?",
 "ground_truth": "2 days per week; 3 from January 2026 according to the annex",
 "relevant_docs": [{"document_id": "teletrabajo_p4", "query_relevance_label": 4},
                   {"document_id": "anexo2026_p1",  "query_relevance_label": 3}]}

16. Retrieval metrics, with numbers

Say that for one question the correct documents are A and C, and the search engine returned [B, A, D, C, E].

These numbers tell you where to look. If Recall@50 is low, the problem is in retrieval (chunking, embeddings, missing BM25), and the reranker won’t fix it. If Recall@50 is high but nDCG@5 is low, the problem is in the reranker.

17. Answer metrics, with numbers

Suppose the system answers: “You get 2 days per week [1], 3 starting January 2026 [2], and you can choose Fridays.”

Microsoft frames it the same way: faithfulness to the context is the precision side (add nothing) and completeness is the coverage side (leave out nothing critical).

Another option is “nugget” evaluation (TREC 2024): you define the atomic facts a good answer must contain and count how many appear.

18. The LLM as a judge

Nobody reviews 10,000 answers by hand, so another model plays the teacher. That judge has known biases. It prefers the first option it sees (position), it prefers long answers (verbosity), and it prefers text from its own model family (self-preference).

To set up a judge you can rely on:

1. A human labels 50–100 cases.
2. Measure judge–human agreement (Cohen’s kappa, accuracy, F1).
3. The judge must be from a different model family than the generator.
4. The judge must explain its score.
5. Scoring claim by claim (RAGChecker) or by nuggets is better than giving one overall score.

How far can you trust it? GPT-4 as a judge reaches more than 80% agreement with humans, the same level as between two humans (Zheng et al., 2023). In TREC 2024, perfect human–GPT-4o agreement was 56%, and 72% when the human was correcting the model’s label. So the LLM judge is reliable for comparing systems and less reliable question by question. ARES combines a few hundred human labels with the automatic judge to produce statistically valid confidence intervals.

19. Evaluation before deployment and in production

A quality gate in continuous integration could require that Recall@10 drops no more than 2 points, faithfulness stays ≥ 95%, and correct “I don’t know” answers stay ≥ 90%. Those thresholds are only examples; the business sets the real ones. In production you sample a percentage of real traffic and evaluate it without a reference answer (faithfulness, answer relevance, context relevance), alongside thumbs-up/thumbs-down votes, latency and cost.

Microsoft recommends a parameter sweep: try combinations and measure which one wins. The numbers below are illustrative, not real results:

DeepEval suggests no more than about 5 metrics per application. MLflow / Databricks recommend using the same evaluators in development and in production. And in production you can only use metrics that don’t need a correct answer.

20. Why it matters: hallucination in real systems

Several of these figures are from 2023–2024 and come from older models, so always cite them with year and model.

Part VII · A production example on Azure

21. Internal policy copilot, step by step

The case is a company with 20,000 employees, with HR, legal and procurement documents in SharePoint and Blob Storage. It needs per-user permissions, citations in every answer, and “I don’t know” when there is no information.

21.1 Architecture

If you know the LangChain + Chroma/Qdrant stack, the pieces map like this:

21.2 One question, end to end

Ana, a manager in Madrid, asked about her contract earlier in the chat. Now she types “so how many days can I work remotely?”

1. The model rewrites the question using the chat history: “remote work days allowed for a manager in Spain”.

2. BM25 and vector search run in parallel and are fused with RRF (k=60). Before scoring, the security filter removes whatever Ana is not allowed to see.

3. The semantic ranker takes only the top 50 and scores them from 0 to 4:

   Score	Meaning
   4	Fully answers
   3	Relevant but incomplete
   2	Partial
   1	Related, answers very little
   0	Irrelevant

   Chunks with a score < 2 are discarded. If none are left, the answer is “I couldn’t find that information”. Microsoft warns that the score distribution can vary slightly, so thresholds should not be too fine-grained.

4. The model generates the answer with citations, using the prompt structure from section 13.

5. A faithfulness check verifies that each sentence is supported by the chunks. If it fails, the answer is regenerated or given with caution.

A complex question (“compare remote work in Spain vs Mexico and tell me which applies if I relocate”) goes to Azure AI Search agentic retrieval, which splits it into subqueries, runs them in parallel, reranks each one with the semantic ranker and merges the results. The LLM-based query planning and answer synthesis are still in preview.

A global question (“which themes recur across all the 2026 policies?”) is where GraphRAG is worth it.

21.3 Evaluation on Azure (Microsoft Foundry)

Scores use a scale of 1 to 5 and pass at 3 by default. Continuous evaluation runs on samples of real traffic (configurable percentage, up to 1000 requests per hour) and sends the results to Application Insights, linked to the traces.

Part VIII · Comparison: Azure vs Google Cloud vs open source

22. Azure vs Google Cloud vs open source

Some products have been renamed recently (verified on 2026-09-23):

• At Google, Vertex AI now appears as Gemini Enterprise Agent Platform, Vertex AI Search is being renamed to Agent Search, and Vector Search 2.0 is now called Agent Retrieval.
• At Microsoft, Azure AI Foundry is now Microsoft Foundry.

Phase 1: Preparing the documents

Phase 2: Answering a question

Phase 3: Generation and guardrails

Phase 4: Evaluation and monitoring

Summary in one picture

Three differences that matter

1. Hybrid search: Azure AI Search includes keyword search. In Google Vector Search you have to generate the sparse vector yourself; if you want Google to manage it, use Agent Search. In open source it depends on the database.
2. Reranker: Azure’s reranks only the top 50. Google’s Ranking API accepts up to 1000 chunks and works with any search engine, even an external one. In open source you control the model, cost and latency, but you also have to operate it.
3. Evaluation: both clouds now offer offline evaluation and continuous evaluation on real traffic, with standard traces (OpenTelemetry). In open source, RAGAS or DeepEval (offline) plus Phoenix, MLflow or Langfuse (production) cover the same ground, but you do the integration yourself.

Appendix · Glossary

Appendix · References

Official documentation (accessed on 2026-09-23)

Microsoft Azure

• Semantic ranker: https://learn.microsoft.com/en-us/azure/search/semantic-search-overview
• RRF in hybrid search: https://learn.microsoft.com/en-us/azure/search/hybrid-search-ranking
• Agentic search: https://learn.microsoft.com/en-us/azure/search/agentic-retrieval-overview
• Document Layout skill: https://learn.microsoft.com/en-us/azure/search/cognitive-search-skill-document-intelligence-layout
• Chunking in Azure AI Search: https://learn.microsoft.com/en-us/azure/search/vector-search-how-to-chunk-documents
• RAG evaluators: https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/rag-evaluators
• Continuous evaluation: https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/continuous-evaluation-agents

Google Cloud

• Ranking API: https://cloud.google.com/generative-ai-app-builder/docs/ranking
• Check Grounding: https://cloud.google.com/generative-ai-app-builder/docs/check-grounding
• Document AI Layout Parser: https://cloud.google.com/document-ai/docs/layout-parse-chunk
• RAG Engine: https://docs.cloud.google.com/gemini-enterprise-agent-platform/build/rag-engine/rag-overview
• Hybrid search in Vector Search: https://docs.cloud.google.com/gemini-enterprise-agent-platform/build/vector-search/about-hybrid-search
• Text embeddings: https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/embeddings/get-text-embeddings
• Online Monitors: https://docs.cloud.google.com/gemini-enterprise-agent-platform/optimize/evaluation/evaluate-online

Elastic

• Semantic search: https://www.elastic.co/docs/solutions/search/semantic-search
• Vector search: https://www.elastic.co/docs/solutions/search/vector
• semantic_text: https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/semantic-text
• ELSER: https://www.elastic.co/docs/explore-analyze/machine-learning/nlp/ml-nlp-elser
• Retrievers: https://www.elastic.co/docs/reference/elasticsearch/rest-apis/retrievers

Papers and reports

Retrieval

• Gao et al., RAG for LLMs: A Survey, 2023/24 — https://arxiv.org/abs/2312.10997
• Cormack, Clarke, Büttcher, Reciprocal Rank Fusion, SIGIR 2009 — https://dl.acm.org/doi/10.1145/1571941.1572114
• Formal et al., SPLADE, 2021 — https://arxiv.org/abs/2107.05720
• Chen et al., BGE-M3, 2024 — https://arxiv.org/abs/2402.03216
• Khattab & Zaharia, ColBERT, 2020 — https://arxiv.org/abs/2004.12832
• Faysse et al., ColPali, ICLR 2025 — https://arxiv.org/abs/2407.01449
• Gao et al., HyDE, 2022 — https://arxiv.org/abs/2212.10496
• Zheng et al., Step-Back Prompting, ICLR 2024 — https://arxiv.org/abs/2310.06117
• Sarthi et al., RAPTOR, 2024 — https://arxiv.org/abs/2401.18059
• Edge et al., GraphRAG, 2024 — https://arxiv.org/abs/2404.16130
• Microsoft Research, LazyGraphRAG, 2024 — https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/
• Gutiérrez et al., HippoRAG 2, ICML 2025 — https://arxiv.org/abs/2502.14802
• Han et al., RAG vs. GraphRAG, 2025/26 — https://arxiv.org/abs/2502.11371
• Asai et al., Self-RAG, 2023 — https://arxiv.org/abs/2310.11511
• Yan et al., Corrective RAG, 2024 — https://arxiv.org/abs/2401.15884
• Jeong et al., Adaptive-RAG, NAACL 2024 — https://arxiv.org/abs/2403.14403
• Singh et al., Agentic RAG Survey, 2025/26 — https://arxiv.org/abs/2501.09136
• Jin et al., Search-R1, 2025 — https://arxiv.org/abs/2503.09516
• Liu et al., Lost in the Middle, TACL — https://arxiv.org/abs/2307.03172
• Li et al., Self-Route, EMNLP 2024 — https://arxiv.org/abs/2407.16833
• Li et al., LaRA, 2025 — https://arxiv.org/abs/2502.09977

Chunking

• Qu, Tu, Bao (Vectara), Is Semantic Chunking Worth the Computational Cost?, 2024 — https://arxiv.org/abs/2410.13070
• Smith & Troynikov (Chroma), Evaluating Chunking Strategies for Retrieval, 2024 — https://research.trychroma.com/evaluating-chunking
• NVIDIA, Finding the Best Chunking Strategy, 2025 — https://developer.nvidia.com/blog/finding-the-best-chunking-strategy-for-accurate-ai-responses/
• Jimeno Yepes et al., Financial Report Chunking, 2024 — https://arxiv.org/abs/2402.05131
• Günther et al. (Jina), Late Chunking, 2024 — https://arxiv.org/abs/2409.04701
• Anthropic, Contextual Retrieval, 2024 — https://www.anthropic.com/engineering/contextual-retrieval (figures verified via a mirror; the original page blocked automated access)
• Bhat et al., Rethinking Chunk Size, 2025 — https://arxiv.org/abs/2505.21700
• IBM, Docling, 2024 — https://arxiv.org/abs/2408.09869

Reranking

• Nogueira & Cho, monoBERT, 2019 — https://arxiv.org/abs/1901.04085
• Thakur et al., BEIR, 2021 — https://arxiv.org/abs/2104.08663
• Sun et al., RankGPT, 2023 — https://arxiv.org/abs/2304.09542
• Pradeep et al., RankZephyr, 2023 — https://arxiv.org/abs/2312.02724
• Su et al., BRIGHT, 2024 — https://arxiv.org/abs/2407.12883
• Abdallah et al., How Good are LLM-based Rerankers?, 2025 — https://arxiv.org/abs/2508.16757
• Agentset, Cohere Rerank 4, 2025 — https://agentset.ai/blog/cohere-reranker-v4
• Qwen, Qwen3 Embedding & Reranker, 2025 — https://qwenlm.github.io/blog/qwen3-embedding/
• Jina, jina-reranker-v3.5, 2026 — https://arxiv.org/abs/2607.18152
• Mixedbread, mxbai-rerank-v2, 2025 — https://www.mixedbread.com/blog/mxbai-rerank-v2
• Jiang et al., LongLLMLingua, ACL 2024 — https://arxiv.org/abs/2310.06839

Evaluation

• Es et al., RAGAS, 2023 — https://arxiv.org/abs/2309.15217
• Saad-Falcon et al., ARES, NAACL 2024 — https://arxiv.org/abs/2311.09476
• Ru et al. (Amazon), RAGChecker, NeurIPS 2024 — https://arxiv.org/abs/2408.08067
• Gao et al., ALCE, EMNLP 2023 — https://arxiv.org/abs/2305.14627
• Zheng et al., Judging LLM-as-a-Judge, NeurIPS 2023 — https://arxiv.org/abs/2306.05685
• Thakur et al., Support Evaluation TREC 2024, SIGIR 2025 — https://arxiv.org/abs/2504.15205
• Yang et al. (Meta), CRAG benchmark, NeurIPS 2024 — https://arxiv.org/abs/2406.04744
• Islam et al., FinanceBench, 2023 — https://arxiv.org/abs/2311.11944
• Magesh et al. (Stanford), Hallucination-Free?, 2024 — https://arxiv.org/abs/2405.20362
• Vectara, Hallucination Leaderboard — https://github.com/vectara/hallucination-leaderboard
• Bao et al., FaithBench, 2024 — https://arxiv.org/abs/2410.13210
• Vectara, Open RAG Eval — https://github.com/vectara/open-rag-eval
• DeepEval, Metrics — https://deepeval.com/docs/metrics-introduction
• Databricks, Scorers and LLM judges — https://docs.databricks.com/aws/en/mlflow3/genai/eval-monitor/concepts/scorers