How Do I Chunk PDFs for RAG Without Losing Citations?

Start with the unit of meaning

Chunk PDFs for RAG by meaning first and size second. Split on headings, paragraphs, lists, and table boundaries. Every chunk should carry its source file, heading path, page range, and bounding boxes when available.

This avoids a common failure: the retriever finds the right sentence, but the application cannot show where it came from.

First-principles view: chunking is a lossy compression step. Its job is to reduce retrieval units without discarding the relationships a reviewer needs to verify the answer.

Pages are not semantic boundaries

A section can start at the bottom of page 4 and finish on page 6. If you create one chunk per page, the definition may be separated from its exceptions, or a table may be split from its caption.

Use page numbers as evidence metadata, not as the main chunking rule. DocuShell Parse PDF provides structured output that is better suited to heading-aware chunking than raw page text.

Build chunks from the document tree

A practical order is:

1. Parse the PDF into structured elements.
2. Remove repeated headers, footers, and empty artifacts.
3. Group paragraphs under their nearest heading.
4. Keep tables and ordered procedures intact where possible.
5. Split oversized groups with a modest overlap.

The heading path matters. A paragraph titled “Exceptions” means little unless the chunk also carries the parent section, such as “Travel Reimbursement Policy.”

Keep citation metadata attached

Store metadata beside the chunk text, not in a separate table that may drift later. A useful record includes the document ID, section path, page start and end, element IDs, and source coordinates.

When one chunk spans two pages, keep both. When a sentence is assembled from multiple elements, preserve every relevant element reference.

Do not bury tables inside prose

A table should normally be its own chunk with its caption and nearby explanation. Long tables can be divided by row groups, but each chunk needs repeated column headers and the same source identity.

See How Do I Extract Tables from a PDF for LLMs? for the table-specific workflow.

Test retrieval like a skeptical reader

Evaluate more than answer quality. For each test question, inspect the returned chunk and open the cited page. Check whether the evidence actually supports the answer.

A RAG system is not source-aware because it displays a page number. It is source-aware when the page and region lead a reviewer to the exact supporting evidence.

A practical chunk record

A production chunk should contain more than text and an embedding. Store a stable chunk ID, document version, heading path, element types, page range, source boxes, and extraction warnings. Add access-control metadata before the chunk enters the index.

For example, a policy chunk might carry “Travel Policy > International Travel > Exceptions,” pages 14–15, and three source regions. If the policy is replaced, the document version lets you remove every derived chunk without relying on fuzzy text matching.

{
  "chunk_id": "policy-v3-international-exceptions-02",
  "text": "Exceptions require written approval...",
  "heading_path": ["Travel Policy", "International Travel", "Exceptions"],
  "pages": [14, 15],
  "source_elements": ["el_443", "el_444"],
  "bbox": [[82, 318, 526, 704], [84, 72, 525, 160]],
  "document_version": "3.0",
  "access_scope": ["employees"]
}

Chunk policy by content type

How much overlap should you use?

Overlap is useful when a paragraph crosses a split, but large repeated windows create near-duplicate search results. Start with structural boundaries. Add only enough overlap to preserve the sentence or list that would otherwise be cut.

Measure retrieval diversity. If the top five results are almost the same paragraph, excessive overlap may be crowding out other evidence. If definitions disappear from later chunks, carry the heading path or a short parent summary as metadata rather than repeating an entire page.

Evaluate chunking separately from generation

Build questions whose answer locations are known. For each question, measure whether the correct chunk appears in the retrieval set before asking a model to write an answer. This isolates chunking and retrieval failures from generation failures.

Include hard cases: answers spanning two pages, table footnotes, definitions referenced later, multi-column pages, and questions with no answer. The RAG faithfulness and citation quality guide provides the next evaluation layer.

Use source-aware PDF parsing before chunking, and follow the page-level citation workflow when presenting results. For broader architecture, see RAG for documentation sites.