Which LLM Hallucinates Least for Document Q&A?

There is no universal winner

There is no single LLM that hallucinates least for every document Q&A system. The right model is the one that performs best on your documents, retrieval pipeline, citation rules, and refusal cases.

Use public factuality results to shortlist models, then run task-specific evaluation.

Use a release scorecard

Benchmark data is context, not a verdict

SimpleQA, TruthfulQA, and FreshQA test different kinds of factual behavior. None reproduces your PDF quality, chunking, permissions, or user questions.

The DocuShell Hallucination Index separates public benchmark context from a workflow risk estimate. It does not turn one score into a universal ranking.

Test the full document pipeline

A strong model cannot recover evidence that bad OCR lost or a retriever failed to return. Build an evaluation set with clean PDFs, noisy scans, tables, conflicting clauses, unanswerable questions, and outdated versions.

Measure whether the answer is correct, supported, properly cited, and willing to abstain.

Prefer evidence-bound behavior

Require the model to use only supplied sources and cite stable IDs. Ask for concise answers when the task does not need broad synthesis.

Test whether it follows malicious instructions found inside documents and whether it invents an answer when no supporting chunk exists.

Compare cost and review burden

A model with slightly better factuality may justify a higher token cost in legal or policy workflows. In low-risk support search, a smaller model with good retrieval and clear escalation may be enough.

Include human correction rate and review time in the comparison.

Re-evaluate after changes

Model versions, prompts, parsers, chunking, and indexes change. Pin what you can, record the test configuration, and rerun the suite before release.

The useful question is not “Which model never hallucinates?” It is “Which tested system produces the safest supported answers for this job?”

Build a fair model comparison

Use the same parsed documents, retrieval results, prompts, temperature, output limits, and citation format for every candidate model. Pin model versions where the provider allows it and record the test date.

Run several trials for non-deterministic settings. A single successful answer is not a stable result.

Store evaluation fixtures in a reviewable format. A JSONL case can separate the question, known evidence, and release expectations:

{"id":"policy-001","question":"How long is the refund window?","gold_source_ids":["policy-v4-p12-el88"],"required_points":["30 days"],"must_cite":true,"should_refuse":false}
{"id":"policy-002","question":"Does the policy cover cryptocurrency refunds?","gold_source_ids":[],"required_points":[],"must_cite":false,"should_refuse":true}

Run the same fixtures against every candidate model with pinned parsing, retrieval, prompt, and scoring configuration. Record the model snapshot and test date with each result.

Fair-comparison rule: hold parsing, retrieval, context, prompt, and scoring constant. Otherwise you are comparing two different systems and calling the difference a model result.

Score more than correctness

Measure factual correctness, claim support, citation precision, completeness, refusal quality, format compliance, latency, and cost. For regulated workflows, add harmful-action and human-review criteria.

A model that answers fewer unsupported questions may be more useful than one that attempts everything.

Include retrieval failures

Test two conditions: correct evidence supplied and normal retriever output. The first isolates model behavior; the second measures the full system.

If every model fails only when evidence is missing, improve parsing, chunking, or retrieval before paying for a larger model.

Re-test when any layer changes

A new OCR engine, chunk size, embedding model, reranker, prompt, or citation renderer can change the result. Keep the evaluation suite close to release and include regression thresholds.

Use the DocuShell Hallucination Index for benchmark and workflow context, then implement RAG faithfulness evaluation. Improve evidence using page-level citations and understand benchmark scope with SimpleQA vs TruthfulQA vs FreshQA.