AI Evaluation Tools

LLM Hallucination Index

Compare model factuality signals and estimate workflow hallucination risk.

Benchmark providerWorkflowAnswering from retrieved document chunks.GroundingRAG retrieval supplies relevant source passages.Source qualitySelectable text with clear headings and page order.Answer styleModerate synthesis with source-aware wording.SensitivityDecisions, customer work, or operational content.

Ranked benchmark reference

SimpleQA factuality where a comparable public score is listed.

7 ranked, 2 unranked

Model	Provider	SimpleQA factuality	Risk band	Best fit	Source
GPT-4.5 preview Preview availability and pricing may differ from production model choices.	openaiOpenAI	62.5% SimpleQA score from the simple-evals README table.	Low risk 37.5% factuality gap	Fact-sensitive short answers when the model is available.	OpenAI simple-evals SimpleQA Verified: 2026-06-20
o3 Use retrieval and citations for document workflows because benchmark accuracy is not a guarantee.	openaiOpenAI	49.4% SimpleQA score from the simple-evals README table.	Low risk 50.6% factuality gap	Reasoning-heavy factual QA with source checks.	OpenAI simple-evals SimpleQA Verified: 2026-06-20
o1 Treat unsupported answers as claims that need evidence.	openaiOpenAI	42.6% SimpleQA score from the simple-evals README table.	Medium risk 57.4% factuality gap	Reasoning tasks where answers can be validated.	OpenAI simple-evals SimpleQA Verified: 2026-06-20
GPT-4.1 Pair with source citations for document Q&A and extraction.	openaiOpenAI	41.6% SimpleQA score from the simple-evals README table.	Medium risk 58.4% factuality gap	General factual workflows with retrieval.	OpenAI simple-evals SimpleQA Verified: 2026-06-20
GPT-4o Use concise answer constraints for factual tasks.	openaiOpenAI	38.8% SimpleQA score from the simple-evals README table.	Medium risk 61.2% factuality gap	Broad assistant tasks with source-backed review.	OpenAI simple-evals SimpleQA Verified: 2026-06-20
Claude 3.5 Sonnet Benchmark row is older than current Anthropic model families.	anthropicAnthropic	28.9% SimpleQA score from the simple-evals README table.	Medium risk 71.1% factuality gap	Document analysis when source grounding is strict.	OpenAI simple-evals SimpleQA Verified: 2026-06-20
Claude 3 Opus Use this row as historical benchmark context, not a current Sonnet/Opus replacement.	anthropicAnthropic	23.5% SimpleQA score from the simple-evals README table.	High risk 76.5% factuality gap	Source-grounded writing where unsupported claims are filtered.	OpenAI simple-evals SimpleQA Verified: 2026-06-20
Gemini 1.5 Flash Do not rank this row without a comparable factuality score.	googleGoogle	n/a The simple-evals table lists n/a for this SimpleQA row.	Unknown risk	Fast long-context workflows that need separate evaluation.	OpenAI simple-evals SimpleQA Verified: 2026-06-20
Gemini 1.5 Pro Do not rank this row without a comparable factuality score.	googleGoogle	n/a The simple-evals table lists n/a for this SimpleQA row.	Unknown risk	Long-context workflows that need separate evaluation.	OpenAI simple-evals SimpleQA Verified: 2026-06-20

Methodology sources

Source context verified: 2026-06-20.

OpenAI simple-evals SimpleQAShort-form factuality benchmark with public SimpleQA scores for selected models.The simple-evals README says new model and benchmark result updates ended in July 2025.TruthfulQABenchmark designed to test whether models imitate common falsehoods across 817 questions.Useful for factuality risk framing, not a live production-risk score.FreshQA / FreshLLMsDynamic QA benchmark focused on current knowledge and false-premise questions.Shows why freshness and false-premise handling matter for live workflows.RAGAS FaithfulnessRAG metric that checks whether generated claims are supported by retrieved context.Useful for document and RAG pipelines where the answer must stay grounded in source text.

Loading tool...

Complete Guide to LLM Hallucination Risk

Hallucination risk is not just a model property. The same model can be reliable on a clean source-backed extraction task and risky on a current-events answer with no retrieval.

DocuShell starts with public benchmark context, then lets you estimate workflow risk for PDF summaries, document Q&A, RAG assistants, and high-consequence extraction tasks.

The goal is not to crown one model. The goal is to identify when you need citations, extractive answers, retrieval refresh, OCR cleanup, refusal rules, or human review.

Benchmark Context

SimpleQA provides public short-form factuality scores for selected models. DocuShell uses those scores where the source lists comparable values and marks unavailable rows as n/a instead of guessing.

TruthfulQA and FreshQA add useful context. TruthfulQA focuses on common falsehoods, while FreshQA focuses on changing facts and false-premise questions. Those benchmarks explain risk patterns, but they are not a substitute for your own workflow evals.

OpenAI simple-evals TruthfulQA paper FreshQA paper

Workflow Risk Signals

A document AI workflow has risk signals that a generic benchmark cannot capture. Clean extracted text is safer than noisy OCR. Citation-required answers are safer than open-ended synthesis. Current facts and regulated domains require tighter controls.

The estimator combines those signals into a planning score. It is intentionally simple and inspectable so teams can debate the assumptions rather than accept a black-box risk label.

Risk reducers: source grounding, required citations, clean text extraction, extractive answers, and explicit uncertainty.
Risk increasers: no retrieval, noisy scans, current events, long-form synthesis, and regulated decisions.

RAG Faithfulness

For RAG systems, the key question is whether each generated claim is supported by retrieved context. Metrics such as RAGAS Faithfulness frame this as supported claims divided by total claims.

That framing maps well to DocuShell document workflows. When a PDF answer cites the exact page or chunk that supports a claim, reviewers can validate the answer faster and unsupported claims are easier to reject.

RAGAS Faithfulness Parse PDF to structured data Estimate LLM costs

pSEO Expansion Opportunities

The index can support future comparison pages for provider, benchmark, and workflow combinations. Examples include OpenAI hallucination risk, Claude factuality benchmark, Gemini RAG hallucination risk, SimpleQA model comparison, and best LLM for PDF Q&A.

Those pages should reuse the same source-backed registry and never invent benchmark values. Each page should answer a narrow search intent, cite the source data used, and link back to the calculator for workflow-specific risk.

How It Works

Step 1

Select a provider filter and review source-backed benchmark rows.

Step 2

Choose workflow, grounding, source quality, answer style, and domain sensitivity.

Step 3

Use the risk score and recommended controls to design safer document AI workflows.

Why This Tool

• Compare public SimpleQA factuality scores where a source lists comparable model results.
• See unranked rows clearly marked when benchmark data is not available.
• Estimate workflow risk from grounding, document quality, answer style, and domain sensitivity.
• Turn risk signals into practical controls such as citations, extractive answers, and review gates.
• Use source links for SimpleQA, TruthfulQA, FreshQA, and RAG faithfulness methodology.

Use Cases

• Choosing an LLM for PDF summarization or document Q&A.
• Estimating hallucination risk before launching a RAG assistant.
• Deciding when legal, policy, finance, or medical answers need human review.
• Comparing factuality signals before paying for AI API usage at scale.
• Writing model evaluation requirements for internal AI tools.

Frequently Asked Questions

Common questions about the Hallucination Index tool, how it works, privacy, file limits, and more.

An LLM hallucination index is a practical way to compare factuality risk. DocuShell separates public benchmark context from workflow risk because hallucination depends on the model, the task, source grounding, source quality, answer length, and domain sensitivity.

There is no universal winner across every workflow. The DocuShell table ranks public SimpleQA factuality scores where comparable source data is available, then pairs that with a workflow risk estimate so document and RAG teams can choose controls instead of relying on one benchmark number.

No. SimpleQA is a short-form factuality benchmark. A higher SimpleQA score suggests stronger factual answer performance on that benchmark, but it is not a universal hallucination rate for production workflows.

The estimator combines workflow type, grounding level, source quality, output style, and domain sensitivity. Retrieval, citations, clean source text, and extractive answers reduce risk. Current events, noisy OCR, verbose synthesis, and regulated domains increase risk.

No. This first release is a browser-side planning tool. It does not accept or upload prompts, PDFs, documents, or retrieved chunks.

No. Use it to plan controls and shortlist evaluation work. Production AI systems still need task-specific tests, source-grounding checks, regression suites, and human review for high-impact content.

Need a walkthrough before you start?

We publish first-party guides for the workflows people actually use, and we explain how those articles are tested, reviewed, and updated.

Read the blog Editorial standards

Source-backed benchmark context

This tool separates benchmark data from workflow estimates. Public benchmark rows link to their sources, unavailable scores stay marked as n/a, and production teams should still run task-specific evaluations before launch.

Editorial standards Privacy policy

Explore More Tools

Compress PDF Webpage to PDF PDF to Word PDF to JSON Markdown to PDF LLM Cost OCR PDF Merge PDF Split PDF Organize PDF Remove pages Rotate PDF Protect PDF