Can an LLM Reliably Judge Another LLM for Hallucinations?

Direct answer

An LLM can judge another LLM well enough to provide a useful hallucination-detection signal, especially for semantic support, contradiction, relevance, and qualitative comparisons. It cannot be assumed reliable by default. Its output is affected by the judge model, prompt, ordering, context length, answer style, domain, and rubric—and it can confidently rationalize the same weak evidence as the model it evaluates.

Use LLM-as-a-judge for questions that genuinely require semantic interpretation. Use deterministic code for source identity, citation location, exact evidence, fingerprints, arithmetic, schemas, and access rules. For high-stakes RAG, combine the two and calibrate the judge on representative human-labeled cases.

The faithfulness, groundedness, and citation-verification comparison explains which evaluation layer owns each question. The hallucinated-citation detection workflow shows the combined architecture.

What LLM-as-a-judge means

LLM-as-a-judge uses a language model to evaluate generated output. The judge may receive:

• the user question;
• retrieved context or verified source evidence;
• the candidate answer;
• a reference answer when available;
• an evaluation rubric;
• requested labels, scores, or explanations.

Typical tasks include factual-support classification, relevance scoring, pairwise preference, style review, safety classification, and claim decomposition. The judge is not observing truth directly; it is producing another model output conditioned on the supplied evidence and rubric.

Why LLM judges can work

Language models encode broad linguistic and factual patterns. Strong judges can recognize paraphrases, contradictions, missing qualifications, and unsupported causal claims that exact matching cannot handle.

They are particularly useful when:

• the claim and evidence use different wording;
• support depends on multiple sentences;
• the answer must be graded for relevance or completeness;
• reference answers allow several valid phrasings;
• human review is too expensive for every test case;
• teams need directional regression signals across prompt or model versions.

Compared with exact string rules, a judge can understand that “rose from 10 to 12” entails an increase while also recognizing that it does not prove why the increase occurred.

Why LLM judges fail

Shared blind spots

Generator and judge models may share training patterns and reasoning weaknesses. A fluent but invalid explanation can persuade both. Using a larger judge may reduce some errors but does not create a truth oracle.

Position and ordering bias

In pairwise evaluation, the order of candidate answers can change preferences. Run swapped-order tests and treat inconsistent results as uncertainty.

Verbosity and style bias

Judges may prefer longer, polished answers even when concise answers are more accurate. Rubrics should state that style is not evidence and should score factual dimensions separately.

Context distraction

Large context windows can include enough loosely related text for the judge to rationalize an unsupported claim. Give the judge only the claim and its relevant evidence when evaluating support.

Prompt and rubric sensitivity

Small wording changes can alter labels or thresholds. Version the full rubric and judge configuration as evaluation code.

Domain weakness

Legal exceptions, clinical qualifiers, financial footnotes, and technical specifications can require expert knowledge. Apparent agreement with ordinary-language evidence may hide a material domain error.

Non-determinism

Even at low temperature, provider implementation, model updates, and token-level variation can affect output. Repeated runs and stable model snapshots help measure this uncertainty.

Core principle: Judge explanations are diagnostic text, not proof. A persuasive rationale should never override a failed deterministic source or calculation check.

What an LLM judge should and should not decide

Moving exact questions into an LLM increases cost and uncertainty without adding value.

A disciplined hallucination-evaluation pipeline

Step 1: define the target failure

Do not ask “is this hallucinated?” without a taxonomy. Separate unsupported claims, contradictions, stale sources, wrong citations, calculation errors, irrelevant answers, and missing coverage.

Step 2: create atomic claims

Decompose compound sentences. The judge should not pass an entire sentence when only one of its three propositions is supported.

Step 3: verify mechanical evidence first

Resolve source IDs, fingerprints, page locators, quotations, values, and table cells. Remove failed citations from the semantic-support path or label them explicitly.

Step 4: constrain judge input

Provide the question, one atomic claim, verified evidence, and a narrow rubric. Do not expose unrelated answer prose that can bias the judge.

Step 5: require structured output

{
  "claim_id": "claim-17",
  "support": "insufficient",
  "reason_code": "causal_link_not_stated",
  "evidence_ids": ["check-v0042"],
  "confidence": "medium"
}

Use a closed set of labels and reason codes. Parse failures should be recorded as evaluation errors, not converted into passing scores.

Step 6: calibrate against humans

Build a representative gold set with domain reviewers. Include straightforward support, paraphrases, negations, exceptions, numerical claims, tables, partial support, and adversarial fluent errors.

Step 7: measure by failure class

Aggregate accuracy can conceal dangerous blind spots. Report precision and recall for unsupported claims, contradictions, numerical errors, and insufficient evidence separately.

Step 8: connect evaluation to policy

Decide what each label does. “Insufficient evidence” may block a final answer; “supported synthesis” may require review; “direct support” may release only after citation grounding passes.

How to validate an LLM judge

Representative sampling

Use the same domains, document structures, languages, answer lengths, and failure types expected in production. Public benchmarks are useful context but do not substitute for application data.

Independent labels

Have qualified reviewers label cases without seeing the judge result. Use adjudication for disagreements and document label definitions.

Agreement metrics

Measure confusion matrices, class-specific precision and recall, and inter-rater agreement. A correlation coefficient alone can hide a judge that ranks answers reasonably but misclassifies the release threshold.

Stability tests

Repeat identical cases, swap pairwise order, paraphrase non-material wording, and vary irrelevant context. Material label changes indicate brittleness.

Leakage controls

Ensure the judge is not rewarded for recognizing benchmark patterns or reference-answer style. Keep evaluation cases separate from prompt-development examples.

Version control

Pin the judge model when possible. Record provider, model snapshot, rubric, schema, decoding parameters, and code version. Recalibrate after any material change.

Thresholds and release risk

The correct threshold depends on the cost of false passes and false blocks. A customer-support assistant may tolerate more review friction than a legal advice system can tolerate unsupported release.

For a release gate, prioritize precision of the “safe to show” class. A judge that catches many bad answers but incorrectly approves a material fraction of unsupported claims is not a sufficient sole gate.

Use three-way decisions when uncertainty matters:

• release: strong support and all deterministic checks pass;
• review: ambiguous support, synthesis, or evaluator disagreement;
• block/abstain: missing, contradictory, stale, or unsupported evidence.

Pairing LLM judges with Ethos

Ethos handles the exact evidence-binding questions. It verifies whether caller-provided citations resolve against source evidence, whether fingerprints are fresh, and whether quotes, values, table cells, or targets match within source capabilities.

The semantic judge handles a different question: does that grounded evidence support the claim and answer the user’s question?

candidate claim + citation
        |
        v
Ethos deterministic grounding
        |
        +-- failed -> block or regenerate
        |
        v
calibrated semantic judge
        |
        +-- direct source fact -> release policy
        +-- synthesis -> review policy
        +-- unsupported -> block

This architecture prevents the judge from silently forgiving a wrong citation because it found similar text elsewhere. It also prevents Ethos from being asked to make semantic claims outside its contract.

Cost and latency controls

LLM evaluation can double or multiply model calls. Control cost by:

• running deterministic filters first;
• judging atomic claims in batches where schema reliability permits;
• evaluating only high-risk or uncertain claims online;
• running broader regression suites offline;
• caching only when inputs, evidence, judge model, and rubric versions are identical;
• routing clear source facts differently from synthesis;
• sending compact verified evidence instead of entire documents.

Do not cache across source-fingerprint or judge-version changes.

Common anti-patterns

• Asking the generator to grade itself with the same context and accepting its confidence.
• Using one public benchmark threshold for every domain.
• Combining relevance, correctness, style, and support into one score.
• Trusting chain-of-thought-style explanations as evidence of correctness.
• Letting a judge override failed source fingerprints or access policy.
• Ignoring parse failures and converting them into neutral scores.
• Updating the judge model without rerunning calibration.
• Reporting average score without false-pass rates at the release threshold.

Definitive verdict

An LLM can be a useful judge, but it is a probabilistic evaluator—not an authority. Use it for semantic support, relevance, nuanced contradiction, and synthesis triage. Validate it against domain experts, measure failures by class, version its configuration, and create a review state for uncertainty.

Use deterministic checks wherever the proposition is exact. DocuShell Parse PDF can support source-aware evidence workflows, Ethos can verify citation grounding, and the DocuShell hallucination index can provide broader model-risk context. The safest system uses each layer only for the question it can genuinely answer.

Primary keyword: LLM as a judge for hallucinations
Optimized meta title: Can an LLM Judge Another LLM Reliably?
Optimized meta description: Learn when LLM-as-a-judge detects hallucinations, where it fails, how to calibrate it, and when deterministic checks are safer.
Proposed URL slug: can-an-llm-reliably-judge-another-llm-for-hallucinations