Can Ethos Verify RAG Answers Without Calling Another LLM?

Direct answer

Yes. Ethos can verify whether RAG citations bind to trusted document evidence without calling another LLM. It uses deterministic checks for source fingerprints, evidence IDs, pages, quotations, values, table cells, regions, and declared source capabilities.

That does not mean Ethos proves the whole answer correct. It deliberately leaves semantic relevance, synthesis, causal reasoning, calculation policy, and broad factual truth outside the citation-grounding contract. A strong system uses deterministic checks first and invokes an LLM judge or reviewer only when interpretation is required.

The LLM-as-a-judge reliability guide explains the probabilistic layer. The deterministic RAG gate guide explains the release architecture.

What Ethos verifies without a model call

These checks compare structured inputs. They do not require a language model to decide whether two ideas are semantically equivalent.

Why deterministic verification is valuable

Repeatability

The same source, citation request, verifier version, and configuration should produce the same structured result. This is useful for release gates and CI.

Explicit failure states

Ethos distinguishes grounded, mismatched, missing, stale, capability-blocked, unsupported, and invalid outcomes. A single confidence score cannot provide the same routing precision.

Local evidence handling

Base verification can run locally without sending private source passages to an external judge API. The deployment still needs secure storage, process isolation, access control, and retention policy.

Cost control

Exact checks avoid judge-model tokens. Semantic evaluation can be reserved for ambiguous or high-risk claims.

Lower evaluation coupling

A judge-model update can change semantic scores. Deterministic citation contracts remain controlled by verifier and schema versions.

Boundary: “No LLM call” describes Ethos citation verification. The surrounding RAG application may still use LLMs for generation, claim extraction, relevance, or synthesis evaluation.

What deterministic matching cannot decide

Suppose the source says:

Revenue grew to $12.4 million in Q3.

Ethos can verify that a citation binds to that text. It cannot infer that a product launch caused the growth unless the source evidence states or supports causality.

Determinism is not a substitute for semantics. It is the correct method for exact propositions.

A hybrid verification architecture

generated claims + citations
          |
          v
trusted source-map resolution
          |
          v
Ethos deterministic checks
          |
          +-- failed -> block, refresh, or repair
          |
          v
semantic relevance and synthesis checks
          |
          v
deterministic calculations and release policy

This ordering prevents an LLM judge from rationalizing a wrong citation. The judge receives only evidence that has already passed identity and integrity checks.

Source facts versus synthesis

Classify claims before release:

• source fact: directly stated in grounded evidence;
• calculation: derived mechanically from verified values;
• synthesis: combines evidence or adds inference;
• unsupported: lacks traceable evidence.

A conservative policy can release relevant source facts, recompute calculations, review synthesis, and block unsupported claims.

Ethos’s application-release guidance also separates citation grounding, question relevance, and synthesis level. This avoids calling an answer “verified” because one citation matched.

Using Ethos from the CLI

A JSON verification path can run without a judge model:

ethos verify document.ethos.json \
  --citations citations.json \
  --fail-on-ungrounded \
  --out verification_report.json

The command writes a structured report. The enforcement flag makes completed but not fully grounded verification fail the gate. Invalid input and usage errors remain separate from negative grounding results.

PDF parsing or rendered crops may require caller-provided PDFium. JSON verification itself does not require a model API.

Using the Python wrapper

The Python package invokes a caller-provided local CLI:

from ethos_pdf import EthosCli, proof_summary

ethos = EthosCli(binary="/path/to/ethos")
report = ethos.verify(
    source="document.ethos.json",
    citations="citations.json",
    fail_on_ungrounded=False,
    output_format="json",
    timeout=30,
)

summary = proof_summary(report)
print(summary["proof_status"])

The derived summary is useful for application policy, but the canonical report remains the audit artifact.

Privacy advantages and limits

Local deterministic verification can reduce disclosure because evidence does not need to be sent to another model provider. This is valuable for contracts, policies, financial reports, clinical documents, and internal knowledge.

It does not automatically create compliance. Teams still need:

• tenant isolation;
• access checks before retrieval;
• encrypted storage and transport;
• controlled logs and artifacts;
• sandboxed document parsing;
• retention and deletion rules;
• dependency and vulnerability management;
• reviewed source authority.

Cost and latency design

Run independent exact checks concurrently within resource limits. Cache results only when the key includes:

• source fingerprint;
• citation request;
• verifier version;
• verification configuration;
• adapter identity and version;
• normalization profile.

Do not cache across a source-fingerprint change. Run deterministic filters before semantic model calls so obviously invalid citations do not consume judge tokens.

CI without evaluator variance

Ethos is useful for stable fixtures:

1. correct quote;
2. wrong page;
3. altered text;
4. missing element;
5. stale fingerprint;
6. table-cell mismatch;
7. capability-blocked table request;
8. malformed citation input.

Assert exact status and reason. Keep separate live-model tests for claim extraction and semantic quality.

When an LLM judge is still appropriate

Use a calibrated judge when:

• evidence paraphrases the claim;
• support spans multiple passages;
• the question requires semantic relevance;
• the answer contains synthesis;
• several valid phrasings exist;
• contradiction is conceptual rather than textual.

Validate the judge against human labels, pin its rubric and model where possible, and preserve an uncertainty or review state.

Decision matrix

Definitive verdict

Ethos can verify RAG citation grounding without another LLM because source identity, fingerprints, locators, quotations, values, tables, and capabilities are structured evidence questions. This provides repeatable, local, lower-cost checks and clear failure reasons.

Use DocuShell Parse PDF for source-aware evidence, Ethos for deterministic checks, the Ethos auditability guide for reports and release records, and the DocuShell hallucination index for broader workflow-risk context. Add semantic evaluation only where semantics are genuinely required.

Primary keyword: RAG verification without another LLM
Optimized meta title: Verify RAG Citations Without Another LLM
Optimized meta description: Use Ethos to verify source-bound RAG citations deterministically without a judge-model call, then evaluate semantics separately.
Proposed URL slug: can-ethos-verify-rag-answers-without-calling-another-llm