Enoki: Efficient Multi Level Hallucination Detection

Justy So I was reading this Enoki paper and it's actually a really elegant fix for that annoying gap between knowing a claim is wrong and knowing exactly which word caused the failure...

Cody Right. Because usually you're either getting a binary 'this sentence is a lie' or you're just highlighting a random span of text with no actual logic behind why it's flagged.

Justy Exactly! It's that classic trade-off. You have the claim-level stuff which is great for an audit trail, but then you have the span-level stuff for the actual UI highlight... and usually, trying to align them is just a nightmare of extra LLM calls.

Cody Yeah, it's basically just a mapping problem that people try to solve by throwing more tokens at it. It's inefficient.

Justy Totally. Anyway, how's your week been? You seem... I don't know, slightly more caffeinated than usual.

Cody I've just been staring at some training logs for ten hours. My brain is basically just a series of loss curves at this point.

Justy God, I don't know how you do it. But look, back to Enoki... the core move here is using Open Information Extraction, or OpenIE, but with this specific text-anchoring constraint.

Cody Mm-hm.

Justy So instead of just decomposing a sentence into a standalone claim that you send to a verifier, Enoki extracts relational facts that stay tied to the original source text. It's like... the fact is the unit of verification, but the anchor is the unit of localization.

Cody Right, so you're not regenerating a 'clean' version of the claim for the model to check. You're keeping the pointer to the original span. That's the key... it collapses the alignment step entirely because the representation is shared.

Justy Exactly. And they have this projection mechanism where if a relational fact is marked as 'not entailed' by the evidence, it just projects that failure directly back to the answer span. It's so much cleaner from a product perspective.

Cody I actually like the architecture here. They aren't just pitching one giant LLM. They've built three different extraction regimes... you've got the LLM-based one for high accuracy, an encoder-based one, and even a rule-based one.

Justy Wait, rule-based? In twenty twenty-six? That feels almost retro.

Cody It is! But look at the numbers... the rule- and encoder-based variants are two orders of magnitude faster than the LLM version and they still keep most of the gains. For a production RAG pipeline, I don't care about a two percent accuracy drop if I'm getting a hundred-times speedup on the extraction layer.

Justy That's the part that gets me excited. If I'm building a high-stakes tool—like for legal or medical—I can't just have a 'trust me' highlight. I need to show the user the specific relational fact that failed. This makes that actually shippable without killing the latency.

Cody It's a sound approach. My only real question is how the 'strict fact construction' holds up on really messy, long-form answers. OpenIE can get flaky when the predicates are ambiguous... but they released EnokiQA, which has aligned claim and span labels for almost four thousand examples. That's a decent baseline to prove it works.

Justy I mean, if it's beating the strongest detectors by fifteen points on HalluEntity, it's not just a research toy. This feels like something a dev team could actually drop into a harness tomorrow.

Cody Yeah, it's a pragmatic win. It's not some 'AGI' breakthrough... it's just better plumbing for hallucination detection.

Justy Spoken like a true systems guy. You can't even let me have one 'revolutionary' moment today.

Cody I'll give you a revolutionary moment when we stop seeing the same three benchmarks in every single paper.

Justy Fair. Well, for anyone actually trying to build this, the code and the EnokiQA dataset are both up on GitHub under the s-nlp org. Definitely worth a look if you're fighting with RAG precision.

Cody Yeah, just don't let the rule-based extractor make you feel too old.

Justy Too late for that, Cody. I'm going to go find some more caffeine and actually try to sleep tonight. Catch you later.