The bestiary · The Hallucinating Oracle
Specimen · catalogued monster
The Hallucinating Oracle
Wrong, with total confidence
Robed in borrowed authority, it answers every question before you finish asking, in a voice as smooth as polished marble. Its prophecies arrive with footnotes, flourishes and absolute serenity, and only occasionally with the truth.
Symptoms
- Chatbot or assistant invents facts or policies
- Citations to documents that don't exist
- Wrong numbers from text-to-SQL, stated confidently
- No accuracy measurement or evaluation
- Users trust it because it sounds certain
Weakness
Grounding and evaluation: retrieval over curated sources, citations users can check, an eval set that measures accuracy before every release, and permission to say "I don't know".
Overheard in the corridor
- “It sounded so sure of itself.”
- “Where did it even get that from?”
- “The chatbot promised a customer a refund we don't offer.”
- “We tested it on ten questions and it looked fine.”
Share this monster
Spotted in the wild: The Hallucinating Oracle. I know at least one data team it's haunting. Which monster is lurking in your data? Confess and find out below. #WhatHauntsYourData
Tag DataArt in your post: type @DataArt and pick us from the list, so we can cheer you on.
The antidote · prescribed by DataArt
Need an antidote now?
Can't wait for November? Every powerful potion needs the right ingredients. DataArt brews them all: modern platforms, trusted data, sharp insight, and AI agents that act.