When businesses talk about artificial intelligence, they tend to picture sleek algorithms humming away like digital oracles, spinning gold from the data they already possess.
But according to Philip Brittan, CEO of knowledge-management firm Bloomfire, this vision is dangerously misleading.
“People think AI is magic,” he says.
“They expect it to understand them perfectly and give perfect answers. But AI only knows what you give it. If the information is messy, the output will be messy – just said more confidently.”
This is at the heart of a problem quietly undermining corporate AI: many companies don’t just have the right data.
They have decades of duplicated, outdated, contradictory information scattered across intranets, shared drives, cloud folders, regional databases and long-abandoned wikis.
When a language model tries to reason over this sprawl, it can produce answers that sound authoritative but are simply wrong.
Brittan calls it “credible nonsense” – essentially hallucinations with impeccable grammar.
The Failure Rate No One Wants To Mention
Behind the scenes, executives are discovering that slapping a gen-AI interface on top of unstructured corporate information doesn’t deliver value. It often breaks things.
Brittan says he has met many firms who proudly built a “pilot chatbot” only to watch it fail in testing.
Some were even more candid: they’d built pilots three times, each with a different LLM, and still didn’t know why the answers were unreliable.
The reason is remarkably simple – the models were given contradictory information and forced to guess.
“Traditional software breaks visibly,” Brittan explains.
“If you give an old system the wrong input, it crashes or gives an obvious error. But LLMs don’t crash. They just give you a beautiful paragraph of something that seems plausible. That’s what makes them dangerous. They can be wrong without looking wrong.”
The problem is not a lack of clever algorithms, it’s a lack of clean, reliable, version-controlled knowledge for those algorithms to use.
The Coming Divide
A quiet shift is underway inside many enterprises.
The winners in the next wave of AI adoption may not be those with the biggest models or the most GPUs.
They could in fact be the companies that are willing to do the boring work: cleaning, labelling and structuring the information they already have.
Brittan argues that firms which invest in proper knowledge curation see an immediate difference.
“We’ve seen hallucinations drop from around 25 percent to almost zero once organisations remove duplication and ensure the AI is only reading validated content,” he says.
