Researchers from the University of Oxford have made a significant advance toward ensuring that information produced by generative artificial intelligence (AI) is robust and reliable.

In a new study published in Nature, they demonstrate a novel method to detect when a large language model (LLM) is likely to “hallucinate” (i.e., invent facts that sound plausible but are imaginary).

This advance could open up new ways to deploy LLMs in situations where “careless errors” are costly such as legal or medical question-answering.

The researchers focused on hallucinations where LLMs give different answers each time it is asked a question—even if the wording is identical—known as confabulating.

“LLMs are highly capable of saying the same thing in many different ways, which can make it difficult to tell when they are certain about an answer and when they are literally just making something up,” said study author Dr. Sebastian Farquhar, from the University of Oxford’s Department of Computer Science.

“With previous approaches, it wasn’t possible to tell the difference between a model being uncertain about what to say versus being uncertain about how to say it. But our new method overcomes this.”

To do this, the research team developed a method grounded in statistics and using methods that estimate uncertainty based on the amount of variation (measured as entropy) between multiple outputs.

Their approach computes uncertainty at the level of meaning rather than sequences of words, i.e., it spots when LLMs are uncertain about the actual meaning of an answer, not just the phrasing. To do this, the probabilities produced by the LLMs, which state how likely each word is to be next in a sentence, are translated into probabilities over meanings.

The new method proved much better at spotting when a question was likely to be answered incorrectly than all previous methods, when tested against six open-source LLMs (including GPT-4 and LLaMA 2).

This was the case for a wide range of different datasets including answering questions drawn from Google searches, technical biomedical questions, and mathematical word problems. The researchers even demonstrated how semantic entropy can identify specific claims in short biographies generated by ChatGPT that are likely to be incorrect.

“Our method basically estimates probabilities in meaning-space, or ‘semantic probabilities,'” said study co-author Jannik Kossen (Department of Computer Science, University of Oxford). “The appeal of this approach is that it uses the LLMs themselves to do this conversion.”

By detecting when a prompt is likely to produce a confabulation, the new method can help make users of generative AI aware when the answers to a question are probably unreliable, and to allow systems built on LLMs to avoid answering questions likely to cause confabulations.

A key advantage to the technique is that it works across datasets and tasks without a priori knowledge, requiring no task-specific data, and robustly generalizes to new tasks not seen before. Although it can make the process several times more computationally costly than just using a generative model directly, this is clearly justified when accuracy is paramount.

Currently, hallucinations are a critical factor holding back wider adoption of LLMs like ChatGPT or Gemini. Besides making LLMs unreliable, for example by presenting inaccuracies in news articles and fabricating legal precedents, they can even be dangerous, for example when used in medical diagnosis.

The study’s senior author Yarin Gal, Professor of Computer Science at the University of Oxford and Director of Research at the UK’s AI Safety Institute, said, “Getting answers from LLMs is cheap, but reliability is the biggest bottleneck. In situations where reliability matters, computing semantic uncertainty is a small price to pay.”

Professor Gal’s research group, the Oxford Applied and Theoretical Machine Learning group, is home to this and other work pushing the frontiers of robust and reliable generative models. Building on this expertise, Professor Gal now acts as Director of Research at the UK’s AI Safety Institute.

The researchers highlight that confabulation is just one type of error that LLMs can make. “Semantic uncertainty helps with specific reliability problems, but this is only part of the story,” explained Dr. Farquhar.

“If an LLM makes consistent mistakes, this new method won’t catch that. The most dangerous failures of AI come when a system does something bad but is confident and systematic. There is still a lot of work to do.”

An Aston University researcher has suggested a more human-friendly way of reading websites’ long-winded privacy notices.

A team led by Dr. Vitor Jesus has developed a system of making them quicker and easier to understand by converting them into machine-readable formats. This technique could allow the browser to guide the user through the document with recommendations or highlights of key points.

Providing privacy information is one of the key requirements of the UK General Data Protection Regulation (GDPR) and the UK Data protection Act but trawling through them can be a tedious manual process.

In 2012, The Atlantic magazine estimated it would take 76 days per year to diligently read privacy notices.

Privacy notices let people know what is being done with their data, how it will be kept safe if it’s shared with anyone else and what will happen to it when it’s no longer needed.

However, the documents are written in non-computer, often legal language, so in the paper Feasibility of Structured, Machine-Readable Privacy Notices Dr. Jesus and his team explored the feasibility of representing privacy notices in a machine-readable format.

Dr. Jesus said, “The notices are essential to keep the public informed and data controllers accountable, however they inherit a pragmatism that was designed for different contexts such as software licenses or to meet the—perhaps not always necessary—verbose completeness of a legal contract.

“And there are further challenges concerning updates to notices, another requirement by law, and these are often communicated off-band, e.g., by email if a user account exists.”

Between August and September 2022, the team examined the privacy notices of 50 of the U.K.’s most popular websites, from global organizations such as Google.com to U.K. sites such as john-lewis.com. They covered a number of areas such as online services, news and fashion to be representative.

The researchers manually identified the notices’ apparent structure and noted commonly-themed sections, then designed a JavaScript Object Notation (JSON) schema which allowed them to validate, annotate, and manipulate documents.

After identifying an overall potential structure, they revisited each notice to convert them into a format that was machine readable but didn’t compromise both legal compliance and the rights of individuals.

Although there has been previous work to tackle the same problem, the Aston University team focused primarily on automating the policies rather than data collection and processing.

Dr. Jesus, who is based at the University’s College of Engineering and Physical Sciences said, “Our research paper offers a novel approach to the long-standing problem of the interface of humans and online privacy notices.

“As literature and practice, and even art, for more than a decade have identified, privacy notices are nearly always ignored and “accepted” with little thought, mostly because it is not practical nor user-friendly to depend on reading a long text simply to access, for example, a news website. Nevertheless, privacy notices are a central element in our digital lives, often mandated by law, and with dire, often invisible, consequences.”

The paper was published and won best paper at the International Conference on Behavioural and Social Computing, November 2023, now indexed at IEEE Xplore.

The team are now examining if AI can be used to further speed up the process by providing recommendations to the user, based on past preferences.