AI Fact Checking

Full Fact’s Public Interest AI Trust Benchmark

People are increasingly turning to AI chatbots, based on large language models (LLMs), for information that shapes their lives. As these tools become embedded in everyday life, the quality and reliability of their outputs become a matter of public interest on par with the accuracy of broadcast media or government statistics.

Full Fact has built a publicly available, auditable benchmark to independently evaluate the output of leading AI models.

By systematically testing these models over time, we can expose any inconsistencies in their responses, and provide the general public with an independent assessment on which AI tools they can trust and ultimately hold tech companies accountable for producing good information.

The first report is out now.

Image of AI chatbot apps on a phone

Wrong on Sunday, right on Monday. Why AI chatbots have a confidence problem

This is the first in a series of reports on the work of Full Fact’s Public Interest AI Trust Benchmark.

Photo of a tape measure

How can we evaluate LLMs? Let us count the ways.

A review of existing benchmarks for Large Language Models.

An artist’s illustration of artificial intelligence (AI)

Full Fact's benchmarking methodology: an overview

A deeper dive into the approach used by Full Fact’s Public Interest AI Trust Benchmark.