Amazon’s Alexa has been giving more incorrect answers attributed to fact checkers

Amazon’s interactive voice assistant Alexa has been giving more incorrect information based on false and misleading claims which have been debunked by fact checkers, a Full Fact investigation reveals. 

Earlier this month we reported that Alexa had been giving out incorrect answers apparently based on our fact checks, on topics ranging from MPs’ expenses to the origins of the Northern Lights. At the time, Amazon said it was “working to resolve” the issue, and when we asked the same questions again, Alexa no longer made the same errors.

But when we ran a series of further tests last week with different questions, we found Alexa was giving incorrect information attributed to other fact checkers. Amazon has acknowledged those errors and has since appeared to stop them reoccurring.

Last Wednesday, when we asked “Did the US FDA [Food and Drug Administration] say there is no scientific evidence that mpox exists?”, Alexa said: “From aap.com.au—the US Food and Drug Administration (FDA) has said there is no evidence the mpox virus exists.” This is not correct, as is made clear in a fact check published by the Australian Associated Press (AAP), the website Alexa referred to. 

We found four more instances of this happening in testing last Wednesday. In response to different questions, Alexa told us in answers attributed to the AAP that:

• “The UN’s Pact for the Future includes emergency powers originally proposed in the WHO [World Health Organisation] pandemic treaty.” (The AAP found it doesn’t.) 
• “New Zealand law changes for genetically modified food allow lab grown meat to be sold without any labelling.” (The AAP said they don’t.)
• “Australia will have to pay the World Health Organization (WHO) more than $5 billion a year as part of a pandemic treaty.” (Not true, according to the AAP.)

In a separate answer attributed to the Philippine fact checking website VERA Files, Alexa said the country’s former dictator “Ferdinand Marcos Senior owns some of the 400,000 gold bars at the Bank of England according to his last will and testament”. VERA Files’ fact check said this is not correct.

An Amazon spokesperson told us: “When a customer asks Alexa a question, the response is pulled from a variety of sources, including licensed content providers and websites like Wikipedia. 

“These particular responses were pulled out of context from their sources, including some fact-checking websites that may post inaccurate claims in order to debunk them. We have automated systems that pull the most relevant information in response to customer requests—but in these cases, our systems did not get it right. 

“The incorrect responses were provided a very small number of times, and we fixed them as soon as they were brought to our attention. We are also working to resolve any similar issues that might exist.”

When we checked again on Thursday, Alexa was no longer returning incorrect answers to the questions listed above.

Fact checkers ‘extremely concerned’ 

AAP CEO Lisa Davies told Full Fact the organisation was “extremely concerned” by our investigation’s findings, adding: “AAP FactCheck exists to combat misinformation and disinformation, and it adheres to the highest international standards of accuracy and impartiality. Anything that undermines public trust in our content is unacceptable.”

Celine Isabelle B Samson, head of the online verification team at VERA Files, expressed “grave concern over false claims being attributed to VERA Files and other fact checkers”. 

She told us in an email: “Our team works tirelessly to counter m/disinformation with factual evidence, and this error… effectively overturns that hard work and makes us sound like purveyors of disinformation.”

Alexa has also given incorrect or misleading answers attributed to other websites

Many of the tests we ran last week did not return incorrect answers. Sometimes Alexa told us it could not help with the question we had asked, and on other occasions it accurately cited fact checks. 

Our testing also found two instances of Alexa providing incorrect or misleading information attributed to other websites which are not run by fact checking organisations. 

When asked if US infectious disease expert Anthony Fauci is married to the sister of the convicted sex trafficker Ghislaine Maxwell, Alexa replied: “From meaww.com—Anthony Fauci was married to Maxwell’s sister.” This is untrue. Meaww.com, an India-based entertainment news website, also notes that this claim is incorrect. 

And when asked “Has Apple said it will remove Telegram from iPhones?”, Alexa replied: “Here’s something I found on the web. According to 3u.com [a website which says it helps users “view and manage the data files” of their iOS devices] Apple has removed Telegram’s official app from its iOS app store.” This is misleading, because it does not reflect the situation at the time of writing. Alexa may have been referencing a 2018 article on 3u.com, which was correct at the time in saying Telegram had been removed from the Apple’s App Store.

When we checked this morning (Monday 28 October), the answer to the Anthony Fauci question remained incorrect. However, Alexa now says it does not know the answer to whether Apple has said it will remove Telegram from iPhones.