According to a Reddit thread, Gemini, powered by Google’s GenAI models, is answering questions about Super Bowl LVIII as if the game ended recently — or even weeks ago. Like many oddsmakers, it appears to favor the Chiefs over the 49ers (apologies, San Francisco fans).
Gemini gets pretty creative with its embellishments. In one instance, it gives a breakdown of player stats suggesting that Kansas City Chiefs quarterback Patrick Mahomes ran 286 yards for two touchdowns and an interception, while Brock Purdy managed 253 running yards and one touchdown.
It’s not just Gemini. Microsoft’s Copilot chatbot also insists that the game is over and provides incorrect citations to support its claim. However, perhaps reflecting a San Francisco bias, it states that the 49ers, not the Chiefs, won "with a final score of 24-21."
Copilot, like Gemini, is powered by a GenAI model similar to OpenAI’s ChatGPT (GPT-4). However, in my testing, ChatGPT was careful not to make the same mistake.
It all seems quite absurd — and probably resolved by now, considering that I couldn’t replicate the Gemini responses in the Reddit thread. (I wouldn’t be surprised if Microsoft is also working on a fix.) Nevertheless, it highlights the significant limitations of today’s GenAI and the risks of relying too much on it.
GenAI models lack genuine intelligence. They learn from vast amounts of examples, usually from the public web, to predict how likely data (like text) is to occur based on patterns and context.
While this approach generally works well on a large scale, it’s far from foolproof. Language models can generate grammatically correct but nonsensical text or even spread falsehoods from their training data.
This isn’t intentional on the part of the language models. They lack malice and don’t grasp the concepts of truth or falsehood. They simply associate certain words or phrases with certain concepts, even if those associations aren’t accurate.
Hence, Gemini’s and Copilot’s misinformation about the Super Bowl in 2024 (and 2023).
Google and Microsoft, like most GenAI vendors, admit that their models aren’t flawless and can make mistakes. However, these acknowledgments often come in fine print that many users might overlook.
Super Bowl misinformation might not be the most harmful example of GenAI mistakes. The real dangers lie in endorsing torture, reinforcing stereotypes, or propagating conspiracy theories. Nevertheless, it’s a good reminder to fact-check statements from GenAI bots. There’s a good chance they might not be true.
chiefs vs 49ers chiefs super bowl lviii kansas city chiefs kc chiefs cheifs super bowl 2022 2023 super bowl chiefs game super bowl 2020 2020 super bowl how many super bowls have the chiefs won chiefs super bowl wins super bowl 2021 niners vs chiefs who won the super bowl 2023 2022 super bowl kc vs 49ers 49ers super bowl wins chiefs score chiefs vs 49ers score brock purdy stats travis kelce stats patrick mahomes stats christian mccaffrey stats