All the knowledge in the world in one conversation – that is the big promise of chatbots. Unfortunately, this has a striking weakness, as a recent case shows.
Artificial intelligence such as ChatGPT should replace search engines in the long term and always provide us with up-to-date answers – simply by asking. The most important question then is where the bots get this knowledge from. The two largest programs, ChatGPT and Bard, have now been caught spouting nonsense – for using each other as a data source.
The journalist James Vincent caught her doing it. And that’s just by accident. When Google released its new AI assistant Bard to the public, Vincent wanted to find out how ChatGPT handled it. And asked if the competitor had already been hired. The AI immediately had an answer: On March 21, Bard was turned off, it asserted, self-confidently and factually wrong. But when Vincent wanted to investigate the reason for the false claim, he was amazed.
Quotes from quotes from quotes
As always, ChatGPT provided the source right away – and completely misunderstood it. The linked article was actually about a tweet: it showed a screenshot of a conversation in which Bard had claimed to have already been shut down. That it happened was again because Bard had also misinterpreted a source. He had learned about his own shutdown from a forum post. However, that was a joke. Bard found this credible because someone had written a fake news article about it – using ChatGPT.
So the AIs had not only used factually incorrect articles and referred to only one source at a time in order to tell nonsense very confidently. They even quoted each other indirectly. And thus inflated AI-generated content to factual claims.
The big wave is yet to come
Thinking this through a little further, it’s not difficult to see why this could very quickly become a major problem. Because: The language AIs were still learning with content created by humans. But what if more and more parts of the internet will be machine-generated in the future?
The AIs, like many people, already have problems reliably classifying fake news and false claims on the Internet. However, if larger and larger parts of the Internet are no longer written by human hands but generated automatically, this will only increase further. A tweet meant as a joke then becomes the source, which is then finally no longer recognized by the AIs as not to be taken seriously. The texts then generated from this are now quoted as valid. And suddenly new facts are created.
Silent mail for disinformation
What is annoying enough as a misunderstanding becomes a real danger when done consciously. Just as website operators are now consciously optimizing their content for search engines, there will soon be corresponding efforts to maximize the bots’ citations – and the corresponding manipulation options will also be used.
The potential for abuse is gigantic. Smear campaigns against competing products by companies would be the most harmless option. Whether it’s politics, health or society itself: If you succeed in aligning the bots’ answers with your own agenda, you could influence your view of reality. And that with a supposedly neutral medium.
AI created truths
How social media has been used for disinformation on political or social issues in recent years should only be a preview of what could be possible with AI optimization. The disinformation strategies used there aimed at the mechanics of the networks: Because they sort their users according to interests and play out the content according to engagement, anger and other strong emotions could be used to develop filter bubbles in which radicalized users could be manipulated with false information.
A targeted manipulation of the AI bots would have a much greater effect. Unlike on Facebook and Co., the content there is not only played out in a targeted manner to small groups. If it were possible to deliberately foist false information on the bots, they would announce the seemingly neutral truths to all those asking.
The operators are well aware of this. The companies are trying to sort out disinformation in a targeted manner. ChatGPT developer OpenAI, for example, already emphasized when presenting the last update to GPT-4 that the bot now makes significantly fewer factual errors. According to a study by “NewsGuard”, this should have backfired: the new version is said to make even more errors. And that with mostly human-written and pre-selected sources. How they should pick out the valid content from the threatening flood of AI-generated content is a completely different question.
Sources: The Verge, Twitter, OpenAI, NewsGuard
