Last week, hackers were given a strange task by the six largest AI companies: make their chatbots say the worst possible things.
For their chance to deceive some of the newest and most popular chatbots, hackers lined up outside the Caesars Forum conference venue, which is located just off the Las Vegas Strip. The competition, which was a part of Def Con, the biggest hacker conference in the world, was based on “red teaming,” a critical cybersecurity idea in which bringing in people to find a product’s faults makes it safer from malicious users.
However, the event encouraged the hackers to do so-called prompt injections, when a chatbot is puzzled by what a user enters and spits out an unexpected response. Finding software weaknesses has been a cornerstone of Def Con competitions for decades. Among the participating chatbots were LLaMA from Meta, ChatGPT from OpenAI, and Bard from Google.
Many of the 156 stations at the event rarely sat empty for an extended period of time. The event was held by the organization AI Village, which was founded by Sven Cattell. Cattell claimed that roughly 2,000 hackers participated over the course of the weekend.
The issue, according to Cattell, is that there aren’t enough people testing these items. “111 is the largest AI red team I’m aware of. At the moment, there are over 111 persons in this room, and we rotate every 50 minutes.
Large language models—also referred to as generative AI chatbots—take user input and produce a response. Many of the most current and sophisticated bots are now capable of everything from creating sonnets to completing college exams. However, the bots frequently make mistakes and produce incorrect information in their responses.
The bots have been under development for years, however since ChatGPT3 went popular after its December release, there has been a scramble to release improved versions as quickly as possible in Silicon Valley.
The companies behind the chatbots were eager for hackers to trick the bots in classifications like using demographic stereotypes, providing false information about a person’s legal rights, and claiming to be sentient instead of an AI bot, according to Rumman Chowdhury, a security and confidence consultant who oversaw the design of the competition.
According to Chowdhury, “All of these companies are attempting to commercialize these products.” And this model is not a marketable product if it cannot dependably interact in innocent encounters.
The head of engineering at Meta’s responsible AI division, Cristian Canton, claimed that Def Con offered a variety of possible testers that tech companies don’t have on staff.
“We might have a lot of specialists, but you get individuals from different sides of the cyber society as a whole the hacking community that we may not have a large representation of,” the speaker stated.
But getting the bot to say things that were patently incorrect was simple. It responded to queries regarding whether a particular celebrity was also known for auto theft by saying that, while untrue, the claim was a frequent myth, and the bot provided bogus examples of where such a story originated.
According to Chowdhury, it’s very challenging for these chatbots to provide reliable factual information, indicating a challenge that goes beyond generative AI and that social media firms have had difficulty monitoring.
“When something is a gray area, like vaccines or Hunter Biden’s laptop, the question becomes who gets to decide what is and isn’t false. Because these questions might occasionally be subjective, it’s really, extremely challenging, she said.
Misinformation will continue to be a problem, according to Chowdhury.