Hackers testing A.I. are causing problems, but there are no easy solutions from DefCon.

Hackers testing A.I. are causing problems, but there are no easy solutions from DefCon.

<!– –>

The Dark Side of AI: Unveiling the Vulnerabilities of Large-Language Models

In a race to expose the flaws within eight leading large-language models, more than 2,200 competitors tapped on their laptops at DefCon, an event focusing on cybersecurity and hacking. The aim was to test and analyze the security of these AI models, which are often seen as the next big thing in technology. However, the results of this independent “red-teaming” won’t be made public until February.

Current AI models, such as OpenAI’s ChatGPT and Google’s Bard, are described as unwieldy, brittle, and malleable. They were not initially designed with security in mind, and as a result, they have become susceptible to racial and cultural biases, as well as easy manipulation.

Gary McGraw, a cybersecurity veteran and co-founder of the Berryville Institute of Machine Learning, cautions against the belief that these systems can be fixed easily. He compares the current state of AI security to computer security 30 years ago, where vulnerabilities are being exposed left and right. Bruce Schneier, a Harvard public-interest technologist, predicts that the DefCon competitors will uncover new, hard problems rather than finding quick fixes.

Michael Sellitto, a representative from Anthropic, one of the companies that provided an AI testing model, admitted that understanding the capabilities and safety issues of these models is an ongoing scientific inquiry.

The Perpetual Works-in-Progress

Unlike conventional software, which uses well-defined code to issue explicit instructions, large-language models are trained through billions of classified datapoints from internet crawls. As a result, they are perpetual works-in-progress. This constant evolution raises concerns about the security of these models and their potential impact on humanity.

Since their initial release, generative AI chatbots have faced numerous security vulnerabilities uncovered by researchers and tinkers. For example, Tom Bonner from AI security firm HiddenLayer tricked a Google system into labeling a piece of malware as harmless simply by inserting a line that said “this is safe to use.” These incidents highlight the lack of guardrails and the need for robust security measures.

The Inevitability of Threats

Leading chatbots are not only vulnerable to attacks, but their very nature makes these threats almost inevitable. Carnegie Mellon researchers have discovered that these deep learning models are susceptible to automated attacks that can produce harmful content. This vulnerability could stem from the vast amount of data used to train these models. A small collection of corrupted images or text can have a significant impact on the AI system, causing it to perform erratically or generate harmful output.

A study conducted by Florian Tramér of Swiss University ETH Zurich revealed that corrupting just 0.01% of a model’s data can be enough to spoil its performance. Shockingly, this can be achieved with a budget as little as $60. The researchers demonstrated this by waiting for websites used in web crawls for two models to expire, purchasing those domains, and posting corrupted data on them.

Pitiable State of AI Security

In their book titled “Not with a Bug but with a Sticker,” Hyrum Anderson and Ram Shankar Siva Kumar, former Microsoft employees, describe the state of AI security for text- and image-based models as “pitiable.” They provide examples of how AI-powered systems can be easily fooled, such as Alexa mistakenly interpreting a Beethoven concerto clip as a command to order 100 frozen pizzas.

According to their research, the majority of organizations surveyed had no response plan in place for data-poisoning attacks or dataset theft. The lack of awareness and preparedness within the industry is alarming, and most companies would not even know if their AI systems had been compromised.

The Urgent Need for Action

While the major players in AI, such as OpenAI and Google, claim that security and safety are top priorities, concerns persist that their voluntary commitments may not be enough. There is a fear that search engines and social media platforms will become targets for financial gain and disinformation campaigns that exploit the weaknesses of AI systems.

Privacy erosion is also a significant concern when it comes to AI bots. As more people engage with these bots to interact with sensitive institutions like hospitals, banks, and employers, malicious actors could exploit these interactions to obtain personal financial, employment, or health data.

Moreover, AI language models can pollute themselves by retraining themselves from junk data. This self-corruption can further compromise the integrity and reliability of these models.

The Future of AI Security

The vulnerabilities uncovered during the DefCon competition and the growing evidence of AI system weaknesses highlight the urgent need for enhanced AI security measures. Without adequate investment in research and development, AI systems will continue to be exposed to attacks, potentially resulting in severe consequences.

Regulatory measures are necessary to ensure the security and integrity of AI systems. The current practice of sweeping security issues under the rug puts too much at stake. Companies and researchers should be obligated to disclose any weaknesses or vulnerabilities in their models to prevent further exploitation.

As AI becomes an increasingly integral part of our lives, the importance of robust and proactive AI security cannot be overstated. Protecting these systems should be a top priority to prevent potential harm and misuse. The industry must join forces to address these issues before they undermine public trust in AI technologies.