AI Hallucinations Pose New Threat to the Software Supply Chain

In a chilling new development for developers and organizations relying on artificial intelligence for coding, researchers have uncovered a novel supply chain threat created by AI hallucinations in code-generating models. This emerging attack vector, dubbed slopsquatting, could allow malicious actors to infiltrate software ecosystems by taking advantage of fictitious packages hallucinated by Large Language Models (LLMs).

The danger lies in the fact that LLMs often "hallucinate" packages—suggesting or inserting references to software dependencies that simply don't exist. In research conducted jointly by the University of Texas at San Antonio, the University of Oklahoma, and Virginia Tech, it was discovered that none of the 16 popular LLMs tested were immune to this phenomenon.

What is Slopsquatting and How Does It Work?

Slopsquatting takes this hallucination flaw and turns it into a weapon. When a LLM suggests a non-existent package during code generation, cybercriminals can quickly register a malicious package under that name. Once published, any developer who accepts the AI-generated code at face value may unknowingly import and run the malicious package. This not only compromises the individual project but could also propagate across entire software supply chains if the infected code is reused or shared.

The study found that out of 2.23 million packages generated across Python and JavaScript test scenarios, nearly 440,000—or about 19.7%—were hallucinated. Of these, a staggering 205,474 were unique fictitious package names. Most hallucinated packages—81%—were unique to the specific model that generated them, pointing to inconsistent behaviors across different LLMs.

Commercial AI models hallucinated packages in at least 5.2% of cases, while open-source models fared significantly worse with hallucination rates hitting 21.7%. Alarmingly, these errors weren’t just one-off flukes. Over half of the hallucinated packages (58%) appeared repeatedly within just 10 iterations, showing a clear tendency for persistence.

The Rising Risk of AI-Generated Code in Software Development

While previous studies have acknowledged the threat of typosquatting—where attackers exploit mistyped or misleading package names—this new slopsquatting attack represents an overlooked and potentially far more dangerous variant. Unlike typosquatting, which preys on human error, slopsquatting leverages the perceived authority and trustworthiness of AI-generated code.

Perhaps most fascinating—and equally concerning—the researchers found that LLMs were capable of recognizing many of their own hallucinations. This suggests an untapped self-regulatory potential that could be used in future safety mechanisms. It also hints at the possibility of deploying in-model detection tools to prevent the distribution of faulty or dangerous code.

How Developers can Protect against AI Package Hallucinations

To counter the threat, the researchers propose a range of mitigations. These include advanced prompt engineering techniques such as Retrieval Augmented Generation (RAG), prompt tuning, and self-refinement. On the model development side, strategies like supervised fine-tuning and improved decoding algorithms could help reduce hallucination rates.

As generative AI continues to transform software development, this study is a stark reminder that convenience can come at a cost. Developers must remain vigilant and critical of AI-generated code, especially when it comes to dependency management. Integrating static analysis tools and manual reviews before installing any recommended packages is now more important than ever.

The threat landscape is evolving rapidly, and as this research shows, so too must our defenses. What once seemed like science fiction—AI imagining software that doesn't exist—has become a very real cybersecurity concern with wide-reaching implications.