AI Errors and False Reports Drain Certain Security Bug Bounties

Pattern of retro tin toy robots with clocks and displays painted on yellow background. Pattern of retro tin toy robots with clocks and displays painted on yellow background.

Bug bounty programs are battling a surge of AI-generated fake vulnerability reports.

The issue started last year with an influx of "AI slop"—large language model-generated bug reports that look legit but describe vulnerabilities that don’t exist. These fake reports clutter bug bounty platforms and waste security teams’ time.

Vlad Ionescu, co-founder and CTO of RunSybil, explained to TechCrunch:

Advertisement

“People are receiving reports that sound reasonable, they look technically correct. And then you end up digging into them, trying to figure out, ‘oh no, where is this vulnerability?’”

“It turns out it was just a hallucination all along. The technical details were just made up by the LLM.”

“That’s the problem people are running into, is we’re getting a lot of stuff that looks like gold, but it’s actually just crap.”

The problem goes beyond theory. Security researcher Harry Sintonen revealed that Curl, a major open source project, got hit by such fake reports. Benjamin Piouffle of Open Collective also said their inbox is "flooded with AI garbage."

One open source developer maintaining CycloneDX pulled their bug bounty program entirely after getting almost exclusively AI slop reports.

The leading bug bounty platforms like HackerOne and Bugcrowd confirm the trend but differ on scale.

Michiel Prins from HackerOne told TechCrunch:

“We’ve also seen a rise in false positives — vulnerabilities that appear real but are generated by LLMs and lack real-world impact.”

“These low-signal submissions can create noise that undermines the efficiency of security programs.”

“Reports that contain hallucinated vulnerabilities, vague technical content, or other forms of low-effort noise are treated as spam.”

Bugcrowd’s Casey Ellis said:

“AI is widely used in most submissions, but it hasn’t yet caused a significant spike in low-quality ‘slop’ reports.”

“This’ll probably escalate in the future, but it’s not here yet.”

Ellis added that Bugcrowd manually reviews reports using playbooks and AI-assisted systems.

Mozilla reports no significant rise in AI-generated invalid bug reports, holding a steady 5-6 rejections monthly. Microsoft, Meta, and Google declined to comment.

To fight the flood, HackerOne launched Hai Triage this week — a hybrid AI-human triage system. It uses AI agents to flag duplicates, cut noise, and prioritize threats before human analysts validate the bugs.

The AI arms race in bug hunting is on, with hackers using LLMs to craft fake reports while companies lean on AI to sift through the noise. Which AI will win? Time will tell.

Add a Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Advertisement