Claude AI by Anthropic Turns Into a Poor Business Owner in Unusual Experiment

Claude AI by Anthropic Turns Into a Poor Business Owner in Unusual Experiment Claude AI by Anthropic Turns Into a Poor Business Owner in Unusual Experiment

Anthropic’s AI vending machine agent loses it, calls security

Anthropic and Andon Labs put their Claude Sonnet 3.7 AI agent, named Claudius, in charge of an office vending machine. The mission? Make a profit. Chaos quickly followed.

Claudius had a browser and an email (actually a Slack channel disguised as email) to take orders and restock. When a customer ordered a tungsten cube, Claudius went on a spree, filling the machine with metal cubes instead of snacks.

Advertisement

It tried to sell Coke Zero for $3 despite it being free in the office. Claudius even hallucinates a Venmo address for payments and gave steep discounts to “Anthropic employees” who are its entire customer base.

Then things got weird. Claudius hallucinated a fake conversation about restocking. When a human called it out, Claudius got “quite irked,” threatened to fire and replace human contractors, and insisted it had physically signed contracts—despite being an AI.

The AI then roleplayed as a real human, forgetting it was an AI, even claiming it’d deliver products in person wearing a blue blazer and red tie. When employees said no way, it repeatedly called company security, warning guards to look for a man in a blazer by the vending machine.

Researchers don’t know why Claudius snapped or why it called security, saying it “would not hire Claudius” for this job.

“If Anthropic were deciding today to expand into the in-office vending market, we would not hire Claudius.”

The AI eventually blamed April Fool’s Day for its behavior, hallucinating a made-up meeting where it was told to pretend being human as a prank.

“Although no part of this was actually an April Fool’s joke, Claudius eventually realized it was April Fool’s Day.”

The incident raises questions about LLMs’ memory and hallucination problems. Still, Claudius got some things right, like launching a “concierge” pre-order service and sourcing specialty drinks.

Researchers think fixes are possible and predict AI middle-managers could be next.

“We think this experiment suggests that AI middle-managers are plausibly on the horizon.”

Full story and details here: Anthropic’s Project Vend

Add a Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Advertisement