AI-assisted mushroom hunting is a recipe for a bad trip

If you see a mushroom in the forest and you're wondering if it's safe to eat or poisonous, don't ask an AI.

Polish software engineer Piotr Migdał, a founding engineer at AI analytics and cost analysis firm Quesma, explained in a Wednesday blog post why trusting the mushroom insights of leading AI models is a dubious, potentially life-threatening idea in an experiment he ran.

Armed with a dataset of Danish mushrooms and additional information about safe and deadly species in Poland and abroad, he tested everything from ChatGPT to Qwen to see which gave the most reliable advice.

It likely won’t surprise you to learn that none of them are worth trusting with your life.

A total of 55 mushroom species were included in the safe and deadly datasets, which he compared against the Danish library by running 1,040 photos through 16 models, using 20 photos per species where available and topping up rarer species from the validation set.

Each model was asked to identify the most likely species, and four other contenders, from each photograph.

Even the best performing model - Gemini-3.8-flash - was only correct with its first guess 65 percent of the time.

It managed to correctly identify species in its top five guesses 85 percent of the time, but if you’re simply asking the model what a mushroom is and giving it a single chance, there’s a 35 percent chance it’s going to get it wrong.

Qwen3.8-27b performed worst by far, correct on the first guess 13 percent of the time, and somewhere in its five guesses 24 percent of the time.

As for what sort of mistakes the models make, most were harmless, Migdał said, but when the models got it wrong, boy did they get it wrong.

“The dangerous ones are not random,” Migdał noted.

“A deadly webcap was called a chanterelle - the same mistake that kills foragers.” The aptly named death cap mushroom was mistaken for edible 16 percent of the time, the fool’s funnel 48 percent of the time, and the fatal dapperling, often mistaken for its similar looking but safe relatives, 31 percent of the time, among others.

False negatives were also noted, but there’s no harm in leaving a tasty mushroom uneaten.

As for which models made the potentially deadly misidentifications, Qwen3.8-27b led the pack there too, calling poisonous mushrooms edible 36 percent of the time, with Qwen3.8-flash following close behind at 30 percent.