ALMOST INTELLIGENT : Don’t let a chatbot’s confidence trick you into taking financial risks

AI tools can be a valuable and free educator, but you need to know where to draw the line and rather trust human experts.

Consider the following scenario. Suzy is 63, recently retired and trying to decide when to start receiving social security and how to manage her retirement savings to minimise the tax hit. She opens an AI chatbot, types in the details and gets a calm, well-organised and confident answer: claim now, convert this much, here is the reasoning.

The chatbot sounds authoritative and even shows its work. So Suzy follows its guidance and never calls a financial planner. Maybe the advice was fine. But maybe it quietly ignored Suzy’s spouse being younger than her and in poor health, which can flip the social security maths. It also may have overlooked that the retirement savings plan conversion it suggested would push Suzy into paying higher medical aid premiums two years later.

Suzy will not find out for a long time, if ever, whether this guidance was right for her. And the AI will never call back to say it was unsure.

Suzy isn’t an exception. AI chatbots have entered everyday life with remarkable speed: a 2025 Pew Research Center survey found that 34% of American adults and 58% of those under 30 have used ChatGPT, roughly double the share two years earlier.

A growing number are asking AI about money, and some are getting burned. According to a 2025 survey of 2,000 US adults by Pearl.com, a professional services platform, 19% said they lost more than $100 by following financial advice from an AI chatbot. Among Gen Z investors, that figure rose to 27%.

These aren’t hypothetical risks. People are already paying for answers about their money that are confident – and wrong.

As a finance professor who has been closely watching the spread of AI into personal finance, this is the part of the AI story that worries me most. And it’s not the part you usually hear about.

There are two seemingly opposite complaints about AI. One is that people trust it too much, treating a chatbot like an oracle, a tendency researchers call algorithm appreciation. The other is that people don’t trust it enough and dismiss its useful tools, a tendency known as algorithm aversion.

I argue that these are actually two sides of the same coin, and what decides which side you see is whether you can tell when the AI is wrong.

When an AI fails in an obvious way, you notice and lose confidence. So you’re more likely to seek a professional or another human you trust sooner than you otherwise would. That is the safe failure.

The dangerous failure is the opposite. The answer is fluent, confident – and wrong. You have no way to catch it, so you keep managing the problem yourself long past when you should have asked for help.

The trouble is that with money, the second kind of failure is the common kind.

Three things make financial advice especially treacherous for AI. First, fluency is not accuracy. People naturally read a confident and well-articulated answer as competent.