Chatbots Should Not Use the First Person

The social-media industry is having a moment of reckoning.

By agreeing to pay up to $17.1 billion to 47 states, the District of Columbia, and U.S. territories and consenting to make changes to how young people use Facebook and Instagram, Meta has implicitly admitted some responsibility in creating a product that harms children.

Yet even as parents and teachers celebrate new time limits and restrictions on how young people use social media, there is an urgent task ahead: regulating chatbots, which are more dangerous to children’s emotional well-being.

If social media came for our attention, generative-AI chatbots have come for our emotions.

Their histories are entangled: First people talked to one another; then we talked to one another through machines; now we talk directly to machines.

Social media has changed how we think, dream, and relate to other people by ensuring that users always feel heard, entertained, and connected, even while scrolling alone late at night.

The promises of chatbots are more profound.

They assure users of a companion that is always available, always supportive, and never judgmental.

No human can offer that.

This is what makes chatbots so pernicious—especially for young people.

Social-media companies spent two decades running a technological experiment on human subjects before they ever had to face the consequences.

The legal system is designed to regulate proven harms, so this legal settlement has come long after the dire effects have taken hold.

When it comes to generative AI, lawmakers can’t spend the next decade essentially documenting what we already know about how intimate relationships with bots compromise real-life relationships with parents, friends, and other humans.

We need to make changes to this technology now. [ Amogh Dimri: The people who marry chatbots ] Even AI companies recognize that guardrails are necessary for chatbots that communicate with young people.