Instinct’s powerful AI assistant is raising privacy and security concerns

Everyone is buzzing about Instinct , an AI personal assistant still in private access, not only for its incredible capabilities, but also for its potential privacy concerns. The agent, a veritable taskmaster, has been praised as feeling “like magic” and being one of the “most exciting launches” since OpenClaw. However, some testers have also raised concerns about the AI agent’s security model and its worrisome terms of service .

To be fair, Instinct is still in private testing for now, so these concerns haven’t yet scaled to the wider public at this time.

Created by a small team led by former Sierra research scientist Noah Shinn , San Francisco-based Instinct is operated by Spear Street Technology, per its terms and California business filings . It’s currently operating in stealth, per PitchBook .

Though Instinct is described by testers as outperforming their expectations, some have also raised concerns about the company’s approach to customer privacy and security. This raises a timely question: are the trade-offs of giving AI this level of access and autonomy actually worth it?

For instance, several people are circulating screenshots from the company’s Terms of Service , which grant Instinct a broad “perpetual and irrevocable” license to “access, use, host, cache, store, reproduce, transmit, display, publish, distribute, and modify” any of the user’s materials, including for training its AI models. The terms also detail how Instinct can receive information from users’ devices, including screen captures, cursor movements, and keyboard inputs.

The terms also allow Instinct to enter into “agreements, commitments, or transactions” on users’ behalf, which would be binding.

One early adopter, Peter Yang, pointed out that Instinct would not delete his Gmail records when asked. (The team later fixed the problem by adding a tool for deleting external data in its settings, he said.)

Another person, Claire Vo, found that Instinct was still summarizing their inbox after disconnecting its access . When she asked Instinct what happened, the bot confirmed that the emails were stored in plain text for later searches.

Many others questioned the security model, too. One tester was a bit worried when they found that Instinct was able to pull a sign-up code from their email inbox to complete a particular task — in their case, booking a table at a restaurant via Resy. And Hello Patient co-founder Alex Cohen wrote that once he found out how easily Instinct could be phished , he deleted his account:

In addition, Moxxie Ventures founder Katie Jacobs Stanton shared that Instinct broke her trust when it sent an email on her behalf without first checking with her.

“We’re trading privacy and control for hyper-personalized AI tools (AI notetakers, personalized AI agents, etc), often without fully understanding the trade,” she remarked on X, summarizing the dilemma posed personal AI agents. “The more powerful these agents become, the more trust matters. Every successful action earns a little more trust.