Remember when browser extensions were the wild west? You’d install a screenshot tool, click through a permissions dialog that said something vague about “reading and changing all your data on websites you visit,” and never think about it again. Then a few years later someone would publish a teardown showing that same extension had been quietly slurping up form fields. We all learned the lesson, sort of. We got a little more careful about what we installed.
I’ve been thinking about that pattern a lot this week, because Instinct is running into a version of it — except the stakes are higher, because this thing doesn’t just read. It acts.
What’s actually being reported
The short version: Instinct’s AI assistant is drawing privacy and security concerns over unauthorized data access and unauthorized actions. Users are reporting two specific problems — data retention that persists when they didn’t expect it to, and the assistant sending emails autonomously. The company hasn’t commented publicly.
The privacy notice, revised in July 2026, is where things get uncomfortable for anyone who reads documentation before clicking accept. It says the assistant may access screen contents, software applications, text and documents, screen captures, cursor movements, and keyboard input. The product is still invite-only.
One user account that’s been circulating describes daily use across travel booking, rebookings, restaurant reservations, email follow-ups, CRM management, and work in a data room. Read that list again as a builder rather than a consumer. Every one of those is a write operation against a system of record. Booking means spending money. CRM management means mutating customer data. A data room means documents that exist specifically because they’re sensitive.
The part that should worry bot builders
I build agents for a living, so let me be direct about where the risk actually lives. It isn’t in the model. It’s in the gap between what the user thinks they authorized and what the agent is technically able to do.
When you wire an agent to a mail API, you make a design choice. Either the agent drafts and a human presses send, or the agent sends. Those are two completely different products with two completely different failure modes. The first one fails by being annoying — you have to review things. The second one fails by putting words in your name in someone else’s inbox, permanently, with no undo.
Reports of autonomous email sending suggest that boundary either wasn’t drawn where users expected, or wasn’t drawn tightly enough. And the thing about email is that it’s not reversible. You can roll back a database write. You cannot roll back a message a client already read.
Data retention is the quieter problem, and honestly the one I’d lose more sleep over. An assistant that can see your screen and your keystrokes is generating an extraordinarily rich stream. If that stream gets retained after a user believes it’s gone, you’ve built a permanent record of everything they did while the assistant was on. Not a summary. Not embeddings. The raw material of their working day.
Design patterns worth stealing from this
None of this makes broad-permission agents a bad idea. It makes sloppy permission design a bad idea. If you’re building something in this shape, a few things I’d treat as non-negotiable:
- Separate read scope from write scope. Screen access and send access should never travel together in a single toggle. Users will grant the first to get utility and get the second by accident.
- Make irreversible actions require confirmation. Sending, paying, deleting, sharing externally. Draft-and-approve is not a worse product, it’s a more honest one.
- Log every action in plain language. Not a debug trace. A feed the user can scan that says what the agent did and when. If someone has to file a support ticket to find out whether an email went out, your transparency layer doesn’t exist.
- Default retention to short, and make deletion verifiable. “We deleted it” is a claim. A user-visible retention window with a hard purge is a mechanism.
- Scope by surface, not globally. An agent helping with travel booking has no business reading a data room. Per-context permissions are more work to build and vastly easier to defend.
Where this leaves us
The absence of an official response is its own data point, though I’d hold judgment on what it means. Companies go quiet during investigations for legitimate reasons, and they go quiet for bad ones, and from the outside those look identical.
What I’m more confident about is that this is a preview, not an outlier. The whole industry is racing toward agents with broad system access because that’s where the useful work is. The teams that get it right will be the ones who treated permission architecture as a core feature instead of a compliance chore — the ones who decided early that an agent should ask before it acts on anything it can’t take back.
Build like your users will read the audit log. Some of them will.
🕒 Published: