CYBER WELFARE

Protect your Digital Privacy

Prompt injection attacks and other threats to your AI assistant’s chats, memory and connected apps

An artificial intelligence assistant feels like a private space: you write, it answers, nobody else is watching. For anyone who wants to take advantage of it, though, that space has a precise value. It holds months of conversations, a memory with details about you and, more and more often, access to your email, calendar or files through connected apps.

This post explains the most common techniques used to read, use or manipulate what an assistant keeps and can reach. Among them are prompt injection attacks: instructions hidden in a web page, a document or an email, which the assistant reads while working for you and may mistake for your own requests. The other techniques are more traditional, from account takeover to fake apps, but all of them become more effective when history, memory and connected apps stay as they were on day one.

It is the threat-side companion to the recommendation to check your AI assistant’s data settings. How signing in with a “Sign in with…” button can open the way to other services is a different topic, covered by the post on attacks through connected apps.

A useful clarification: this post explains how these abuses work from the point of view of the people on the receiving end, so they can be recognised and prevented. They are not operating instructions.

Starting scenario

Your assistant holds a year of history: work drafts, a house move, a trip, a letter to the managing agent of your building. The memory remembers your job, your town and the names of a couple of relatives. You connected your email so that it can draft replies, and a file storage service for a trial, months ago. You have shared a few conversations with a link.

None of these choices, on its own, is a problem. Together, though, they make up a rich archive and a tool able to act in your name. And anyone who manages to get into the account, or to get the assistant to read their instructions, finds everything ready.

When you connect an app for convenience, it is natural not to think about this. Anyone who wants to take advantage, on the other hand, counts on exactly that.

Why default settings increase the risk

All the techniques described here have one thing in common: they do not need to persuade you to hand over anything new; they use what the assistant already has or can already do.

The longer the history, the more there is to read. The richer the memory, the easier it is to build a believable message about you. The more apps are connected, and with more permissions, the more a wrong instruction can turn into a real action. The UK’s National Cyber Security Centre (NCSC), in a blog post of 8 December 2025, explains that current language models do not enforce a security boundary between instructions and data inside a prompt, and that it is very possible prompt injection may never be totally mitigated: the realistic goal is to reduce its likelihood and impact.

1. Account takeover and reading chats and memory

In plain terms. Someone gets into your assistant account and reads what it holds.

How it works. The routes are the same as for any account: a reused password that turned up in someone else’s data breach, a phishing message imitating the sign-in page, a device left unlocked. Once in, the intruder finds the history, the memory and the connected apps neatly organised, ready to browse.

Why the settings make it easier. A history with no time limit and a memory never reread turn one sign-in into a complete archive of your recent life.

Possible impacts. Personal and work data read by strangers; information used for targeted scams; altered memories that shape future answers.

Warning signs. Sign-in alerts from devices you do not recognise; conversations you do not remember starting; new lines in the memory.

How to protect yourself. Use a password you do not use anywhere else and turn on two-step verification. Choose a period for the history and reread the memory. On a shared device, sign out.

2. Hidden instructions in pages, documents and emails

In plain terms. Content you ask the assistant to read contains hidden sentences addressed to it, not to you.

How it works. This is the technique known as indirect prompt injection. When you ask it to summarise a page, analyse a document or reply to an email, the assistant reads that text together with your request. If the text contains instructions, perhaps written so as to be invisible to the eye, the model may treat them as requests. If the assistant is connected to your email or files, a hidden instruction may try to make it look for information, send it out or save a false memory. Providers are adding defences, but the NCSC warns that the problem has no complete fix.

Why the settings make it easier. An assistant that reads only what you type can at worst give a bad answer. One connected to email and files, with permission to send or edit, can do something concrete.

Possible impacts. Messages sent without your knowledge; data copied out of connected services; memory altered with false details.

Warning signs. An action on email, calendar or files you did not ask for; an answer that changes subject or nudges you towards a link; a new memory you do not recognise, appearing after the assistant read outside content.

How to protect yourself. Connect only the apps you use, with minimum permissions: read rather than send, when that is enough. When the assistant proposes an action, read it before confirming. Be more cautious about having it read content of unknown origin, especially when apps are connected.

3. Shared links made public

In plain terms. A conversation shared through a link ends up in front of people it was not meant for.

How it works. Many services let you create a link to a conversation to show it to someone. Anyone with the link can read it. If the link is forwarded, or posted in a group or on a forum, whoever finds it reads everything the conversation contained, including documents and personal details. Anyone gathering information for a scam knows these pages can hold a lot of it.

Why the settings make it easier. Links often stay live until you revoke them, even when you have forgotten creating them.

Possible impacts. Addresses, phone numbers and other people’s data exposed; scams built on real facts from your life.

Warning signs. Someone tells you they have read one of your conversations; you receive messages quoting details found only in a chat.

How to protect yourself. To show some text, copy the part you need rather than sharing the whole conversation. Review the list of active links, if the service shows one, and revoke the ones you do not need.

4. Fake apps and extensions that harvest chats

In plain terms. An app or a browser extension presents itself as useful for AI, but collects your conversations.

How it works. An extension is a small program that adds features to the browser. Some ask for permission to read every page you visit: with that permission they can read chat pages too. Other apps imitate the name or look of a well-known assistant to get installed and ask for access to your account. In both cases, conversations can flow out to whoever made the program.

Why the settings make it easier. The more conversations stay in the history, the more there is to collect when the pages are opened.

Possible impacts. Chats, documents and memories copied by third parties; account credentials stolen.

Warning signs. An extension asking for permission to read and change your data on every website you visit; an app with a name similar to a well-known service but from a different developer; a request for account access from a program you do not remember.

How to protect yourself. Install apps and extensions only from official sources and check the developer, as the recommendation on checking an app is genuine explains. Remove extensions you do not use.

5. Data leaks at the provider

In plain terms. A problem in the service’s systems makes visible data that should have stayed private.

How it works. Like any online service, an AI assistant can suffer a technical error or an intrusion. Italy’s data protection authority, the Garante, in a statement of 31 March 2023, reported that a widely used AI service had suffered a data breach on 20 March involving users’ conversations and subscribers’ payment information.

Why the settings make it easier. You cannot prevent an incident at the provider. You can, however, decide how much there is to expose: a short history and a minimal memory reduce what an incident can touch.

Possible impacts. Titles or contents of conversations visible to others; account data exposed.

Warning signs. A message from the service about a security incident; news from official sources.

How to protect yourself. Keep the history for as long as you need it, do not upload whole documents with identity details when an extract will do and, after an announced incident, change your password following the service’s instructions.

6. Connectors with too many permissions

In plain terms. A connected app can do more than you need, and that extra is available to anyone who manages to steer the assistant.

How it works. A connector is the link between the assistant and another service. If you grant read, edit and send when you only needed to read, every error, every hidden instruction and every unauthorised sign-in has all those permissions available. The NCSC cites a principle that is useful for users too: when a model processes content from someone else, its privileges should drop to those of that someone.

Why the settings make it easier. Connections stay active until you remove them, including those made for a trial.

Possible impacts. Emails sent, appointments changed, files shared or read without your knowledge; data ending up in the other service and following its rules.

Warning signs. Connected apps you do not remember; send or edit permissions you do not use; actions you did not ask for.

How to protect yourself. Review the list of connected apps, disconnect the ones you do not use and reduce the others’ permissions. The logic is the same as in the recommendation on checking what your apps can do on your phone.

Summary table

TechniqueMain riskWarning signsEffective defences
Account takeoverChats and memory read by strangersUnknown sign-ins, new memoriesUnique password, two-step verification, short history
Hidden instructions (prompt injection)Actions in your name, altered memoryActions you did not ask for, answers that change subjectMinimum permissions, confirming actions, caution with outside content
Shared links made publicConversations readable by anyoneContacts quoting your chatsCopy the text, revoke links
Fake apps and extensionsChats harvested by third partiesBroad permissions, unknown developersOfficial sources, unused extensions removed
Provider data leaksData exposed by an incidentMessages from the serviceShort history, trimmed documents, password changed
Over-permissioned connectorsMore power than neededForgotten apps, send permissionsDisconnect, reduce permissions

What they have in common

Six different techniques, three defences that work for almost all of them:

  • Less archive — a history that lasts as long as you choose and a memory you have reread reduce what anyone can read or use.
  • Less power — few connected apps, with minimum permissions, limit what a wrong instruction can do.
  • A protected account — a unique password and two-step verification keep out anyone who should not get in.

These are not advanced measures: they are settings you can check in a few minutes.

Protection checklist

  • ☐ The assistant account has a unique password and two-step verification
  • ☐ The history has a period I chose
  • ☐ I have reread the memory and deleted what I do not recognise
  • ☐ Connected apps are only those I use, with minimum permissions
  • ☐ I read the actions the assistant proposes before confirming them
  • ☐ I have no active shared links I do not need
  • ☐ My extensions and apps come only from official sources

How this connects to the Cyber Welfare Framework

PillarWhat this content contributes
AwarenessUnderstanding that an assistant reads everything you have it read, and that outside content may contain instructions
SkillsRecognising the techniques, from account takeover to prompt injection and over-permissioned connectors
Secure BehaviourKeeping little archive, little power and a protected account, and confirming actions only after reading them

Reference level: FL3 — Autonomous, with elements of FL2 — Beginner in the parts on history, memory and account protection.

Conclusion

The techniques described here do not require getting into your phone. They rely on what the assistant already has, an archive of conversations and a memory, and on what it can already do, through connected apps.

That is why the most effective defence is not giving up the assistant, but choosing: less archive, less power, a protected account. To find out whether your assistant already keeps more than you think, the next step is the signs your AI assistant remembers too much; for a broader view of your digital habits, you can start from the digital resilience self-assessment.

Something to think about. If a web page could ask your assistant to do something, what could it do today with the apps you have connected?

Related resources

Short reads from the Resources section, for anyone who wants to focus on a single aspect:

Related content

Start with the first step: the Cyber Welfare Programme guides you free of charge, one recommendation at a time.