It is almost always the first question we get, sometimes before we even talk about the project: “if we send our contracts, customer records or internal procedures to an AI model, will that data be used to train it? Will it end up with someone else?” It is a healthy question. The answer is rarely a simple yes or no: it depends on the tool, the contract behind it, and how the integration was built.
This article walks through what actually happens to data sent to an LLM, the differences between offerings that really matter, and the habits we apply on every project so our clients stay in control of their information.
The journey of a piece of data, step by step
When a document is sent to a language model hosted by a provider, it does not “vanish” into a black box. It follows a fairly standard path, and each step raises its own security question:
- Transit: the request travels from your application to the provider’s servers, normally encrypted (TLS). This is rarely where the risk lies.
- Processing: the model reads the content to generate a response. At this point the data is decrypted and processed on the provider’s infrastructure, in a given geographic region.
- Retention: depending on the offering, the request and response may be kept for some time, typically for abuse monitoring, sometimes longer for other purposes.
- Reuse: this is the real fear, that your content is used to improve or train the provider’s future models.
The last two steps are where offerings differ radically, and where most of the confidentiality question is decided.
Consumer apps and APIs: two different worlds
The most common confusion comes from lumping together the chat app an employee uses with a personal account and the API access a company builds into its tools. They are two different products, with different terms.
On free or individual consumer plans, conversations may, depending on settings and providers, be used to improve models, and are often kept for a long time. Control then depends on a setting the user has to remember to switch off, which is no guarantee at company level.
On business plans and API access from the major providers, the default is now the opposite: data sent is not used to train models, and retention is limited, often to around thirty days for abuse monitoring. Some providers even offer zero retention under certain conditions. These commitments are written into a contract, and that makes all the difference.
The question is not “do we use OpenAI?”, but “under which contract, with which settings, and for which data?”.
The real risk is not always where you expect it
In the audits we run, the main risk is rarely the official API integration, which is usually well governed. It is far more often in uncontrolled usage: a salesperson pasting a client contract into a chat app on a personal account, a browser extension that promises to summarize your emails, a free “AI-powered” tool whose terms nobody has read.
Another frequent blind spot is technical logs. An application that is well integrated on the provider side may still record every prompt and response in its own logs, in a third-party monitoring tool, or in a test database. The sensitive data is then no longer at the model provider, but scattered across your own infrastructure, often with no defined retention period.
Where is your data physically?
For a European company, where processing happens matters, for both regulatory and trust reasons. Several options exist today:
- Major US providers with European hosting: some offerings, especially through the large clouds, let you process and store data in EU regions. This is real progress, but the provider remains subject to US law, which can be a concern for the most sensitive data.
- European providers: companies such as Mistral offer capable models hosted in Europe, under European law.
- Self-hosted models: open models can run on your own infrastructure or a private cloud. No data leaves, but operations, security and performance become your responsibility.
There is no universal right answer. The right choice depends on how sensitive the data is, the level of performance you need, and your ability to run infrastructure.
The habits we apply on every project
As a team that works remotely on our clients’ data, confidentiality is not optional for us: it is what trust is built on. Here are the principles we apply systematically.
- Classify data before choosing the tool: public, internal, confidential, personal. The choice of model and hosting follows from this classification, never the other way round.
- Use only business access under contract: API or enterprise plans, with no reuse of data for training and a signed data processing agreement (DPA).
- Minimize what is sent: the model only needs the relevant passage, not the whole document. The less you send, the less you expose.
- Pseudonymize where possible: replace names, emails or numbers with identifiers before sending, then reinsert them into the response.
- Control the logs: define what is recorded, where, and for how long, including in monitoring tools.
- Review terms regularly: provider policies change. A setting that was valid a year ago needs to be checked again.
A concrete example
Take an internal assistant that answers staff questions from the HR knowledge base of a 200-person company. The most sensitive documents contain personal information about employees. A naive approach would send all relevant documents to a consumer model. A controlled approach separates the content: general procedures go through a contracted API hosted in Europe, while personal data is excluded from the index or pseudonymized upstream. Logs keep the question and the source used, but not the full content of answers, and are purged automatically after 30 days.
The extra cost of this approach is small compared to the project, a few more days of design. But it completely changes the conversation with the DPO, management and employee representatives.
Our recommendation
Sending data to an LLM is neither dangerous nor harmless by nature: it is an architecture decision, to be made with full knowledge of the facts. Before any project, ask three simple questions: what data will actually be sent, under which contract, and where it will be stored. If any of these three answers is unclear, that is where to start, before writing a single line of code.