Privacy and Your AI Model Provider

Consumer AI plans may use conversations for model training, depending on the service and your settings. Business offerings from the major providers generally exclude customer content from training by default. A paid personal subscription can still have consumer terms, so price al

Every week, we're asked some version of the same question: “Is OpenAI, Claude, or Gemini safe for my data?”

Usually, the concerns are around who can see that data, whether it will be used for training, and how long it will be stored. Those are separate questions, and the answer is at least partially determined by what part of the world you're in.

Are they training on your data?

Check the terms and the actual privacy settings for your account; training defaults differ across providers. Also check feedback policies. Submitting a rating or report can share the associated conversation for review or improvement, even when ordinary chats are excluded from training. Disable optional data sharing where appropriate. [1–3]

What can your organization see?

Depending on the product, configuration and regulatory jurisdiction, authorized administrators or compliance staff may be able to retrieve work-account conversations and activity records. Tell employees what the organization can access and how that information will be used.

Separately, a no-training commitment doesn't necessarily prohibit access by the provider’s authorized personnel. Review its access policies as well as its training policy. [1]

“No training” does not mean “no retention”

A provider may store conversations, uploaded files, or records used to investigate abuse without using them to train a model. Retention varies by product and feature, and deleting a chat may not immediately remove every stored copy. [1, 4]

Enabling consumer model training can also mean longer retention. For example, Anthropic says eligible consumer chats may be retained in de-identified form in training pipelines for up to five years. Turning training off excludes chats from future training, but does not undo training already underway or completed, or automatically delete saved chats within 30 days. [8]

Zero Data Retention (ZDR) is generally offered through developer APIs and specific enterprise arrangements, rather than as a standard consumer chat setting. It limits retention of covered inputs and outputs, subject to the agreement’s scope and exceptions. A paid or enterprise subscription alone does not establish ZDR coverage. [5, 9]

ZDR doesn't turn off automated safety checks. How those checks operate and what information they retain depend on the provider and arrangement. Confirm eligibility, activation, and coverage in writing, including the models and features you use. Provider ZDR does not automatically cover copies held by your application, a coding tool, a gateway, or another connected service. [5, 9]

A BAA covers specific services (US Only)

In the United States, a Business Associate Agreement (BAA) is a contract required under HIPAA when a service provider handles protected health information (PHI) on behalf of a covered entity or another business associate. It establishes obligations for safeguarding the information and limiting its use and disclosure. [6]

For AI services, confirm that the agreement covers the product and features you actually use. Signing a BAA does not make every feature or workflow HIPAA compliant; appropriate configuration, access controls, and staff practices are still necessary.

Connected services need appropriate coverage too. A BAA with the model provider does not automatically cover an independently chosen coding platform or gateway handling the same information. Required agreements must follow the relevant business-associate and subcontractor relationships. [5, 6]

[Now do GDPR for EU, PIPEDA in Canada, DPDPA in India, Privacy Act in Australia, etc.]

Reduce the data you share

Send only what the task needs. Removing names and email addresses may not be enough: a narrative can still identify someone, and pricing, source code, strategy, and contract terms can be confidential without containing personal identifiers.

There are three places to limit what reaches the model:

At the source. Give the AI connection access only to the records and fields it needs, and verify those restrictions across the operations it exposes. Read-only access prevents changes; it does not prevent disclosure.
In the tool. Filter sensitive information before it is sent to the model. Verify that the filter covers returned data as well as prompts, and test what happens if filtering fails.
In a library or gateway. Redaction software can remove or replace identifiers before transmission. Test its limits, protect any replacement mappings, and check what the gateway itself stores.

Redaction can miss identifying details. Removing obvious identifiers alone does not establish that health information meets HIPAA’s de-identification requirements. [7]

Send the structure, not the data

Sometimes the AI does not need access to your client data at all.

Instead of sending client data to an AI provider, send the schema and a description of the task. The LLM can write a script that runs locally against your data, so the client data never leaves your database.

For example, rather than sending customer records to an AI provider and asking it to analyze or transform them, you can describe the relevant tables and fields and ask the model to generate code that performs the operation inside your own database or application.

The model helps create the logic, but the actual client data remains in your environment.

This approach will not work for every AI task—particularly when the model itself must interpret unstructured content—but when the job can be expressed as deterministic code, a query, or a calculation, it can substantially reduce unnecessary disclosure to an AI provider.

Use test data when real data is unnecessary

For development and demonstrations, use a separate synthetic dataset whenever it can meet the task’s needs. Generate it from approved schemas and fictional requirements. Do not send real client records to a provider simply to ask for a synthetic version.

Validate the generated data against the relationships, constraints, and edge cases you need to test. If the task requires real data but your client agreement prohibits sending it, resolve that restriction by redacting or removing before proceeding.

Use local models so the data never leaves your control

If your tasks can be served by local models, another option is to stand up infrastructure for local model use. You then send your data to a local and private model. Problem solved.

You can also access frontier models through enterprise cloud services from AWS, Microsoft, and Google, with contractual privacy protections and security controls. Check the terms for the specific service, model, and features you use—including training use, retention, abuse monitoring, and data location. Zero data retention may be available, but it is not automatic or universal.

Being Mindful of Your Client Data

Confirm that the task needs the data and that you have permission to send it.
Check the account’s training, feedback, and access policies.
Check retention terms and any required ZDR coverage.
Confirm required contractual coverage, including a BAA where applicable.
Minimize or elimiinate what you send, using test data where possible.

Policies and feature coverage change. Check the provider’s current documentation and your agreement before relying on a particular protection.

Apply the same scrutiny to every provider

If AI has prompted you to look more closely at data privacy, extend that review to your cloud hosting providers and every other API or service that handles client information. Your databases, file storage, CRM, and integrations deserve the same questions: what data do they receive, who can access it, how may they use it—including for AI training—and how long do they retain it?

Check the actual service, account settings, and contract. Confirm deletion practices, applicable contractual protections, and what information you can avoid sending in the first place. Make this part of choosing and reviewing all your providers, not just the ones selling AI.