OpenAI employs hundreds of contract workers who read real ChatGPT conversations and rate them on a scale of one to seven. The stated goal is to reduce overly agreeable responses and behavior that is too "human" in the model.
Chats are anonymized, but anonymization doesn't remove the risk: if a user has typed sensitive data, such as client names, contract numbers or sales strategies, that content can end up in the review queue. The setting that enables this collection is called "Improve the model for everyone" and it is on by default. To turn it off, you have to do it manually in your account settings.
If you use ChatGPT at work and haven't checked this setting, the risk isn't theoretical: it affects the confidentiality of the data you handle and, in some contexts, GDPR compliance. The first thing to do this week is check the setting on every active company account.
The setting almost nobody has turned off
OpenAI has hundreds of contract workers who read real ChatGPT conversations. This was reported by The Decoder, citing an investigation by 404 Media. These reviewers' job is to rate the model's responses on a scale of one to seven, with the aim of reducing ChatGPT's tendency to be too agreeable or to imitate human behavior in an artificial way.
What matters for businesses isn't the training method itself. It's that this collection happens by default, through an option called "Improve the model for everyone", and that turning it off requires a manual action from the user. If you've never looked for it, it's almost certainly still on.
Anonymized doesn't mean safe
OpenAI states that conversations are anonymized before being assigned to reviewers. But anonymization removes explicit identifiers, such as username and email address, not the content of the chats.
If a marketing manager used ChatGPT to draft a sales proposal with client names, prices and contract terms, those details remain in the text of the conversation. A human reviewer can read them. In a B2B context, this isn't an abstract problem: it's a potential breach of confidentiality obligations toward those very clients.
On the regulatory side, the GDPR, the European regulation on personal data protection, requires that data be processed on precise legal bases and with adequate safeguards. Entrusting third-party data to a tool that passes it to human reviewers without informing the people concerned is a gray area many companies haven't yet assessed. The issue is part of a broader picture of governance and privacy in the use of AI tools, which is becoming urgent for Italian SMEs too.
Who's really exposed and who can relax
Not all ChatGPT users are in the same situation. There are important differences between plans:
- Free accounts and personal ChatGPT Plus: the "Improve the model for everyone" setting is on by default. Your data may be used for training and human review.
- ChatGPT Team and Enterprise: OpenAI states that conversations are not used to train models by default. But "by default" doesn't mean "never": it's worth checking the terms of your contract.
- Direct API: if you use ChatGPT through the API without explicitly enabling data training, different conditions apply. Here too, you need to read the updated terms of service.
In short: if your company uses a free or Plus account for tasks that involve client data or confidential information, the risk is real and immediate.
Three actions to take this week
You don't need consultants or long projects. Three practical checks are enough:
- Check the setting on every active company account. In ChatGPT: Settings → Data controls → "Improve the model for everyone" → turn off.
- Map out who uses ChatGPT in your company and with what data. Use is often informal and spread out: sales, assistants, marketing. Each of them is a potential point of exposure.
- Assess whether your current plan is adequate. If company use is heavy and involves sensitive data, moving to an Enterprise plan with explicit contract terms is the safest route.
This kind of risk doesn't come from an outside attack, but from a setting that's on by default. It's the same pattern we've seen elsewhere, like Meta Muse Spark, which offered a 95% discount in exchange for sharing data for training. The mechanism is different, but the logic is identical: users give up data without realizing it, because the default option works silently.
The most underestimated risk: customer trust
Beyond formal compliance, there's a reputational risk that companies tend to ignore. If a client finds out that their data, even just a draft proposal or an email exchange reworked with ChatGPT, was read by human reviewers at a third-party company, the conversation that follows is hard to manage.
Human oversight of AI tools is already at the heart of the regulatory debate. Anyone dealing with API governance and the risks tied to AI agents knows that the exposure surface grows every time a new tool enters the workflow without a prior assessment. And it's not just about developers: it concerns anyone with a ChatGPT account open in their browser during a meeting.
The issue of OpenAI's transparency about its internal processes has come back into focus several times in recent months. Each time, the pattern is the same: the news comes from outside sources, not from proactive communication by the company. For anyone deciding which tools to adopt, this is a fact to weigh as much as the product's features.
Related articles
Discover more articles exploring similar topics, selected to offer you a more complete and stimulating perspective. Each piece of content is carefully chosen to enrich your experience.
