OpenAI Contractors Read ChatGPT Conversations Under Project Lily

SAN FRANCISCO — OpenAI’s internal evaluation initiative, known as “Project Lily,” uses hundreds of third-party contractors to review consumer conversations on ChatGPT, according to internal documentation and leaked prompts obtained by digital investigative outlet *404 Media*.

The contractors read, summarize, and score full conversational threads initiated by consumer users. Their work forms part of Reinforcement Learning from Human Feedback (RLHF), a standard technique for aligning large language models with human preferences. Technology companies across the artificial intelligence industry frequently use third-party data annotation vendors to manage networks of human reviewers who rank model outputs.

For each assigned conversation, a Project Lily contractor first reads the user’s original prompt and writes a concise summary of what the user was requesting. After the system generates potential outputs, the reviewer examines four distinct responses produced for that prompt.

Each response must receive at least three specific text selections, with the contractor explaining why the selected segments are effective or problematic. Internal training materials identify tone and formatting violations, including excessive emoji use that is considered “misaligned” when it does not fit the context of the prompt.

Contractors also assign every response a numerical score from 1 to 7. A score of 1 means the output is unusable, while 7 indicates that it requires virtually no adjustment. OpenAI’s internal instructions say ChatGPT responses should sound natural and should not claim artificial personal experiences or emotions.

The disclosure has renewed attention in San Francisco and elsewhere to the way generative artificial intelligence developers handle consumer data while training and refining language models. It also highlights differences in user disclosure among companies. Google, which operates Gemini, and Anthropic, the developer of Claude, state in public privacy documentation that human reviewers may access and analyze sampled, anonymized chat logs.

OpenAI’s primary privacy policy said personal data could be used to improve system performance, but did not explicitly tell consumer users that human contractors might read full conversational logs. OpenAI said in response to the report that conversations pass through an automated privacy filtering process before being sent to contract reviewers.

That filter is intended to identify and sanitize personally identifiable information, including full names, contact details, and identification numbers. OpenAI acknowledges that automated filtering systems can sometimes fail to detect sensitive information embedded in unstructured text. Prompts containing medical details, legal matters, or financial records can therefore remain intact when delivered to external human reviewers.

The data-management practices come amid regulatory and technical scrutiny of OpenAI. In March 2023, Italy’s Data Protection Authority, the *Garante per la protezione dei dati personali*, temporarily suspended ChatGPT in Italy over concerns about compliance with the European Union’s General Data Protection Regulation (GDPR). The authority specifically cited the lack of a legal basis for processing user data to train algorithms.

OpenAI later resumed service in Italy after adding age-verification controls and forms that allow European users to object to data processing under GDPR rules. That same month, the company fixed a bug in an open-source Redis client library that briefly exposed user chat histories, billing details, and the initial messages of active conversations to other logged-in users.

After the Redis incident, OpenAI introduced new data controls in April 2023. The controls allow users to disable conversation history and prevent their data from being used for model training. In July 2023, the U.S. Federal Trade Commission opened a broad inquiry into OpenAI and issued a Civil Investigative Demand seeking detailed documentation about data security practices, consumer privacy protections, and methods for preventing exposure of personal information.

OpenAI introduced ChatGPT Enterprise in August 2023 to address institutional compliance standards, followed by ChatGPT Team. Both products automatically exclude workspace conversations and enterprise data from model training pipelines by default. Standard consumer accounts, including free tiers and individual ChatGPT Plus subscriptions, remain opted into data training and reviewer pipelines unless users manually change the setting.

Users can prevent future interactions from being processed for model training or reviewed by human contractors through the ChatGPT mobile application or web interface. They select the profile icon in the lower-left corner, open “Settings,” and then go to “Data Controls.” There, the “Improve the model for everyone” toggle can be switched off.

OpenAI notes that the change applies only to future conversations started after the setting is disabled. It does not retroactively remove conversations that were previously processed or ingested into training pipelines before the setting was changed.