Microsoft Copilot sometimes hands your intimate photos over to real people

Microsoft has humans rate the image edits produced by Copilot in order to train its model. Hundreds of contractors would thus compare the photo submitted by the user with the AI-generated version. The photos reach them unaltered, complete with faces, loved ones, and everything we thought was reserved for the machine.

Faces that are never blurred

The setup concerns Copilot’s image editing tool, the one that makes a portrait look younger, removes a passerby, or changes a background.Each reviewer receives the instruction typed by the user, the original image, and the AI’s proposal, then rates how faithfully the task was performed. This work falls under model training, a separate undertaking from the moderation of prohibited content, and recruitment is handled at least in part through Prolific, an online microtask platform.

Read also: Microsoft unveils its new Copilot, a “super app” for all its AIs

“Faces are never censored, and many requests are sexual in nature, with questionable consent”, one of these reviewers recounts. Another says they came across a batch of “eight photos taken under skirts”, which they reported to their management.

When questioned, the company denies nothing and refers to its terms of use: customer data are used “notably to improve our products and enforce our code of conduct”. No commitment to blurring or anonymization accompanies this response, nor does it specify whether users can object to such review or how long the images are retained. The scale of the operation would amount to at least hundreds of reviewers, a figure Microsoft does not confirm.

Seven years after Siri, the same controversy

The industry had already stumbled in this area in July 2019, when Apple subcontractors recounted listening to Siri recordings, including medical conversations and intimate moments. The company had suspended its program one week after the revelations. In France, the case is continuing before the courts: after a complaint filed by the Human Rights League in February 2025, the Paris prosecutor entrusted an investigation to the Anti-Cybercrime Office, a procedure revealed in October of the same year.

Copilot is not alone in this reliance on hidden labor. Two weeks before these revelations, a separate investigation described hundreds of subcontractors reading ChatGPT conversations, while Anthropic likewise acknowledged that it also used human review. Image editing adds another dimension: faces now join the text.

👉🏻 Follow tech news in real time: add 01net to your sources on Google, and subscribe to our WhatsApp channel.

Source: 404 Media