Big Tech companies intensify efforts to combat AI-generated false content in U.S. elections

OpenAI, Anthropic, Google, and Meta have strengthened their strategies to prevent artificial intelligence (AI) tools from being used to manipulate the United States legislative elections, which are being held this Tuesday (the 3rd). The companies have assembled specialized teams to monitor suspicious activity and attempt to contain disinformation campaigns capable of undermining confidence in the democratic process.

At OpenAI, more than a dozen employees work in an operations center set up at the company’s office in San Francisco (USA). The setup represents a significant expansion of the company’s efforts during the 2024 presidential election, when AI capabilities were less advanced and ChatGPT had only one-fifth of the billion-user base it currently has.

“In 2024, the most accessible technology was text.“Now, we are not monitoring just texts,” said Josh Lawson, OpenAI’s head of civic and strategic risks and leader of the operation, to The New York Times.

The current landscape includes threats ranging from deepfake videos and cyberattacks to foreign and domestic influence operations. With AI tools, this type of campaign can be produced and circulated from a sequence of prompts.

Anthropic also sets up a monitoring center

• OpenAI is not the only company strengthening its defenses. Anthropic, the company behind the Claude chatbot, has set up a dedicated operations room on the issue for the first time, bringing together teams of analysts in San Francisco, New York (USA), and Washington (USA);
• Google and Meta have also pledged to step up efforts to prevent their AI tools from being used to undermine confidence in the democratic process;
• At OpenAI, the election center began operating on the Thursday before the vote. The San Francisco team works jointly with a unit in Washington and brings together dozens of people, according to company representatives;
• Employees analyze prompts sent to ChatGPT that are automatically flagged for possible attempts to produce false text or images about the elections;
• One example would be using the chatbot to devise an influence campaign based on false allegations of election fraud, such as accusations of ballot manipulation. The material could be distributed on social media through fake accounts;
• When it identifies malicious activities intended to circulate on specific platforms, OpenAI says it notifies the companies responsible for those services.

AI heightens the risks of electoral manipulation

A survey by the Brennan Center for Justice, an organization affiliated with New York University School of Law, identified a contradictory picture.

The most popular chatbots generally resist requests related to conspiracy theories about elections. At the same time, AI tools regularly produce texts, images, and videos that can be used to weaken confidence in voting.

The concern is also related to cuts to social media security and moderation teams in recent years. After Russian interference in the 2016 presidential elections, Meta created an operations center for the 2018 legislative elections.

Since then, however, Meta and other platforms have reduced the teams responsible for security and combating electoral disinformation. The decision came amid pressure from Republicans and, from 2025, the Trump administration, which accused the companies of censoring conservative opinions.

This retreat coincided with the expansion of AI tools, heightening concerns about the technology’s impact on elections.

Specialists and former U.S. government officials also warned about the ability of these resources to make manipulation campaigns more convincing, while the Trump administration dismantled federal teams that had previously worked to counter threats to the electoral process.

“It would be unwise to think that this threat has disappeared and that technology is not making these threats more powerful and convincing,” said Lauren Protentis, a former member of the U.S. National Security Council, where she supervised matters related to malicious foreign influence during the Joe Biden administration.

OpenAI has already identified operations linked to other countries

OpenAI had already reported attempts by Russia, China, and Iran to use ChatGPT in foreign influence operations. The company says it disrupted two such campaigns in the month prior to the U.S. election.

However, up to that point, the company had not disclosed any specific operation related to the November legislative elections.

OpenAI’s chief investigator, Ben Nimmo, said the goal is to identify these initiatives before they reach a significant audience.

“We are trying to detect operations before they have the opportunity to gain traction,” he said.