Representatives from OpenAI and Meta warned members of the U.S. judiciary on Thursday (15) about the growing difficulties of identifying fake videos and audio produced by artificial intelligence (AI). According to experts, the so-called deepfakes are becoming so sophisticated that it will be increasingly difficult to verify the authenticity of audiovisual evidence presented in trials.
The warning was issued during a meeting in Boston of the Advisory Committee on Evidence Rules of the United States Judicial Conference. Since 2023, the group has discussed how to deal with the risks of AI-generated content in the courts and what criteria should be applied to confirm the validity of this type of evidence.
Deepfakes make evidence verification more difficult
Shira Anderson, OpenAI public policy attorney, said that advances in AI image generation make it increasingly difficult to develop scientific methods capable of identifying artificial content.
“AI images are getting better and better, and I don’t think there will be a scientific way to detect them in the future,” Anderson told the committee.
Devin Norder, Meta’s director of product management, also highlighted the advancement of the technology.According to him, content produced by the most advanced AI models can already be indistinguishable, to human observers, from authentic media or other artificially generated materials.
Norder added that thecost and time required to create such content are very low. This increases concern about the possibility of false materials being presented as legitimate evidence in court proceedings.
Few cases identified so far
Despite the warnings, there are still few records of court cases involving deepfakes in the United States. In March, the Federal Judicial Center, a research arm of the U.S. judiciary, reported that only 15 of the 931 judges who responded to a survey had encountered cases in which one party challenged the presentation of evidence because they suspected it was a deepfake.
Jesse Furman, a federal judge in Manhattan and the committee’s chair, said there is a significant amount of academic analysis and forecasting on the technology’s impact on the courts, but concrete cases are still limited.
At Thursday’s meeting, Furman appointed a subcommittee to examine the issue in greater depth. However, experts from industry, academia, and government stressed that the challenge should not be treated as a distant possibility. For them, the courts’ ability to prevent the use of false evidence is likely to become increasingly difficult.
Detection tools have limitations
Companies such as OpenAI and Meta are developing tools and services to help verify the authenticity of visual content.In addition, new laws in California and the European Union require the use of digital watermarks designed to facilitate the identification of deepfakes.
Anderson, however, warned that there is no universal standard for detecting these watermarks or confirming the authenticity of images and videos. Since each company adopts its own systems, the available tools do not allow for the reliable verification of all content.
The OpenAI representative also said she believed that identification mechanisms will continue to evolve but will not be entirely infallible.
Judges advocate caution in drafting rules
Valerie Caproni, a federal judge in Manhattan and a member of the committee, expressed caution about the immediate creation of specific rules for AI-generated evidence. According to her, there is still a lack of data indicating that deepfakes have become a widespread problem in the courts.
Caproni also noted that current rules and laws already prohibit the presentation of fraudulent evidence and allow judges to assess whether particular material has been falsified.
The judge emphasized that evidence manipulation is not a new problem in the judicial system. Therefore, the discussion concerns determining whether existing rules are sufficient to address the additional challenges posed by artificial intelligence.
Ana Figueiredo is a technology editor at Olhar Digital. She has a degree in Journalism from the Federal University of Uberlândia (UFU).
