Facebook CEO Mark Zuckerberg sounded an optimistic note three years ago when he wrote about the progress his company was making in automated moderation tools powered by artificial intelligence. “Through the end of 2019, we expect to have trained our systems to proactively detect the vast majority of problematic content,” he wrote in November 2018.
But as recently as March, internal Facebook documents reveal the company found its automated moderation tools were falling far short, removing posts that were responsible for only a small fraction of views of hate speech and violence and incitement on the platform. The posts removed by AI tools only accounted for 3–5 percent of views of hate speech and 0.6 percent of views of violence and incitement.
While that’s up from 2 percent of hate speech views two years ago, according to documents turned over to The Wall Street Journal by whistleblower Frances Haugen, it’s far from a vast majority. One of the company’s senior engineers wrote in 2019 that he felt the company could improve by an order of magnitude but that they might then hit a ceiling beyond which further advances would be difficult.
“The problem is that we do not and possibly never will have a model that captures even a majority of integrity harms, particularly in sensitive areas,” he wrote. “Recent estimates suggest that unless there is a major change in strategy, it will be very difficult to improve this beyond 10-20% in the short-medium term.”
To arrive at these estimates, Facebook takes a sample of posts, applies its AI moderation tools to them, and then asks human moderators to assess the AI’s accuracy. It then uses that fraction to estimate how much hate speech or violence and incitement is missed platform-wide.
“When you consider that we miss 95 percent of violating hate speech, you realize that it might actually take 100 violations for that group to accrue its five strikes,” one data scientist said in a 2020 note that was reported by BuzzFeed and WSJ.
