An analysis has found that AI can perform 41.3% of actual freelance work at a level equal to or higher than human experts. This assessment evaluated the ability to complete practical projects from start to finish, not just simple knowledge or reasoning capabilities. Compared to the highest automation rate of 2.5% for the same indicator a year ago, this indicates a significant improvement in AI's practical execution ability.
The Remote Labor Index (RLI), an AI agent evaluation metric based on real freelance projects first introduced last year by the U.S. AI Safety Center and Scale AI, has released its latest results.
The evaluation included tasks from various fields such as game development, product design, architecture, data analysis, and video production.
The RLI was constructed based on 240 actual projects completed by professionals. Of these, 230 were used for private evaluation, and 10 were provided as public examples. The total workload exceeds 6,000 hours, and the total value of the projects is over 140,000 dollars. Some tasks required over 100 hours of work or cost more than 10,000 dollars.
AI was provided with the necessary materials and tools for the tasks, and the execution cost per project was limited to a maximum of 30 dollars. Three evaluators directly assessed whether the AI's output was at a level suitable for delivery to actual customers, comparing it to the completed work of human experts.
In the latest evaluation, Anthropic's 'Claude 5.2' recorded the highest automation rate at 41.3%. 'Claude 5.5' followed with 21.3%, and 'GPT-6 Astral' came in at 20.8%. The performance of Claude 5.2 is more than double that of the previous generation, 'Claude 5.1', which had a 17.9% automation rate.
The tasks successfully completed by AI were concentrated in areas such as sound effect production, image generation, report writing, and data retrieval. On the other hand, issues such as insufficient quality of the output, inability to complete the task, and failure to meet the required file formats also emerged.
When the RLI was first introduced in 2025, the highest record was 2.5% by Manus. Since then, as model performance has continuously improved, the percentage of projects completed by AI at the level of human professionals has rapidly increased. This shows that AI is developing in the direction of performing complex practical tasks as a whole, beyond its ability to handle individual knowledge or coding problems.
However, 41.3% does not mean that 41.3% of all freelance jobs have disappeared. It refers to the percentage of evaluated projects where AI outputs were recognized as equal to or better than human professionals' work. Actual work also includes tasks not covered by this index, such as communication with customers or team collaboration.
Reporter Park Chan, cpark@aitimes.com
