Chinese Baidu unveiled an automated research system for AI development in which AI independently determines research directions and repeats experiments. Following its breakthrough of existing AI training benchmark records, the company demonstrated technological progress toward recursive self-improvement (RSI), while increasing the inference speed of its own model under development by more than sixfold in certain environments.
Baidu announced its AI automated research system, “ERNIE Evolve,” through its official blog on the 9th (local time). It is a system that repeats experiments and evaluations after AI selects a research direction and modifies code, finding ways to improve performance.
ERNIE Evolve consists of a “Director” that determines the research direction and multiple AI agents that conduct actual experiments. The Director sets research goals by consulting existing papers, code, and past experimental results, after which each agent independently seeks ways to improve performance and conducts experiments.
In each experiment, a “Selector” chooses the next improvement target, and an “Executor” modifies the code and runs tests. An evaluator and reviewer then determine whether performance has improved and filter out cheating methods that exploit overfitting or flaws in the evaluation criteria. Results that pass validation are used in the next experiment.
A notable feature is that it conducts multiple research projects simultaneously and accumulates experimental results in an “Experience Bank.” It records not only successful experiments but also failed processes, which are used to determine the next research direction. This enables the AI to go beyond simply modifying code and discover new ways to improve performance based on accumulated experience.
Baidu first validated the system with three public benchmarks evaluating AI training algorithms, training speed, and GPU code optimization. As a result, it surpassed the previous best records in three of the four evaluation metrics in total.
In “NanoChat AutoResearch,” which measures AI training performance, it recorded 0.8763, lower than the previous record of 0.8878. The lower the figure, the better the performance. In “NanoGPT Speedrun,” which evaluates AI training speed, it also shortened the time required to reach the target performance from 39.9 seconds to 38.9 seconds.
In the GPU kernel optimization benchmark “SOL-ExecBench,” it recorded a higher score than Recursive and Tensor’s automated research system “Hyra 1.0,” but fell short of the overall record.
Baidu particularly emphasized that it applied the system to actual AI model development beyond benchmarking.Applying ERNIE Evol to the inference system of the next-generation model under development, “ERNIE 5.5 Flash,” increased text-generation speed for specific low-latency workloads to 6.24 times the previous level on the Chinese-made Kunlunxin “P800” AI chip.
These results were achieved through four rounds of automated research without changing the model itself or its deployment environment. Previously unavailable optimization capabilities were activated, and inefficiencies in computational processing and data transmission were improved. The revised code also passed correctness verification for deployment in actual services.
In a GPU environment as well, approximately three days of automated research increased throughput for inference tasks requiring consistent results from identical inputs by 33.7%. In a separate inference optimization effort, text-generation speed improved by 14.9% while maintaining accuracy. Baidu described this as the first step toward applying RSI to actual AI research and development.
However, ERNIE Evol does not constitute a complete implementation of RSI. It conducts research within goals and evaluation criteria set by humans, and the results confirmed so far have focused primarily on optimizing training algorithms and inference software. The performance figures are based on Baidu’s own evaluations and have not undergone independent validation by an external institution.
Baidu did not release the entire automated research system, but published the optimization code and experiment logs generated on public benchmarks on GitHub.
Baidu plans to expand the scope of automated research to model-training systems and enable AI to develop not only research hypotheses and experimental methods but also performance-evaluation criteria. The company said it will maintain a control framework in which humans oversee research goals and permissions and can halt the system when necessary.
Reporter Im Dae-jun ydj@aitimes.com
#AD176958096310.ad-template { margin:auto; position:relative; display:block; clear:both; z-index:1; padding-bottom:20px; } #AD176958096310.ad-template .col { float:left; text-align:center; width:100%; } #AD176958096310.ad-template .col .ad-view { position:relative; } #AD176958096310::after {content:"";display:block;clear:both;}
