what is ai agent benchmark

Credit: Pixabay/CC0 Public Domain In artificial intelligence, 2025 marked a decisive shift. Systems once confined to research labs and prototypes began to appear as everyday tools. At the center of this transition was the rise of AI agents—AI systems that can use other software tools and act on their own. While researchers have studied AI for more than 60 years, and the term "agent" has long been part of the field's vocabulary, 2025 was the year the concep

Copy link Facebook X Whatsapp Reddit Pinterest Flipboard Email Share this article 1 Join the conversation Add us as a preferred source on Google Newsletter Subscribe to our newsletter Over at Tom's Hardware Premium, we've been publishing a slew of stories centered around our themed AI Chip Design Week, where you can learn more about how artificial intelligence is reshaping the chipmaking world. Each article in our series of stories is free to access for a

Editor | Yang WenRobotics is ushering in its own Computer Use moment.Yesterday, Anthropic released Fable 5.2, bringing new advances. Compared with its predecessor, its physical robot control capabilities improved markedly, enabling it even to complete tasks on which it had never previously undergone specialized training.In the RoboDojo Sim simulation evaluation, Fable 5.2 scored 64.9%, far exceeding GPT-6 Astra (29.8%) in Anthropic's internal test under th

Copy link Facebook X Whatsapp Reddit Pinterest Flipboard Email Share this article 12 Join the conversation Add us as a preferred source on Google Newsletter Subscribe to our newsletter After a few revisions of his position on AI-assisted contributions to the Linux kernel, Linus Torvalds has posted his own first LLM-assisted patch. The development branch purports to fix an old issue with massive cache contention when many CPUs are reading the same file — a

Last week, Kimi K3.1 Pro, an LLM from the Beijing, China-based company Moonshot AI, debuted to a surprising amount of fanfare, particularly if you care about benchmarks, which are meant to be standardized measurements for models’ capabilities. It charted at a very respectable—in fact somewhat shocking—fourth place on the marquee Artificial Analysis benchmark called Intelligence Index v4. It also claimed the top spot on Artificial Analysis’ Terminal Bench 4

Shutterstock / fizkes Microsoft’s new AI is surpassing the abilities of human doctors — according to its own study, at least. The research, which was peer-reviewed and published in the prestigious journal Science, showed that the medical model, dubbed the Microsoft AI Diagnostic Orchestrator (MAI-DxO), was over twice as accurate than generalist physicians at diagnosing patients. And unlike previous studies, this one forced the AI to approach each case like

@axzxs2001(suwei gui)2026 Developer AI Usage Report: 83% Are Already Using AIC#AIOpenAIAgent0 2026 Developer AI Usage Report: 83% Are Already Using AI Last updated at 2026-11-11 Today, if you ask programmers, “Do you use AI at work?”, the answer is essentially a foregone conclusion. The questions genuinely worth asking are several others. What are they using it for? How deeply are they using it? How much do they trust it? How widely are the much-discussed

Copy link Facebook X Whatsapp Reddit Pinterest Flipboard Email Share this article Join the conversation Add us as a preferred source on Google Newsletter Subscribe to our newsletter Nvidia and Oracle have announced general availability of Nvidia’s Vera Rubin NVL72 rack and Vera CPUs on Oracle Cloud Infrastructure (OCI) Superclusters at Oracle AI World on Tuesday. The announcement comes hot on the heels of Google Cloud announcing general availability for Ve

Xinzhiyuan reports The moment Claude 5.2’s “Twin Stars” were released, the AI world was buzzing again! This morning, Anthropic finally unveiled its next-generation dual flagship models—Fable 5.2 and Mythos 5.2. Not only did it complete the formalization of Fermat’s Last Theorem in 30 hours, Anthropic’s new dual champions, Fable 5.2 and Mythos 5.2, have genuinely grown hands this time! The four pre-release “real-world challenge tests” just released by the o

Accelerating the Evolution of Urban Intelligent Computing Centers in Eastern China Toward E-AIDC Recently, Damo Technology and Shanghai Pudong Environmental Protection Energy Development Co., Ltd. (hereinafter referred to as “Pudong Environmental Protection Energy”) officially signed a strategic cooperation agreement. The two parties will make in-depth deployments across four key areas: data center energy-efficiency optimization and intelligent management

(Photo=Baidu) Chinese Baidu unveiled an automated research system for AI development in which AI independently determines research directions and repeats experiments. Following its breakthrough of existing AI training benchmark records, the company demonstrated technological progress toward recursive self-improvement (RSI), while increasing the inference speed of its own model under development by more than sixfold in certain environments. Baidu announced

Whether you’re looking for your next investment, discovering solutions for your business, or gathering ideas to take back to your team, there’s plenty to explore today. Today’s focus spans AI, with speakers from Nvidia, Waabi, Flock Safety, and Ricursive Intelligence, alongside Benchmark’s full partnership. Conversations will cover robotics, AI hardware, public safety, and venture investment. Still deciding whether to come? Get your pass and join us at 50%

OpenAI researcher allegedly pressured mathematician to drop Anthropic co-author from math breakthrough paper Matthias Bastian View the LinkedIn Profile of Matthias Bastian Sep 8, 2026 Nano Banana Pro prompted by THE DECODER Key Points Mathematician Tristan Buckmaster says OpenAI pressured him after he and Levent Alpöge made progress on the Navier-Stokes equations using AI models. OpenAI claimed an internal model had produced a roughly 100-page proof follow

Adobe is testing how well AI agents can use editing tools like a Photoshop professional would Jonathan Kemper View the LinkedIn Profile of Jonathan Kemper Nov 1, 2026 Nano Banana Pro prompted by THE DECODER Key Points Adobe Research and UC San Diego built EditAgent, an AI system that breaks image editing tasks into sub-steps and pairs a general-purpose editor with specialized tools for things like sharpening or zooming. A standard language model evaluates

No AI agent scores above 12 percent on SemiAnalysis' new real-world work benchmark Manuel Uth Nov 5, 2026 Nano Banana Pro prompted by THE DECODER Key Points Analytics firm SemiAnalysis has released GDPx, a benchmark that tests AI agents on 206 tasks in realistic work environments and professional software over several hours. The scores are low. Claude Opus 5.5 leads the field with just 12 percent, followed by Google's significantly cheaper Gemini 4 Carbon

Published on October 22, 2026 at 10:26 AM Tech Artificial intelligence Anthropic unveils Claude Duet, an AI agent that works for you like Muse (Meta) and Dots (OpenAI) AI at your service 3 min Nicolas Lellouche {"@context":"https:\/\/schema.org","@type":"Person","@id":"https:\/\/www.numerama.com\/#\/schema\/person\/e1c5c505bb96ef4588035c09ccaf93e2","name":"Nicolas Lellouche","url":"https:\/\/www.numerama.com\/author\/nicolaslellouche\/","description":"Fasc

@alibabacloud-saAftershock: A Disaster Simulator for Evaluating the Coordination of a Qwen Agent SocietyTutorialAIDevelopersQwenAI Hackathon0Posted at 2026-11-09TAG40@@Award-winning project in the Global AI Hackathon Series (organized by Qwen Cloud) Project Overview A disaster occurs in a virtual city. Various missions unfold on the map: a flooded residential area, a collapsed school building with disaster victims trapped inside, a hospital whose generator

Reported by Xinzhiyuan After AI agents began taking over terminals, security issues also entered the execution environment. When an agent can read files, write scripts, run commands, and continuously adjust its next action based on system feedback, a single incorrect decision may affect actual file and system state. Enabling agents to promptly identify potential risks while completing tasks has become an important issue in agent safety research. Long-horiz

Ripple-65B Preview delivers strong agentic coding results as a fully Apache 2.0 open US model Matthias Bastian View the LinkedIn Profile of Matthias Bastian Nov 6, 2026 AI chip design firm Ricursive Intelligence has trained an open language model. Ripple-65B Preview was pretrained on 12 trillion tokens and, according to the company, achieved a score of 80.52 on the TerminalBench 2.1 coding benchmark, the best result among open US models. Ripple partly uses