what is ai agent benchmark

Nine AI agents tried autonomous research on open questions from US national labs and none passed the experts' bar Jonathan Kemper View the LinkedIn Profile of Jonathan Kemper Nov 7, 2026 Nano Banana Pro prompted by THE DECODER AI News Without the Hype – Curated by Humans Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section. Sub

AI auditors catch most agent misbehavior but flood reviewers with false alarms, new study finds Manuel Uth Nov 7, 2026 Nano Banana Pro prompted by THE DECODER AI News Without the Hype – Curated by Humans Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section. Subscribe now { "@context": "https://schema.org", "@type": "NewsArticle

AI coding agents still need human cleanup on nearly three out of four tasks in a real-world test Jonathan Kemper View the LinkedIn Profile of Jonathan Kemper Nov 8, 2026 ChatGPT-5.6 prompted by THE DECODER AI News Without the Hype – Curated by Humans Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section. Subscribe now { "@contex

Called Box Content Retrieval, the new feature allows agents to answer queries far more effectively by relying on the embedded metadata created by Nemotron. Box says the upshot for users is faster, more accurate, and less expensive queries. Nvidia’s VP of generative AI software, Kari Briski, told TechCrunch that she expects queries to be answered four times faster, with 20% fewer errors and 90% fewer tokens used. The system will be available to all of Box’s

Meta particularly emphasized Ember’s cost-per-task, using Artificial Analysis’s composite benchmark score to show Ember as cheaper and smarter than competitors like Anthropic’s Haiku and OpenAI’s Luna. Image Credits:Meta Superintelligence Labs At 10 cents per million input tokens, Ember is significantly cheaper than frontier competitors like Haiku and Luna. Not only that, it’s cost-competitive with open source competitors like DeepSeek V4.1 Flash and GLM 5

Why a new report on Anthropic automating AI development is alarming researchersAI is dramatically speeding up Anthropic’s research, even as the company calls for slowing down AI development.AI is dramatically speeding up Anthropic’s research, even as the company calls for slowing down AI development.Anthropic CEO Dario Amodei. Photo: Jason Henry/Bloomberg via Getty Images is a London-based reporter at The Verge covering all things AI and a Senior Tarbell F

One codebase can run on all chips. Author | Wu Haiming Editors |Li Na, Cen Feng 01The Evolution of Grok Bot:From “Chatbox” to “Cloud Computer”Musk recently added new capabilities to Grok Bot. (Leiphone(official account: Leiphone))On October 7, he said on X that Grok Bot would integrate with top-tier third-party models such as Claude Opus 5.5, Midjourney, and Suno; and more recently, it has undergone a fundamental paradigm shift:The main bot will identify w

41.3% of design items clear the safety threshold, yet complex engineering projects remain bogged down by tightly coupled constraints. Authors | Zheng Jiamei Cen Feng Editor | Cen Feng In engineering, 91% correctness can sometimes be indistinguishable from zero ability to deliver.A cable tray crossing fire-protection piping at the top of a server room can render dozens of pages of correct calculations worthless as a deliverable. The current-carrying capacit

99.4% Takes the Global Top Spot! This Chinese AI Was Trained by World Champions 2026-11-06 13:01:47 Source: QbitAI In one stroke, it broke the CyberGym world record at the "AI Security World Cup" Yishui, reporting from Aofei Temple QbitAI | Official WeChat Account QbitAI 99.4%, No. 1 in the world. Wow—who would have thought that the Silicon Valley firms always basking in the spotlight and expounding on AI security would...? Now, the one breaking the AI sec