what is ai llm benchmark

“Drunk” AI is terrible at keeping secrets AI models taught to write like drunk people became easier to jailbreak and more likely to leak secrets shared in confidence. That is the finding of UNSW Sydney researchers Anudeex Shetty, Aditya Joshi and Salil Kanhere, published in their paper “In Vino Veritas and Vulnerabilities.” “The key research question from the natural language processing (NLP) side for me was, how do we get LLMs drunk?” said Aditya Joshi, a

Getty / Futurism A recent survey found that millions of Americans are asking AI chatbots for medical advice, often instead of consulting human doctors. That’s despite researchers continuing to find severe flaws plaguing large language model-based tools that can purportedly offer summaries of medical records and dole out health advice based on simple text prompts. For one, hallucinations remain a massive unsolved problem, from AI models generating detailed

The rapid proliferation of AI chatbots has made it difficult to know which models are actually improving and which are falling behind. Traditional academic benchmarks only tell you so much, which has led many to lean on vibes-based analysis from LM Arena. However, a new study claims this popular AI ranking platform is rife with unfair practices, favoring large companies that just so happen to rank near the top of the index. The site’s operators, however, s