“Abusive or cruel”: Anthropic forbids you from mistreating Claude

Warnings about the uncontrolled development of artificial intelligence have multiplied in recent months. Bill Gates notably explained that this technology, if misused, could cause « one billion deaths », while OpenAI, Google, and Anthropic are working on their own AI cop.While Sam Altman (OpenAI) estimated that certain negative consequences of AI still had to be accepted, Dario Amodei (Anthropic) has consistently advocated strict control over this technology. Anthropic has just updated Claude’s usage policy to prevent excessive and malicious uses of its AI models.

New rules to combat malicious uses of Claude

This update to Claude’s usage policy aims to address high-risk cases of abuse, including election interference, weapons development, surveillance, and uses in healthcare and finance. From now on, any « abusive or cruel, persistent and pointless behavior » toward Claude is strictly prohibited. The company explains that if such behavior is detected, the ongoing conversation will be closed instantly. However, the new policy does not mention bans.

The company also clarifies that this update to its usage policy « only applies to extreme cases, when users act cruelly and repeatedly toward our models, for no apparent reason. It does not concern commonplace forms of frustration, resistance, dark creative themes, or testing and research on the models ».

Election fraud, people tracking, weapons manufacturing and robotics

With this new policy, Anthropic seeks to combat propaganda content generated by AI, particularly in the context of political and commercial campaigns. Anthropic says it is prohibiting any form of fraud or disruption of the electoral process. This includes, in particular, the dissemination of false information about candidates or voting procedures, the impersonation of candidates or election officials, and attempts to discourage participation in elections. It is worth noting that these changes come just a few weeks before the U.S. midterm elections.

Anthropic also announces new restrictions concerning people tracking using AI, while reporting increased use of Claude to identify and track “political dissidents”. From now on: “Claude cannot be used to decide or recommend who should be investigated, arrested, or charged as part of a law enforcement or judicial proceeding. We also prohibit the use of Claude to design or improve surveillance tools”.

Dario Amodei’s company has always strictly prohibited the manufacture of weapons using its AI systems. However, it says it has observed a growing number of requests to develop guidance systems and weapon control systems.The screw is therefore tightened at this level. Finally, the robotics sector is developing rapidly. Anthropic now states that when its AI models are « connected to hardware performing autonomous physical actions and liable to cause injury », a « qualified operator must be able to observe the equipment and stop it if necessary ». In other words, a potentially dangerous robot integrating Claude must be closely monitored.

Ultimately, Anthropic once again positions itself as one of the leading American AI players in favor of stricter regulation of this technology.

👉🏻 Follow tech news in real time: add 01net to your sources on Google, and subscribe to our WhatsApp channel.

Source: Anthropic