Anthropic is partnering with an Ireland-based information technology company to start evaluating its frontier artificial intelligence models.
The AI firm announced its “non-exclusive” partnership with Accenture on Friday after news surrounding AI development hit an all-time high last week. Accenture is the first company with which Anthropic is partnering on an independent evaluation of frontier AI. Anthropic said it will start collaborating with additional independent evaluators, who will be announced soon.
Anthropic and Accenture are each investing at least $1 billion to build out “embedded evaluation” over the next five years. The practice is extremely new, as Anthropic CEO Dario Amodei first introduced the concept in a policy essay released last weekend.
Amodei disclosed that embedded evaluation is the first step in Anthropic’s three-stage plan to “pace the frontier.” He defines embedded evaluators as those “who have employee-like access to verify safety practices and report incidents” involving AI models. Amodei said his company is “unilaterally committing” to this step, as seen in the Friday announcement.
“Unlike today’s external evaluators, embedded evaluators will work inside AI companies, with access comparable to an employee’s,” Anthropic said in a statement. “That access allows them to watch models take shape in training, follow the decisions that govern how those models are built and deployed, and speak directly to employees. From this vantage point, embedded evaluators can assess how a company operates, verify that it is keeping its safety commitments, and identify blind spots. They can also report incidents and give the public a more informed account of benefits and risks.”
“To be clear, independent embedded evaluators do not reduce our accountability, but help to make it more verifiable,” the frontier AI lab said. “The safety of our models remains our responsibility.”
Anthropic said it expects other frontier AI labs to work with several embedded evaluators simultaneously. Accenture will likely work with more than one AI company in the future.
“Given the importance and urgency of this work, Anthropic will fund Accenture’s work directly,” Anthropic said. “We are also in dialogue with METR and other nonprofit evaluators to pilot elements of embedded evaluation using their own funding. Ultimately, we believe frontier AI needs an ecosystem of evaluators operating with shared standards.”
NEWSOM ENTERS AI POLITICAL FIRESTORM WITH ‘KILL SWITCH’ EXECUTIVE ORDER
OpenAI also committed itself to embedded evaluation as a way to keep frontier AI in check. OpenAI CEO Sam Altman called Amodei’s concept a “great idea.”
Concerns about AI safety were renewed last week after an AI researcher at Anthropic quit, citing an existential fear among AI developers that the very technology they’re building will advance to the point of potential human extinction in a few years. More cynical observers wondered whether the former employee’s public resignation was a ploy meant to push Congress into further regulating AI models.