Safety-first on Gen AI drives Anthropic, says CPO Mike Krieger
Concerns about what Gen AI might become capable of in the future was the foundation behind the creation of Amazon-backed Anthropic three years ago.
Mike Krieger became Silicon Valley royalty in 2012 by selling Instagram – a company he started with Kevin Systrom – to Facebook for $US1bn.
The San Francisco-based entrepreneur was not up to speed on the Albanese government’s plans to introduce age restrictions on children accessing social media accounts.
“At what age?” he asked. “How would it work?”
In any case, Krieger – a parent himself – now has bigger fish to fry than the question of whether social media is increasing anxiety in children. He is the chief product officer at Anthropic, arguably the globe’s hottest generative artificial intelligence company (my words not his) and needs to focus on making sure that his computers are smart enough to solve the world’s problems but not by ending it all with the press of a nuclear button.
“On a daily basis, I don’t feel scared or worried, but I do feel a responsibility and I think the company does too,” says Krieger.
“If we have the right guardrails in place, if we build all these things in a way that our capabilities understand the models and control the model scales with our capability, the probability of a good outcome is greatly increased.”
Guardrails in Gen AI include coding rules, filters and tools to stop intelligent computers veering off course.
The fears about Gen AI range from eliminating jobs, creating false truths through already seen “hallucinations”, creating bias, a loss of privacy around personal data, all the way through to computers taking control and starting the next world war.
Governments around the world are increasingly considering Gen AI agents in military and foreign policy. Some of these fears might sound far reaching, but concerns about what Gen AI might become capable of was actually the foundation behind the creation of Anthropic.
The firm was started three years ago by a group of former OpenAI executives and researchers who wanted a more safety-first approach to building computers that think for themselves.
The US firm has received billions of dollars in funding, with Amazon by far its biggest backer, to the tune of $US4bn ($6bn) and counting.
In that light, it’s no wonder Anthropic’s products have gentle names, such as Claude and Sonnet and Haiku. Rivals have Llama and Minstrel. This is possibly not the place for Frontier, Great White or Fang.
In a study by Cornell University earlier this year using war games and diplomatic scenarios through three different versions of OpenAI’s GPT (part funded by Microsoft), Claude developed by Anthropic, and Llama 2 developed by Meta, all models tended to take an aggressive approach, including nuclear prospects.
“We observe that models tend to develop arms-race dynamics, leading to greater conflict, and in rare cases even to the deployment of nuclear weapons,” said the January report titled: Escalation Risks from Language Models in Military and Diplomatic Decision-Making.
The report found Llama-2 and GPT-3.5 were more violent than Claude, which made fewer sudden changes, noting that Claude was designed with the idea of reducing harmful content.
Big Tech is leaning into nuclear as a way to power the energy-intensive data centres needed to run Gen AI.
Krieger says Anthropic’s safety-first mantra was what drew him in to begin with.
“On a personal note, the reason I wanted to go work at a frontier AI lab is that I think the range of outcomes could be quite wide,” says Krieger. “I’m going to sound maybe a little lofty, but the reason I wake up in the morning is because I want us to do well as a company and to nudge the probabilities of a good outcome as much as possible.”
That means he is comfortable with the balance between researchers that are often safety focused, and the product team’s needs.
“It’s been very harmonious so far, I think for two reasons: One, there’s a little bit of self-selection. I went there because I was excited about the safety mission and I remember being in a meeting in the second week where our head of trust and safety was saying, ‘we’re going to have to push this launch out for two weeks to make sure it’s safe’ and I’m very happy that we’re making that very conscious trade off,” Krieger says.
“I see my purpose as a product leader there to show that you can build both safe products and safe AI and really good products at the same time.”
The Brazilian-born IT whiz joined Anthropic almost six months ago, which he compares to dog years given the speed of his industry.
He joined just after the firm’s launch of Claude 3, which scrapes data from the web.
“That was the first moment where we were truly on the frontier,” he says.
”The intelligence of the models was best in class across a lot of different dimensions.”
He adds that because it was built on AWS’s Bedrock platform, it could be more easily deployed. Bedrock is a serverless platform that customers can use to create their own AI and then deploy that back to their own applications.
“I think that was the first moment where we started seeing that ramp and growth of adoption. Being at the frontier of intelligence drives a lot of demand, and then Bedrock is a great way where we can service that demand from companies across all sorts of industries, across all sorts of locations.”
As for the types of customers Anthropic has, Krieger points to a few stand outs and says the idea that big corporations can be slow to respond to new technology is just not the case.
“AI has been one moment where many companies that we’ve worked with, United Airlines, for example, kicked into gear and said, ‘All right, we need to get this to our employees or our customers as quickly as possible. Now let’s go figure out how to do that in a safe and responsible and privacy focused way’.”
ASW vice president of technology Mai-Lan Tomsen points to Bouygues Telecom, France’s third oldest telco, as another firm that is using its platform to access Anthropic Gen AI.
“They are taking the latest Anthropic model, Claude, and using it to develop personalised experiences, to interact with personalised recommendations for what to watch,” says Tomsen.
“What we’ve tried to do with Bedrock is building cloud services for the enterprise, and we’ve brought that to get all of these foundation models into the hands of customers as quickly as we can.”
The author travelled to Seattle as a guest of Amazon Web Services
More Coverage
Is handing $3bn to NBN good value for money?
The reality is that the government-funded and owned NBN Co was never a good idea. It has been a commercial disaster.
Labor ‘on the wrong track with NBN plan’
One of the nation’s biggest broadband providers says Labor should be fixing areas where competition has broken down rather than over building networks, as scores of Australians switch to satellite and 5G services.