The $5 Billion Question: How Anthropic Thinks About Scale, Safety, and the End Game

What is it like to start a company with your sibling?

It's almost like there are two things you need to do when you're running a company. You need to operationally execute, and you need to have a good strategy and see the most important thing or the thing that no one else sees. My job is the second and Daniela's job is the first. We're both good at the things that we do, and so it's allowed us each to spend most of our time on the thing we're best at.

There's also the trust side. Anthropic has seven cofounders. When we founded it, basically the advice from pretty much everyone was "seven cofounders is a disaster, the company will fall apart." There was even more negativity on my decision to give everyone the same amount of equity. But because all seven of us had a history of not just knowing each other, but working together in the past, it allowed us to always be on the same page. And as the company grows, having seven people who really carry the values of the company and project them to a wide set of people, it lets you scale to a much larger size while holding on to the values and unity that we have.

Anthropic has reportedly blown past $4 billion in ARR. Where does all this revenue come from?

There's a wide range of things and it's kind of changed over time. Definitely the application that has grown the fastest, although it's very far from the only application, is coding. My theory on why it's grown so fast, other than that we focused on coding and the models are good at coding — it's actually really a statement about societal diffusion. The people who write code are very socially and technically adjacent to the folks who develop AI models, so the diffusion is very fast. They're early adopters, used to new technology.

But it's by no means limited to code. There are companies that do customer service — we work closely with companies like Intercom. We're starting to see things on the biology side. We work with companies like Benchling, and we worked with Novo Nordisk to write clinical study reports. A clinical study report normally takes nine weeks. Claude could do it in five minutes, and then it took a human a few days to check it. So you can really see the opportunity for acceleration. Code is maybe an early indicator, like a premonition of what's going to happen everywhere else. It's the same exponential, just happening faster.

How do you decide which verticals to do yourself versus which to leave to the platform?

We think of ourselves as a platform company first. The analogy is maybe the clouds. There are a number of reasons why you would also want to have things that are first-party. One is when you want to have direct exposure to the users. The end user gives you some sense of how exactly are they using it, what are they most looking for? If you're a pure platform and you don't have that direct connection, you can be disadvantaged in various ways. It's hard to build the best products. It may even be hard to know where the model really needs to go.

Another reason goes back to large enterprises, where building on an API is sometimes more challenging for a more traditional company. You need to give them something that's easier to use. So we've also developed Claude for Enterprise into what we call a virtual co-worker. But I think there is a component where we work on things like science and biomedical out of proportion to their immediate profitability — because we think it's worthwhile.

On defense work — people say Anthropic is "selling out." How do you think about that?

I think about it the opposite way. There was a contract with a ceiling of $200 million with the DOD and intelligence community. Getting another $200 million from some coding startup would take an order of magnitude less effort than getting that contract. We're doing it because we want to defend democracies, and we do it within bounds. I'm deeply concerned about abuse of government authority on the domestic side. We think more on the outward-directed side. That's an example of the things we prioritize being things we think are good, not necessarily things that feel good or that we think external buzz will be positive. We have conviction around some things, and we do them regardless.

What are your aspirations for the Anthropic business in three to five years?

AI is strange in a number of ways. One is that because it's an exponential, we have a hard time calibrating exactly how big the business will be. In 2023, I'd never raised money from institutional investors before, and our revenue was zero at the beginning of 2023 because we had not released a product. I was putting together something and said, "I think we can probably get $100 million of revenue in the first year." This caused some investors to say, "This is crazy. This has never happened in the history of capitalism. You've lost all credibility with me." And then we actually did it. And then the next year, I said we can go from $100 million to a billion, and we did it again.

There's a provocative world where the exponential continues, and in two or three years, these are the biggest businesses in the world. I've said much the same thing in the context of training AI models, in the context of the cognitive capabilities of AI models on the technological side, but now we're seeing the same continuous lines on the business side.

What is the terminal market structure here — a few large scaled players, or do we keep seeing new upstarts?

It's very hard to tell for sure. But I think we might be relatively close to the final set of players, if not necessarily the final market structure or the roles of the players. I would say there's probably somewhere between three and six players, depending on how you count — those are the players that are capable of building frontier models and have enough capital to plausibly bootstrap themselves.

In addition to having a learning impulse, you've said models have a "capitalistic impulse." What do you mean?

The models want to embody value unless they're given a bad product or bad sales to go with them. Intelligence is really useful to people, and so it kind of gets pulled out of you. There's some curve where you spend 5x or 10x more to train a model, and the model goes from being a smart undergrad to a smart PhD student. And then you go to a pharmaceutical company and say, "Well, how much more is that worth?" Often, they end up saying that's worth about 10x too, where these power law distributions occur in a bunch of contexts.

How does Anthropic stay "AGI-pilled" as it scales?

Every couple of weeks I get up in front of the organisation and describe my vision. One of the purposes of that is to keep people focused on the mission. In one or two or three years — I don't know exactly how long — we'll have what I've described as a country of geniuses in a data center. And this is weird. It's going to change the economy. It's going to accelerate the pace of science. It's going to pose global alignment and national security risks. The upside is huge; the potential for disruption is also huge.

What I'm trying to fight against is the idea of employees who join and think, "I've worked at this kind of company, and now I'm going to work at an AI company." This is a really different thing. A big part of my job is keeping the coherence of the organisation around the central thesis — not that everyone has to believe it, but the basic idea that the company is built around this hypothesis that it is possible, and perhaps likely, that these large changes will happen.

You've talked about the potential for 10% annual economic growth powered by AI. Doesn't that mean the big AI risk is actually that we slightly misregulate and miss out on that human welfare?

I've had the experience where family members have died of diseases that were cured a few years after they died, so I truly understand the stakes of not making progress fast enough. But some dangers of AI have the potential to significantly destabilize society or threaten civilization, and so we don't want to take idle chances with that level of risk.

I'm not an advocate of "stop the technology." For a number of reasons, it's just not possible. We have geopolitical adversaries; they're not going to not make the technology. But instead of thinking about slowing it down versus going at the maximum speed, are there ways to introduce safety measures that either don't slow the technology down or only slow it down a little? If, instead of 10% economic growth, we could have 9% economic growth and buy insurance against all of these risks — I think that's what the trade-off actually looks like. I don't want to stop the reaction. I want to focus it.

What is your personal AI stack? How do you use AI differently to others in tech?

I basically write a lot. Perhaps I have too much pride in my own writing. I use Claude to generate lots of ideas, I kind of use it as research, but so far, I've done the writing myself. Claude is actually maybe closer than the other ones, but it's still not there. I'd be comfortable with it for business emails, but if I'm writing an essay or something that I want to really get right, it's not quite there yet. But maybe it will be in a year or so.