This is a free article made possible by Big Technology subscribers with no advertising included. To support my independent tech journalism, please consider subscribing for $8 per month.

Subscribe Today

AWS CEO Matt Garman Talks Amazon’s Big Bets In AI Chips, Reasoning, And Nuclear Energy

The Amazon Web Services leader shares Amazon's AI strategy and plans, including his willingness to work with OpenAI and details on his partnership with NVIDIA.

Amazon made a slew of big AI announcements early Tuesday. It said its Trainium2 chip would be generally available for AI training and inference. It debuted a new reasoning tool to limit hallucinations. It said it was building a new “ultracluster” with hundreds of thousands of GPUs that Anthropic — which it just invested $4 billion in — will use for AI training, among others. And it shared that it built a new foundational set of AI models, called Nova.

A day before the news hit, I sat down with AWS CEO Matt Garman at the company’s re:Invent conference in Las Vegas, Nevada to talk through Amazon’s AI strategy and plans for the future. During the conversation, Garman put all this news in context, though he danced a bit around Nova.

Below is our conversation, edited lightly for clarity and length, with revealing insights from Garman about Amazon’s AI push and noteworthy remarks about his willingness to partner with OpenAI. We also talk nuclear energy, the state of AWS itself, and Amazon culture.

You can read below or listen to the full podcast on Apple Podcasts, Spotify, or your app of choice (that’s coming tomorrow at 6 a.m. eastern, but please subscribe!) This interview has been edited for clarity and length. Please forgive the typos, I was moving fast.

Alex Kantrowitz: You're the kings of building data centers. Nobody does it better than AWS. That said, Elon Musk took 122 days to build a 100,000 plus GPU cluster in Memphis named Colossus, and used it to train xAI’s latest model. Does this show that scaling data centers is now core to competing in AI? And do you view it as validation, or something else?

Matt Garman: Well, look, we've been building data centers for almost two decades now, and it's less that we were out there kind of bragging to the press about it.

You can brag, what's your size of yours?

Well, more what we do is provide infinite scale for customers. And so our goal is for largely the customers not have to think about these things, right? And so we want them across their compute, across their storage, across their databases to able to scale to any size.

Take something like s3 as an example. It's an incredibly complex, very detailed system that keeps your data, keeps it durable, and scales infinitely, right? And customers largely just put data in there. Don't think about it. And so today, s3 actually stores 400 trillion objects, an enormous number that's hard to even get your head around, but it's something where we just keep scaling and we keep growing for our customers.

I just think about AI now, these are power hungry, massive data centers for sure. And AWS is adding tons and tons of compute all the time for our customers. Largely, what we think of, though, is less about how fast can you build one particular cluster — the absolute size of AWS dwarfs any other particular cluster out there — but we're focused on how do we deliver the compute that customers need to go build their applications.

Take somebody Anthropic as an example. Anthropic has the what are widely considered to be the most powerful AI models out there today in their Claude set of models. It’s using our next generation Tranium2 chips, and this cluster that we're building for them in 2025 will be five times the size the number of exaflops that they use to train the current generation of models, which are by far the most powerful ones out there. It's going to be five times that. Five times that size next year, all built on Trainium to deliver hundreds of thousands of chips in a single cluster for them so they can train the next generation.

That's the type of thing where we work with customers, understand what's interesting for them, and then help them scale to whatever level they need. And that's just one of our customers, of course, right? We have hundreds and hundreds of other customers.

Here's my point. You're so good at this, right? Look at what you just talked about, in terms of helping Anthropic scale. That would lead me to believe that Amazon would have its own cutting edge model, one that would lead, and be better than the OpenAIs and the Anthropics. This is your core competency, and this is what makes the models run. So why hasn't that happened?

Our core competency is about delivering compute power for all of the people that need it. And for a long time, we've been very focused on — How do we build the capabilities to let our customers build whatever they want? And sometimes they’re areas that Amazon also builds, and other times they're not areas that Amazon builds.

So you think about whether it's in the database world, or you think about in the storage world, or you think about the data analytics world, or you think about the ML world. We build this underlying compute platform that everybody can go build upon, and sometimes we build services that compete with others out there in the market.

That's the kind of the beauty of AWS, that our goal is to build that infrastructure. We want this platform that everybody can go build the broadest set of applications possible out there.

But I'm thinking about it for AI specifically. And in the world that you play in, you have Google. They have their own model. They sell cloud services. You have Microsoft, they have a deal with OpenAI. This is where a lot of the growth is coming from…

I just think it's fundamentally the wrong way of thinking about it. A lot of times people are thinking about — there's just going to be this one model, and I want to have the one model that's going to be the most powerful, and the one model to rule them all. And as you've seen over the last year, there isn't one model that's the best at everything. Like there's models that are really good at reasoning. There are models that are great for that provide open weight so that people can bring their own data and fine tune them and distill them and create kind of completely new models from that, that are completely custom for customers — and for that you may want to use a Llama model, or you may want to use a Mistral model.

There's customers who really want to build the world's best images, and they might use something like a Stability or they might use something like a Titan model. There's customers that need really complex reasoning, and they might use an Anthropic model.

There's a whole ton of these operating out there, and our goal is — How do we help customers use the very best. It doesn't have to be one thing. It's not just one. And we don't think that there's one best database. We don't think there's one best compute platform or processor. We don't think that there's one best model. It's across that whole set. And that's been our strategy, and customers have really embraced that strategy and they're thinking about how they go build production applications. They want the stability, the operational excellence and the security that they get with AWS, but they also want that choice. It's incredibly important for them, and I think choice is important for customers, no matter if they're building generative AI applications, no matter if they're picking a database offering, no matter if they're picking a compute platform. They want that choice, and that is something that AWS has from the very earliest days, really leaned into, and I think it's an important part of our strategy, and it's maybe not the strategy that others have. Maybe others say it's just this one, and this is the one that we're going to lean into. But it's not the strategy that we've picked.

Our choice is around choice, and it's part of why we have the broadest partner ecosystem as well. It's why, as you walk the halls here in re:Invent it's filled with partners who are building their business on top of AWS. We're leaning in and helping our customers accelerate their journeys to the cloud, their modernization efforts, their AI efforts. I think that is a lot of what makes AWS special.

I'm gonna move off this in a moment, but the reason why I'm asking these questions is because you do have at least one bet that big foundational models are going to matter — that's the 4 billion you just invested in Anthropic.

And I think that the strategy that AWS has makes a lot of sense. The Bedrock strategy, with lot of different models that developers can choose and build on top of. But you also are limited by the fact that OpenAI is not there. I don't think Google's there. So wouldn't it make sense in parallel to the Bring Your Own model strategy to also use this capacity that you have to scale infrastructure to get in the game yourself.

I'll never say never, right? I think there's an interesting idea. And then we'll we never close any doors. I think we're always open to, frankly, a whole host of things. We're always open to having OpenAI be available in AWS someday, or having Gemini models be available in AWS someday, and maybe someday we will spend more time focused on our own models. For sure, I think, I think all of that is open.

Part of what I think makes AWS special is we're always open. Take our announcement earlier this year about partnering deeply with Oracle, about making Oracle databases available in AWS. Lots of people would said, ‘Oh, that's never going to happen. It's against your strategy.’ Our strategy is to embrace all technologies, because we want anything that customers can use. We want them to be available and to be able to use it inside of AWS. And look, sometimes it happens today, sometimes it happens tomorrow, sometimes it happens weeks from now, months from now, years from now. But that is our goal, is to make all of those technologies available for our customers.

I'm going to parse your language a little bit. You said that you're always open. You might be open to having OpenAI on Bedrock within AWS. Are you talking to them? Would you want to ask them to come on board?

There's nothing to announce there today. But I'm saying if customers want that, that's something that we would want, and we'd love to make it happen at some point.

Let's speak about Anthropic. What does the $4 billion that you just invested in Anthropic get you? And how does that make you differentiated from other cloud providers?

We make the investments in Anthropic because we think it's a it's a good bet. They have a very good team. They've made some incredible traction in the market, and we really like where they're innovating. And so we thought that's a good investment.

We’re definitely Claude heads on show.

It's a fantastic product, right? And Dario and team are very good, and they continue to attract some of the best talent out there in the market today. The other thing that we get from that is a deep collaboration on Trainium. We've made a big bet on Trainium as an additional option for customers.

These are the Amazon-built chips that developers can use to train their own models with.

That's right. And so today, the vast majority of of AI processing, whether it's inference or training, is done on NVIDIA GPUs. And we're a huge partner of NVIDIA. We will be for a long time. And they make fantastic products. When Blackwell chips come out, I think people are very excited about that next generation platform.

But we also think that customers want choice, and we've seen that time and time again. We've done that with general purpose processors. We have our own custom general purpose processor called Graviton. And so we actually went and built our own AI chips. The first version was called Trainium, and we launched Trainium1, a couple years ago, in 2022 and we’re just announcing the general availability of Trainium2 here at re:Invent.

So that's news that's happening this week?

Yes that's in my keynote. We'll have Trainium2, and Trainium2 gives really differentiated performance. We see 30 to 40% price performance gains versus our instances that are GPU powered today. So we're very excited about Trainium2, and customers are really excited about that.

And what Anthropic gives us, back to your question, is a leading frontier model provider that can work deeply to build the largest clusters that have ever been built with this new technology, where we can learn from them, right? And just learn what's working, what's not, what are the things we accelerated so that Trainium3 and Trainium4 and Trainium5 and Trainium6 can all get better as we as we continue to go and the software associated with GPUs gets better, or the accelerators gets better as well.

One of the things where people who have tried to build accelerated platforms before have fallen down, is the software support has not been as good as NVIDIA. NVIDIA’s software support is fantastic, and so that's a big area where they're helping us as well as we iron out the kinks and try to figure out how we make sure that developers can start to use these Trainium2 chips in a very seamless way, in a high performance way. So we learn a lot from Anthropic. As big users, they're literally leaning in and help us learn. And they get that scale and cost benefit of running on this price performance platform that gives them a huge win. We think then from that investment, we can both benefit as they deliver better and better and better models over time.

NVIDIA doesn’t get upset that you're trying to build the your own chips? They do have a supply issue, but how does it work with them?

No, no, I have a great relationship with NVIDIA and Jensen and this is a thing that we've done before. We have a fantastic relationship with Intel and AMD, and we produce our own general purpose processors. And it's a big world out there, and there's a lot of market for and for lots of different use cases. That's not one is going to be the winner, right? There's going to be use cases where people are going to want to use GPUs, and there's going to be use cases where people are going to find training do the best case. There are use cases where people find that our Intel instances are the best choice for them. They're ones where they find that AMD instances are the best choice for them. And there's increasingly a large set where they find Graviton, which is our purpose built general purpose processor, is the right fit for them, and it doesn't mean that we don't have great relationships with Intel and NVIDIA or Intel and AMD. And it means we'll continue to have a great relationship with NVIDIA, because for them and for us, it's incredibly important for NVIDIA processors and GPU powered processors to perform great on AWS, and so we are doubling down our investment to make sure that NVIDIA performs outstanding in AWS. We want it to be the best place for people to run GPU based workloads, and I expect it will continue to be for a really long time.

What's the buying process like with NVIDIA? You want as many chips as you can get, I would imagine you would come up against Elon, who buys truckloads. You have Zuckerberg, who has been buying lots with Meta. So do you have to jostle with the other companies to get NVIDIA chips, or do you get every exact quantity you want?

NVIDIA is very fair about how they go about it. They're very fair in dealing with us. We give long term forecasts, and they tell us what they can supply, and we all know that there's been shortages in the last couple of years. Specifically as demand has really ramped up, and they've been great about ensuring that we get enough to support our joint customers as much as possible.

What about inference? Last time I heard you speak, you said that the activity within Gen AI is 50% training, 50% inference. Does that ratio still hold? And how are you going to put the chips out there to allow companies to be able to do cheaper inference? Because the issue with generative AI is it works well, but it's so expensive that companies take proof of concepts and only 20% actually make them out into production.

Yeah, it's absolutely the case. We're still probably seeing about that ratio of 50/50, I think it's more inference to than training. And increasingly we'll see more and more of the workload shift that way. Cost is a super important factor that many of our customers are definitely worried about and thinking about on a daily basis.

A lot of people went and did a bunch of these Gen AI capability tests where they did proof of concepts, and they launched hundreds of proof of concepts across the enterprise without really paying attention to what the value was going to be, or anything like that. And they say, well, the ROI is not really there. They're not really integrated into my production environment. They're just kind of these proof of concepts that I'm not getting a lot of value out of, and they're expensive, as you mentioned.

So two things that people are thinking about is, one, how do I lower the cost so that I make that the cost much lower to run? And that's to your point about cost of inference. And two, how do I actually get more value out of that? So the ROI equation just completely shifts and makes more sense. And it turns out it's probably not all hundred of those. It's probably two or three or five of those that are really valuable.

There's a few things that we're doing to help people lower costs. Originally, we had a small chip called Inferentia that would run really fast, lightweight inference. Now, as you're running models that have billions, tens of billions, hundreds of billions of parameters, they're way too big to fit on these small inference chips. And effectively they're running on the same training chips, like they're the exact same things. And so you run inference today on H100s, H200s, or you run inference today on Tranium2s or Tranium1s. And so we may come out over time with other inference inferentia chips, but they're really using a lot of that same architecture, and they're still really large servers.

And so we actually expect that Tranium2 is going to be a fantastic inference platform. It's going to be a fantastic inference platform. We actually think it'll be about 30 to 40% cheaper compared to the leading GPU based platforms, that's a pretty big price decrease.

And then there's a couple there's also announced at re:Invent we're launching automated model distillation inside of Bedrock. And what that lets you do is you can take one of these really large models that's really good at answering questions, you can feed it all your prompts and things for the specific use case you're going to want, and it'll automatically tune a smaller model based on those outputs and teach a smaller model to be an expert only in the area that you want, with regards to reasoning and answering.

So you can get these smaller, cheaper models, say, like a Llama 8b model, as opposed to llama 400b model: Cheaper to run, faster to run, and you can get it to be an expert at the narrow use case that you want it to be.

And so that, combined with a cheaper infrastructure, is one of the things that is really going to help people lower their costs and be able to do more and more inference in production

Those small models seem to be the cost solution. Sounds like you're a believer.

That's right. So absolutely.

One more question about Nvidia. You've tested the new Blackwell chip. Is it the real deal?

We have. They're working on getting the yields up and getting it into production, but we're excited about that. And also, at re:Invent we're going to announce that the p6 which is the Blackwell based instance, that's coming early next year, and we're excited about that. We're expecting about two and a half times the compute performance out of a Blackwell chip that you get out of an h100. And so that's a pretty big win for customers.

So you're on board with Jensen's, “The more you spend, the more you save,” slogan?

That's right. That team has executed quite well, and they continue to deliver huge improvements in performance and we're happy to make those available for customers.

Okay, should we talk about ROI?

Sure

It’s the two year anniversary of ChatGPT. All these companies have rushed to put generative AI in their products. To this point, there's a couple of things that have worked well. AI for coding, AI that is a customer service Chatbot, AI that can read unstructured documents and make sense of them.

Yep, those are the three big ones.

I haven't heard much more outside of that. We're talking about something that's added trillions of dollars, potentially, to public company market caps, something that has had the largest VC funding round, and then probably the subsequent three after that. So are the three examples that I listed, enough to make this worth the money?

No, definitely not, but they're super valuable right now, and they're just the tip of the iceberg. And that's the thing. It's like, you just have to look at the rest of the iceberg. And on those three look, I think those are actually massive opportunities by themselves. We have a number of announcements here at re:Invent around Q Developer and making developers and their whole life cycle more valuable.

The first generation was just code suggestions, right? Code suggestion is super valuable. It made developers much more efficient being able to code. It turns out, developers, on average, code about one hour a day. The rest of their day is spent doing documentation. It's spent doing writing unit tests. It's spent doing code reviews. It's spent going to meetings. It's spent upgrading existing applications, doing all that stuff.

So as part of that, we're actually launching a bunch of new agents that do all of those things for you. You can type in /test, and it'll actually automatically write unit tests for you. As you're sitting there coding. You can have a Q developer agent, build, write documentation for you as you're writing code, and so you can have really well documented code in your data. You don't have to go think about it. It'll even do code reviews and look for where you have risky parts of your code, where you maybe have open source or parts that you should go look at and think about what the licensing rules are around, where you may want to think about how you're deploying stuff, things that you would expect out of somebody doing a code review for you before you go do a deployment. Q can now do all that for you.

Same on the contact center side, right, we're doing a ton of announcements around Connect, which is our contact center in the cloud offering, making it much more efficient. So for customers to get a ton out of that contact center, all powered by generative AI. And to your point, those use cases get more and more valuable as you add more capabilities.

If you think about where things are going, if you think about how I talked about code generation, moving to a bunch of the value, it's adding agents in there so they can do a bunch of these things right now. It's not just giving you code suggestions. It's actually going and doing stuff for you, right? It's writing documentation for you. It's helping you identify and troubleshoot where you have operations issues. And it says, oh, you have an operations issue. And it can look and understand your whole environment. You can interact with it, and you and Q together can go and look and say, Ooh, it looks like some permissions over here were broken. And if you go fix those,it'll fix your application. So saving tons of time across that whole development life cycle.

As AI gets to be more integrated into the core of what a business the core of what you do, that’s where you get the value.

There's a startup that we work with called evolutionary scale, and they use AI to try to discover new proteins and molecules that may be more applicable to solving certain diseases. Right now, instead of being able to find tens or hundreds of new molecules a year, you can now find hundreds of thousands of different proteins and test and figure out which are the most likely to be successful and get drugs to market much faster. And that's a huge amount of additional revenue.

So if you think about models and capabilities that can do that, whether it's in health care and life sciences, whether it's in financial services, whether it's in automate manufacturing and automation, every single industry, in our view, is going to be completely remade by generative AI at its core. And that's where you where you get that huge value.

Another question about this, I was speaking with a developer friend who said, “Yes, AI can code, the problem is when you trust things to generative AI, something breaks, you've lost the skill set to go in and fix it, because you've relied so much on the artificial intelligence.” What do you think about that? Isn't that a problem?

What’s three times four?

Twelve

You have Excel, but you still know how to multiply.

This is different than multiplication. This is different.

It is different. But the key parts of coding are not the semantics around writing language, right? The key about thinking about how you break down a problem, how you creatively come up with solutions. And I think that doesn't change, right? The tools change. They can make you more efficient, but you're the developer, that core of what the developer actually does is not going to change.

Increasingly, I think developers are going to get to do the things that are exciting. They're going to get to the creative work. They're going to get to figure out how to go solve those interesting problems, and they're going to be able to move much faster, because they don't have to worry about writing documentation. And someday, if it breaks, they probably will know how to write documentation, and we'll figure out a fix that. It’s not rocket science, it's just things they don't necessarily want to do today.

You're a believer in reasoning. AWS has some news this week that you're going to have an automated reasoning test where it checks for hallucinations before an answer goes out. Another issue when it comes to ROI is how can I trust it? It always comes out with wrong answers? So talk a little bit about your announcement and how reasoning might solve this.

Yes, it’s different reasoning than you might be thinking about too. So automated reasoning is a form of artificial intelligence that it's been around for a while, and it's a thing that Amazon has adopted pretty significantly across a number of different places. It uses mathematical proofs to prove that something is operating as you intended. We actually use it internally to make sure when you change permissions, that it's behaving as expected. This AI system has this mathematical proof that can go: Okay, all the places that permissions are applied across the surface area that's too large for you to actually go check everything. It can prove that they're applied in the right way, because it knows how the system is supposed to operate. And it can go kind of mathematically prove, yes, your permissions mean you can access this bucket or you can't access this bucket.

We took that and we said, can we apply that to AI to eliminate hallucinations? And so turns out, not universally, but for limit for for selected use cases where it's important that you get the answer right, you can. And so we do is… Say, you're an insurance company, right? And you want to be able to answer questions about people that say, “Hey, I have this problem. Is it covered?” And you don't want to say yes when the answer is no, or vice versa. And so that is the one where it's pretty important to get that right. What you do is you upload all your policies and all your information into the system, and we'll automatically create these automated reasoning rules. And then there's a process you go through that's a couple minutes, kind of 10,15, 20, 30, minutes, where you, as the developer, answer questions of how it's supposed to interact right you, and you tune it a little bit. Say, yep, that's how you'd answer that type of question, or no, or that's what this means. It'll ask you questions, and you kind of interact with it.

Now if you go ask it a question, you say, Hey, I, you know, I ran my car through my garage door, is that covered by my insurance policy? it'll actually produce a response for you. And then it'll tell you that yes, this is provably correct, that the answer is yes. And here are the reasons why, it’s in the documentation I have and why I feel confident in that. Or it'll tell you actually I don't know the answer. Here's some suggested prompts that I recommend you put back into the engine to see if you can get the answer correct. It'll give you kind of tips and hints on how you can re-engineer your prompts or ask additional questions to come back until you get an answer that's probably correct by automated reasoning.

So by this kind of mechanism, you're systematically able to actually mathematically prove that you got the right answer coming out of this and completely eliminate hallucinations for that area. It doesn't mean that we've eliminated hallucinations — it’s just for that area.

This is similar to what Marc Benioff talked about on the show last week, where he said that, because companies have large stores of information within his platform, agents will be able to go in and pull it out, and then present it, and help create the linkage to go from step a to step b. I think both you and Benioff have the idea that agents will be something I'm going to interact with when I'm speaking with a company, and that's going to happen before consumers get them?

Yeah, I think that that's right. I think that agents are going to be a really powerful tool. Another thing that we're launching this week — one of the things that agents today are quite good at is doing relatively simple tasks, right? What they're very good at, actually, is tasks that are pretty well defined in a particular narrow slice, and go accomplish something.

And so what a lot of people are doing is starting to launch a bunch of agents: One that's very good at going and doing one particular task, another one that's good at another task, another one's a good another task. But increasingly, you actually need those agents to interact with each other, right?

So we have an example in my keynote where if you're thinking about, should I launch a coffee shop? Here, you might have an agent that goes out and investigates what the situation is in a particular location. You might have another agent that goes and looks at the competitors in that area. You may have another agent that goes and does a financial analysis of that particular area, another one that looks at the demographics of that zone, etc.

And that's great. So now you have a dozen agents that go and do a bunch of these things. Saves you some time, but they actually interact with each other, right? Like the demographics may imply you change your financial analysis as an example. And so that's super hard. And then if you want to do it across 100 different locations, see where the best one is. That's also hard to do like and it's super hard to coordinate,

So we launched a multi-agent collaboration capability where you basically have this kind of super agent brain that can actually help collaborate across all of them, help pass data back and forth between them. And so we think that this is going to be a really powerful way for people to really accomplish much more complicated things out there in the world. There's a fundamental model under the covers that's driving a bunch of this reasoning and breaking these jobs into into individual parts, and then the agents go and actually accomplish a bunch of this work.

Let's talk about nuclear energy. You have invested $500 million in a company called X-energy. You're also part of a wave of companies that are effectively reanimating nuclear energy in the United States. Should we really believe in this moment for nuclear? On one hand, it’s cleaner than fossil fuels. On the other hand, we don't really know what happens with nuclear waste. We can't get rid of it.

Yeah, I think nuclear is a fantastic option for clean energy. It is a carbon zero energy that has a ton of potential. And as you look about the energy needs over the next couple of years, and really the next couple of decades, whether it's from technology or broadly in the world, or there's electric cars, or just the general electrification of lots of things in our world, we're going to need a lot more energy.

We at Amazon are one of the biggest investors in renewable energy in the world. In the last five years, we've done over 500 renewable projects where we've added and paid for new energy to the grid, whether they're solar or wind or others. And we'll continue to continue to invest in those projects. I think they're super valuable. And there's probably not going to be enough of those soon enough for us to really get to where we want to get, from a clean energy perspective.

So I think nuclear is a huge portion of that. There's always the fear mongering from like back in the 60s and 70s, of what nuclear used to be. Nuclear is an incredibly safe technology today. It's much different today. Turns out, technology has changed in the last 50 years. It's improved a lot, and so there is a ton of improvements in that space, and we think that it is a very safe, very eco-friendly energy source that is going to be critical for our earth if we're going to keep for our world as we keep ramping our energy needs. You're going to have solar, you're going to have wind, and nuclear is going to play an important role and we're excited about what that potential looks like.

You mentioned X-energy. We do think that, somewhere in 2030 and beyond, these small modular reactors, which is what x-energy builds, are going to be a huge component of this. But today, all these nuclear plants that people build are really large impementations, right? They're multi-billions and billions of dollars to go build these energy plants, they produce lots of energy, which is great, but obviously all that energy is in one location, and then you have to invest in a ton of transmission to get the energy to the actual place you need it to go. And they're big projects.

These small modular reactors are much smaller. You can actually produce them, almost like you produce gas turbines, like in a in a factory type setting, eventually, and you can put them where you need them, right? So you can actually put them next to a data center where transmission is not going to have to be an important factor. And so we think that's a great solve for a portion of the world's energy needs and it's one of the components of an energy portfolio that we're very excited about.

On the economy, AWS had a few quarters of stagnant growth. Part of that was because the economy was in a rough moment. Everybody was looking for efficiency and you made some deals with customers to help get their bills down. How to things look like right now? Is the economy back, or are companies still in efficiency mode?

By the way, it wasn't even just deals, we went and proactively jumped in with our customers and helped them figure out how they could reduce their bills. And we looked where they could consolidate resources, where they could move to cheaper offerings, where they could maybe do more with less. And we were really proactive about helping customers reduce those costs, because we thought, from our view, it was important for them as they thought about how they got their economics in the right place> And it was the right thing to do for them, and built that long term trust.

Now, customers, a lot of them have been optimized right, and there's only so much you can kind of squeeze into an optimized place, and customers are still looking for optimizations, but a lot of that work has been done, and they're using some of that optimization to help fund some of the new development that they want to do. A lot of that is in the area of AI.

So some of those optimizations they did are helping them fund some of that work that's moving more of their workloads to the cloud, that's letting them go and build new AI experiences in AWS, and so that is where you've seen our growth start to come back up as a percentage basis. Some of that is customers leaning into those new experiences and doing some of those more of those modernization migrations.

On culture, Andy Jassy recently emailed the company, and said “There have been pre meetings for pre meetings for the decision meetings, a longer line of managers feeling like they need to review a topic before it moves forward, owners of initiatives feeling less like they should make recommendations because the decision will be made elsewhere.” Was that going on within AWS, and what is the process to change that?

Yeah, I think it's across across Amazon. So it wasn't specific to the rest of Amazon. It was definitely inside of AWS, too. And it's a it's kind of a natural evolution. We have these leadership principles inside of Amazon, we have 16 different leadership principles, things like being customer obsessed and really understanding the customer. And in order to really understand the customer, you've got to be close to the customer. And the more layers you have, the more removed you are from customers.

We went through an area of explosive growth of just the number of people and the size of the company and the size of the business. And so throughout that, we didn't always have the organizational structure exactly right. And so it's, you know, we we believe that a flatter organizational structure is better. The closer you are to the customers, the better decisions you're going to make, the faster decisions are you going to make. And you really want ownership to be pushed down to the people who really are making some of those decisions.

When you have a very kind of hierarchical organization where people don't feel like they have that ownership to make decisions, you go slow. And for us, speed really matters. And it was just highlighting some observations we had where, he's incredibly thoughtful on these points, and which I appreciate. We've had a lot of debate here where nothing was broken, but you could see, like, really early warning signs or stress around it.

And for us, culture is so important, and doing the things in the right way, being that customer obsessed, having the right level of ownership be so important is what makes Amazon so special.

It's not like there was any burning problem. And we obviously could have just said, done nothing, and kind of let things go for a while. But for us…

it's not the Amazon way.

It's not the Amazon way. And so we we're just being proactive, identifying that. Like, hey, look, this is super important for us, and so let's just be aware of it. Let's be upfront about it, think about it, and be very intentional as we think about organizational structures and things, about where we can and I think all of that has been received really well.

So you can be a big company but not have big company culture.

That's right.

Last one before we go. You've said that less than 20% of all workloads have moved to the cloud so far. What is the max number that can be?

I think that that percentage can flip. And it could be 80/20, versus 20/80, where it is today, or even less. I think there's a massive number of applications that just haven't moved.

And if you think about the line of business applications, the workloads that are in telco networks, the workloads that are running inside of hospitals. It's not even just traditional data center workloads, but there's a lot of these other workloads that be much more valuable. They'd be much more connected. They'd be much more able to take advantage of advancements in AI, if they were connected into the cloud world and running there.

And so I think that there's a huge opportunity for us to continue to expand what it means to be in the cloud and to continue to migrate many of these workloads that just haven't moved. And so there's a massive opportunity. Flipping that percentage over time could be an interesting opportunity for us. And the size of the pie is getting bigger too. I think that's the other exciting thing about generative AI, is that to total amount of compute workloads are actually significantly accelerating too

And timeline to flip?

I still think we're still ways out for the whole thing to flip. There's just a massive amount of workloads out there, but we'll keep working on them and keep going as fast as we can.

Matt, great to meet you. Thanks so much for coming on the show.

Yeah. Thank you.

Thank you for reading Big Technology! Paid subscribers get our weekly column, breaking news insights from a panel of experts, monthly stories from Amazon vet Kristi Coulter, and plenty more. Please consider signing up here.

Upgrade To Paid

© 2024 Alex Kantrowitz
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe