DHH · 2025-07-12 · source ↗ · whole document (9)
AWS
`3:19:26`
That if he could just make 2% when the other guy was trying to make 4%, he would end up with all the money. And on volume, he would still win. So I thought that was the operating ethos for AWS. It turns out that's not true at all. AWS by the way operates its almost 40% margin. So just in that, there's a clue that competitors are not able to do the competitive thing we like about capitalism, which is to lower costs and so forth. So the cloud pitch in my optics, it's fundamentally false. It did not get easier first of all. I dunno if you've used AWS recently, it is hella complicated. If you think Linux is hard, you've never tried to set up IAM rules or access parameters or whatever for AWS. — AWS is always difficult. It was always complicated. — Well, I think it's gotten even more difficult. But yes, now some of that is it's difficult because it's very capable and you have a bunch of capacity on tap. And there are reasons I don't think they're good enough to justify how complicated
`3:20:26`
the whole jing-a-ma-jing has become. But what's certainly true is that it's no longer easier. It's not easier to use AWS than it is to run your own machines, which we learned when we pulled out the cloud and didn't hire a single extra person. Even though we operate all our own hardware, the team stayed exactly the same. So you have this three-way pitch, right? It's gonna be easier, it's gonna be cheaper. Certainly wasn't cheaper. We've just proved that by cutting our spend on infrastructure by half to two thirds. And it's gonna be faster. The last bit was true, but way too many people overestimated the value of that speed. If you need a thousand computers online in the next 15 minutes, nothing beats the cloud. How would you even procure that? If we just need another 20 servers, it's gonna take a week or two to get boxes shipped on pallets delivered to a data center and unwrapped and racked, and all that stuff, right? But how often do we need to do that? And how often do we need to do that
`3:21:27`
if buying those servers is way, way cheaper so we get vastly more compute for the same amount of money. Could we just buy more servers and not even care about the fact that we're not hyper optimized on the compute utility? That we don't have to use things like automatic scaling to figure things out because we have to reduce costs? Yes, we can. So we went through this journey over a realization in early 2023 when I had finally had enough with our bills. I wanted to get rid of them. I wanted to spend less money. I wanted to keep more of the money ourselves. And in just over six months, we moved seven major applications out of the cloud in terms of compute caching databases to works onto our own servers. A glorious, beautiful new fleet bought from the king of servers, Michael Dell, who really, by the way, is another icon on my, I saw he just celebrated 41 years in business. 41 years this man has been selling awesome servers
`3:22:28`
that we've been using for our entire existence. But anyway, these pallets arrive in a couple of weeks and we rack 'em up and get everything going. And we were out. At least with the compute part. We then had a long multi-year commitment to S3 because the only way to get decent pricing in the cloud, by the way, is not to buy on a day-to-day basis, not to rent on a database basis, but to bind yourself up to multi-year contracts with computes often a year. That was in our case and with storage is was four years. We signed a four-year contract to store our petabytes of customer files in the cloud, to be able to get something just halfway decent affordable. So all of these projects came together to the sense that we're now saving literally millions of dollars, projected about 10 million over five years. It's always hard. How do you do the accounting exactly, and TOC this, that and the other thing. But it's millions of dollars. But it's not just that. It's also the fact that getting out of the cloud
`3:23:30`
meant returning to more of an original idea of the internet. That the internet was not the sign such that three computers should run everything. It was a distributed network such that the individual nodes could disappear and the whole thing would still carry on. DARPA had designed this such that the Russians could take out Washington and they could still fight back from New York, that the entire communication infrastructure wouldn't disappear because there was no hub and spoke. It was a network. I always found that an immensely beautiful vision. That you could have this glorious internet and no single node was in control of everything. And we've returned to much more of a single node controlling everything idea with these hyperscalers. When US-East-1, the main and original region for AWS goes offline, which has happened more than a few times over the years. Seemingly a third of the internet is offline. Like that in itself is just an insult to DARPA's design. It just detract from the fact