Scaling

`1:14:51`

And some of those limits you hit with dynamic typing are actually, by the way, just limits you hit when you write 5 million lines of code. I think the Shopify monolith is about 5 million lines of code. At that scale, everything breaks because you're at the frontier of what humans are capable of doing with programming languages. The difference in part is that Ruby is such a succinct language that those 5 million, if they had been written in, let's just say Go or Java would've been 50 or 25. Now that might have alleviated some of the problems that you have when you work on huge systems with many programmers. But it certainly would also have compounded them trying to understand 25 million lines of code. — So the thing does scale. That's a persistent myth that it doesn't scale Shopify and others. But Shopify thinks a great example. By the way, I love Shopify and I love Tobi, amazing. — You gotta have Tobi on. — Yeah, for sure. — Just talking to him this morning. — For sure, he's a brilliant. I got to hang out with him in the desert somewhere, I forget in Utah.

`1:15:52`

He's just a brilliant human. And Shopify, shopify.com/lex has been supporting this podcast for the longest time. I don't think actually Tobi knows that they sponsor this podcast. I mean this is a big company, right? — It's a huge company. I think just under 10,000 employees, market cap of 120 billion, GMV of a quarter of a trillion every quarter. — And he's involved with the details though. — He is, very much so. Funny story about Tobi. Tobi was on the Rails core team back in the mid 2000s. Tobi himself wrote Active Merchant, which is one of the frameworks for creating shops. He wrote the Liquid templating language that Shopify still uses to this day. He has a huge list of contributions to the Rails ecosystem, and he's the CEO of the company. Yeah, I think it's just... It's very inspiring to me because it's such at the opposite end of what I like to do. I like to chisel code with my own hands most of the day.

`1:16:53`

He runs a company of almost 10,000 people that is literally like world commerce depends on it. A level of criticality I can't even begin to understand. And yet we can see eye to eye on so many of these fundamental questions in computer science and program development. That is a dynamic range to be able to encompass Rails being a great tool for the one developer who's just starting out with an idea who don't even fully know everything. Who is right at the level where PHP would've been a good fit in those late '90s because yeah, I could probably upload something to an FTP server, and so on. Rails does have more complexity than that, but it also has so much longer runway. The runway goes all the way to goddamn Shopify. That is about the most convincing argument I can make for sort of dynamic range that we can do a lot of it. And even having said that, Shopify is the outlier of course. I don't think about Shopify

`1:17:53`

as the primary target when I write Rails. I think of the single developer. Actually, I do think about Shopify. But I don't think about Shopify now. I think of Shopify when Tobi was writing Snow Devil, which was the first e-commerce store to sell snowboards that he created, that was the pre- Shopify Shopify. He created all by himself. And that was possible because Ruby on Rails isn't just about beautiful code. It's just as much about productivity. It's just as much about the impact that an individual programmer is able to have. That they can build system where they can keep the whole thing in their head and be able to move it forward such that you can go from one developer sitting and working on something and that something is Shopify and it turns into what it is today. When we talk about programming languages and we compare 'em, we often compare 'em at a very late stage. Like what is the better programming language for let's say Twitter in 2009 when it's already a huge success. Twitter was started on Ruby on Rails. They then hit some scaling problems. It was a big debacle at the time.

`1:18:55`

They end up then I think writing it in some other language, which by the way I think is the best advertisement ever for Ruby on Rails, because nothing fucking happened for 10 years after they switched over, right? Essentially zero innovation. Some of that was because they were doing a long conversion, and all of the early success in part came because they had the agility quickly change and adopt and so forth. That's what startups needs. That's what Shopify needed. That's what Twitter needed. That's what everyone needs. And that's the number one priority for Ruby on Rails. To make sure that we don't lose that. Because what happens so often when development tools and program language driven by huge companies is that they mirror their org chart. React and everything else needed to use that is in some ways a reflection of how Meta builds Facebook. 'Cause of course it is. Because of course it's an destruction of that. I'm not saying React isn't a great tool and that can't be used by smaller teams. Of course it can. But it's born in a very different context

`1:19:58`

than something like Ruby on Rails. — Let me say this a small aside because I think we might return to Shopify and celebrate it often. Just a sort of personal note. This particular podcast has way more sponsors and sponsors that want to be sponsors than I could possibly ever have. And it's really, really important for me to not give a shit, and to be able to celebrate people. Like I celebrate people. I celebrate companies. And I don't care that they're sponsoring. I really don't care. I just wanna make that very explicit 'cause we're gonna continue saying positive things about Shopify. I don't care. Stop sponsoring. It doesn't really matter to me. But yeah, I just wanna make that explicit. So, but to linger on the scaling thing with the Twitter and the Shopify, can you just explain to me what Shopify is doing with the YJIT? What did they have to try to do to scale this thing? Because that's kind of an incredible story, right?

`1:20:59`

— Yeah, so one of the great contributions that Shopify has made to the entire Ruby ecosystem, not just Rails but in particular Rails is YJIT. So YJIT is their compiler for Ruby. That just makes everything a lot more efficient and at Shopify scale eking out even a five, 10% improvement in Ruby's overhead and execution time is a huge deal. Now, Shopify didn't need YJIT. Shopify was already running on the initial version of Ruby that was, I think 10 times slower than what we have today. If you look back upon the Ruby 186, that Tobi probably started on just as I started on, and that was enough to propel Shopify to the scale that it has today. A lot of the scaling conversation in is lost in a failure to distinguish two things. Scale is kind of one package we talk about when there are really multiple packages inside of it.

`1:21:59`

One is runtime performance, latency. How fast can you execute a single request? Can it happen fast enough that the user will not notice? If your Rails request takes a second and a half to execute, the user's gonna notice. Your app is gonna feel slow and sluggish. You have to get that response time down below, let's say at least 300 milliseconds. I like to target a hundred milliseconds as my latency. That's kind of performance. How much performance of that kind of latency can you squeeze out of a single CPU core, that tells you something about what the price of a single request will be. But then whether you can deal with 1 million requests a second like Shopify is doing right now. If you have one box that can do a thousand requests a second, you just need X boxes to get up to a million. And what you'll actually find is that when it comes to programming languages, they're all the same in this way. They all scale largely beautifully horizontally. You just add more boxes.

`1:23:01`

The hard parts of scaling a Shopify is typically the program language. It's the database. And that's actually one of the challenges that Shopify has now is how do you deal with MySQL at the scale that they're operating at? When do you need to move to other databases to get worldwide performance? All of these things. The questions about scaling Ruby are economic questions. If we are spending so and so much on application servers, if we can get just 5% more performance out of Ruby, well we could save 5% of those servers and that could filter down into the budget. Now that analysis concludes into basically one thing. Ruby is a luxury language. It's a luxury, the highest luxury in my opinion. It is the Coco Chanel of programming languages. Something that not everyone can afford. And I mean this in the best possible way. There are some applications on the internet

`1:24:01`

where each request has so little value, you can't afford to use a luxurious language like Ruby to program in it. You simply have to slum it with a C or a Go or some other low level language or Rust, talk about line noise there for... — It's like the thrift store of languages. — Exactly, where you need kind of just... you need a very low level to do it. You can't afford to use a luxury language to use to build it with. That's not true of Shopify. It wasn't true of Basecamp even back in 2004. It's not been true of 99% of all web applications ever created because the main cost component of 99% of web applications, it's not CPU course. It's wet course. It's human course. It's human capacity to understand and involve systems. It's their personal productivity. I did a calculation once when someone had for the 400th time said that, "Oh, if you switch from Ruby to some faster language,

`1:25:02`

you could save a bunch of money." And I calculated it out that at the time, and I think the last time I did this calculation was almost a decade ago. We were spending about 15% of our operating budget on Ruby application service. So for me to improve my cost profile of the business by seven percentage points, I'd have to pick something twice as fast. That's quite hard. Versus if Ruby and Ruby on Rails was even 10% more productive than something else, I would move the needle far more because making individual programmers more productive actually matters a lot more. This is why people are so excited about AI. This is why they're freaking out over the fact that a single programmer in Silicon Valley who makes $300,000 a year can now do the work of three or five, at least in theory. I haven't actually seen that fully in practice, but let's just assume the theory is correct if not now, then in six months. That's a huge deal. That matters so much more

`1:26:02`

than whether you can squeeze a few more cycles out of the CPU, when it comes to these kinds of business applications. If you're making unreal engine rendering stuff like Tim Sweeney you had on. Yeah, he needs to really sweat all those details. The Nanite engine can't run on Ruby. It's never going to. It would not meant for that, fine. These kinds of business applications absolutely can and everything that people are excited about AI for right now, that extra capacity to just do more. That was why we were excited about Ruby back in the early 2000s. That was be because I saw that if we could even squeeze out a 10% improvement of the human programmer, we'd be able to do so much more for so much less. — Probably argue about this, but I really like working together with AI, collaborating with AI. And I would argue that the kind of code you want AI to generate is human readable, human interpretable. If it's generating Perl golf code,