Best AI coding models

`2:31:29`

I kid you not, in just under 45 minutes, it was like Am I heard all? I'm finished. I've checked everything. I have reduced the startup time from 86 milliseconds to two milliseconds. I have sped up the execution time by 9.6 times, I believe it was. The executable is three megabytes. Do you wanna run it?

`2:32:02`

... and I don't know why I'm surprised, because this translation job is something we've known for a while that AI is pretty good at, but it was still staggering to me that I could one-shot a full translation of a Python library I had been using for a year, that others had been using for much longer, and turn it into a Rust executable without knowing any Rust, without looking at the Rust code at all, and produce this executable that I then told the agent right after, "This is great. Ship it." It packaged it up as a new package. It told me, "What do you want it to call?" "Uh, let's call it TTFX. Let's create a new Git repo." It sets up the Git repo. "Let's create a new package, build package for our build system." It puts that up. "Let's push it out. Let's open the pull request to the Omarchy itself so we switch around from TTE, the Python implementation, to TTFX." It does all of it, and I'm just sitting there. And again, I'm already at this point fully delirious with agent acceleration, and I still had to lean back and I go like, "This is AGI, isn't it? This is what AGI looks like. If we have... If these moments are just what it is all the time, this is AGI."

`2:33:21`

I did it on all the models. First, I did it... Actually, funny thing. So I ran out of Fable tokens about two-thirds through, and it just automatically switched over to Opus 5 and kept going and finished the job. Part of the reason why I think it was able to do that was that the first thing Fable did was create a plan, and it was a really detailed plan. I think it had eight separate steps of, "Here you do it, and here, how you analyze it, and here you run the effects," and so on. I didn't review the plan at all. I didn't change the plan. It just made the plan so Opus could take over. And then I thought, "Well, if Opus can finish the job, maybe some of the other agents could finish it, too." So the first thing I did was I gave it to... I think I gave it to Sol. And Sol finished the job, too. It took twice as long, so it took, I think, about an hour and a half. But here's the kicker. The per token cost, I didn't pay per token. I have a max subscription to Claude, right? So it did it within the subscription. It used all my tokens, so I had to switch over to Opus 5. But if I had paid per token, it would've been 550 bucks, I think, to do the whole thing. And I thought for a second, "Holy shit. What a steal." If I personally had to learn Rust well enough to be able to do this translation, I'm looking at a nine-month job here. I can pay 500 bucks to have this translation happen, and suddenly I get a 10x execution speed up. This is ama- I would totally pay 500 bucks for this.

But competition. So I give it to Sol. Same plan. To be fair, I didn't ask Sol to do the plan. I just took the Fable plan, gave it to Sol. Sol, in an hour and a half, and I think $46 worth of token, repeated the task. Did the same thing. And I thought, "Well, blimey, that's amazing." Then I got greedy. So I asked GPT Luna, which is this crazy cheap model that OpenAI has as well, "Can you do it?" Absolutely not. First of all, it didn't even wanna start the task. I think something happened I don't know. I think it was in the spring, where we didn't need these slash goal things anymore. The agents could just automatically keep going in a loop if you told it not to stop-... and so forth. So Sol could do that. Fable could do that. But Luna couldn't. So I had... I think I did 12 prompts. Kept telling it to do it, and eventually I sorta got it started. The first thing it did was to cheat. So the first thing it, it looked outside its own directory, saw that there was already another imple- implementation, and just did a short wrapper around that and said, "I'm done." Hilarious. But it couldn't finish because it made just a couple thing. But I mean, okay, so it can't do that. Then I gave it to Grok-... 46. And I had used Grok 45 a little bit, and I thought like, "Ah, I mean, it's cool that there's others trying, but, like, I'm not gonna use it," because it felt quite far behind. Grok 46 fucking completes the task. 10x speed up, same size executable.