Best AI coding models

`2:24:28`

I have not had a single problem on my Linux machine since the beginning of this year that an agent could not diagnose. That was not true a year and a half ago. A year and a half ago, I was still searching on forums-... to find answers to esoteric questions. Now the agents know the source code of not just the Linux operating system, but every single piece of software I have on that box. The agent has access to the source code of all of it. In fact, this was one thing I built in just before we, uh, we shipped. So once you set up your default agent-... Omarchy Quattro has a crash watcher. If any app on your machine crashes, it'll pop up a little thing, ask you whether you want your AI to diagnose the problem.

`2:25:14`

Unbelievable. I've seen things where there's an error in some subsystem. Some, some... Not even Linux, just some app I installed. The agents start digging through the logs and looks up systemd log. Then it checks out the damn source code. Of the application that crashed, pins down that it's in this Rust file line 472 that there's an unbounded, unwrapped variable that overflowed or whatever it is. Then offers you whether you wanna file a bug report with all this detail. I'll give you one amazing anecdote. So I was working with this guy JDX, who's working on mise. This, this package manager for fast-moving development tools. This is how we manage all the agent software and so forth. Normal package managers are not built to be updated seven times a day. And all of these agent harnesses are updated about seven times a day, so we needed an out-of-band package manager, and mise just turns out to be perfect for this. Anyway, it had a small bug. There was some race condition which agents are very good at finding because when you start running multiple agents at the same time, they will suss out all these race conditions you have in your underlying infrastructure that was never triggered by a human trying to do manual thing. So it finds this issue, right? I tell I tell the agent, "Hey, can you post this as a bug report to JDX's GitHub?" And unfortunately, right before that, I had had to do a QA run on Omarchy itself with eight different agents. It had found 28 real issues-...

that it needed to file. Well, it went to GitHub and tried to file all 28 issues at the same time, which it did in about, uh, I don't know, 12 seconds. GitHub, not unreasonably, marked that as probable spam and banned-... my Omarchy bot. And then I couldn't do that. So I was blocked from the bot having access to GitHub, so I just told it, "Hey, do you know what? Just email JDX. Here's his email address." I had already set it up with hey.com-... an email address. We have a CLI that's in beta right now. So I set it up with that so it can send to email. That's how it gives me reports about outstanding issues and PRs. So it sends, um, JDX this email about the bug it found, and he was like, "Wait, this is the first time I've gotten a bug report in unreleased software." Because it had downloaded the source code to mise, saw that he had already fixed the bug, but that it would not fix the problem entirely in software they had not shipped yet. Pinpointed the problem, and he was like, "Damn it-... I got a bug report before we even cut a release."

`2:27:54`

... believable AGI levels of mind-blowing stuff.

`2:28:10`

They are so incredibly good at this, and this is actually interesting because I was on the perhaps same side of that, a little skeptical about can it reason about all these things. And Mikhail, who's the CTO at Shopify, ran a scientific study on this. They had... And this was, I think, late last year or early this year, where he had agents go back through all the incidents, both outages and, and other problems that Shopify had had in production, trace that back to the PR that was merged, and find out whether PRs that had been reviewed by human or PRs that had been reviewed by agents were of higher quality. Well, lo and surprise-... the PRs that have been reviewed by agents caused far fewer issues in production. And this was with models we had six months ago. At this point, 100% in the majority of domains we work in today, agents are better at finding bugs.