Future of programming

`4:07:15`

I'm already seeing human-like concepts of consciousness. This is what, to me, this remarkable situation, as you say. You give it this vague, fuzzy intent-... and somehow it knows exactly what you mean. Or even better, it improves upon what you said and delivers what you really wanted that you could not articulate yourself. If that's not glimmers of-... consciousness, what is? This was one of the-- I think it was an interview with Sutton or maybe one of the other original guys talking about this sense that intelligence is perhaps not as constrained just to the specific one spot. No, it's the interaction. It's the weights. And I don't know. Whatever's true. What I can observe is that it feels close enough, or not even close enough, identical to the appreciation I have for other forms of consciousness, mostly human. And this is one of the reasons it's so delightful to work with. Now, that's not the same to say as are LLMs the end station for AI. I mean, I know there are people talking about world models and other forms of, of AI, and I'm not an expert in any of that, and I think we should have this productive criticism, as should always be present in science, that for the longest time, neural nets were kind of on the outs, right? Like they were interested in the symbolic route. And- Neural nets walked a couple of decades into darkness without any funding and without any attention, and then suddenly we realized that we had the blind alley and we switched over.

So we should also have the humility. As amazing as the LLMs are now, it could be that they eventually plateau. We haven't seen any evidence of it yet, and I think this is also why we're seeing this absolute gobsmacking levels of investment, because so far the scaling laws are true, and the more billions are poured in, the more intelligence comes out.

`4:09:48`

I find that the cynic approach to this is the meme, where you see the guy in front of the computer who's like, um, "Whatever. Are you gonna destroy the world?" And then the computer says, "I'm gonna destroy the world." And then the guy says, "Oh," right? Like, this is so cynical that they're just aping us. They're just telling us what was in the training data or whatever. When I interact with the agents, I was just doing this yesterday, or this morning, I was working on this, AmaBot system, right? And it's doing this coordination with multiple workers, and I had multiple agents running at the same time. And at one point, the main agent who's doing the coordination steps on the toe of another agent and-... kind of ruins their work. Its ability to convey regret and being sorry was uncanny. Again, I don't know, is that a real projection of true remorse? But I don't know. With most humans-... they express some level of regret. Is that true remorse? Hard to tell, but maybe it also just doesn't matter.

`4:11:26`

There totally is that. I see it all the time in, in the traces where it just reasons wrong, right? It went down one alley and then it realizes that was a blind alley and it's gotta go back, and so they're like, "Oh yeah, I got this wrong," and then it'll provide the reasons for why it think it got these things wrong. And you're like, this is uncannily human-like. This is indistinguishable from the kind of consciousness you would recognize in a human. Again, does that mean that it truly is or isn't? I don't need the answer to that question actually to be able to appreciate what we have in this moment.

`4:13:04`

I think where it's gonna be harder is once we put these systems into-

`4:13:08`

Yes. Humanoid robots. Which just happens to be the plot of every sci-fi movie ever, which is also an amazing premonition, not only the Terminator movies and Skynet and that going wrong, but then also just Blade Runner, right? Insertion dates, expiration times, Tyrell Corporation.