Future of programming

`4:04:56`

I think that's exactly spot on, and this is the fundamental misunderstanding that a lot of programmers have of AI, is that they wish it was deterministic. No, no, no. Temperature is the most beautiful part of the AI setup. The fact that it is not deterministic, the fact that creativity requires little tweaks in the road, that the human brain, if it was perfectly deterministic, would not be the creative brain that it is. And the main charge against AI both-- It's funny. It's a contradiction. There's both the charge that it's not deterministic and therefore bad, and also that it is not creative. It's one or the other, bro. Either it's non-deterministic and therefore creative, therefore, to some degree random, or it's... or it's not creative. So we, we can-- Both of those charges can't be true at the same time, and I have fully come to embrace the fact that the same prompt won't produce the same response every time. You can't step at the same river twice, and it is the most beautiful part of the whole interaction. It's what makes it so human. In fact, this is one of the things I've been thinking about in my own sort of meta-analysis, is just how much of my own personal brain works like next token prediction. I sit down to write an essay. I have this vague, fuzzy premise I want to convey, and I sit down at the keys, and I could not tell you what the next token was gonna be in advance.

The tokens just come out, and I'm astonished how the similarities seem so great, and this is also why I don't have any trouble at all recognizing these breakthroughs of creativity that I've seen with my own eyes that AI is capable of right now. 'Cause my creative moments come through the same kind of next token prediction with a bit of temperature sprinkled in for random effect.

`4:07:15`

I'm already seeing human-like concepts of consciousness. This is what, to me, this remarkable situation, as you say. You give it this vague, fuzzy intent-... and somehow it knows exactly what you mean. Or even better, it improves upon what you said and delivers what you really wanted that you could not articulate yourself. If that's not glimmers of-... consciousness, what is? This was one of the-- I think it was an interview with Sutton or maybe one of the other original guys talking about this sense that intelligence is perhaps not as constrained just to the specific one spot. No, it's the interaction. It's the weights. And I don't know. Whatever's true. What I can observe is that it feels close enough, or not even close enough, identical to the appreciation I have for other forms of consciousness, mostly human. And this is one of the reasons it's so delightful to work with. Now, that's not the same to say as are LLMs the end station for AI. I mean, I know there are people talking about world models and other forms of, of AI, and I'm not an expert in any of that, and I think we should have this productive criticism, as should always be present in science, that for the longest time, neural nets were kind of on the outs, right? Like they were interested in the symbolic route. And- Neural nets walked a couple of decades into darkness without any funding and without any attention, and then suddenly we realized that we had the blind alley and we switched over.

So we should also have the humility. As amazing as the LLMs are now, it could be that they eventually plateau. We haven't seen any evidence of it yet, and I think this is also why we're seeing this absolute gobsmacking levels of investment, because so far the scaling laws are true, and the more billions are poured in, the more intelligence comes out.

`4:09:48`

I find that the cynic approach to this is the meme, where you see the guy in front of the computer who's like, um, "Whatever. Are you gonna destroy the world?" And then the computer says, "I'm gonna destroy the world." And then the guy says, "Oh," right? Like, this is so cynical that they're just aping us. They're just telling us what was in the training data or whatever. When I interact with the agents, I was just doing this yesterday, or this morning, I was working on this, AmaBot system, right? And it's doing this coordination with multiple workers, and I had multiple agents running at the same time. And at one point, the main agent who's doing the coordination steps on the toe of another agent and-... kind of ruins their work. Its ability to convey regret and being sorry was uncanny. Again, I don't know, is that a real projection of true remorse? But I don't know. With most humans-... they express some level of regret. Is that true remorse? Hard to tell, but maybe it also just doesn't matter.