Future of programming

Like I've been programming in English for the last three months. I've been reading a lot of code, but I'm programming in English. I'm telling the computer what to do, and I'm using natural language, and it is shockingly even more delightful. If there is one programming language more beautiful than Ruby, it is the English language.

`4:02:36`

Beautiful. It's beautiful. And I mean, the reason I say this in part is I always loved writing. I always just loved the English language-... for the sheer beauty of it for the sheer intricacy, for the depth of it. I mean, Ruby is a very expressive programming language, but it can't hold a candle to English. I mean, all the poetry and literature in the world that's been expressed through the English language. I mean, I like a beautiful code poem, but I mean, the real deal in English is just on a different level.

`4:04:56`

I think that's exactly spot on, and this is the fundamental misunderstanding that a lot of programmers have of AI, is that they wish it was deterministic. No, no, no. Temperature is the most beautiful part of the AI setup. The fact that it is not deterministic, the fact that creativity requires little tweaks in the road, that the human brain, if it was perfectly deterministic, would not be the creative brain that it is. And the main charge against AI both-- It's funny. It's a contradiction. There's both the charge that it's not deterministic and therefore bad, and also that it is not creative. It's one or the other, bro. Either it's non-deterministic and therefore creative, therefore, to some degree random, or it's... or it's not creative. So we, we can-- Both of those charges can't be true at the same time, and I have fully come to embrace the fact that the same prompt won't produce the same response every time. You can't step at the same river twice, and it is the most beautiful part of the whole interaction. It's what makes it so human. In fact, this is one of the things I've been thinking about in my own sort of meta-analysis, is just how much of my own personal brain works like next token prediction. I sit down to write an essay. I have this vague, fuzzy premise I want to convey, and I sit down at the keys, and I could not tell you what the next token was gonna be in advance.

The tokens just come out, and I'm astonished how the similarities seem so great, and this is also why I don't have any trouble at all recognizing these breakthroughs of creativity that I've seen with my own eyes that AI is capable of right now. 'Cause my creative moments come through the same kind of next token prediction with a bit of temperature sprinkled in for random effect.

`4:07:15`

I'm already seeing human-like concepts of consciousness. This is what, to me, this remarkable situation, as you say. You give it this vague, fuzzy intent-... and somehow it knows exactly what you mean. Or even better, it improves upon what you said and delivers what you really wanted that you could not articulate yourself. If that's not glimmers of-... consciousness, what is? This was one of the-- I think it was an interview with Sutton or maybe one of the other original guys talking about this sense that intelligence is perhaps not as constrained just to the specific one spot. No, it's the interaction. It's the weights. And I don't know. Whatever's true. What I can observe is that it feels close enough, or not even close enough, identical to the appreciation I have for other forms of consciousness, mostly human. And this is one of the reasons it's so delightful to work with. Now, that's not the same to say as are LLMs the end station for AI. I mean, I know there are people talking about world models and other forms of, of AI, and I'm not an expert in any of that, and I think we should have this productive criticism, as should always be present in science, that for the longest time, neural nets were kind of on the outs, right? Like they were interested in the symbolic route. And- Neural nets walked a couple of decades into darkness without any funding and without any attention, and then suddenly we realized that we had the blind alley and we switched over.