Natural language is full of nuances and implicit cultural and domain specific connotations. LLMs work because they have been trained on massive corpus of materials that encode some (but not all) of those hidden meanings.
With specification driven development, the prompts need to be very detailed in order to produce exactly what is required.
skeledrew · 2026-10-23 01:26:00 UTC
Humans also work because they've been trained on a massive corpus of materials, spanning like an average 15 year period or more. And humans too need to be prompted, sometimes everyday, with meetings and whatnot to produce exactly what is required.
LoganDark · 2026-10-23 01:36:15 UTC
I used to do software dev without meetings. It worked just fine. I would send whatever I did once it was done, and get direction on where to take it next. Maybe this breaks down in companies with more than a few leaders, or where the leaders can't otherwise keep in sync.
Also gave me plenty of opportunities to go above and beyond. I miss when that felt safe.
andsoitis · 2026-10-23 01:57:18 UTC
> I used to do software dev without meetings. It worked just fine. I would send whatever I did once it was done, and get direction on where to take it next.
You didn’t collaborate with other engineers? Or what if the spec you were given was unclear in places? Forgive me, but the style you describe sounds incredibly collaborative, super transactional.
LoganDark · 2026-10-23 02:39:55 UTC
Funny story actually, there were a few other engineers but not very much collaboration. They were resistant to sharing source code or algorithm details, so I had to essentially reverse engineer how their other products worked. I was good enough at it that I came up with several features they didn't know were possible. They eventually ended up letting me go and cloned my design, which I thought was weird, but it was developed for them in the first place so not a big deal.
Exoristos · 2026-10-23 02:46:53 UTC
The lengths to which devs will go not to collaborate.
LoganDark · 2026-10-23 03:34:08 UTC
To be fair, it's a very secretive industry, and betrayals are quite common. Worse, a lot of those betrayals are done on pure impulse without even thinking through the consequences first, so you can't actually build incentives against it because they'll just be ignored.
skeledrew · 2026-10-23 02:00:31 UTC
Maybe those were some very detailed requirements you were getting, the deliveries were straight-forward, or you already knew the space pretty well. Or you're the only dev there. Somehow most teams just keep tripping over each other and need near constant management to ensure they're delivering what the stakeholders expect.
LoganDark · 2026-10-23 02:44:00 UTC
They weren't particularly detailed, but I did know the space quite well. I was a heavy user of other products of that type, and knew very well what I wanted out of one.
brabel · 2026-10-23 11:28:32 UTC
The problem is that your case is extremely rare. Most people work on systems that they never would use themselves or even know existed if not for working on it.
LoganDark · 2026-10-23 13:06:10 UTC
Wherever I work, I try to turn the system into something I would use myself. I am not always successful in this, but it drives a lot of my UX decisions, and it seems to work fairly well.
dahart · 2026-10-23 02:09:30 UTC
This depends entirely on how well the requirements have been communicated, right? If you could go for long periods without meetings on large projects, it means either you had great PMs or you’re very good at collecting relevant info and planning in advance. Or, it could mean that your target was quite flexible and open to interpretation. I don’t doubt it, and I’ve known some developers who could do what you did, deliver something great without that much back and forth. I’ve also known many developers (probably most TBH) who will focus on problems the customers don’t have and endlessly go down rabbit holes that have no bottom. Heads down software engineering for big-O and data structures seems to be a lot more fun for some people than fixing weird hard to repro bugs with vague repro instructions that customers complain about. Many very good devs still need to check in and iterate and get some steering, especially if the requirements have not been well or fully explained… which in my own experience has been more or less always. ;)
LoganDark · 2026-10-23 02:46:46 UTC
Maybe my definition of meetings is different than yours. To me, meetings are voice or video calls where multiple people talk about progress and direction, raise issues to the others, etc. and the leaders can make calls across the whole team at once. Staying in constant communication doesn't count as much because it's more async and incremental.
dahart · 2026-10-23 16:59:41 UTC
You’re right, my definition sounds different. Your comment left me with the impression that you meant you weren’t talking to anyone. Meetings in my book are any scheduled discussions, regardless of how many people are involved. 1:1s are still meetings.
LoganDark · 2026-10-23 20:16:55 UTC
1:1 meetings are still meetings, but casual messages about progress and direction (without a scheduled time block or synchronous call or whatever ) don't count to me.
skydhash · 2026-10-23 02:00:21 UTC
> Humans also work because they've been trained on a massive corpus of materials,
Massive? My introduction to programming (from noob to beginner level) has been like only 10 books worth of content. Including some alternative pathways like 3D, security, system administration,... To this day, I don't think I've reached 50 books of materials. And If I counted everything it would be still small.
Human learning is very efficient (not time wise, but cost wise, and even training data wise).
senordevnyc · 2026-10-23 02:05:28 UTC
Only if you ignore the billion years of pretraining…
skeledrew · 2026-10-23 02:24:35 UTC
You're forgetting all the schooling before that, which taught you how to read, do math, etc. And the general daily experiences of just being a human being interacting with the world. Magnitudes more data taken in than any language model.
DonaldPShimoda · 2026-10-23 02:29:44 UTC
> Magnitudes more data taken in than any language model.
Ah, yes, I can fondly recall reading and internalizing every word available on the internet when I was young...
nvme0n1p1 · 2026-10-23 02:35:25 UTC
A picture is worth a thousand words. How much video data was your brain trained on?
skeledrew · 2026-10-23 02:40:38 UTC
You should check out how much raw data just your eyes process for every second they're open (although a significant amount is discarded before it gets to the vision center, but still). Then consider the other senses also constantly doing their thing.
LoganDark · 2026-10-23 02:48:48 UTC
The vast majority of people don't internalize most things. Some internalize even less. It can become a big issue with things like dissociative disorders. (I have some experience with that)
skeledrew · 2026-10-24 01:27:32 UTC
LLMs don't internalize per se (assuming something akin to conscious intent). The foundation of a model is comparable to a human's daily experiences of just being, then the fine-tuning is comparable to formal schooling, and finally the prompting is comparable to meetings for workplace tasks.
LoganDark · 2026-10-24 02:53:35 UTC
I said nothing of LLMs. I don't compare the process of LLM training to any form of human education because they are fundamentally dissimilar. LLMs are engineered mathematics and cannot self-train yet.
skeledrew · 2026-10-24 04:30:43 UTC
There are of course parallels. Anyone who's done ML should be able to see it at least. But yes, there are fundamental differences in methodology due to the mode and level of access: with a human, the only way is to go through the senses and there's no - easy - way to affect the learning substrate; with a LLM, data is directly injected into the substrate, which is also human-architected.
LoganDark · 2026-10-25 03:28:20 UTC
Organic beings have more than just the senses, they also have higher level concepts and complex internal incentives. This is more important than the senses being the only method of access, I think.
Language models currently are mutated in the direction of exactly producing the desired output data, but gradually enough that they generalize across inputs in a way that appears understanding. It is more similar to compression, though, at least before a randomizing sampling strategy is applied during decode.
Anyway, circling back to my original topic: no organic being has the capability to integrate all the data arriving from their senses! They typically get more and more selective with experience, picking out only the pieces of information that are still important. Some are more selective than they're supposed to be, by choice or not (such as with dissociative disorders). In any case, the raw quantity of data coming through the senses becomes almost entirely irrelevant.
groundzeros2015 · 2026-10-23 01:39:13 UTC
Yep, LLMs work to the degree you don’t care shout the choices made. That’s why it’s great for demos and starter templates.
But for mature software eventually every single detail matters.
montag · 2026-10-23 01:46:03 UTC
After reading this essay, I wonder: what if program code was withheld from training data? Then how useful would LLMs be for programming?
hbarka · 2026-10-23 01:22:55 UTC
There’s em dashes in that 1978 article.
timcobb · 2026-10-23 01:34:22 UTC
It can only mean one thing.
gnabgib · 2026-10-23 01:35:48 UTC
Get out the Binford 6000, it's Tool Time?
soulofmischief · 2026-10-23 01:51:40 UTC
Unreadable.
kevin_thibedeau · 2026-10-23 01:57:25 UTC
The typed original [1] has double hyphens that have been converted to dashes. His typewriter (Hermes Ambassador) was fixed pitch.
Wasnt the original famously in written with Fountain pen? And the typed out version reproduced much later?
kevin_thibedeau · 2026-10-23 02:17:10 UTC
He typed them until 1979 when his typewriter stopped working correctly.
voxelghost · 2026-10-23 04:22:10 UTC
Well - in this case the EWD in TFA - has a footer that says.
transcribed by Tristram Brelstaff
revised Fri, 19 Nov 2010
And so seem to be the case with most of the other EWDs I looked at as well.
Jtsummers · 2026-10-23 15:58:57 UTC
That's the HTML version, which Dijkstra did not himself produce. People took the handwritten and typed EWDs and transcribed them for the web. If you look at the PDF scan of the source document for this one it was not transcribed by Tristram Brelstaff, it was typed by Dijkstra.
I'm not native speaker but the TOC smells llm to me
jtbaker · 2026-10-23 14:06:13 UTC
Yeah looks like it. Was just trying to search up a writeup of it, was pretty sure I remembered reading that a while ago.
DonHopkins · 2026-10-23 03:12:29 UTC
Luca Cardelli created the handwritten typeface Dijkstra font in the late 1980s, to mimic the distinctive and precise handwriting of his colleague Edsger W. Dijkstra.
(You have to click on the "Fonts" tab, there's no direct link to it.)
Dijkstra [.ttf (2013 updated version by Reuben Thomas)] [.zip (old PC)] [.hqx (old Mac)] [Preview.pdf]
A beautiful handwriting font of unknown origin, first popularized by E.W.Dijkstra. Excellent for slides that appear to be carefully written, and for short, deep technical notes.
Oh look at the last paragraph on the first page. Manually justified:
In order to make machines significantly easier to use, it has been
proposed (to try) to design machines that we could instruct in our native
tongues. This would, admittedly, make the machines much more complicated,
but, it was argued, by letting the machine carry a larger share of the
burden, life would become easier for us. It sounds sensible provided you
blame the obligation to use a formal symbolism as the source of your dif-
ficulties.
How does one even achieve that with a manual typewriter?
kevin_thibedeau · 2026-10-23 04:24:48 UTC
It's coincidence in this case. Proportional typewriters were made but this wasn't one of them.
archargelod · 2026-10-23 01:59:48 UTC
It's not just em dashes, it's 1: typewriter em dashes aka `--` and 2: usage of them in pairs, like parenthesis, that is completely different from how LLMs use them.
adamrezich · 2026-10-23 02:13:10 UTC
Literally stop the presses
nickm12 · 2026-10-23 02:47:50 UTC
Proof that Djikstra was an AI model!
anonymous908213 · 2026-10-23 01:39:06 UTC
> (You have only to look at the indeed alarming amount of on close reading meaningless verbiage in scientific articles, technical reports, government publications etc.)
It is perhaps for the best that Djikstra did not live long enough to witness this problem become orders of magnitude worse.
jeanpah · 2026-10-23 01:42:18 UTC
This article shows every now and then, and it's a must read. Indeed the natural language programming requires massive clusters of gpus just to make a half baked crud app
nickm12 · 2026-10-23 01:43:32 UTC
I send this article to everyone who tells me how English is now a high-level programming language, the next level of abstraction up the chain from machine language to <their favorite language>. This analogy tends to be followed by "Just like most programmers never look at assembly code, looking at higher-level source code is going to become something only a few people do".
Dijkstra's arguments tend not to change their minds. I think the analogy takes hold because there is a kernel of truth in it. AI models now give us the magical ability to program in natural language, but the implication that an AI model is a form of compiler is deeply flawed. AI models don't replace compilers, they replace software developers. The reason software developers rarely look at assembly code is that compilers (and emulators and VMs) are able to precisely preserve the semantics of the original program.
A natural language spec is not a program, and there are many possible programs that can meet the spec. Software engineers and AI models turn specs into code, but the code is not the spec I think the analogy clouds our thinking about the value of programming languages as a tool for formally describing what the computer should do and enforcing properties of the program that should not be left to either humans or AI models to uphold.
indrex · 2026-10-23 02:05:22 UTC
You mean the English to code layer is stochastic while code to assembly is not?
This stochastic nature existed before AI. Humans are stochastic too. Two engineers convert specs to code in two different ways. I would argue the distribution is wider among humans than among agents.
DonaldPShimoda · 2026-10-23 02:27:24 UTC
Arguing that two humans' distinct interpretations of a single specification is stochastic is patently absurd and requires, I think, some shred of evidence.
I would argue that, in the absence of such evidence, we cannot conclude that the English-to-code conversion process is stochastic; it is merely not deterministic, which is altogether different.
CamperBob2 · 2026-10-23 03:45:38 UTC
Would they write the same code, if given the same spec? No? How about over multiple trials? No? Even the individuals' own code would differ from one trial to the next?
Well, then, their interpretations of the spec are stochastic. Good enough for me, anyway.
DonaldPShimoda · 2026-10-23 06:29:31 UTC
That's not what "stochastic" means. You might want to look that up before you keep (mis)using it in conversations about actual stochastic processes.
CamperBob2 · 2026-10-23 16:10:25 UTC
The term 'stochastic' refers to characteristics or properties that are measured and/or understood on a statistical basis rather than on an individual basis, because they are driven or biased by random processes. What's your understanding of the term, if you're not working from a widely-accepted definition?
DonaldPShimoda · 2026-10-24 00:58:47 UTC
Two people interpreting a specification in different ways isn't stochastic because human processing is not a statistical process in any meaningful way. Or, to put it another way: given a specification, you could not reasonably enumerate all the ways two humans might interpret it, much less assign statistical likelihood to those outputs.
nickm12 · 2026-10-23 02:44:44 UTC
As I said, "AI models don't replace compilers, they replace software developers" so yes, I agree that AI models are like humans.
while both LLMs and humans are not 100% deterministic, I don't think that's the interesting part. It's both true that there are many assembly language programs that implement a C program and that there are many valid programs (in any programming language) that correctly implement a natural language spec. But there's a significant qualitative difference between the two types of translation and that would be true regardless of whether the translations are stochastic.
sorokod · 2026-10-23 07:56:08 UTC
The stement to which we are replying will (has?) launched a 1000 startups.
It would be interesting to state sane measurments that determine a "distance" between a spec and software that claims to implement it
sorokod · 2026-10-23 04:57:21 UTC
I would argue the distribution is wider among humans than among agents.
Please go ahead and do.
chrisjj · 2026-10-23 07:39:58 UTC
> Two engineers convert specs to code in two different ways
So? Two compilers convert source to binary in two different ways.
kasey_junk · 2026-10-23 10:08:10 UTC
The same compiler will convert source to binary in different ways! Hermetic builds are hard!
Even if we allow for hermetic builds in some cases in many many other cases your runtime will recompile your code based on its observations about your environment and that’s frequently what we want.
This fetishism of “determinism” seems crazy to anyone that has had to spend time really digging into modern JIT based compilation.
chrisjj · 2026-10-23 11:13:35 UTC
> This fetishism of “determinism” seems crazy to anyone that has had to spend time really digging into modern JIT based compilation.
Copium.
cowboylowrez · 2026-10-23 11:45:35 UTC
lol I saw the exact same JIT non determinism argument on reddit. The thing is, if JIT cannot replace a hotspot with a 100 percent functionally identical snippet of code then the JIT is just in time wrong and will be either patched, repaired or orphaned.
So all of a sudden all focus is on the word "determinism", and before even going down this route you have to agree on what aspect of the subject are you declaring to be "deterministic?"
Its sort of like chatting with libertarians, as soon as you agree that taxation is theft and democracy is tyranny then all problems melt away!
chrisjj · 2026-10-23 12:08:28 UTC
> So all of a sudden all focus is on the word "determinism"
Yup. Pure obfuscation.
The point is reliability.
cowboylowrez · 2026-10-23 13:43:19 UTC
yeah I've never personally used a language that when compiled, has a runtime that has a roulettes wheel chance of going hacking lol
betterunix2 · 2026-10-23 14:45:47 UTC
Then you have never used C, C++, or several other popular languages that have ambiguous semantics and undefined behavior in the standard.
chrisjj · 2026-10-23 15:16:55 UTC
Show us a C UB that went off hacking a website.
betterunix2 · 2026-10-23 15:35:32 UTC
Does the AT&T long distance network outage count?
cowboylowrez · 2026-10-23 19:29:09 UTC
of course not lol
betterunix2 · 2026-10-23 14:40:37 UTC
"Time flies like an arrow" is a valid English sentence that one person can find numerous ways to interpret. A programming language that has the kind of ambiguous syntax that natural languages have would not be useful.
What makes agents useful is not simply that they interpret natural language, it is that they are interactive and ambiguous instructions can be corrected when the agent's output is not what the programmer intended. That is not the same as writing out a spec and giving it to a developer to implement.
chrisjj · 2026-10-23 15:16:02 UTC
And does the "agent" say when the agent's output is not what the programmer intended?
betterunix2 · 2026-10-23 15:27:54 UTC
No, the programmer does. Human beings are still in the loop.
chrisjj · 2026-10-23 16:27:21 UTC
So might be corrected if the agent's output is found not what the programmer intended.
glimshe · 2026-10-23 03:21:14 UTC
Your acquaintances aren't changing their minds because the arguments are weak. Both the article and your comment assume that the only mode of interaction for natural language is a very high level description of programmer intent.
You can interact with AI and say "code me a calculator" or "now declare a array of booleans to map to each of the keys in the UI and set the corresponding array position to true when the key is pressed".
These commands are written in natural language at such different levels of abstraction that is hard to put them in the same bucket. It would be fair to call the latter an imperative language with the LLM in the role of a compiler. That's very similar to how I use AI.
soco · 2026-10-23 08:09:45 UTC
The latter is also very far from specifying a calculator. And based on what we see in the "Show HN" projects, also very far from what the average person process is of using AI for generating software.
nmeofthestate · 2026-10-23 08:35:08 UTC
If AI is natural language programming then they already had it in 1978, because AI is in fact "asking a programmer to write code for you". And you can even ask them to do it at different levels of abstraction!
nmeofthestate · 2026-10-23 08:38:59 UTC
It's the same as "natural language bricklaying" where you make walls appear by speaking instructions to a brickie whom you're paying.
coldtea · 2026-10-23 08:49:21 UTC
>If AI is natural language programming then they already had it in 1978, because AI is in fact "asking a programmer to write code for you"
That would just be the "I" part, since was missing the A for artificial: you asked a naturally made person.
And yes, nobody argued that they didn't have I in 1978, so I'm not sure what the argument is supposed to prove.
ninkendo · 2026-10-23 11:18:44 UTC
> I'm not sure what the argument is supposed to prove.
I’m guessing it’s supposed to show that if you have to have arbitrary levels of intelligence to interpret natural language, then nothing’s changed between now and 1978 when considering whether “natural language programming” is a thing. Just because the intelligence is artificial now doesn’t really change the answer.
coldtea · 2026-10-23 13:13:29 UTC
>Just because the intelligence is artificial now doesn’t really change the answer.
Not only it changes the aswer, it's literally the whole of the question. When people talked of the posibility or imposibility of “natural language programming” they *always* meant "natural language instructions which *a machine* translates to running code".
They didn't have to spell it out, since nobody doubted a human programmer can write code when given natural language instructions - that was already a thing since ENIAC, if not since Ada Lovelace.
That's what we have now, and that's what changed between now and 1978 when considering whether “natural language programming” is a thing.
ninkendo · 2026-10-23 15:10:50 UTC
These two statements are distinct:
- You can program in natural language
- You can tell an AI what to do in natural language, and it can gather requirements, sketch up a design, write code, search the web for error messages, and iterate on the program until it works.
For certain things people care about, they’re equivalent (I tell the computer what I want and out pops a program) but for certain other things, they’re absolutely not (natural language doesn’t map deterministically into machine instructions.)
When people talked about natural language programming decades ago, they were talking about a system that compiles natural language into machine code. They weren’t talking about a system that autonomously gathers requirements, decides what frameworks to use, searches the web to look up error messages, iterates on ideas, etc… these things are so different from “natural language maps to machine code” that I don’t think we’re going to have a useful discussion if you think “I tell the computer what I want” is the only aspect of this worth looking at.
coldtea · 2026-10-23 16:15:26 UTC
>They were talking about a system that compiles natural language into machine code. They weren’t talking about a system that autonomously gathers requirements, decides what frameworks to use, searches the web to look up error messages, iterates on ideas, etc… these things are so different from “natural language maps to machine code” that I don’t think we’re going to have a useful discussion if you think “I tell the computer what I want” is the only aspect of this worth looking at.
This confuses two things still:
People wanting to have natural language to machine code via AI, just AI that was implemented the way they thought was achievable then (symbolic AI).
People wanting to have natural language style programming language, similar to later AppleScript or some 4GL languages.
Natural language programming, or natural language processing, as it was called, was tied to the first, and the AI research of the time, from the beginning:
So, yeah, they didn't have LLMs, but they pretty much understood such a program or "compiler" would need to understand world facts and natural language to even work.
ninkendo · 2026-10-24 17:55:37 UTC
> Natural language programming, or natural language processing, as it was called
NLP has nothing to do with programming. The Wikipedia article you linked doesn’t even mention programming once. NLP is old-school AI like you get when you try and cram all language into an ontology and try to “parse” it in an unambiguous way. It’s how Siri worked from the beginning, and it’s a complete dead-end. (I have some background here, I was an engineer on Siri for over 10 years.)
The fact that a compiler using old-school NLP would need a complete understanding of the world and all relevant facts in order to make natural language “process-able” is the reason the whole thing was a failure and why LLM’s are needed to make the idea work. You can’t just go from “parsing language” to “make me a TODO list app” without having an arbitrarily intelligent system to understand all the relevant background that such a statement implies.
When people realized human-level intelligence is necessary to make natural language programming work, is basically when the idea died. And now we have LLM’s and (what many would define as) human level intelligence, and its ability to program is just one side effect of many of it being truly artificially intelligent.
nmeofthestate · 2026-10-23 18:08:39 UTC
You misunderstood my comment - I meant that if instructing a programmer to program something is "natural language programming" (what AI agentic coding is), then they had "natural language programming" in 1978. I wasn't saying they had AI in 1978.
TheOtherHobbes · 2026-10-23 12:56:46 UTC
The magic sauce in intelligence is context-awareness. If a prompt is somewhat general ("make me a calculator") an AI should either ask questions, or make reasonable assumptions, or both.
IME LLMs often make reasonable assumptions, but sometimes they don't and you have to backtrack and say "No, what I meant was..."
When the process works it's almost magical because the LLM successfully anticipates your needs, and implements your request in a much fuller way than you might have done yourself.
When it doesn't it makes a mess you have to clean up.
But generally I've been impressed by how well the leading LLMs will Do The Thing from a brief description, interpolating useful and correct details without being told to.
Compilers don't work like that. They're translators, not (entry level) inventors with independent agency.
js8 · 2026-10-23 17:06:46 UTC
Some compilers can do that, but only in a very controlled way. For example Haskell's deriving or inference of the correct method called on typeclass. Or the Prolog/Datalog/SQL - they will also fill in many details on how to arrive at an answer.
alterom · 2026-10-23 15:06:09 UTC
> AI is natural language programming then they already had it in 1978, because AI is in fact "asking a programmer to write code for you". And you can even ask them to do it at different levels of abstraction!
There are a few little differences between human beings and computers that you should know about.
For example, humans are sentient, have agency and can be held accountable.
So when you give humans a bad spec, you can still get what you want, because, with some luck, you're simply delegating the task of thinking and making choices to another human, who, by the very premise, understands the language you are communicating in, and can work out your intents with you.
The whole "this meeting could've been an email" thing, you know.
That meeting could've been a pull request if whoever needed a thing to be done could figure it out on their own.
LLMs can be helpful for figuring things out, sure.
But "sudo make me a sandwich" style of vibe coding is just removing the obstacles to enshrining ambiguities and bad decisions in code.
ragequittah · 2026-10-24 14:49:43 UTC
>But "sudo make me a sandwich" style of vibe coding is just removing the obstacles to enshrining ambiguities and bad decisions in code.
I'm of the opinion that if you think it will stay this way you haven't been paying attention. "sudo find me a security flaw" already embarrasses top security researchers who have been working on the same code for decades. "sudo decompile this program/firmware/game" is doing absolute wondrous things right now even if there are still some flaws. If it can do these things better / significantly faster than humans already, and if it keeps improving at the rate it has been "sudo make me an operating system" that spits out something actually pretty good is probably within a decade.
alterom · 2026-10-25 19:53:18 UTC
Your two examples are taking something that's already built, and are extracting a feature (security flaw) or translating (decompiling).
The Kolmogorov complexity, at best, remains unchanged.
"sudo make an OS" is a different kind of transformation.
It's as much programming as downloading an Ubuntu image, and you don't need to wait a decade to do that.
ernst_klim · 2026-10-23 14:24:14 UTC
> Both the article and your comment assume that the only mode of interaction for natural language is a very high level description of programmer intent.
No, the article explicitly says that the difference is formal vs informal language.
> Programming remained the use of a formal symbolism and, as such, continued to require the care and accuracy required before.
One can define an operational semantics for (at least a subset of) the programming language (even for sloppy one, since they are still very small and restricted). One can prove the semantics preservation theorem formally, or reason somewhat in a similar way informally.
With natural language you kinda lose it, and we don't even touch the fact that models are probabilistic.
Comments
With specification driven development, the prompts need to be very detailed in order to produce exactly what is required.
Also gave me plenty of opportunities to go above and beyond. I miss when that felt safe.
You didn’t collaborate with other engineers? Or what if the spec you were given was unclear in places? Forgive me, but the style you describe sounds incredibly collaborative, super transactional.
Massive? My introduction to programming (from noob to beginner level) has been like only 10 books worth of content. Including some alternative pathways like 3D, security, system administration,... To this day, I don't think I've reached 50 books of materials. And If I counted everything it would be still small.
Human learning is very efficient (not time wise, but cost wise, and even training data wise).
Ah, yes, I can fondly recall reading and internalizing every word available on the internet when I was young...
Language models currently are mutated in the direction of exactly producing the desired output data, but gradually enough that they generalize across inputs in a way that appears understanding. It is more similar to compression, though, at least before a randomizing sampling strategy is applied during decode.
Anyway, circling back to my original topic: no organic being has the capability to integrate all the data arriving from their senses! They typically get more and more selective with experience, picking out only the pieces of information that are still important. Some are more selective than they're supposed to be, by choice or not (such as with dissociative disorders). In any case, the raw quantity of data coming through the senses becomes almost entirely irrelevant.
But for mature software eventually every single detail matters.
[1] https://www.cs.utexas.edu/~EWD/ewd06xx/EWD667.PDF
It shipped as a free PostScript font with NeWS:
https://www.donhopkins.com/home/NeWS/fonts/Dijkstra.ps
https://lucacardelli.name/indexartifacts.html
(You have to click on the "Fonts" tab, there's no direct link to it.)
Dijkstra [.ttf (2013 updated version by Reuben Thomas)] [.zip (old PC)] [.hqx (old Mac)] [Preview.pdf]
A beautiful handwriting font of unknown origin, first popularized by E.W.Dijkstra. Excellent for slides that appear to be carefully written, and for short, deep technical notes.
https://lucacardelli.name/Artifacts/Fonts/Pc/dijkstra.ttf
https://lucacardelli.name/Artifacts/Fonts/Pc/Dijkstra.zip
https://lucacardelli.name/Artifacts/Fonts/Mac/Dijkstra.sea.h...
https://lucacardelli.name/Artifacts/Fonts/Dijkstra.pdf
It is perhaps for the best that Djikstra did not live long enough to witness this problem become orders of magnitude worse.
Dijkstra's arguments tend not to change their minds. I think the analogy takes hold because there is a kernel of truth in it. AI models now give us the magical ability to program in natural language, but the implication that an AI model is a form of compiler is deeply flawed. AI models don't replace compilers, they replace software developers. The reason software developers rarely look at assembly code is that compilers (and emulators and VMs) are able to precisely preserve the semantics of the original program.
A natural language spec is not a program, and there are many possible programs that can meet the spec. Software engineers and AI models turn specs into code, but the code is not the spec I think the analogy clouds our thinking about the value of programming languages as a tool for formally describing what the computer should do and enforcing properties of the program that should not be left to either humans or AI models to uphold.
This stochastic nature existed before AI. Humans are stochastic too. Two engineers convert specs to code in two different ways. I would argue the distribution is wider among humans than among agents.
I would argue that, in the absence of such evidence, we cannot conclude that the English-to-code conversion process is stochastic; it is merely not deterministic, which is altogether different.
Well, then, their interpretations of the spec are stochastic. Good enough for me, anyway.
while both LLMs and humans are not 100% deterministic, I don't think that's the interesting part. It's both true that there are many assembly language programs that implement a C program and that there are many valid programs (in any programming language) that correctly implement a natural language spec. But there's a significant qualitative difference between the two types of translation and that would be true regardless of whether the translations are stochastic.
It would be interesting to state sane measurments that determine a "distance" between a spec and software that claims to implement it
Please go ahead and do.
So? Two compilers convert source to binary in two different ways.
Even if we allow for hermetic builds in some cases in many many other cases your runtime will recompile your code based on its observations about your environment and that’s frequently what we want.
This fetishism of “determinism” seems crazy to anyone that has had to spend time really digging into modern JIT based compilation.
Copium.
So all of a sudden all focus is on the word "determinism", and before even going down this route you have to agree on what aspect of the subject are you declaring to be "deterministic?"
Its sort of like chatting with libertarians, as soon as you agree that taxation is theft and democracy is tyranny then all problems melt away!
Yup. Pure obfuscation.
The point is reliability.
What makes agents useful is not simply that they interpret natural language, it is that they are interactive and ambiguous instructions can be corrected when the agent's output is not what the programmer intended. That is not the same as writing out a spec and giving it to a developer to implement.
You can interact with AI and say "code me a calculator" or "now declare a array of booleans to map to each of the keys in the UI and set the corresponding array position to true when the key is pressed".
These commands are written in natural language at such different levels of abstraction that is hard to put them in the same bucket. It would be fair to call the latter an imperative language with the LLM in the role of a compiler. That's very similar to how I use AI.
That would just be the "I" part, since was missing the A for artificial: you asked a naturally made person.
And yes, nobody argued that they didn't have I in 1978, so I'm not sure what the argument is supposed to prove.
I’m guessing it’s supposed to show that if you have to have arbitrary levels of intelligence to interpret natural language, then nothing’s changed between now and 1978 when considering whether “natural language programming” is a thing. Just because the intelligence is artificial now doesn’t really change the answer.
Not only it changes the aswer, it's literally the whole of the question. When people talked of the posibility or imposibility of “natural language programming” they *always* meant "natural language instructions which *a machine* translates to running code".
They didn't have to spell it out, since nobody doubted a human programmer can write code when given natural language instructions - that was already a thing since ENIAC, if not since Ada Lovelace.
That's what we have now, and that's what changed between now and 1978 when considering whether “natural language programming” is a thing.
- You can program in natural language
- You can tell an AI what to do in natural language, and it can gather requirements, sketch up a design, write code, search the web for error messages, and iterate on the program until it works.
For certain things people care about, they’re equivalent (I tell the computer what I want and out pops a program) but for certain other things, they’re absolutely not (natural language doesn’t map deterministically into machine instructions.)
When people talked about natural language programming decades ago, they were talking about a system that compiles natural language into machine code. They weren’t talking about a system that autonomously gathers requirements, decides what frameworks to use, searches the web to look up error messages, iterates on ideas, etc… these things are so different from “natural language maps to machine code” that I don’t think we’re going to have a useful discussion if you think “I tell the computer what I want” is the only aspect of this worth looking at.
This confuses two things still:
People wanting to have natural language to machine code via AI, just AI that was implemented the way they thought was achievable then (symbolic AI).
People wanting to have natural language style programming language, similar to later AppleScript or some 4GL languages.
Natural language programming, or natural language processing, as it was called, was tied to the first, and the AI research of the time, from the beginning:
https://en.wikipedia.org/wiki/Natural_language_processing
So, yeah, they didn't have LLMs, but they pretty much understood such a program or "compiler" would need to understand world facts and natural language to even work.
NLP has nothing to do with programming. The Wikipedia article you linked doesn’t even mention programming once. NLP is old-school AI like you get when you try and cram all language into an ontology and try to “parse” it in an unambiguous way. It’s how Siri worked from the beginning, and it’s a complete dead-end. (I have some background here, I was an engineer on Siri for over 10 years.)
The fact that a compiler using old-school NLP would need a complete understanding of the world and all relevant facts in order to make natural language “process-able” is the reason the whole thing was a failure and why LLM’s are needed to make the idea work. You can’t just go from “parsing language” to “make me a TODO list app” without having an arbitrarily intelligent system to understand all the relevant background that such a statement implies.
When people realized human-level intelligence is necessary to make natural language programming work, is basically when the idea died. And now we have LLM’s and (what many would define as) human level intelligence, and its ability to program is just one side effect of many of it being truly artificially intelligent.
IME LLMs often make reasonable assumptions, but sometimes they don't and you have to backtrack and say "No, what I meant was..."
When the process works it's almost magical because the LLM successfully anticipates your needs, and implements your request in a much fuller way than you might have done yourself.
When it doesn't it makes a mess you have to clean up.
But generally I've been impressed by how well the leading LLMs will Do The Thing from a brief description, interpolating useful and correct details without being told to.
Compilers don't work like that. They're translators, not (entry level) inventors with independent agency.
There are a few little differences between human beings and computers that you should know about.
For example, humans are sentient, have agency and can be held accountable.
So when you give humans a bad spec, you can still get what you want, because, with some luck, you're simply delegating the task of thinking and making choices to another human, who, by the very premise, understands the language you are communicating in, and can work out your intents with you.
The whole "this meeting could've been an email" thing, you know.
That meeting could've been a pull request if whoever needed a thing to be done could figure it out on their own.
LLMs can be helpful for figuring things out, sure.
But "sudo make me a sandwich" style of vibe coding is just removing the obstacles to enshrining ambiguities and bad decisions in code.
I'm of the opinion that if you think it will stay this way you haven't been paying attention. "sudo find me a security flaw" already embarrasses top security researchers who have been working on the same code for decades. "sudo decompile this program/firmware/game" is doing absolute wondrous things right now even if there are still some flaws. If it can do these things better / significantly faster than humans already, and if it keeps improving at the rate it has been "sudo make me an operating system" that spits out something actually pretty good is probably within a decade.
The Kolmogorov complexity, at best, remains unchanged.
"sudo make an OS" is a different kind of transformation.
It's as much programming as downloading an Ubuntu image, and you don't need to wait a decade to do that.
No, the article explicitly says that the difference is formal vs informal language.
> Programming remained the use of a formal symbolism and, as such, continued to require the care and accuracy required before.
One can define an operational semantics for (at least a subset of) the programming language (even for sloppy one, since they are still very small and restricted). One can prove the semantics preservation theorem formally, or reason somewhat in a similar way informally.
With natural language you kinda lose it, and we don't even touch the fact that models are probabilistic.