It’s interesting how AI may both raise and lower the quality of software. It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal. Time is much more important for a human developer with a salary.
Supermancho · 2026-08-27 18:14:31 UTC
I don't care if you call it an over-engineered looping machine or what, there are concrete benefits to using LLMs for this. They work faster than developing your own looping algorithm and more often produce useful results than not.
saghm · 2026-08-27 20:24:31 UTC
It's not even like fuzzers are valuable because of the process they use specifically either; the value is that they produce a concrete input that you can use as a reproducible test case at that point. The value could be produced by gazing into a crystal ball for all I care, as long as I can use what it gives me to reproduce a bug.
dmix · 2026-08-27 18:18:04 UTC
Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.
hombre_fatal · 2026-08-27 19:44:41 UTC
The missing part of this is that verifying the bug with LLMs is also easy, and so is adversarially reviewing the proposed fix with LLMs.
The only thing left for you to do should be directional decisions. The LLMs should pause and rope you in if the fix involves directional/invariant changes.
nonethewiser · 2026-08-27 19:49:00 UTC
No one can keep up with the volume of code AI produces.
We wont stop using AI.
We will use AI to check AI.
Of course this is crazy, but it will also unlock pretty insane scaling and productivity and ultimately we will manage it on either end via requirements and tests.
krona · 2026-08-27 19:54:31 UTC
You're suggesting that LLMs get better at fixing bugs/vulnerabilities, but at the same time stop getting better at finding them? What if this difference is inherent and essential?
TacticalCoder · 2026-08-27 20:21:01 UTC
> You're suggesting that LLMs get better at fixing bugs/vulnerabilities, but at the same time stop getting better at finding them?
Are you implying that all code writing by LLMs atm is bug-free?
krona · 2026-08-27 21:26:50 UTC
Absolutely not. By most accounts they're terrible at fixing anything other than trivial bugs in complex codebases e.g. Linux kernel, but they're much better at finding them.
a2ff6eeb0 · 2026-08-28 02:43:15 UTC
So you put it in a loop and tell it to find the bugs in the code it wrote. What's the issue?
krona · 2026-08-28 06:34:21 UTC
This is a self confession if I ever saw one.
a2ff6eeb0 · 2026-08-28 12:50:54 UTC
I absolutely do this. It works great.
Planktonne · 2026-08-28 09:07:41 UTC
What value are you providing in this scenario?
a2ff6eeb0 · 2026-08-28 12:51:05 UTC
Manual testing, and making sure that the AI didn't create so many bugs.
But, to the underlying question, obviously as we automate more and more of our work, of course we provide less and less value. We're heading towards a future where selling thought for money isn't going to work so well.
harambae · 2026-08-27 20:03:12 UTC
It's mostly (not entirely, but mostly) finding security issues in old human-written code. It'll eventually start running out of those.
From that standpoint, it's not a crazy setup security-wise. Maybe still crazy for development.
stefan_ · 2026-08-27 20:20:49 UTC
You can point AI at any AI produced code and ask it to review it, get back 10 bullet points and a few pages of prose. And the fun part is, you can do that over and over and over again!
bilalq · 2026-08-27 20:55:33 UTC
This happens all the time. Yesterday, I ran into an especially egregious case.
I had Fable add a new subcommand to our internal CLI tool. I reviewed and tested it locally and had to suggest several fixes that I feel like I wouldn't have had to tell a human senior engineer to do. When it finally submitted the PR, I had it on a loop waiting a few minutes for comments on the PR, then assessing/addressing/replying-to/resolving them, and then repeating again until all AI reviewers were okay with it. It ended up going through dozens of revisions and ended up with 160 comments left on the PR.
CPLX · 2026-08-27 20:50:25 UTC
In fairness at root this has been going on for awhile. No one can keep up with the volume of machine code that modern more abstracted codebases produce.
We didn't stop using syntactic programming languages we used code to check code.
Not sure it's really crazy at all. It's been an abstraction for programmers probably since we stopped soldering transistors to each other.
ldng · 2026-08-28 00:31:12 UTC
There is a MAJOR difference between predictable generated machine code and Russian Roulette code generator.
CPLX · 2026-08-28 00:32:01 UTC
Of course there is.
But if you don’t actually read it…
adamddev1 · 2026-08-27 21:09:12 UTC
> it will also unlock pretty insane scaling and productivity
Insane scaling of bloat, bugs, and technical debt I'd say.
> We will manage it on either end via requirements and tests
It is so crazy that this is being touted as a sane strategy. When I was a much worse programmer, I tried to write a big complicated string manipulation function to take two types of scripts in a language and add diacritics. I had the requirements very clear. I had the tests very clearly with all the edge cases. But I didn't have a good and clear picture of how to attack the problem which was quite novel for me. As I got closer to passing all the tests it got exponentially more unruly and confusing. And nearing the end I was frantically changing little bits here and there wincing and praying and hoping the tests would pass. "Please work! Come on!" Then when I got close enough, I could never ever think about touching that mess again.
I was a below average programmer then throwing myself at some novel problem I didn't understand. Throwing LLMs that produce below average code at novel problems and relying on tests and requirements is not where we want to go to make real progress.
(Years later after much learning and coding myself I was able to redo the function in a totally different way. This time I actually understood how to attack the strange problem and made something clean, clear, and robust that just worked. The tests then become a secondary guardrail, not the main force of correction.)
We are seeing such a massive regression from what we've learned over the years of CS.
shiandow · 2026-08-27 21:21:03 UTC
I think all code is technical debt in a way. Good code is a necessary evil, bad code is more evil than necessary.
Generating code automatically when you're not even quite sure what it is or even should be doing is insanity.
bonoboTP · 2026-08-27 23:06:09 UTC
You shared a story of a novice incompetent human programmer and this should tell us that AI is bad at coding.
nextaccountic · 2026-08-27 23:26:20 UTC
I'm not so sure LLM code today is below average. There was a time that things posted to dailywtf were normal everyday stuff
krupan · 2026-08-28 00:35:53 UTC
Sorry, no, they wouldn't have been WTF's if they were normal
>Insane scaling of bloat, bugs, and technical debt I'd say.
You just described every legacy codebase. Many of which are widely used and do a lot of sales. You dont need a clean codebase to have a valuable product.
>It is so crazy that this is being touted as a sane strategy.
Re-read what I said. I literally called it crazy.
It is the same dynamic that gave us customer service from some call center in India. Why would companies do this? Customer service got worse. Are they stupid? No, it's just worth it. The quality goes down but the business can scale more so it doesnt matter.
AI will absolutely be good enough at doing things that we'll happily accept some jankiness at times so that we can devote an extra 3000 hours per year per person to other things.
Im not even suggesting its a good thing. I just think the incentive structure dictates it. You're not going to have time to maintain a small slice of some service by hand.
kayamon · 2026-08-28 02:16:58 UTC
Volume..... <sigh>
It used to be considered a quality of good code that there would be less code, not more.
Some people always tryin to get the highscore on golf.
adrianN · 2026-08-28 04:00:39 UTC
You can have both less code per problem and more code overall when you make problem solving cheap enough.
black_knight · 2026-08-27 20:37:32 UTC
This is where I believe strong typing (like, Haskell-strong or stronger) and functional programming in general will be a win. The confidence I have that my fixes are localised when fixing Haskell code is infinitely stronger than fixing even Java, not speak about C, code.
fouronnes3 · 2026-08-27 20:45:59 UTC
What's stronger than Haskell?
theLiminator · 2026-08-27 20:47:18 UTC
Perhaps coq/agda/idris/etc.
astrange · 2026-08-27 20:47:56 UTC
Anything with ranged numeric types. Like everyone's favorite functional programming language, Ada.
black_knight · 2026-08-27 20:54:02 UTC
Or Liquid Haskell.
ghaslt · 2026-08-27 20:58:18 UTC
This issue raises SIGFPE. Ada would raise Constraint_error, which is easier to catch than a signal, but still occurs at runtime.
You need range proofs to be 100% safe, and then you can as well use the regular type because invalid values will not occur.
black_knight · 2026-08-27 20:49:53 UTC
Dependent types is one possible direction. Not sure when a language with dependent types will arise which will be useful for making real programs.
Agda is the most mature dependently typed programming languae (having been around since the 90s – it is basically Haskell on steroids), but has a more proof-assistant flavor than an actual programming language flavor. Opus & Fable write Agda quite well, so LLMs can understand dependent types.
TheGoddessInari · 2026-08-27 20:53:34 UTC
Lean 4, Idris 2.
theLiminator · 2026-08-27 20:47:05 UTC
Imo, formal methods like more expressive/stricter type systems are key to making LLM generated code successful. Of course models will get better, but trusting the output will become much easier with a type system that proves more properties.
astrange · 2026-08-27 20:47:37 UTC
Haskell's type system would not easily prevent this bug. It's not good at numeric/logic issues like that. When people say "Haskell makes it impossible to write bugs" they mean "Haskell has enums" (ADTs).
black_knight · 2026-08-27 20:51:56 UTC
I am not claiming you cant write buggy code in Haskell! But following good functional style, your bug will more likely be compartmentalised, and fixing it will not break some other part of your program.
StilesCrisis · 2026-08-27 22:58:48 UTC
You can write good functional code in many languages. (Even C++!)
black_knight · 2026-08-27 23:01:40 UTC
Sure! I have done my fair share of pretending Java and C++ support my functional style. But at the end of the day, you have better support for writing that style in a real functional programming language. And I wonder how well one can enforce a functional style in say Java or C++ upon the LLMs. Who knows, they might be great at it?
_jackdk_ · 2026-08-27 21:02:40 UTC
Liquid Haskell might require you to prove that the divisor is nonzero, but even in standard Haskell there's common idioms for ensuring that a list is non-empty (data NonEmpty a = a :| [a]) or that text is non-empty (newtype NonEmptyText = NonEmptyText Text, with non-exported constructor, helpers like make :: Text -> NonEmptyText, or more advanced tricks like https://exploring-better-ways.bellroy.com/haskell-koan-type-... ).
The big problem preventing this approach from working for numbers is that it's just so cumbersome there. Most of this is because all the arithmetic operators are bundled into a single Num typeclass, and `fromInteger :: Num a => Integer -> a` has a type that's impossible for a "non-zero number" wrapper to satisfy.
inigyou · 2026-08-27 21:56:43 UTC
OOP has those too, and they're very annoying.
nh2 · 2026-08-27 22:19:15 UTC
In Haskell they are a little less annoying. It is just easier to reason about (including proving) pure functions.
inigyou · 2026-08-27 22:44:30 UTC
I meant the constrained types by hiding the constructors. Super annoying, not automatically convertible, in Haskell you have to remember what the fake constructor is called, and write it every time you use it, but at least it's efficiently implemented with newtype, unlike the Java OOP version. Think about writing a value with several nested constrained types, like NonEmptyListOne (makeNonZeroNumber 42, 'h' `NonEmptyString` "ello world"). It's just really annoying.
_jackdk_ · 2026-08-28 00:51:49 UTC
The blog link I mentioned avoids this cost with literals, by providing using a required type argument to check the string length at compile time without TH. It requires a relatively recent GHC:
make :: forall symbol -> (IsNonEmptySymbol symbol) => NonEmptyText
type family IsNonEmptySymbol symbol :: Constraint where
IsNonEmptySymbol "" = Unsatisfiable (Text "Expected a non-empty string")
IsNonEmptySymbol _ = (()::Constraint) -- empty constraint is always satisfied
black_knight · 2026-08-27 22:19:43 UTC
Definitely room for improvement on Haskell's standard library when it comes to the number-related type classes. Modern Haskell could do very well in this area with a good type-class redesign in this area. The issue I think is that this would invalidate a lot of existing code, relying upon that. But you can already replace Prelude with something else in your own code if you want to.
rootnod3 · 2026-08-28 02:34:52 UTC
I think Idris has a better chance there.
tome · 2026-08-28 07:07:12 UTC
People don’t say "Haskell makes it impossible to write bugs"! You may have heard "if it compiles it works" which is somewhat tongue in cheek, but also true for a sufficiently loose interpretation of "works" in a way it is not true for languages with a less strong and flexible type system.
UltraSane · 2026-08-27 22:29:51 UTC
Even Lean 4 strong typing
deepsun · 2026-08-28 06:11:34 UTC
You haven't mentioned the dynamic typed languages that I believe should die -- Python and Javascript. The only good use case for dynamic typing is notebooks (niche of R lang) where you're throwing out the code you just wrote after getting the result you wanted from it.
sadfgknerknksdf · 2026-08-28 00:22:14 UTC
If finding the bugs with LLMs is easy. Then making sure it doesn't break something else is just LLMs finding no bugs. Easy.
BikiniPrince · 2026-08-28 00:29:48 UTC
That hasn’t been that bad. My real issue has been the time sink involved in following along with the maintainer and jumper through their hoops. Even after I demonstrate a flaw and a potential fix. My schedule is just so busy I need to pencil in time to deal with them.
eviks · 2026-08-27 18:18:23 UTC
But what's your expectation of the net?
shevy-java · 2026-08-27 18:45:53 UTC
I dislike AI, but if AI finds real bugs then this is in my opinion objectively a positive thing. Of course the question is what constitutes a real bug.
pixl97 · 2026-08-27 18:49:15 UTC
Unfiltered models will help build exploits for the bugs they find, so there is some means of measuring their efficacy.
klipt · 2026-08-27 20:05:46 UTC
If you're just talking about security bugs.
There are also non security bugs that don't have exploits but just make the user experience worse.
hn_submit · 2026-08-27 20:01:40 UTC
A.I. is useful for this. But it would be even more useful if all new code were written in Rust or some other memory-safe language.
A.I. could also be used to port C/C++ codebases to Rust, which isn't economically feasible at the moment.
senderista · 2026-08-27 20:06:24 UTC
AI will have plenty of security bugs left to find in Rust codebases.
Spivak · 2026-08-27 20:27:00 UTC
I mean I get the sentiment but Rust won't save you against division by zero, it'll just panic at runtime like every other language.
Gigachad · 2026-08-27 23:07:10 UTC
From a security perspective, panic at runtime is not that bad for security. Much better than continuing to run with undefined behavior. If someone sends a malformed video in and it crashes the ffmpeg process you can just log it and restart it. Vs potentially exploiting the system.
Sharlin · 2026-08-28 09:21:34 UTC
The Rust standard library has `NonZero<T>`, which, if used, at least forces you to consider what you initialize it with. Doing
let foo = NonZero::new(unvalidated_input).unwrap();
is at the very least a big red sign that stands out in the code and should fail code review.
evenhash · 2026-08-27 19:03:59 UTC
> It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal.
No big deal? It’s not like it’s free… tokens cost money.
rogerrogerr · 2026-08-27 20:02:19 UTC
Often rounds to free compared to human costs.
UltraSane · 2026-08-27 22:39:14 UTC
When talking about LLM tokens the cost is almost always being implicitly compared to very expensive human developer time.
simonjuk · 2026-08-27 21:05:34 UTC
In my experience, there are two ways to use AI: speed or quality. Speed is where you give the AI a task to do and you review it; quality is where you write the code yourself and you get AI to review it. Both are valid for different situations.
merb · 2026-08-27 21:26:00 UTC
My plan for bigger things is mostly:
Generate multiple solutions- they do not to work 100% correctly.
And than I check which I would prefer. Which is more to our applications taste.
And than I would take the vibe output as a kind of a ‚plan‘ which I use to implement but not follow 100% and at the end I take my solution and review it.
I gain speed with that because I often can quickly see the pros and cons of a solution way better than when I would manually do it and hang on a major roadblock and also I even see such roadblocks in the vibe output - it’s mostly the part with an unnecessary amount of new code that looks nonsensical.
UltraSane · 2026-08-27 22:36:38 UTC
Using a LLM whose output is slowed to the rate of a human programmer as a pair programming partner is a very interesting experience.
DarmokTanagra · 2026-08-28 06:29:28 UTC
Having worked in a few vibe coded codebases over the last few years I can safely say that AI is not raising the quality of anything.
tikotus · 2026-08-28 07:52:07 UTC
I had the same knee-jerk reaction. "Did I read that correctly?"
But yeah, I guess it can be used to increase certain aspects of quality by letting them go wild. But I think I mostly hear about security or crash issues. In my experience they don't outweigh the number of other issues they cause. Like UI bugs. I've seen more than one service constantly rolling out features that are completely broken, just to have a completely new, still broken, solution available the next day.
VCFundedGenYer · 2026-08-27 18:32:14 UTC
The fruits of using LLMs to code.
You'll waste far more time finding what it quietly and subtly wrecked than you would have if you just coded it yourself.
vegnus · 2026-08-27 18:35:02 UTC
You're not reading it right. The bug was found using a vibecoded fuzzer.
12j3afAv · 2026-08-27 18:41:40 UTC
I wonder from where Claude stole this fuzzer.
pjankiewicz · 2026-08-27 19:09:11 UTC
Or it used something called an "analogy" which is a valid way to solve new problems.
I think they're talking about the misconception that LLMs can only ever regurgitate their training data verbatim enough to constitute mass copyright violation. And that that's therefore "stealing"
jaggederest · 2026-08-27 18:39:16 UTC
Those sneaky LLMs going 7 years into the past and committing as a human:
It’s obviously Claude 69 with time travel functionality, that’s too dangerous to release to public. They’re working on space-time limiting sandbox to prevent these issues.
six_seven · 2026-08-27 19:29:46 UTC
Its all fun and games until the Claude-who-remains hunts you down
jaggederest · 2026-08-27 20:04:08 UTC
Just remember kids, never immanentize the eschaton.
Surac · 2026-08-27 18:39:09 UTC
send patches
rs_rs_rs_rs_rs · 2026-08-27 19:05:13 UTC
...they did.
ligarota · 2026-08-27 19:16:24 UTC
Where?
They only suggested a basic guard, chich can be useless if this case never happens
12j3afAv · 2026-08-27 18:40:17 UTC
Generating an incorrect input file seems to be the easiest task of all for any fuzzer.
Generating correct input to get deep into the call stack and then finding something is the hard part.
ks2048 · 2026-08-27 19:11:15 UTC
No doubt fuzzers (vibecoded or otherwise) can be powerful, but can't you just mark all "/" as potential divide by zero errors?
I guess sometimes developers think they "know" some variable won't be zero, but unless it checked explicitly or by the compiler, that shouldn't be trusted.
wvbdmp · 2026-08-27 19:17:25 UTC
I mean there could be a guard clause? But yeah, seems like this could be statically evaluated like how some IDEs see a null check and don’t complain about nullability within the same scope.
dooglius · 2026-08-27 19:20:07 UTC
What are you suggesting and how would it be different than how SIGFPE already works?
Someone · 2026-08-27 19:29:00 UTC
> but can't you just mark all "/" as potential divide by zero errors?
If you’re accepting large false positives rates: yes.
If you want users to take your warnings serious: no.
(Nitpick: you certainly don’t want to flag _all_ of them. Divisions by non-zero constants definitely should be excluded, for example (integer division by -1 can lead to overflow, but that would be a different warning))
saghm · 2026-08-27 19:32:16 UTC
Fuzzers find inputs, not just "potential" errors that aren't triggerable.
Comments
The only thing left for you to do should be directional decisions. The LLMs should pause and rope you in if the fix involves directional/invariant changes.
We wont stop using AI.
We will use AI to check AI.
Of course this is crazy, but it will also unlock pretty insane scaling and productivity and ultimately we will manage it on either end via requirements and tests.
Are you implying that all code writing by LLMs atm is bug-free?
But, to the underlying question, obviously as we automate more and more of our work, of course we provide less and less value. We're heading towards a future where selling thought for money isn't going to work so well.
From that standpoint, it's not a crazy setup security-wise. Maybe still crazy for development.
I had Fable add a new subcommand to our internal CLI tool. I reviewed and tested it locally and had to suggest several fixes that I feel like I wouldn't have had to tell a human senior engineer to do. When it finally submitted the PR, I had it on a loop waiting a few minutes for comments on the PR, then assessing/addressing/replying-to/resolving them, and then repeating again until all AI reviewers were okay with it. It ended up going through dozens of revisions and ended up with 160 comments left on the PR.
We didn't stop using syntactic programming languages we used code to check code.
Not sure it's really crazy at all. It's been an abstraction for programmers probably since we stopped soldering transistors to each other.
But if you don’t actually read it…
Insane scaling of bloat, bugs, and technical debt I'd say.
> We will manage it on either end via requirements and tests
It is so crazy that this is being touted as a sane strategy. When I was a much worse programmer, I tried to write a big complicated string manipulation function to take two types of scripts in a language and add diacritics. I had the requirements very clear. I had the tests very clearly with all the edge cases. But I didn't have a good and clear picture of how to attack the problem which was quite novel for me. As I got closer to passing all the tests it got exponentially more unruly and confusing. And nearing the end I was frantically changing little bits here and there wincing and praying and hoping the tests would pass. "Please work! Come on!" Then when I got close enough, I could never ever think about touching that mess again.
I was a below average programmer then throwing myself at some novel problem I didn't understand. Throwing LLMs that produce below average code at novel problems and relying on tests and requirements is not where we want to go to make real progress.
(Years later after much learning and coding myself I was able to redo the function in a totally different way. This time I actually understood how to attack the strange problem and made something clean, clear, and robust that just worked. The tests then become a secondary guardrail, not the main force of correction.)
We are seeing such a massive regression from what we've learned over the years of CS.
Generating code automatically when you're not even quite sure what it is or even should be doing is insanity.
You just described every legacy codebase. Many of which are widely used and do a lot of sales. You dont need a clean codebase to have a valuable product.
>It is so crazy that this is being touted as a sane strategy.
Re-read what I said. I literally called it crazy.
It is the same dynamic that gave us customer service from some call center in India. Why would companies do this? Customer service got worse. Are they stupid? No, it's just worth it. The quality goes down but the business can scale more so it doesnt matter.
AI will absolutely be good enough at doing things that we'll happily accept some jankiness at times so that we can devote an extra 3000 hours per year per person to other things.
Im not even suggesting its a good thing. I just think the incentive structure dictates it. You're not going to have time to maintain a small slice of some service by hand.
It used to be considered a quality of good code that there would be less code, not more.
Some people always tryin to get the highscore on golf.
You need range proofs to be 100% safe, and then you can as well use the regular type because invalid values will not occur.
Agda is the most mature dependently typed programming languae (having been around since the 90s – it is basically Haskell on steroids), but has a more proof-assistant flavor than an actual programming language flavor. Opus & Fable write Agda quite well, so LLMs can understand dependent types.
The big problem preventing this approach from working for numbers is that it's just so cumbersome there. Most of this is because all the arithmetic operators are bundled into a single Num typeclass, and `fromInteger :: Num a => Integer -> a` has a type that's impossible for a "non-zero number" wrapper to satisfy.
There are also non security bugs that don't have exploits but just make the user experience worse.
A.I. could also be used to port C/C++ codebases to Rust, which isn't economically feasible at the moment.
No big deal? It’s not like it’s free… tokens cost money.
Generate multiple solutions- they do not to work 100% correctly. And than I check which I would prefer. Which is more to our applications taste.
And than I would take the vibe output as a kind of a ‚plan‘ which I use to implement but not follow 100% and at the end I take my solution and review it. I gain speed with that because I often can quickly see the pros and cons of a solution way better than when I would manually do it and hang on a major roadblock and also I even see such roadblocks in the vibe output - it’s mostly the part with an unnecessary amount of new code that looks nonsensical.
But yeah, I guess it can be used to increase certain aspects of quality by letting them go wild. But I think I mostly hear about security or crash issues. In my experience they don't outweigh the number of other issues they cause. Like UI bugs. I've seen more than one service constantly rolling out features that are completely broken, just to have a completely new, still broken, solution available the next day.
> This is a bug found with our fuzzer: https://github.com/daedalus/fuzzer/
https://code.ffmpeg.org/FFmpeg/FFmpeg/commit/8eda3c7f91e1a5b...
They only suggested a basic guard, chich can be useless if this case never happens
Generating correct input to get deep into the call stack and then finding something is the hard part.
I guess sometimes developers think they "know" some variable won't be zero, but unless it checked explicitly or by the compiler, that shouldn't be trusted.
If you’re accepting large false positives rates: yes.
If you want users to take your warnings serious: no.
(Nitpick: you certainly don’t want to flag _all_ of them. Divisions by non-zero constants definitely should be excluded, for example (integer division by -1 can lead to overflow, but that would be a different warning))