This is insane. Can we switch the linked page to this?
tomhow · 2026-10-23 20:55:15 UTC
We updated the main URL to this and added other related links to the thread's top text.
rafaelero · 2026-10-23 17:13:18 UTC
That's nuts.
rudolftheone · 2026-10-23 17:13:48 UTC
Hallu Bench Hard: K3 at 23.4%, K3.1 at 8.3% which is very nice improvement!
volf_ · 2026-10-23 17:13:52 UTC
That's quite an impressive jump
brianritz · 2026-10-23 17:14:28 UTC
Maybe this was why the “AI trade” was weak this week.
throwa356262 · 2026-10-23 17:14:58 UTC
Ah, the version available to us mortals is nerfed in cybersecurity just like the US models :(
anderber · 2026-10-23 17:32:25 UTC
It seems that GLM-5.3 is still the go-to for cybersecurity.
Aboutplants · 2026-10-23 17:15:11 UTC
If those are true then wow
TuxSH · 2026-10-23 17:15:15 UTC
> For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness. We will continue to apply appropriate controls at every training stage, of which cyber-related data identification and restriction is one example.
Lame.
AspireOne · 2026-10-23 18:14:09 UTC
I like it. If the increase in unrestricted cyber capabilities can be slowed down, please be it. The world needs to get ready, and that takes time. Way more time than even a slow progress would allow.
Unless you want to see more of what happened the past few days.
I think the argument that "cyber capabilities should be democratized, thus keep producing them now, fast, and for everybody" is a very naive one.
jrflo · 2026-10-23 20:19:06 UTC
Yes, thank you Moonshot for not throwing open Pandora's box and seeing what happens security wise.
0xbadcafebee · 2026-10-23 20:24:11 UTC
> The world needs to get ready
How will it do that without a model that's good at security? You gonna have humans manually secure the millions of lines of new AI-generated code, in addition to the existing code? Suddenly people will start doing the job they haven't been doing up to now? Not only are we not capable of auditing all the code, it would take decades to finish, and new code will be coming out at the same time. We can't catch up by hand.
The only way to fight this fire is with fire. We need AI hacking tools to find our security holes quickly and patch them. Otherwise we're just churning out more insecure software, and the only people who can defend against it are the governments who don't want to patch the holes (because they want to be able to exploit them). Without open tools, we all lose.
AspireOne · 2026-10-23 22:10:25 UTC
I disagree with this completely.
2001zhaozhao · 2026-10-23 20:56:44 UTC
I am glad that at least one Chinese company seems to be listening to AI cybersecurity concerns and did concrete steps that actually reduce the problem. This was probably at least somewhat costly to them in terms of general model capabilities as well. They'd probably have gotten slightly better coding capabilities if not for removing the cyber training data and tasks.
AspireOne · 2026-10-23 22:08:46 UTC
I am glad about it as well. Don't get me wrong, I love OSS, I love personally being able to get my hands on the frontier stuff and frontier cybersec capabilities, and I dislike a few entities having exclusivity on it. Everybody does. But that absolutely doesn't mean it's the correct approach. I think people are blinded a little bit by general principles and by personal feelings, rather than operating from a point of pragmatism, consequences, and the big picture.
Also, to note: I don't claim to have the answers either. I don't even hold a strong position on this issue yet. I'm just sharing some of my intuitions, and reacting to people who, as far as I'm concerned, are making this issue all too simple.
TuxSH · 2026-10-23 21:46:23 UTC
Most low- and medium-handing fruit vulns can be found and exploited by GLM and some knowledge already.
Kneecapping cyber vulns only serve governments, Pandora's Box is already open for cyber-related stuff.
AspireOne · 2026-10-23 22:02:42 UTC
The Pandora box can expand even further, and it matters. If it didn't matter, like you seem to be suggesting, then by the same logic there isn't a reason to push for further open-source development.
> Kneecapping cyber vulns only serve governments.
Uhh, no? Kneecapping cyber vulns server most of us. There are negatives to kneecapping it, but let's not pretend like it's perfectly okay and safe for everybody in the world to have a magic box that can hack anything and cause havoc all over the world. The more powerful, cheap, and accessible this magic box is, the worse the consequences.
This is a hard problem and there's real tension - like you said, the playing field not being leveled is a problem (major companies / entities / govts having exclusivity on the best cyber abilities etc), but everybody having access to it is a problem as well.
So let's not pretend like this is a simple problem, and don't say that it only serves the govt. That's very silly.
LinXitoW · 2026-10-23 22:57:04 UTC
How does "the world" get ready, when these limits are so hard core that Opus regularly stops working when I'm just trying to find and fix security exploits in my apps?
Currently available, unhindered models can already find a bunch of exploits, not because they're smart, but because they're cheap and smart enough to just keep going for hours or days until it finds something.
The defenders, especially in the FOSS space, only have one big advantage: Unified scale. If defender A finds a vuln, then defender B doesn't need to find it again. Attackers generally act alone, so they need to do all the work themselves.
If we remove a bunch of defenders, the attackers are back at it again.
CamperBob2 · 2026-10-23 19:47:48 UTC
If I were them, I'd release the weights and claim I had nerfed them for safety, having done no such thing in reality.
moecables · 2026-10-23 17:16:35 UTC
Great! next week we'll get Astra 6.1 and Fable 5.2
onlyrealcuzzo · 2026-10-23 19:01:07 UTC
Probably something like:
MiniMax -> Qwen -> DeepSeek (pro) -> Gemini (Carbon) -> Fable (to beat Carbon) -> Astra (to be slightly worse than Fable but cheaper).
I doubt we'll see any major US release besides Grok in the next 3 weeks.
Would be nice if we got a Terra that's competitive with Gemini Flash and DeepSeek Flash.
The fun part is, we're already at the point where we are seeing something frontier-ish almost every single week.
The really fun part will be when they all start adding System 1-3 natively (IMO) - which I think will be the next release cycle.
henry700 · 2026-10-23 19:05:24 UTC
> Would be nice if we got a Terra that's competitive with Gemini Flash and DeepSeek Flash.
GPT 6.1 Sol is already the new Terra IMO. Same pricing as old Terra and competitive performance with even Astra on strictly coding-oriented tasks.
onlyrealcuzzo · 2026-10-23 19:08:40 UTC
It's almost like there's a dimension besides time which is important - speed, which Sol 6.1 moved vastly in the wrong direction on...
I don't mind speed actually, codex 5.2 was phenomenal for me, but everyone was whining about speed so OpenAI made 5.3 dumber and faster. Just go work on something else while the prompt runs!
thefourthchime · 2026-10-23 21:20:36 UTC
I doubt we'll see Grok either. After the 4.7 disaster, I doubt they want to be embarrassed with 4.8.
aenis · 2026-10-24 07:36:53 UTC
Would you care to elaborate? Grok is the only major model that I haven't tried. Was it that bad?
simondotau · 2026-10-24 10:39:55 UTC
The problem with Grok 4.7 is that it's not amazing at anything, and didn't even retain the blazing fast performance of Grok 4.5 which made it awesome for highly interactive workflows.
But despite all the negative coverage, I can say with confidence that Grok 4.7 is a great code reviewer. You know who says that? Every other model. A lot of my time is spent orchestrating code reviews. I will give the same prompt to three models, then give each model the responses from the other two models to assess and rank them. Finally I give the rankings and meta-rankings to Opus to synthesise a final report.
Right now, for the code I'm dealing with (embedded c++, python, swift, and CFML) I'm fairly consistently seeing every other model rate Opus 5.5 first, with Grok 4.7 a very close second, often finding real issues other models miss.
iyonn · 2026-10-24 07:19:20 UTC
what do you mean with system 1-3?
wolttam · 2026-10-23 17:17:18 UTC
It's a bit fascinating to me how all of these labs have rolled their own (it sounds like) completely unique RL environments.
I wonder how much the industry will centralize around a few key patterns, and if we'll start to see some RL framework projects gain traction and be used by the bigger labs
> First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains behind that of frontier proprietary models. Second, we intend to progressively raise the cyber capability available in open weights.
So they admit that they could do more, but they won't?
segmondy · 2026-10-23 17:20:11 UTC
They don't want to be banned, and maybe their best offensive cyber capability models will be for their govt.
Danox · 2026-10-23 19:23:11 UTC
That really does go without saying…
pixl97 · 2026-10-23 20:26:29 UTC
It seems a good portion of HN doesn't seem to realize that
Onavo · 2026-10-23 17:20:37 UTC
The more interesting question is how much more performance can be squeezed out by post training the open weights through RL.
scottyah · 2026-10-23 17:22:02 UTC
Or it's a cop out. I could totally outrun Usain Bolt, but by design I don't push to set new World Records. I will increasingly show faster times though.
stalfie · 2026-10-23 17:42:35 UTC
Well, in the publication they claim to have identified specific cybersecurity post training tasks that nerfed capabilities when they removed them, which seems like a reasonable methodology.
Seeing as no one besides blackhats want frontier cybercrime capability to be open source, this seems like a very sensible thing to do.
So they have published a reasonable means, and they have a motive. I'd put my money on "probably true".
ApolloFortyNine · 2026-10-23 17:49:14 UTC
The problem is, as anyone trying to address security concerns in claude can attest to, the guardrails trigger even when trying to address real bugs. Good cyber defense requires an understanding of the offense.
How you avoid the "fix my code" attack, the code being someone else's code, I do not know. But nerfing the model doesn't seem like the best path forward either.
zozbot234 · 2026-10-23 17:53:34 UTC
This is not about guardrails, it's just not post-training the model on tasks that specifically relate to cyber-offense. It's a very lightweight intervention, and it makes the model less of a target for bad actors.
tough · 2026-10-23 18:15:56 UTC
claude code "auto-mode" uses a smaller sonnet-like model for classifying, which means it's basically dumb, and the harness biases it into denials
CBLT · 2026-10-23 17:54:28 UTC
Please don't change "cybersecurity" to "cybercrime" from one sentence to the next as a rhetorical flourish. It comes off as disingenuous and makes it harder to get your real point across.
zozbot234 · 2026-10-23 18:02:17 UTC
"Cybersecurity" as a general term often includes offensive "cyber" which is what is being talked about here.
bbg2401 · 2026-10-23 20:36:48 UTC
But the person being addressed specifically used the term cybercrime.
stalfie · 2026-10-23 21:20:16 UTC
Noted. I also did it intentionally to sound a bit spicy, which seemed like a good idea at the time. So I am guilty, and I regret my actions.
LtdJorge · 2026-10-23 18:16:23 UTC
It would also be very sensible to offer a BlackHat Pro version to national officials, for them.
InsideOutSanta · 2026-10-23 18:17:28 UTC
> Seeing as no one besides blackhats want frontier cybercrime capability to be open source
Sure. The problem is that there is no practical difference between "cybercrime capability" and "cybercrime defense capability."
All of my PRs go through an adversarial security review by GLM-5.3, because it's the only smart-enough model with a good understanding of cybersecurity issues that won't regularly refuse the job. Removing that ability from frontier models helps nobody except cyber criminals.
stalfie · 2026-10-23 21:03:19 UTC
I'm not sure I follow. Let's say that GLM remains the frontier of open source cyber capabilities because Kimi is nerfed. Your situation, and the situation of your attackers, remain exactly the same. You run GLM to audit your code, they do the same looking for exploits.
Let us say that un-nerfed Kimi was released, and it turns out to be at the absolute frontier of cyber capabilities, Mythos 5.5 level, a step ahead of anything else. What now? Well now your situation is exactly the same as before, just that auditing and hacking got more expensive. You run a Kimi security audit, they do the same to try to pwn you. However, now the cabal of Linux kernel defenders with secret access to Mythos 5.5 all of a sudden have no advantage, where before they used to have one. And even worse now they are in a literal race to be the first ones to find every bug that Kimi is capable of finding, and patching it yesterday.
So for you, this makes no difference in practice, your capabilities are not elevated, and neither are those of the attackers. The only result of the other scenario is an overall defender disadvantage.
dannyw · 2026-10-24 00:44:37 UTC
It’s highly likely that everyone has software or firmware made by Chinese engineer running in their house.
These defenders aren’t supposed to have access to the “defender models”.
InsideOutSanta · 2026-10-24 08:12:09 UTC
What your argument is missing is that I'll ship security issues that would have been caught otherwise.
You're assuming that these won't matter because attackers might not have better models than I do, but that doesn't make sense. The assumption that attackers can't find the issues because I didn't find them at release is wrong. Even if they never get better models (which they will), current models might find flaws I missed, and they also have other means of finding flaws in my software.
The best defense is to make software as secure as possible regardless of assumed attacker capabilities at the point of release.
ben_plotke · 2026-10-23 18:20:15 UTC
> Seeing as no one besides blackhats want frontier cybercrime capability to be open source
I'm not interested in black hat anything, but I am very invested in making sure the products I work on are not compromised. Given that state agencies have "frontier cybercrime capability", I would like to have the same.
stalfie · 2026-10-23 21:17:23 UTC
I wrote a few lengthier comments on some other similar replies. Summarized my argument is that it seems to me to be strictly better that select defenders have an advantage, rather than everyone being equalized. If you're not one of the privileged few that might be frustrating, but open source models upgrade the attack and defense by the same amount, so open source cyber capabilities being at the frontier only results in a defender disadvantage.
Replying here also as I'm curious about the counterargument.
tkz1312 · 2026-10-23 19:31:37 UTC
I am a defender, and I really really want to have access to high quality models that are good at finding vulns and hacking. Leaving them in the hands only of government agencies and orgs with enough resources to do extensive training / fine tuning seems like the absolute worst possible case for the future security of global software systems.
stalfie · 2026-10-23 20:43:09 UTC
Well, and I am genuinely curious here, wouldn't defender and attacker capabilities being accessible equally by both parties be objectively worse than having a limited amount of privileged defenders? Of course, you could have every piece of code go through an adversarial audit before it's pushed, but in an equal world you would have to always devote as much or more resources to this as your attackers. And historically speaking people just don't do this in practice.
In the scenario where only "verified" defenders get access to the best cyber capabilities, unverified plebian defenders and blackhats are on the same level exactly as described above, there's just also a few defenders that get to run things on godmode.
So to me, it seems that the only result of frontier cyber capabilities is removing a defender advantage. Although that advantage is not for everyone, it can only exists as long as capabilities are unequally distributed. Am I missing something?
Avamander · 2026-10-23 21:19:29 UTC
> Well, and I am genuinely curious here, wouldn't defender and attacker capabilities being accessible equally by both parties be objectively worse than having a limited amount of privileged defenders?
Well, not really. Because those privileged defenders will never be able to predict what needs defending. So we have a few privileged "defended" places and the rest is left for privileged attackers to plunder.
stalfie · 2026-10-23 21:35:48 UTC
Privileged attackers?? So that would be career cybersecurity experts with Project Glasswing/Daybreak/Googlewhatever access, using it to perform actual attacks? Because that's what privileged means in this context. Access to frontier models not available to the public.
Avamander · 2026-10-23 21:38:51 UTC
> So that would be career cybersecurity experts with Project Glasswing/Daybreak/Googlewhatever access.
Yes. You really think access those models will not be given for offensive operations in each country? Of course they will be.
stalfie · 2026-10-23 21:53:34 UTC
Well, seeing as Claude literally snitches on everyday people planning murder and Houthis building rockets I would assume that every user in project Glasswing is heavily and very effectively monitored. I have to admit that I genuinely do not think that what you're hypothesizing is happening right now. I would bet good money that the amount of people with Glasswing access doing actual Blackhat work is currently zero.
Perhaps in the future this will change, but even if that is the case, that's still a win for the defenders. Having a time advantage, even for a few months, is still a win.
Edit: Actually, the CIA/FBI/NSA/Etc.. does probably have access and are probably trying to hack Iran and so forth, so my comment above is probably technically wrong in that respect.
Avamander · 2026-10-23 22:05:00 UTC
> I would bet good money that the amount of people with Glasswing access doing actual Blackhat work is currently zero.
I think it's not "blackhat" in that sense because it's officially sanctioned. Same goes for equivalent models overseas.
The only thing these embargos achieve is giving massive nations a huge advantage, but leaves the average country and person exposed.
stalfie · 2026-10-23 22:14:13 UTC
I edited my comment above before realizing this is what you meant. And its a valid point I concede. Still, I disagree that advantaging this activity is the only thing these embargoes achieve. I would argue that they legitimately also give the Linux kernel, cURL and so on an advantage. And as long as that is true, this still makes it an overall win in my books, even if it also enables officially sanctioned offensive activities. I'm not a US citizen, but I would still rather get pwned by the CIA than both the CIA and Shinyhunters.
tkz1312 · 2026-10-24 04:21:47 UTC
Sounds like you would have been in support of the US export bans on cryptography back in the day ;)
tkz1312 · 2026-10-23 21:32:46 UTC
Because the people I want to defend against includes those with enough resources to train and fine tune models with strong offensive capabilities themselves.
stalfie · 2026-10-23 21:45:44 UTC
Ah, now that's an interesting point. However, wouldn't those groups just fine tune Kimi K3.1 pro no matter if it's nerfed or not, leaving you in the same position as before?
That is, unless the nerf is easier to correct as compared to "upgrading" the models offensive capabilities. So a scenario where it is easy to "restore" offensive capabilities, but hard to "enhance" them beyond the labs best attempt. So is that the world we're living in?
tkz1312 · 2026-10-24 04:20:51 UTC
The groups with a lot of resources are always going to have the best tools available. The question is what tools do I (a person with limited time and resources) get access to. I am in a better position if the delta between my tooling and their tooling is minimized.
verdverm · 2026-10-23 19:59:41 UTC
Black Forest Labs did similar with vision capabilities
> Or it's a cop out. I could totally outrun Usain Bolt, but by design I don't push to set new World Records.
The benchmarks show Kimi K3.1 Pro outrunning practically all models. It's weird to hear someone like you accusing them of making up excuses for not lagging behind in discussions on how they are ahead of the pack.
p-e-w · 2026-10-24 00:24:56 UTC
Not only is that not weird, it’s Occam’s Razor in action. “We could crush the competition but we are holding back” is a laughable excuse that I’m sure you wouldn’t buy in other contexts.
locknitpicker · 2026-10-24 06:54:45 UTC
> "We could crush the competition but we are holding back” is a laughable excuse that I’m sure you wouldn’t buy in other contexts.
I don't think you understand. Benchmarks show just that, that they "are crushing the competition".
I mean, remember Fable? A couple of months back it was lauded as leaps and bounds above everything, that needed to be contained at all costs? Well, apparently Kimi K3.1 Pro outperforms it.
What industrial level of cope do you need to drag in order to look at the results and claim they need to make excuses about anything at all?
Mattwmaster58 · 2026-10-23 17:23:16 UTC
This doesn't seem particularly notable and is in line with Anthropic and OpenAI's approach of restricting release/access to more capable models.
_factor · 2026-10-23 17:44:14 UTC
It is notable in that it’s the first time an open Chinese company openly stated they were reducing intelligence for “safety”.
“Our lock can be picked by the robot.”
We ban the robot instead of fixing the lock. It’s shortsighted.
JoshTriplett · 2026-10-23 17:54:13 UTC
If we had a technique that would defeat almost every lock in the world, we might be prudent to delay releasing that until people have had time to change the locks.
Same general benefit as coordinated disclosure, except it's a big game-theory problem where we're likely to eventually have sufficiently capable defectors.
vikramkr · 2026-10-23 18:45:57 UTC
they're pretty clearly framing it as holding back on the robot and letting is lose slowly to give everyone time to update their locks
senderista · 2026-10-23 23:28:10 UTC
They are directly responding to new Chinese export controls on cyber capabilities.
revolvingthrow · 2026-10-23 17:26:51 UTC
Yeah that's a huge downside. That the model does the task without tripping over its own feet, like discovering a vulnerability the model itself accidentally built 15 minutes ago and triggering its own guardrails, made the Chinese models attractive to use for security-adjacent things. Also useful to red-team your own stuff. While I can understand the reasoning behind it (and I guess having a model that's really good at both offense and defense released into the ether could lead to CCP asking some very pointed questions if Moonshot managed to get blamed for some incident stemming from their model's misuse) it's still a huge disappointment.
I don't want to give US labs the results of my DNA tests and my family tree going back to the 16th century just to secure my shitty app, but if the Chinese are bowing out as well it seems the only alternative is to admit that some script kiddie with a bunch of tokens will just pwn you now and then.
0xWTF · 2026-10-23 17:40:30 UTC
> I don't want to give US labs the results of my DNA tests
My friend, you realize that if you have had blood drawn for a genetic test, there is a very decent probability that the sequencing was done by a Chinese lab operating in the US, right? The Chinese probably have more US genomes that the US does.
Also, illumina used to ship products with hardcoded S3 access, so purchasers of used equipment could gain access to sequencing information generated by the original owners. Too many links to cover all of it, here's the Gemini summary:
If the Chinese want it, they have it. BGI does an enormous amount of sequencing. My own genome is entirely public (or was, and will be again when/if Personal Genome Project comes back) so you can watch and see if it harms me.
TrainedMonkey · 2026-10-23 17:29:06 UTC
On the surface that seems like a responsible thing to do
WarmWash · 2026-10-23 17:40:20 UTC
All it takes is one idiot who happens to have the resources to "pWm Teh Interwebs" to scare both the US and Chinese governments.
woadwarrior01 · 2026-10-23 17:46:21 UTC
Kimi is the closest Chinese equivalent of Anthropic.
Comments
Chinese models never agreed to the pacing anyway lol.
It appears to be generally frontier-level.
Lame.
Unless you want to see more of what happened the past few days.
I think the argument that "cyber capabilities should be democratized, thus keep producing them now, fast, and for everybody" is a very naive one.
How will it do that without a model that's good at security? You gonna have humans manually secure the millions of lines of new AI-generated code, in addition to the existing code? Suddenly people will start doing the job they haven't been doing up to now? Not only are we not capable of auditing all the code, it would take decades to finish, and new code will be coming out at the same time. We can't catch up by hand.
The only way to fight this fire is with fire. We need AI hacking tools to find our security holes quickly and patch them. Otherwise we're just churning out more insecure software, and the only people who can defend against it are the governments who don't want to patch the holes (because they want to be able to exploit them). Without open tools, we all lose.
Also, to note: I don't claim to have the answers either. I don't even hold a strong position on this issue yet. I'm just sharing some of my intuitions, and reacting to people who, as far as I'm concerned, are making this issue all too simple.
Kneecapping cyber vulns only serve governments, Pandora's Box is already open for cyber-related stuff.
> Kneecapping cyber vulns only serve governments.
Uhh, no? Kneecapping cyber vulns server most of us. There are negatives to kneecapping it, but let's not pretend like it's perfectly okay and safe for everybody in the world to have a magic box that can hack anything and cause havoc all over the world. The more powerful, cheap, and accessible this magic box is, the worse the consequences.
This is a hard problem and there's real tension - like you said, the playing field not being leveled is a problem (major companies / entities / govts having exclusivity on the best cyber abilities etc), but everybody having access to it is a problem as well.
So let's not pretend like this is a simple problem, and don't say that it only serves the govt. That's very silly.
Currently available, unhindered models can already find a bunch of exploits, not because they're smart, but because they're cheap and smart enough to just keep going for hours or days until it finds something.
The defenders, especially in the FOSS space, only have one big advantage: Unified scale. If defender A finds a vuln, then defender B doesn't need to find it again. Attackers generally act alone, so they need to do all the work themselves.
If we remove a bunch of defenders, the attackers are back at it again.
MiniMax -> Qwen -> DeepSeek (pro) -> Gemini (Carbon) -> Fable (to beat Carbon) -> Astra (to be slightly worse than Fable but cheaper).
I doubt we'll see any major US release besides Grok in the next 3 weeks.
Would be nice if we got a Terra that's competitive with Gemini Flash and DeepSeek Flash.
The fun part is, we're already at the point where we are seeing something frontier-ish almost every single week.
The really fun part will be when they all start adding System 1-3 natively (IMO) - which I think will be the next release cycle.
GPT 6.1 Sol is already the new Terra IMO. Same pricing as old Terra and competitive performance with even Astra on strictly coding-oriented tasks.
But despite all the negative coverage, I can say with confidence that Grok 4.7 is a great code reviewer. You know who says that? Every other model. A lot of my time is spent orchestrating code reviews. I will give the same prompt to three models, then give each model the responses from the other two models to assess and rank them. Finally I give the rankings and meta-rankings to Opus to synthesise a final report.
Right now, for the code I'm dealing with (embedded c++, python, swift, and CFML) I'm fairly consistently seeing every other model rate Opus 5.5 first, with Grok 4.7 a very close second, often finding real issues other models miss.
I wonder how much the industry will centralize around a few key patterns, and if we'll start to see some RL framework projects gain traction and be used by the bigger labs
P.S. weights by November 4th
> First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains behind that of frontier proprietary models. Second, we intend to progressively raise the cyber capability available in open weights.
So they admit that they could do more, but they won't?
Seeing as no one besides blackhats want frontier cybercrime capability to be open source, this seems like a very sensible thing to do.
So they have published a reasonable means, and they have a motive. I'd put my money on "probably true".
How you avoid the "fix my code" attack, the code being someone else's code, I do not know. But nerfing the model doesn't seem like the best path forward either.
Sure. The problem is that there is no practical difference between "cybercrime capability" and "cybercrime defense capability."
All of my PRs go through an adversarial security review by GLM-5.3, because it's the only smart-enough model with a good understanding of cybersecurity issues that won't regularly refuse the job. Removing that ability from frontier models helps nobody except cyber criminals.
Let us say that un-nerfed Kimi was released, and it turns out to be at the absolute frontier of cyber capabilities, Mythos 5.5 level, a step ahead of anything else. What now? Well now your situation is exactly the same as before, just that auditing and hacking got more expensive. You run a Kimi security audit, they do the same to try to pwn you. However, now the cabal of Linux kernel defenders with secret access to Mythos 5.5 all of a sudden have no advantage, where before they used to have one. And even worse now they are in a literal race to be the first ones to find every bug that Kimi is capable of finding, and patching it yesterday.
So for you, this makes no difference in practice, your capabilities are not elevated, and neither are those of the attackers. The only result of the other scenario is an overall defender disadvantage.
These defenders aren’t supposed to have access to the “defender models”.
You're assuming that these won't matter because attackers might not have better models than I do, but that doesn't make sense. The assumption that attackers can't find the issues because I didn't find them at release is wrong. Even if they never get better models (which they will), current models might find flaws I missed, and they also have other means of finding flaws in my software.
The best defense is to make software as secure as possible regardless of assumed attacker capabilities at the point of release.
I'm not interested in black hat anything, but I am very invested in making sure the products I work on are not compromised. Given that state agencies have "frontier cybercrime capability", I would like to have the same.
Replying here also as I'm curious about the counterargument.
In the scenario where only "verified" defenders get access to the best cyber capabilities, unverified plebian defenders and blackhats are on the same level exactly as described above, there's just also a few defenders that get to run things on godmode.
So to me, it seems that the only result of frontier cyber capabilities is removing a defender advantage. Although that advantage is not for everyone, it can only exists as long as capabilities are unequally distributed. Am I missing something?
Well, not really. Because those privileged defenders will never be able to predict what needs defending. So we have a few privileged "defended" places and the rest is left for privileged attackers to plunder.
Yes. You really think access those models will not be given for offensive operations in each country? Of course they will be.
Perhaps in the future this will change, but even if that is the case, that's still a win for the defenders. Having a time advantage, even for a few months, is still a win.
Edit: Actually, the CIA/FBI/NSA/Etc.. does probably have access and are probably trying to hack Iran and so forth, so my comment above is probably technically wrong in that respect.
I think it's not "blackhat" in that sense because it's officially sanctioned. Same goes for equivalent models overseas.
The only thing these embargos achieve is giving massive nations a huge advantage, but leaves the average country and person exposed.
That is, unless the nerf is easier to correct as compared to "upgrading" the models offensive capabilities. So a scenario where it is easy to "restore" offensive capabilities, but hard to "enhance" them beyond the labs best attempt. So is that the world we're living in?
https://bfl.ai/blog/capable-open-and-safe-combating-ai-misus...
The benchmarks show Kimi K3.1 Pro outrunning practically all models. It's weird to hear someone like you accusing them of making up excuses for not lagging behind in discussions on how they are ahead of the pack.
I don't think you understand. Benchmarks show just that, that they "are crushing the competition".
I mean, remember Fable? A couple of months back it was lauded as leaps and bounds above everything, that needed to be contained at all costs? Well, apparently Kimi K3.1 Pro outperforms it.
What industrial level of cope do you need to drag in order to look at the results and claim they need to make excuses about anything at all?
“Our lock can be picked by the robot.” We ban the robot instead of fixing the lock. It’s shortsighted.
Same general benefit as coordinated disclosure, except it's a big game-theory problem where we're likely to eventually have sufficiently capable defectors.
I don't want to give US labs the results of my DNA tests and my family tree going back to the 16th century just to secure my shitty app, but if the Chinese are bowing out as well it seems the only alternative is to admit that some script kiddie with a bunch of tokens will just pwn you now and then.
My friend, you realize that if you have had blood drawn for a genetic test, there is a very decent probability that the sequencing was done by a Chinese lab operating in the US, right? The Chinese probably have more US genomes that the US does.
https://pandorareport.org/the-gryphon-scientific-biosecurity...
Also, illumina used to ship products with hardcoded S3 access, so purchasers of used equipment could gain access to sequencing information generated by the original owners. Too many links to cover all of it, here's the Gemini summary:
https://share.google/aimode/4JLsHpeRCWlVqU8qT