GLM 5.2 beats Claude in our benchmarks

After installing, do a `n8 build` to build the image, then `n8 --danger --provider opencode interactive` to launch it in a container.

Signup for GLM-5.2 here: https://z.ai

I think they give $5 trail credits to test with any of the open weight models.

When I posted the comment I was both the first commentor as well as the first person to upvote the submission. That matters. My name is ALSO on the open source repo that allows Opencode to be run in a container.

That's transparency, maybe not here, but on a clickthrough to Github it is immediately obvioius.

Not sure a project nobody knows or uses is much better in this regard?

Not that it would make any sense.

US imposing export restrictions on a model from China?

The weights are already available and downloaded, is it going to be a crime to have them, run them, make them available? Constitutional rights still exist (I hope)

Now you're getting it! Commerce will call it a munition and those harboring it as harboring illegal/foreign munitions.

No business will take the hit, so they will quickly deplatform the models.

No end user has the GPU capacity to use GLM 5.2 or similar models at full precision so the government will call the problem "mostly solved." But they might choose to "make examples" out of a few people using p2p software to download the weights if they choose to.

I'm for making software better instead of banning it based on what the rich and powerful claim.

I suspect the real fear is that open weight models undermine the financials and token prices they thought were going to pay off their ludicrous spending because they have all raced and raised hardware prices.

That would be the rational thing to do.

> financials and token prices

I do not think the government thinks this deeply. Market manipulation might be a rational, if unethical reason to ban open source models.

But this admin banned Anthropic models to "own the libs." They will continue to ban what they want for whatever reason they want. I don't think those reasons will be particularly coherent.

We're still in the middle of the cambrian explosion.

If Anthropic was capable of developing Opus 4.49-4.5 2H 2025.... then any company with a research team capable of reading all the papers and press releases will be capable of producing Opus 4.8 by the end of 2027, either raw model competency, or in a harness like claude code (or better with both). I guess what I am trying to say is that Opus 4.5 does not represent the edge of agentic capability, merely somewhere in the thick meaty layer of "functional and achievable".

We can draw the line at Sonnet 4.6 in the US but much like encryption export restrictions in the 1980s, the line drawn will be laughably low within a few years and simply unthinkable in a decade.

Yeah. Illegal numbers.

If you classify AI as a weapon which seems to be the direction that we are all heading towards, they yes first amendment rights won't likely apply.

The reason GLM-5.2 hasn't been banned is that despite these cherry picked use cases, GLM-5.2 isn't even close to Opus in all use cases. These vibe benchmarks are ran by companies that are not part of the cyber services offered by Anthropic and OpenAI where they can use the models without the safeguards and refusals so their actual cyber capabilities can be utilized.

These guys that wrote the article compared a gimped Opus to GLM-5.2, knew full well it's misleading, and got the clicks regardless. They don't have enough clout to be a part of something like Project Glasswing, GPT Cyber, etc.

Any prohibition on open source models will do nothing to fix the problem.. since attackers will never feel bound to the law. All advanced models must be available for defensive purposes.

If the real motive is profit, then open source models are likely simply not a viable means to that end.

But that's the whole point.

Fall out of favor with the admin and you lose access to the good American models, aren't allowed to use Chinese ones, and fall prey to the attackers and behind your competitors.

Which would be fine, but as we know, people securitize the crap out of their investments these days, and least some people probably leveraged themselves on some US AI companies, so now the risk is spreading outside of the sector to the economy in general, which is made worse by the sheer amount of spending on AI.

If huggingface or whatever is forced to take down open source licensed weights, there’s always bittorrent.

Export controls are one thing, but the US doesn’t really have import controls, and there’s no copyright issue, so DMCA, etc don’t come into play.

It’d take the courts years to decide how to contort the law to ban open weight models, and by then, it’ll be too late (and also pointless).

The others are a waste of taxpayer money. Extraordinarily low return on investment (kill-on-investment?)

Yes, you get your free model, but the cost of this is not developing your own capability and tying your fate to a country which may or may not have your best interests as a nation in mind.

This is just the deindustrialization that occurred in my home region (the American Midwest) playing out on a global scale in different sectors. It was originally driven by the Japanese, who, to their credit, acted more as partners than competition. Eventually that desire for larger margins went to China, and now you basically can't build anything of consequence without at least some Chinese parts, because there's "no economic case" for it. This means that you have to play Beijing's game if you want access to any sort of modern market.

You see this happening with Volkswagen's restructuring, next you'll see it with non-American, non-Chinese AI.

Over the last decade, the US has been way more unreliable than China. There's been a near constant negative impact from the US doing something.

At least with China, we are very good at winning trade wars with them here in Australia.

Or, on a more local note, an Australian automotive worker who worked for a company that figured out 10 years ago that they wouldn't be able to pay him a decent wage, compete with the then-upcoming Chinese EVs, and remain profitable.

There is no good guys in general, and whataboutism and making the scope bigger doesn't help.

The thing is that if the models you are building on are open source whether hosted on chinese / american / whatever service at least give you an option to switch provider easier vs a fable / chatgpt 5.6 that gets banned for none americans etc...

2 years ago america would have had the branding/perception advantage but right now that is well and truly gone...

So far as I know, there have not been any offensive American operations in the West Bank or Gaza. Do you have sources for any?

> There is no good guys in general, and whataboutism and making the scope bigger doesn't help. The thing is that if the models you are building on are open source whether hosted on chinese / american / whatever service at least give you an option to switch provider easier vs a fable / chatgpt 5.6 that gets banned for none americans etc...

Then it'd make far more sense to use a native open-source AI project than to use Chinese AI. Personally, I've been looking into Mistral AI for my own uses.

> 2 years ago america would have had the branding/perception advantage but right now that is well and truly gone...

Hardly. Let's not pretend that much of the world, particularly Europe, hasn't had a long and storied history of staring down its nose at Americans. At a certain point you just stop caring, regardless of your political persuasion.

Stop pretending there’s some type of moral high ground there isn’t. Disgusting.

I don't understand what your point is? This seems like a perfect example of comparative advantage - Australia can produce iron ore cheaper than anywhere else in the world and even when China launched a trade war against Australia the Australian economy kept growing.

There wasn't even any bump in unemployment from the closing of the car industry.

Once that trade war was settled, Australia got cheaper cars, China got cheaper iron ore and both economies won.

The rational behavior on both parts there is in stark contrast to current US policy, which is unpredictable and capricious.

> You might feel differently if you were a Filipino or Vietnamese fisherman whose family relied on the income from the stocks of the South China Sea, or a Uighur person living in Western China, or a Ukrainian soldier who has to deal with drones built with Chinese components, or a democracy advocate in Hong Kong, or arguably, a person who had plans for 2020-2021.

This seems like a random list of complaints about China and I agree with them in general.

I think you'll find most major powers have similar complaints. There certainly are against the US - I think you might find that both the Philippines and Vietnam(!) have fairly mixed feelings about the US for example.

man you're gonna be disappointed when you learn where the components for Ukrainian drones come from (spoiler alert, it's China 95% of Ukrainian drone manufacturers use Chinese components. Both Ukrainian and Russian drones are Chinese components glued together, the vendors in China literally stagger Russian and Ukrainian buyers on the factory floors to not have them run into each other). The largest trade partner of Vietnam and the Phillipines is China.

The kind of thinking that assumes that rivalry implies deglobalization or bloc politics is exactly what's 30 years out of date. It's projecting how Americans think on the entire world, but that's not how the world works any more. The rest of the world continues to globalize, even through war.

America is undergoing Sovietization and erecting an Iron Curtain, and China ironically enough is simply doing what the US used to do. If Americans think the rest of the world will follow them into isolation they're going to make the same discovery the Russians did in the last century.

I’ll happily pay a 100% tariff on open weight models, and there are no regulatory hurdles for them to jump through (yet).

I’m sceptical they could find the legal framework to do this even if they wanted to

They have legal authority to (a) prevent export of US goods/services; (b) ban imports of physical goods; (c) ban transactions (including purchasing services or license agreements) with foreign firms

But I’m not aware of any legal authority which lets them ban US firms from running a Chinese-developed open source AI model in the United States, if they are at arms length from the vendor, and aren’t using it for government contracts or regulated applications

Possibly they could order HuggingFace/etc to suspend Chinese accounts. But if someone in the US (or a third country) downloads the model from China then reuploads it to a US server, completely independently of the vendor - where is the legal hook to prohibit that?

This would be extremely heavy handed and probably end up accelerating the loss of the virtual US monopoly of payment network. The reast of the world isn't going to let the US dictate that only they get the frontier models whether their US made or otherwise

Can they actually though? Do they have legal authority to tell a payment processor that it has to block transactions of a legal US company, just because the company is hosting a Chinese-developed open source model? I’m sceptical

And what about companies (e.g. AWS) that let you “bring your own model”?

That's sanctioning specific individuals for specific acts they performed which the US claims contravene its interests and those of its allies.

I don't agree with the ICC sanctions, but it really can't be compared with the proposal "sanction any company, even US domestic entities, which use a Chinese-developed open source model".

In fact, I think part of what enables the US to sanction them (under US law) is the fact they are neither US citizens nor residents; if they were US citizens living in the United States, I don't think the President would have the legal authority to impose those kinds of sanctions.

They could sanction Hetzner–because it is a German firm based in Germany. I don't see how they could sanction a US firm based in the US whose owners and staff were US citizens.

Also, the 5th Circuit Court of Appeal decision Van Loon v Treasury (Nov 2024) is relevant–it held that IEEPA (the law used to sanction ICC officials) couldn't be used to sanction the Tornado Cash smart contract system, since open source code wasn't "foreign property" under IEEPA.

I agree, my only caveat is that the current administration has shown it's willing to go beyond aggressive regulatory interpretations to questionable and outright implausible interpretations. As we've seen recently, the federal courts and SCOTUS are overturning most of these but that can take a year or more to resolve. The one positive light is they seem to push the hardest on certain culture war issues (immigration, voting, districting, etc). AI doesn't seem like a core hot button issue for the White House and there is a strong pro-AI / business faction.

Which I guess makes what semgrep sells obsolete. Unless they have built a pareto-optimal point in terms of capabilities and token usage maybe?

Secondly these are "just" IDORs, arguably the easiest class of vulnerabilities.

Thirdly it compares to GPT 5.5 and Opus 4.8.

No, we don't have Mythos at home.

mythos is <10% ahead of gpt 5.5 on all benchmarks, which it gains by being several times the size of opus. had it been economical to provide, it would've been released to the public on day one instead of the marketing circus those effective altruism clowns had exhibited. admitting that it costs >1000% to run inference on a <10% better model would've been very damning.

do you have a source for this claim? i thought LLM providers earn high margins from inference (charged by token). is this no longer the case?

The only ones who seem to profit are the ones running smaller Chinese models. Even NVIDIA seems to have to "reinvest" their profits into sponsoring companies to buy their cards now.

no one has a source, because no one knows closed model parameter counts. we have only heuristics which strongly indicate that Mythos is simply a big fucking model that any other lab could make an equivalent of.

> No, we don't have Mythos at home.

That's still useful. To paraphrase the kids these days, GLM5.2 is in the room with us, today. Mythos is not. And for us in the EU, it's even more complicated, as Mythos might be with us in the room one day, and go poof the next day, on the whims of political entities that we have 0 control over.

Knowing where open, accessible, local models are is important. We know they're behind. But there comes a time when "good enough" is useful. Even if they're "just IDORs" today, and even if they're behind SotA today.

As someone else said above, GLM5.2 (and other models in the same tier like kimi, dsv4, etc) is / are slowly becoming "good enough" to assist in automated repo prepare work (download, install, test, edit, re-test, etc). And that translates in RL traces ready to be trained into the next generations. That might be more important than x% behind on benchmarks.

Beats which model in Claude? Whenever a "benchmark" doesn't put precise model numbers in their headlines I am immediately skeptical. Either they don't know the difference (bad) or they are benchmarking against weaker models (misleading, also bad).

It's like when studies say "AI is bad at X" and they used GPT-3.5 in current year.

How are we supposed to stay skeptical of everything if we read anything!?

GLM 5.2 is already capable enough to assist in self-training which is similar to what we saw happen with frontier models and they appear to be getting there at a significantly lower cost than openai/anthropic.