DSpark: Speculative decoding accelerates LLM inference [pdf]

Guessing the timing isn't accidental. Demonstrated openness vs harsh regulation

Strange timeline, though this only works because it’s aligned with Xi’s goals.

Mistral...don't fumble this

this is definitely where things are going. the enormous "eat the world" models have extreme diminishing returns by comparison.

I second ccusage, it's nice

> Local-first session search, analytics, insights, and token use statistics for coding agents, supporting Claude Code, Codex, and more than 20 other agents.

solid piece of software

It's drastically reduced my AI spend. I went from spending $40/day to $10/day.

https://github.com/esengine/deepseek-reasonix

I've heard others say that Deepseek tends to be smarter on specific problems but that Mimo tends to more well-rounded.

> As with V4-Flash, we treat this point as an indication that DSpark sustains useful throughput under an interactivity target that the baseline cannot efficiently support. At matched system capacities, DSpark delivers 57% to 78% faster per-user generation.

Reminds me of the flawed solution in scaling servers in 2017 that use memory-intensive technologies by adding even more servers to solve the problem. (It just increases costs.)

Rather than doing that, think about which critical parts of your app can be written in a more performant technology.

Fast forward to 2026, now you can see who is just throwing more money at the problem to create even more problems where as DeepSeek is giving us optimized solutions.

I know exactly who I would pay attention to, and it is absolutely not Anthropic.

The last year has shown that’s not true anymore (even for web servers).

A&O no longer have the most to justify their high valuation. The only thing they can do now is to get the government forbid the Chinese models.

Hopefully the experts here can offer insight. The above is just my hunch and I’m not a specialist in this field.

The American companies, from my impression don’t involve themselves with such lowly “hacks” because they have so much money to just push forward with doing everything on big heavy models that run on the most cutting edge nvidia chips that they can, the moment, kinda sorta get on demand (I say that in some degree of jest).

They don't develop them because they don't collaborate publicly anymore.

Where would the whole industry be if Google never allowed publishing the transformers paper?

It's not a coincidence that the American AI industry grew fastest in capability when it was the most open.

integrating your own work with the latest public advances takes resources. For one or two small changes this is manageable, but the further you diverge from the public, the cost of maintenance rises exponentially if you want to continue to integrate public advances. when you publish your meaningful advance, you offload the maintenance burden onto everyone else (and they only have to pay a linear cost rather than an exponential one) as it's integrated by default in new work.

In most cases, the (exponential) maintenance cost of integrating public advances with secret ones exceeds the value of the public advances, so most that undertake this strategy of advancing the open frontier in secret don't attempt to integrate continually, but instead try to make a breakaway sprint in isolation to grab a few sticky customers before the unstoppable wave of the public frontier catches up.

This is a pattern commonly seen in university research departments when researchers switch into product development mode, most of these projects are a sprint to advance away from the public frontier once a good idea is found and they do good work and find a few customers for a little while. But if you check back in a few years you won't find an advanced research department but a zombie IP company that brings in a steady income via IP enforcement and a small number of customers for whom switching is too expensive.

How do you know they aren't doing this stuff? Something has to account for them leading the industry.

So, despite hiring the cream of the crop of math graduates, who could read the papers of free academia, but whose own result the free world could not access - they fell behind.

I have a theory explaining why. I think it's because science is an interactive process. NSA cryptographers could read papers, but they couldn't talk openly with the authors of those papers, because of secrecy demands - even asking question might indicate what they were working on. You can easily imagine them spending months on something they could have avoided by going to the original authors and getting told "Oh, we tried that for a long time, it doesn't work".

Whether that theory is right or not, cryptography is a concrete example of a domain where public research with fewer resources beat private research with a lot more resources.

One would have to conclude that there is little correlation b/w openness and progress speed. Sometimes open is faster, sometimes it isn't.

Is this happening? These open models have been a generation or two behind the closed models for quite a while now. They've been keeping pace but clearly behind.

But the word pioneer comes from French pionnier, literally “foot soldier”, a soldier who goes ahead to prepare the way.

If you don't publish you may be advancing, but you're not preparing anyone's way.

Multi-head Latent Attention (MLA), Multi-Token prediction, MoE architecture are some of the most famous examples.

MTP is from Meta

Another DeepSeek advance that the west are copying is DeepSeek Sparse Attention (DSA)

[1] R. A. Jacobs, M. I. Jordan, S. J. Nowlan, G. E. Hinton, Adaptive mixtures of local experts. (1991)

[2] M. I. Jordan, R. A. Jacobs, Hierarchical mixtures of experts and the EM algorithm. (1993)

[3] L. Xu, M. Jordan, G. E. Hinton, An alternative model for mixtures of experts. (1994)

[4] S. Waterhouse, D. MacKay, A. Robinson, Bayesian methods for mixtures of experts. (1995)

[5] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, J. Dean, Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. (2017)

It's more a cultural thing. Sharing progress is just in their blood.

Revealing optimizations similar to these would pretty much reduce their competitive position.

I suspect their tune will change if they ever take the lead..

US labs in Google, Meta and SpaceX are not leading, none of them managed to build something on par with GLM 5.2.

Care to explain to me why they still don't collaborate and still choose to do it in private?

Those are mostly for embedded devices and the current "sponsor" is Apple.

From a practical POV having all the training data, training infrastructure, and training know-how wouldn't help you either unless you could afford to spend the millions of dollars (hundreds of millions for a SOTA model) in compute to train it each time they released a new training set, in which case you're only talking about the big commercial companies. "open source for the people" just does not apply.

But even if I can't build it from source locally, being able to see what went into the model is an important part of what open source is about.

Yes, but for whatever reason this usage seems to have stuck. Open weights is definitely a better name. I assume the reason "open source" has stuck is because you can download and use it for free, but "open source" was always intended to be about "free as in speech", not "free as in beer". That said, I remember when the term "open source" was invented, and it was always a bit different, more commercially aligned, than the goals of the FSF.

> But even if I can't build it from source locally, being able to see what went into the model is an important part of what open source is about.

True. Unfortunately LLMs have become such a big money and closed enterprise (the opposite of OpenAI and Anthropic's altruistic founding principles) that it's hard to see these commercial models releasing their training data, especially since this data is the closest thing they have to a moat other than the cost of training.

The most valuable training data right now seems to be "reasoning data", and the need for this at least may disappear as AI moves beyond pre-trained language models to smarter systems capable of learning for themselves, and that can actually reason, not need to parrot reasoning data.

Particularly when it comes to tool use w/ self-distillation it can be done without any data... have a tool the model doesn't know? a teacher model RTFMs and the source code, and helps the student learn to get it right.