GPT-6.1 Terra

The pelicans place it nicely between Sol and Luna in terms of both pricing and results.

The only thing that pelican have is "models where not trained on this" maybe that true months (or years ago) now it's on every HN post and these AI labs are monitoring HN

At least unless the Terminal Bench 4.0 score is more indicative than the other ones, because that one shows it clearly worse than 6.1 Sol for both performance and cost.

My expectation is Fable 5.5 is next overall improvement, and my bet is OpenAI's "Bel" will be the next competitive one (or, their lack of safety culture will bite them/us in the process).

There's very few stuff that needs Astra/Opus level intelligence.

Luna 6.1 would be amazing.

I'm optimistic about them tackling frontier intelligence next. Astra 6.1 please?

The "cost per task" metric is just a weaselly way to sound like they didn't just double the cost for most users, and aren't having their proverbial lunch money stolen. They can no longer claim to provide more tokens, more capable models, or more usage in general. So they made up a new metric where "task" could be defined to mean whatever they need it to mean.

They had to become VERY specific in their wording to present any kind of value proposition that will entice people.

[0] https://newsletter.semianalysis.com/p/anthropic-subscription...

Nope to both. Not even close.

The only way it works is if you define "task" in a way that OpenAI is good at it and Anthropic isn't.

Second of all, barring more cuts, yes I do think the $20 and $100 plans do deliver more tasks because the models have gotten cheaper in the meanwhile (like this very announcement).

> Are they delivering 50% more than Anthropic per token?

So say the public benchmarks. Pull up artificial analysis, “Cost (USD) to run all evaluations in the Artificial Analysis Intelligence Index”:

Sol 6.1 (max): $1,082 Opus 5.5 (max): $8,708 Sonnet 5.5 (max): $7,260

>7x difference in cost for the same tasks.

The original plan doesn't exist anymore. To get the same usage you now need a $500/month plan. The new plan that costs $200 is different in every way that counts. It was actually a 150% price increase for people who want or need what the old plan used to provide.

Don't get me wrong, I genuinely LIKE OpenAI. It's probably worth it even at the new price. I just hate the rhetoric designed to Goodhart past the math.

> the models have gotten cheaper in the meanwhile (like this very announcement).

That is not good news for plan users. Plan users now get less of something that also has less worth to begin with. It would only be good news if they got more of it.

For example, if I buy the "unlimited car wash" package for $19.99/month and the car wash lowers the price per wash from $10 to $1 my subscription becomes less valuable, not more. The breakeven has moved, so I now need to wash my car more than 20 times before the package is justified.

If the car wash then increased the price to $50/month would it be wise for me to keep the package? Would lowering the price per wash make the package more valuable to me somehow? No, it would do the opposite.

I’m not sure I see how. Subsidy got cut by half. But in the meantime 6.1 came out which uses 25% fewer tokens than 5.6 and costs 1/3 as much per task.

So for 1/2 the budget you use to have with 5.6 on the old plan, you can do more with 6.1. I don’t see that as a problem.

Once you are inserting models into non-creative (and software development is creative work) it is imperative to start creating your own small evals. Not just to check how different models perform in that equation, but to catch model drift over time.

5x is calculated in terms of API costs, nobody pays those extreme Anthropic prices outside enterprise users.

> like they didn't just double the cost for most users

Most users aren't on their $200 plan. Most users (including on the $200 plan) continue being subsidized.

Ignore value then. Anthropic still provides about 3x as many raw tokens. The 20x\200 plans provide about 10B for openai versus almost 30B for Anthropic.

Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.

> Most users aren't on their $200 plan.

Fair enough. I should have said most heavy users.

> Most users (including on the $200 plan) continue being subsidized.

They continue to be fifty percent less subsidized, I guess.

Look, I don't fault them for trying to become profitable. I fault them for the weird weaselly business speak about cost per task. It's BS game playing and we all know it.

I'm honestly rooting for them, and hope it all works out. For now though I'm pretty salty that my bill jumped by 150% in one month. Gaslighting about made up metrics just makes me more angry.

We agonize over cost per task to the extreme for services/functionality built atop of APIs and can absolutely guarantee you that it is the real world metric that counts most right after “is the model good enough for the task at hand,” and furthermore that, yes, in fact, it is quite often that we see certain models/providers easily using 3x or more tokens on average for the same tasks (or even the exact same task) than a supposedly similarly matched model from another provider. Once you become intimately familiar with the thinking traces from specific models, you can preempt this to some extent with manipulations to the system prompt (essentially heading off entire thinking branches) but the nuking of reasoning traces in the name of preventing against distillation attacks has made this harder.

> Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.

You don’t have to speculate. Pull up artificial analysis, pick either opus or sol and scroll down to total tokens and to total price. Opus (and sonnet) use 4x the tokens and 7x the dollars.

That difference alone removes the difference that semi analysis found.

Fair point. I just tried it, and I stand corrected. Claude uses almost exactly 3x as many tokens.

In the end Claude produced 2.25x more code, and used 3.01x more tokens (ignoring the cached system prompt, etc). It's answer was also much more substantial and feature rich - including many features I did not ask for.

I gave them both the prompt "Without searching the internet please write and test a python script that accepts a year (in any common format) as input and provides the date that easter fell or will fall on in that year. The output should be in discordian date format. You may not use any libraries."

The thing I found most interesting were the unit tests. Claude implemented the core code using both the Meeus–Jones–Butcher algorithm AND Gauss, but only used Gauss as a sanity check to ensure that it's primary code worked. Sol just validated that it matched some dates it could remember. That explains a good chunk of the almost doubled output, and it was a reasonable thing to do, though neither answer was "wrong."

Token usage looked like this (based on parsing it out of the jsonl, neither provider seems to make it easy to get real billing information per project/conversation.

   Tool   Fresh input Output Cache-excluded Cached reads Gross total
   ----   ----------- ------ -------------- ------------ -----------
   Codex  24,119      7,729  31,848         109,312      141,160
   Claude 54,802      41,093 95,895         553,252      649,147

I believe oh-my-pi will be able to ingest your Claude and codex conversations, count tokens, and then assign API-equivalent rates to them. May be worth looking into.

It's very likely this is a next checkpoint of GPT-6 "Sol", over-optimised for the web chat and offered to Codex subscribers as a "please go away" offering.

If this was half the price, I'd be very interested, but at 1/5/0.05? Feels like another panic release because they can't get Astra 6.1 out of the door...

But like the original release of Terra, it's still a weird middle child between the small-model-that-got-substantially-better (Luna) and the large-model-that-got-significiantly-cheaper (Sol). The TerminalBench 4.0 cost-per-task results are a bit confusing and I'm not sure why you would pick Terra over Sol in that case. Overall this release seems like a flex on Sonnet 5.5: a model which is having its own middle child problems between the godly Opus 5.5 and the surprisingly capable Haiku 5.5.

It is having improvements in other fields than coding while being more cost efficient.

If all I want is coding at a cheaper price, then deepseek is a good model for that.

I am more than okay to have a bifurcation of models good for coding and models good for everything other than coding. I strangely imagined it many years ago.

Everyone is suggesting OpenAI to release a model close to opus 5.5 yet it's releasing this. On one hand we can imagine that maybe their training cycle wasn't able to produce this whereas gpt 6.1 terra was made.

But on the other hand, I also now wish to see an OpenAI model which is good for just pure coding, like an absolute beast at it while being cheap (so the opposite of terra)

ironically, one of the few appreciations that I might give to OpenAI if they pushed this model to prod and are facing some pushback as developers like us want/only see coding models. They could've benchmaxed it and it would've been slightly irrelevant. I'd much rather see reality.

At the very least, this new model won't make things more expensive for anybody and would only make use-cases cheaper depending on what they are, so it's a fine deal for me.

I also wonder maybe this is why open source labs aren't targeting this space and why the only focus of AI labs is coding nowadays. Everything else just doesn't seem so worth it from investor, even end user, training perspective.

Maybe this is the reason why we have gotten almost an exponential growth in coding capabilities as that became the most (if not only) capability that truly really matters.

To not wish for discussion to go to a tangent, nonetheless this could also explain why our work as a software engineer is going through a slight crisis of meaning as these AI models have gotten impressive at coding.

As an (aspiring) developer (currently in 1st year college), I don't wish for AI to replace coding even more for me or to push it even more or to suck the joy out of it even further. So I am more than happy for these models to stay as they are and rather make advancements elseplace. Ironic because it is us developers who use AI as well and we want cheaper workflows and so we would also want models to be cheaper and better at coding. There are some fine lines of contradictions and irony in all of it. I'd much rather also have open-weight models of such terra-like models but I imagine that because of the reasons that I mentioned above, we are most likely gonna see it the most focused on coding but still open-weights is pretty close.

The more choices the better (though choice-fatigue is a real thing), I hope I was able to explain some of the nuance I feel about this particular launch.

Destroyed by Anthropics crazy decision to price it x5 after 100k context.

At least this my discovery when I did evals on my reviewers skills. 10k LOC is high end for high effort or around 150k context using opus. You can push towards 15k but that’s on max effort with about 400-500k ctx.

Fast models like haiku do not even need 1M ctx in my view

First we had:

> Luna (Small), Terra (Medium), Sol (Big).

Which was fairly smooth after you got used to their planets.

But then they added Astra as the "Bigger," which was mostly still self-coherent:

> Luna (Small), Terra (Medium), Sol (Big), Astra (Bigger)

But then things go confusing, they released Sol-6.1 which sometimes is superior to Astra, and discontinued Terra which made it look like this:

> Luna (Small), Sol-6.1 == Astra

But with Sol-6.1 costing the old Terra-6.0 price, while, as I said performing as good as the old "Bigger" model in many non-3D/UI situations.

Now they're re-introduced Terra, as a 6.1 release, making this even more confusing.

Did Microsoft's chief branding officer move over to OpenAI?

But minor credit to Anthropic, at least they've mostly stuck to their four model names (Haiku, Sonnet, Opus, and Fable). You mostly know where a model is MEANT to be positioned from the name alone. OpenAI cannot keep it consistent from release to release.

But, yes, currently Opus Vs. Fable has the same issue as Sol-6.1 Vs. Astra.

Hmm? They've been using the Luna/Terra/Sol/Astra scale for a few versions now and as far as I can tell it maps directly onto Haiku/Sonnet/Opus/Fable, in terms of the kind of thing it's meant to convey (if not in terms of the scale of differences).

Both companies have screwed up their naming royally. Maybe that's intentional - I'm sure it doesn't hurt Anthropic to have people using Fable a bit longer than they need to and racking up a huge bill in the process.

It's like iPhone 17 being faster than iPhone 16 Pro. Because it's newer and benefits from advances in chip architecture, despite being a lower model tier. Sometimes different iPhone models are released at different times.

I think the names are more of a reference to price. Luna is always cheaper than terra which is cheaper than Sol which is cheaper than Astra (for the same model number).

Per token, yes. But per task it's not even necessarily the case depending on how much thinking is needed.

But the model names are linear by price.

Yes, a more modern smaller model can beat an older bigger model at some tasks, that's what can be confusing.

“Test them all”

The lineup of models is still the same.

There are four models: luna, terra, sol, and astra.

Pricing increases for each level. This new 6.1 Terra is 50% cheaper than Sol 6.1.

They are lowering prices for newer releases, so yes if you compare Sol 6.1 to 5.6 Terra it won't look right, but why are you doing that? Look at the same release version for comparable numbers.

Each model improves in successive releases but they maintain pricing and capability coherency within a release version.

Yes, Sol 6.1 is supposed to rival Astra for some work, which is what you'd expect from a later versioned release model against a prior release bigger model.

Assuming they release Astra 6.1 at some point, I expect it'll be better than Sol 6.1, at a higher price.

I don't understand what people are asking for here? No reductions in pricing? Different model names on every version release? Not releasing any models for a version number until they're all ready?

  - Model name
  - Version number
  - Effort level

Given the operations of the model are opaque, can you see how someone (most people?) might be confused?

After that you just have model "strength" and version number. But of course strength is only comparable at equal version number and the two axes are almost parallel.

Anthropic also does this, perhaps to a lesser extent.

I see a ton of criticism and not a single person coming up with anything better, unless it's to take away options, like removing effort level.

What about buying Intel CPUs? A car? Electrical components?

If you're not just using this at a consumer level to ask about cooking recipes (and if that's your use case, you really don't care about API pricing, or any of this, and can just use the default), and are using it professionally, then this seems like a low bar of effort to understand what you're buying.

I do grant you the relationship between model and effort level has an unclear relationship, but there are also tradeoffs in most complex technical products that are not always clear; this is exacerbated by the fact that this product is stochastic.

Why are engineers using this as a professional tool expecting this to be bought like a consumer gadget and then blaming the maker?

Why is it ok for engineers to have confusing names and ways to interact with something?

How am I supposed to decide between these options: https://imgur.com/a/bXHWowL

How am I supposed to keep up with the changes every few months?