p.enthalabs

OpenAI models coming to Amazon Bedrock: Interview with OpenAI and AWS CEOs

stratechery.com · Read Story HN original

https://aws.amazon.com/bedrock/openai/

https://www.aboutamazon.com/news/aws/bedrock-openai-models

https://openai.com/index/openai-on-aws/

https://x.com/amazon/status/2049178618639839427

Comments

OpenAI frontier models coming to Bedrock soon?
> Starting today, @awscloud and OpenAI are bringing the latest OpenAI models to Amazon Bedrock, launching Codex on Amazon Bedrock, and launching Amazon Bedrock Managed Agents, powered by OpenAI (all in limited preview). AWS and OpenAI will continue to bring the latest advances to Amazon Bedrock—so the models and agents you build with today continue to benefit from new breakthroughs as they arrive.

https://x.com/amazon/status/2049178618639839427

We've updated the title above to make that clearer.

Since the product doesn't seem to be available yet, and the other links are all press releases, we'll leave the interview up as the main link.

As someone who works at big tech and spends countless hours in meetings hoping to get some small feature coordinated for deployment across two teams, I can't imagine the amount of meetings and 6-pagers that were involved in running these models on bedrock's hardware.
at this level they just decide and spin up a swat team to execute it in a couple weeks without politicking. the bureaucratic ways, reviews are just for the low levels, to keep them busy with feature scraps while they mostly do operations
Depends on how its implemented, but Amazon already did add gpt-oss-20b so if the model is similar enough to the OSS variant of GPT, it might not have been as complicated as you might think.
I imagine there's lots of custom kernels and optimization...

Openai hasn't been publishing innovations for quite a while.

Well that didn't take long.
Remember that models on different inference platforms might not necessarily give exactly the same results, adding another axis of non-determinism to development. Things like quantization, custom model serving silicon, batching, or other inference optimizations might mean a model from the original provider performs differently from the hosted one :/

This paper isn't the exact same scenario, since it's an auditable open weight llama model, but shows the symptoms of this: https://arxiv.org/pdf/2410.20247

Claude got a looooot more buy in with a lot of privacy-concerned orgs I work with because they could access it through their "trusted" intermediate Amazon. OpenAI has been banned and is not trusted. I'm not sure that I agree with these orgs' legal teams' assessments, but they definitely read the terms of service far closer than I did.

We will see if this changes the equation, but it feels like OpenAI is pretty far behind and playing catch up on all fronts. Though to be honest, "pretty far behind" is like 2-8 weeks in the AI world, so it may not matter a ton, it's mostly perception. And for me and my information bubble, perception of OpenAI is rock-bottom due to Sam Altman. From appearing unethical to appearing unhinged with demands from fabs and everything else, I'm not a fan.

They're also not focused exclusively only on building an LLM, they have video and image generation too. Anthropic has one single focus, and this is why they are usually at the very top in the SWE benchmarks.
IMHO the benchmarks aren't useful, and ranking among the frontier models is mostly noise. The extra features around the coding agent have a much bigger impact on productivity than having to provide slightly more specification and guidance to the models; a 90% success rate versus a 92% success rate on the tasks I ask it to do is far more influenced by what I say than what the model is capable of.
Isn't it the case that OpenAI and Anthropic regularly just swap for whoever is at the top of the latest benchmarks? They're also so close in scores that it's effectively a wash anyways.

What OP is referring to is Anthropic aligning with corporate terms and conditions early, positioning themselves to be effectively resold by AWS rather than requiring orgs to procure them directly. This is huge in the enterprise world because the processes to get broad approval are generally far smaller and shorter for "just another AWS service" compared to a whole new vendor.

The thing they are really wildly behind on is a business model. They are losing wild amounts of money per customer and it is hard to see how the competitive situation is going to allow them to fix that.
Given the scaling hurdles Claude Code / Opus is having, those Anthropic customers might leave to Codex. I'm _this_ close.
I'm getting pretty close too, but I wouldn't switch to Codex I'd switch to one of the open agents that can use any backing LLM. My reasoning is that if I'm willing to pay the cost of the small changes in usage, I might as well switch to an open source agent that I can add my own convenience features to, like remote sessions and phone-based operation.
Codex is open source and allows any model to be configured.
Many thanks for that info!
What would be subscription customers, no? Rather than Bedrock or per-api customers? Many of the companies running on Bedrock or by-use have per day limits above the max monthly subscription costs.
Codex is pretty good. Its friction to switch but I think it’s sensible being across multiple AI toolchains.
This doesn't mean you have the raw model weights, right? That's still entirely hidden / opaque?

You can just run "air gapped" inference?

Is this only of interest to enterprise customers already on AWS (who want "air gapped" behavior)? Is there any other use case for this?

This will be more expensive than calling OpenAI directly, right?

This is for people who don't trust openAI with their data, but do trust Amazon.

But it also is for Devs in a company who already have a blanket agreement with Amazon, but would have an uphill battle signing an agreement with openAI.

This would be a nice compliance win. One less sub-processor and all our data is already on AWS so less worrying about sending it off somewhere else
The market might be increasingly hard on AI startups in general as enterprises adopt providers like Amazon Bedrock and refuse to sign other deals.