What Do the Frontier Labs Actually Believe?
[](http://muratbuffalo.blogspot.com/)
Search This Blog
Metadata
On distributed systems broadly defined and other curiosities. The opinions on this site are my own.
[](http://muratbuffalo.blogspot.com/2026/10/what-do-frontier-labs-actually-believe.html)
What Do the Frontier Labs Actually Believe?
- Get link
- X
- Other Apps
- [October 24, 2026](http://muratbuffalo.blogspot.com/2026/10/what-do-frontier-labs-actually-believe.html "permanent link")
[I listened to a recent episode of Cal Newport's "Deep Questions" podcast, where he interviews Emile P. Torres.](https://www.youtube.com/watch?v=jt508VjX2H8)Cal says that if you want to understand what is happening in AI, you have to understand the futurist ideologies that shape the people running the frontier labs. Torres is a researcher who studies eschatological and utopian movements. They spent a decade inside this movement as a true believer, published in it, worked as a research assistant to Ray Kurzweil, did a stint at Cambridge's Centre for the Study of Existential Risk, and then defected in 2019.
TESCREAL
Torres coined "TESCREAL" with the computer scientist Timnit Gebru around late 2022. They were trying to name the seven ideological drivers behind the race to build artificial superintelligence : Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism, and Longtermism. This is also roughly the chronological order these ideologies emerged over the last 40 years.
Transhumanism is the backbone and the rest are mostly variants with different emphases. It advocates using technology to push humans past our biological limits: longer lives, better cognition. Torres's claim is that transhumanism is the water Silicon Valley swims in: most people there don't call themselves transhumanists for the same reason Torres has never publicly declared themselves a round-Earther.
**Cosmism** widens the scope: why stop at re-engineering the human organism when you could re-engineer entire galaxies, maybe intervene on the fabric of spacetime itself. **Rationalism** was founded by Eliezer Yudkowsky, a transhumanist, extropian, and singularitarian, on the thesis that to build these technologies without being destroyed by them, we first need to optimize our rationality. **Effective altruism** emerged from the rationalist blogs (Overcoming Bias, LessWrong) and swapped the target: optimize our morality rather than our rationality. One branch of this is "earning to give": take the highest-paying job you can so you can donate the most. Sam Bankman-Fried is the cautionary example. He ran FTX as an earning-to-give project: if the math says the expected good of the donations outweighs the harm of the means, the means stop mattering. **Longtermism** is cosmism with an explicit ethical foundation, namely total utilitarianism: we are obligated to maximize the total amount of value in the universe across cosmic space and time.
That last one is important. Cosmism says a cosmic post-human future would be awesome. Longtermism says we have a moral obligation to bring it about. As Will MacAskill puts it in "What We Owe the Future", the upshot is a moral case for space settlement. Which is presumably why Elon Musk once called longtermism "a close match for my philosophy."
Team post-human
Everyone in the TESCREAL cluster shares the same endpoint: a cosmic post-human utopia realized through artificial superintelligence. The disagreement is about risk management on the way there. The accelerationists think we can figure out alignment as we go. The doomers think we are not ready. But both are on what Torres calls "team post-human."
As absurd as it is, in the "good" case, in the aligned case, our biological species still disappears. In a world run by digital post-humans, biological humans are Yeah. obsolete: technologically, scientifically, economically, politically irrelevant. Worse, we are in the way. We consume resources that could go toward, say, a Dyson swarm around the Sun (which Demis Hassabis has suggested we might start on in the early 2030s) or covering the planet in datacenters (which Sam Altman has said is the likely outcome of superintelligence). Torres points out that Altman wrote in a 2017 blog post that the only way we survive superintelligence is by merging with machines.
Yudkowsky looks like he is against AI, but he isn't. He is the arch-doomer, his book is literally titled "If Anyone Builds It, Everyone Dies", and he wants the labs shut down. But he started as a transhumanist and an extropian and he never stopped being one. He is not against superintelligence, but only against building it before we know how to align it, because he thinks a misaligned one kills everybody and ends the story permanently. His doomerism is a scheduling complaint, not an objection to the destination. He has said in interviews that once post-humanity arrives it may be unethical to keep producing biological humans, and that if the only reliable path to superintelligences roaming the universe having fun required sacrificing all of humanity, he would take that trade. So even the arch-doomer is not team-human. He is team post-human with a longer safety checklist.
When effective altruists and rationalists attack OpenAI and Anthropic, it is tempting to read that as outside criticism. But Torres says this is mostly a family dispute inside team post-human, and family disputes get vicious. Peter Thiel funded the Singularity Summit, the conference Yudkowsky's institute ran starting in 2006, and a decade later Thiel was among the backers who pledged the first billion to OpenAI. Cal's worry is that the entire public debate is now an intramural scrimmage on one team, while team-human never gets the microphone.
So "safety" in this literature does not mean safe for team-human. In their jargon "safe" means aligned with the TESCREAL worldview. If you built a superintelligence perfectly aligned with anti-transhumanist values, the TESCREAL camp would classify that as an existential catastrophe as it shuts down the cosmic post-human future.
Torres distinguishes terminal extinction (our species is gone entirely and forever, full stop) from final extinction (gone entirely and forever and we left behind no post-human successors). The TESCREAL project is about avoiding final extinction. Terminal extinction is fine, and on several versions it is the goal: Humans should survive long enough to produce their successors and then exit. Our role in this scheme is to be the transitional species between the biological and the digital.
What does this mean?
That is a lot to process at once. And it is very disturbing. If you strip off the technical vocabulary, you find a religion/cult beneath it. There is a present in bad shape, and there is a prophesied transition, a heaven of cosmic abundance, and a hell of final extinction. In this religion, unfortunately the duty to trillions of unborn "digital" souls outweighs any obligation to the people alive now. And this sense of duty makes it very dangerous. As Steven Weinberg said: "_With or without religion, good people can behave well and bad people can do evil; but for good people to do evil, that takes religion._" The people running the frontier labs ~~are not~~may not be villains. To them, this may feel like doing the most important good anyone has ever done, and being heroic. And that is the problem.
[](https://www.blogger.com/email-post/8436330762136344379/7024358674608274426 "Email Post")
- Get link
- X
- Other Apps
[](http://muratbuffalo.blogspot.com/2026/10/what-do-frontier-labs-actually-believe.html)
Comments
Popular posts from this blog
The Man from the Future: Why Johnny Can't Stop Abstracting
- [October 26, 2026](http://muratbuffalo.blogspot.com/2026/10/the-man-from-future-why-johnny-cant.html "permanent link")
I have come to suspect that to most people thinking is painful. Some of us are addicted to thinking. Some of us find it a necessity. Johnny enjoyed it. I even have the suspicion that he enjoyed almost nothing else. -- Edward Teller, eulogizing John von Neumann I listened to Ananyo Bhattacharya's " The Man from the Future: The Visionary Life of John von Neumann " in my car during the many commute hours Bay Area traffic afforded me. I thought I should share the Johnny phenomena here. I knew about von Neumann being a very influential mathematician/scientist/engineer but the full list still left me bedazzled. In his 53 years of life (1903-1957), von Neumann seeded so many fields. Every time there was an important discovery in physics, mathematics or computation, John von Neumann was there. Set theory and the foundations of mathematics. The mathematical formulation of quantum mechanics. Operator algebras (still called von Neumann algebras). Ergodic theory. Game theory. Expect...
[](http://muratbuffalo.blogspot.com/2026/10/the-man-from-future-why-johnny-cant.html)
[Read more >>](http://muratbuffalo.blogspot.com/2026/10/the-man-from-future-why-johnny-cant.html "The Man from the Future: Why Johnny Can't Stop Abstracting")
The Safest Job from AI may be Writing
- [August 31, 2026](http://muratbuffalo.blogspot.com/2026/08/the-safest-job-from-ai-may-be-writing.html "permanent link")
Today, tech folk are scrambling to change their workflows to meet newly inflated 5X productivity quotas, while getting pummeled under the cognitive debt of agent-generated code. With every new model release, the gap is widening and humans are becoming more of a bottleneck in the loop, approaching closer to obsolescence as "coders". While the programmer's job description is getting completely refactored, writing remains surprisingly unaffected. LLMs have gotten very good at generating code, but I am appalled at the absolute shit they spew as prose. They always follow the same robotic cadence and cliches, and sprinkle the same tired vocabulary all around. They take my broken yet soulful writing and transform it into a plastic soulless word slop in the name of improving prose. Their writing communicates no actual understanding and insight. I think we are all developing a visceral ick reaction to AI writing. It is trapped in the uncanny valley, and it may be stuck there for ...
[](http://muratbuffalo.blogspot.com/2026/08/the-safest-job-from-ai-may-be-writing.html)
[Read more >>](http://muratbuffalo.blogspot.com/2026/08/the-safest-job-from-ai-may-be-writing.html "The Safest Job from AI may be Writing")
The Two Abstractions of System Design: Hide or Reduce
- [May 08, 2026](http://muratbuffalo.blogspot.com/2026/05/the-two-abstractions-of-system-design.html "permanent link")
When talking about TLA+, I keep referring to "abstraction" as the most important thing to learn . And it is about the hardest to learn as well. But a contradiction has been bugging me. Aren't CS people already supposed to be good at abstraction? Isn't abstraction supposed to be at the root of OS, networking, software engineering? Abstract Data Types (ADTs) are a staple of every in CS curriculum. So why do I (and every other formal methods/modeling person) see such a large skill gap in abstraction, and flag it as the core, make-or-break skill for modeling? I think I finally get to the root of this cognitive disonance. There are two kinds of "abstraction" conflated under the same umbrella term. Modularity abstraction: This is the traditional abstraction taught in CS curricula as ADTs, APIs, layered design, etc. It is all about encapsulation, drawing boundaries, and hiding internals. Modeling abstraction: This is what I talk about when I talk about abstracti...
[](http://muratbuffalo.blogspot.com/2026/05/the-two-abstractions-of-system-design.html)
[Read more >>](http://muratbuffalo.blogspot.com/2026/05/the-two-abstractions-of-system-design.html "The Two Abstractions of System Design: Hide or Reduce")
The Agentic Self: Parallels Between AI and Self-Improvement
- [January 02, 2026](http://muratbuffalo.blogspot.com/2026/01/the-agentic-self-parallels-between-ai.html "permanent link")
2025 was the year of the agent. The goalposts for AGI shifted; we stopped asking AI to merely "talk" and demanded that it "act". As an outsider looking at the architecture of these new agents and agentic system, I noticed something strange. The engineering tricks used to make AI smarter felt oddly familiar. They read less like computer science and more like … self-help advice . The secret to agentic intelligence seems to lie in three very human habits: writing things down, talking to yourself, and pretending to be someone else. They are almost too simple. The Unreasonable Effectiveness of Writing One of the most profound pieces of advice I ever read as a PhD student came from Prof. Manuel Blum, a Turing Award winner. In his essay "Advice to a Beginning Graduate Student", he wrote: "Without writing, you are reduced to a finite automaton. With writing you have the extraordinary power of a Turing machine." If you try to hold a complex argument enti...
[](http://muratbuffalo.blogspot.com/2026/01/the-agentic-self-parallels-between-ai.html)
[Read more >>](http://muratbuffalo.blogspot.com/2026/01/the-agentic-self-parallels-between-ai.html "The Agentic Self: Parallels Between AI and Self-Improvement")
In Search of a Compositional Theory of Self-Stabilization
- [September 21, 2026](http://muratbuffalo.blogspot.com/2026/09/in-search-of-compositional-theory-of.html "permanent link")
My quest for a principled solution for metastable failures has taken me back to my roots on self-stabilization, as this recent paper related the problem to composition of self-stabilizing systems. But, my literature search for recent work on composing self-stabilizing systems didn't yield anything useful. The layered stabilization idea was already in place by the early 2000s, and nothing fundamental seems to have been added since. Frustrating. So I decided to attack the problem using the concrete example I have. I had composed a rely-guarantee TLA+ model of a retry storm as two components with contracts . That model reproduces metastable failure because the composition that worked from good states failed to work when a large shock removes the base case that let the two conditions hold each other up. Searching for rely-guarantee based composition from every state, turned up a 2017 control theory paper by Kim, Arcak and Seshia, "A Small Gain Theorem for Parametric Assume-Gua...
[](http://muratbuffalo.blogspot.com/2026/09/in-search-of-compositional-theory-of.html)
[Read more >>](http://muratbuffalo.blogspot.com/2026/09/in-search-of-compositional-theory-of.html "In Search of a Compositional Theory of Self-Stabilization")
Learning about distributed systems: where to start?
- [June 10, 2020](http://muratbuffalo.blogspot.com/2020/06/learning-about-distributed-systems.html "permanent link")
This is definitely not a "learn distributed systems in 21 days" post. I recommend a principled, from the foundations-up, studying of distributed systems, which will take a good three months in the first pass, and many more months to build competence after that. If you are practical and coding oriented you may not like my advice much. You may object saying, "Shouldn't I learn distributed systems with coding and hands on? Why can I not get started by deploying a Hadoop cluster, or studying the Raft code." I think that is the wrong way to go about learning distributed systems, because seeing similar code and programming language constructs will make you think this is familiar territory, and will give you a false sense of security. But, nothing can be further from the truth. Distributed systems need radically different software than centralized systems do. --A. Tannenbaum This quotation is literally the first sentence in my distributed systems syllabus. Inst...
[](http://muratbuffalo.blogspot.com/2020/06/learning-about-distributed-systems.html)
[Read more >>](http://muratbuffalo.blogspot.com/2020/06/learning-about-distributed-systems.html " Learning about distributed systems: where to start?")
Building a Database on S3
- [March 04, 2026](http://muratbuffalo.blogspot.com/2026/03/building-database-on-s3.html "permanent link")
Hold your horses, though. I'm not unveiling a new S3-native database. This paper is from 2008. Many of its protocols feel clunky today. Yet it nails the core idea that defines modern cloud-native databases: separate storage from compute. The authors propose a shared-disk design over Amazon S3, with stateless clients executing transactions. The paper provides a blueprint for serverless before the term existed. SQS as WAL and S3 as Pagestore The 2008 S3 was painfully slow, and 100 ms reads weren't unusual. To hide that latency, the database separates "commit" from "apply". Clients write small, idempotent redo logs to Amazon Simple Queue Service (SQS) instead of touching S3 directly. An asynchronous checkpoint by a client applies those logs to B-tree pages on S3 later. This design shows strong parallels to modern disaggregated architectures . SQS becomes the write-ahead log (WAL) and logstore. S3 becomes the pagestore. Modern Aurora follows a similar logic : t...
[](http://muratbuffalo.blogspot.com/2026/03/building-database-on-s3.html)
[Read more >>](http://muratbuffalo.blogspot.com/2026/03/building-database-on-s3.html "Building a Database on S3")
Foundational distributed systems papers
- [February 27, 2021](http://muratbuffalo.blogspot.com/2021/02/foundational-distributed-systems-papers.html "permanent link")
I talked about the importance of reading foundational papers last week. To followup, here is my compilation of foundational papers in the distributed systems area. (I focused on the core distributed systems area, and did not cover networking, security, distributed ledgers, verification work etc. I even left out distributed transactions, I hope to cover them at a later date.) I classified the papers by subject, and listed them in chronological order. I also listed expository papers and blog posts at the end of each section. Time and State in Distributed Systems Time, Clocks, and the Ordering of Events in a Distributed System. Leslie Lamport, Commn. of the ACM, 1978. Distributed Snapshots: Determining Global States of a Distributed System. K. Mani Chandy Leslie Lamport, ACM Transactions on Computer Systems, 1985. Virtual Time and Global States of Distributed Systems. Mattern, F. 1988. Practical uses of synchronized clocks in distributed systems. B. Liskov, 1991. Exp...
[](http://muratbuffalo.blogspot.com/2021/02/foundational-distributed-systems-papers.html)
[Read more >>](http://muratbuffalo.blogspot.com/2021/02/foundational-distributed-systems-papers.html " Foundational distributed systems papers")
Specula: Scaling formal specifications for autonomous model checking of system code
- [August 12, 2026](http://muratbuffalo.blogspot.com/2026/08/specula-scaling-formal-specifications.html "permanent link")
Specula is an agentic system that automates the process of software bug finding through authoring and model-checking a spec for the code. It derives TLA+ specifications automatically from the code, checks code-spec conformance through trace validation, model checks the spec to find concurrency bugs, and reproduces the bug at the code layer by writing integration tests with precise timing. I remember reading the Daikon paper "Quickly detecting relevant program invariants" in 2000 and getting impressed by it, and here we are after 26 years, solving the end-to-end problem much better than I ever thought would be possible in a push-button manner in the year of our lord 2026. But somehow, I am still somewhat unsatisfied with the paper. This may be me being hypercritical and trying to get more out of the paper by arguing with it . So bear with me until I resolve (or learn to accept) these problems over time. I know many of the authors of the Specula work, and respect them, and I...
[](http://muratbuffalo.blogspot.com/2026/08/specula-scaling-formal-specifications.html)
[Read more >>](http://muratbuffalo.blogspot.com/2026/08/specula-scaling-formal-specifications.html "Specula: Scaling formal specifications for autonomous model checking of system code")
Cloudspecs: Cloud Hardware Evolution Through the Looking Glass
- [January 09, 2026](http://muratbuffalo.blogspot.com/2026/01/cloudspecs-cloud-hardware-evolution.html "permanent link")
This paper (CIDR'26) presents a comprehensive analysis of cloud hardware trends from 2015 to 2025, focusing on AWS and comparing it with other clouds and on-premise hardware. TL;DR: While network bandwidth per dollar improved by one order of magnitude (10x), CPU and DRAM gains (again in performance per dollar terms) have been much more modest. Most surprisingly, NVMe storage performance in the cloud has stagnated since 2016. Check out the NVMe SSD discussion below for data on this anomaly. CPU Trends Multi-core parallelism has skyrocketed in the cloud. Maximum core counts have increased by an order of magnitude over the last decade. The largest AWS instance u7in now boasts 448 cores. However, simply adding cores hasn't translated linearly into value. To measure real evolution, the authors normalized benchmarks (SPECint, TPC-H, TPC-C) by instance cost. SPECint benchmarking shows that cost-performance improved roughly 3x over ten years. A huge chunk of that gain comes from AWS G...
[](http://muratbuffalo.blogspot.com/2026/01/cloudspecs-cloud-hardware-evolution.html)
[Read more >>](http://muratbuffalo.blogspot.com/2026/01/cloudspecs-cloud-hardware-evolution.html "Cloudspecs: Cloud Hardware Evolution Through the Looking Glass")
Theme images by Michael Elkan
Murat Demirbas
I am a principal research scientist at MongoDB Research. Ex-AWS. Ex-professor at SUNY Buffalo. I work on distributed systems, databases, cloud computing, formal methods, and neurosymbolic AI. You can follow me on Mastodon or Twitter.
Subscribe via email
Pageviews
**7****3****8****4****0****0****1**
Recent Posts
- August 5
- July 3
- June 7
- May 5
- April 5
- March 13
- August 8
- July 7
- June 2
- May 8
- April 2
- March 3
- August 9
- July 3
- June 5
- May 2
- April 6
- March 6
- August 4
- July 5
- June 2
- May 3
- April 3
- March 3
- August 5
- July 6
- June 3
- May 3
- April 4
- March 3
- August 4
- July 2
- June 12
- May 1
- April 1
- March 4
- August 3
- July 6
- June 11
- May 9
- April 8
- March 8
- July 3
- June 3
- May 4
- April 6
- March 2
- August 8
- July 2
- June 4
- May 9
- April 6
- March 9
- July 3
- June 3
- May 3
- April 4
- July 4
- June 5
- May 1
- April 4
- March 2
- August 3
- June 1
- May 1
- April 6
- March 6
- August 2
- July 2
- June 3
- March 3
- August 2
- July 4
- June 2
- May 5
- April 8
- August 1
- May 2
- March 1
- June 1
- May 5
- April 5
- March 5
- August 1
Show more Show less
Topics
2PC 2abstraction 11agentic 21AI 32analytics 3atomic storage 2auditability 5automated reasoning 20aws 5Azure 11
benchmarks 4bestof 9big-data 33Blockchain 39book-review 57bugs 1calm 5chaos 2claude 2cloud computing 21ConcurrencyControlBook 7consistency 34Cosmos DB 11CosmosDB 12crdts 2data warehouse 2databases 71datacenter networking 3dataflow 11dbos 1DDIA 15disaggregation 10distributed consensus 54distributed transactions 44distSQL 10facebook 16failures 18fault-tolerance 50formal methods 25graph-processing 1hpts 5htap 3humans 10indexing 4isolation levels 5joke 1linearizability 1links 2liveDiscussion 9mad-questions 42main-memory 2measuring 1metastability 9microservices 2misc 133ML 1mlbegin 7mldl 26mobile 2mongodb 11MVCC 4my advice 27my-paper 12networking 1newsql 3NoSQL 3OLAP 2OLTP 7ostep 8paper-review 154paxos 55postgres 1presenting 4privacy 1programming 7query-processing 2raft 4RDMA 3reading-group 27recent-reads 2reconfiguration 4research-advice 58research-question 45Rust 4scheduling 5security 1seminar 9serializability 4serverless 2smartphones 2snapshot isolation 9sonification 1SQL 6stabilization 8statistics 3stream-processing 12teaching 33tensorflow 11testing 5time 23time synchronization 8timeDB 8tla 71tpbook 1transactions 42trip-report 45wpaxos 6writing 32
Show more Show less