1M Durable Workflows per Second on Postgres
!Image 4: logo
[](https://www.cookiebot.com/en/what-is-behind-powered-by-cookiebot/?utm_source=banner_cb&utm_medium=referral&utm_content=v2)
- Consent
- Details
- [[#IABV2SETTINGS#]](https://www.dbos.dev/blog/one-million-workflows#)
- About
This website uses cookies
We use cookies to personalise content and ads, to provide social media features and to analyse our traffic. We also share information about your use of our site with our social media, advertising and analytics partners who may combine it with other information that you’ve provided to them or that they’ve collected from your use of their services.
Show more
[#GPC_BANNER_ICON#]
[#GPC_TOAST_TEXT#]
Consent Selection
**Necessary**
- [x]
**Preferences**
- [x]
**Statistics**
- [x]
**Marketing**
- [x]
Details
* Necessary 17- [x] Necessary cookies help make a website usable by enabling basic functions like page navigation and access to secure areas of the website. The website cannot function properly without these cookies.
- Cookiebot 2Learn more about this provider**CookieConsent[x2]**Stores the user's cookie consent state for the current domain**Maximum Storage Duration**: 1 year**Type**: HTTP Cookie
- HubSpot 1Learn more about this provider**cookietest**This cookie is used to determine if the visitor has accepted the cookie consent box.**Maximum Storage Duration**: Session**Type**: HTTP Cookie
- LinkedIn 2Learn more about this provider**bcookie**Used in order to detect spam and improve the website's security. **Maximum Storage Duration**: 1 year**Type**: HTTP Cookie **li_gc**Stores the user's cookie consent state for the current domain**Maximum Storage Duration**: 180 days**Type**: HTTP Cookie
- hs-analytics.net hs-banner.com hubspotusercontent-na1.net linkedin.com 12**__cf_bm[x12]**This cookie is used to distinguish between humans and bots. This is beneficial for the website, in order to make valid reports on the use of their website.**Maximum Storage Duration**: 1 day**Type**: HTTP Cookie
* Preferences 2- [x] Preference cookies enable a website to remember information that changes the way the website behaves or looks, like your preferred language or the region that you are in.
- Cookiebot 1Learn more about this provider**CookieConsentBulkSetting-#**Enables cookie consent across multiple websites**Maximum Storage Duration**: 1 year**Type**: HTML Local Storage
- LinkedIn 1Learn more about this provider**lidc**Registers which server-cluster is serving the visitor. This is used in context with load balancing, in order to optimize user experience. **Maximum Storage Duration**: 1 day**Type**: HTTP Cookie
* Statistics 6- [x] Statistic cookies help website owners to understand how visitors interact with websites by collecting and reporting information anonymously.
- HubSpot 4Learn more about this provider**__hssc**Identifies if the cookie data needs to be updated in the visitor's browser.**Maximum Storage Duration**: 1 day**Type**: HTTP Cookie **__hssrc**Used to recognise the visitor's browser upon reentry on the website.**Maximum Storage Duration**: Session**Type**: HTTP Cookie **__hstc**Sets a unique ID for the session. This allows the website to obtain data on visitor behaviour for statistical purposes.**Maximum Storage Duration**: 180 days**Type**: HTTP Cookie **hubspotutk**Sets a unique ID for the session. This allows the website to obtain data on visitor behaviour for statistical purposes.**Maximum Storage Duration**: 180 days**Type**: HTTP Cookie
- Matomo 2Learn more about this provider**_pk_id#**Collects statistics on the user's visits to the website, such as the number of visits, average time spent on the website and what pages have been read.**Maximum Storage Duration**: 1 year**Type**: HTTP Cookie **_pk_ses#**Used by Piwik Analytics Platform to track page requests from the visitor during the session.**Maximum Storage Duration**: 1 day**Type**: HTTP Cookie
* Marketing 14- [x] Marketing cookies are used to track visitors across websites. The intention is to display ads that are relevant and engaging for the individual user and thereby more valuable for publishers and third party advertisers.
- Google 2Learn more about this providerSome of the data collected by this provider is for the purposes of personalization and measuring advertising effectiveness. The provider may use the IP Addresses for ads measurement and ads personalization.
**_ga**Used to send data to Google Analytics about the visitor's device and behavior. Tracks the visitor across devices and marketing channels.**Maximum Storage Duration**: 2 years**Type**: HTTP Cookie **_ga_#**Used to send data to Google Analytics about the visitor's device and behavior. Tracks the visitor across devices and marketing channels.**Maximum Storage Duration**: 2 years**Type**: HTTP Cookie
- HubSpot 1Learn more about this provider**__ptq.gif**Sends data to the marketing platform Hubspot about the visitor's device and behaviour. Tracks the visitor across devices and marketing channels.**Maximum Storage Duration**: Session**Type**: Pixel Tracker
- YouTube 11Learn more about this provider**__Secure-ROLLOUT_TOKEN**Used to track user’s interaction with embedded content.**Maximum Storage Duration**: 180 days**Type**: HTTP Cookie **__Secure-YEC**Stores the user's video player preferences using embedded YouTube video**Maximum Storage Duration**: Session**Type**: HTTP Cookie **__Secure-YNID**Used to track user’s interaction with embedded content.**Maximum Storage Duration**: 180 days**Type**: HTTP Cookie **LAST_RESULT_ENTRY_KEY**Used to track user’s interaction with embedded content.**Maximum Storage Duration**: Session**Type**: HTTP Cookie **LogsDatabaseV2:V#||LogsRequestsStore**Used to track user’s interaction with embedded content.**Maximum Storage Duration**: Persistent**Type**: IndexedDB **ServiceWorkerLogsDatabase#SWHealthLog**Necessary for the implementation and functionality of YouTube video-content on the website. **Maximum Storage Duration**: Persistent**Type**: IndexedDB **TESTCOOKIESENABLED**Used to track user’s interaction with embedded content.**Maximum Storage Duration**: 1 day**Type**: HTTP Cookie **VISITOR_INFO1_LIVE**Tries to estimate the users' bandwidth on pages with integrated YouTube videos.**Maximum Storage Duration**: 180 days**Type**: HTTP Cookie **YSC**Registers a unique ID to keep statistics of what videos from YouTube the user has seen.**Maximum Storage Duration**: Session**Type**: HTTP Cookie **yt-icons-last-purged**Necessary for the implementation and functionality of YouTube video-content on the website. **Maximum Storage Duration**: Persistent**Type**: HTML Local Storage **YtIdbMeta#databases**Used to track user’s interaction with embedded content.**Maximum Storage Duration**: Persistent**Type**: IndexedDB
- Unclassified 7 Unclassified cookies are cookies that we are in the process of classifying, together with the providers of individual cookies.
- Amazon 6Learn more about this provider**__test__**Pending**Maximum Storage Duration**: Session**Type**: HTTP Cookie **_reb2bgeo**Pending**Maximum Storage Duration**: Session**Type**: HTTP Cookie **_reb2bloaded**Pending**Maximum Storage Duration**: Session**Type**: HTTP Cookie **_reb2bsessionID**Pending**Maximum Storage Duration**: Session**Type**: HTTP Cookie **_reb2buid**Pending**Maximum Storage Duration**: Session**Type**: HTTP Cookie **_reb2buid**Pending**Maximum Storage Duration**: Persistent**Type**: HTML Local Storage
- HubSpot 1Learn more about this provider**hubspot-modern-theme**Pending**Maximum Storage Duration**: Persistent**Type**: HTML Local Storage
Your consent applies to the following domains:
List of domains your consent applies to:console.dbos.devwww.dbos.dev
Cookie declaration last updated on 10/27/26 by Cookiebot
[#IABV2_TITLE#]
[#IABV2_BODY_INTRO#]
[#IABV2_BODY_LEGITIMATE_INTEREST_INTRO#]
[#IABV2_BODY_PREFERENCE_INTRO#]
[#IABV2_LABEL_PURPOSES#]
[#IABV2_BODY_PURPOSES_INTRO#]
[#IABV2_BODY_PURPOSES#]
[#IABV2_LABEL_FEATURES#]
[#IABV2_BODY_FEATURES_INTRO#]
[#IABV2_BODY_FEATURES#]
[#IABV2_LABEL_PARTNERS#]
[#IABV2_BODY_PARTNERS_INTRO#]
[#IABV2_BODY_PARTNERS#]
About
Cookies are small text files that can be used by websites to make a user's experience more efficient.
The law states that we can store cookies on your device if they are strictly necessary for the operation of this site. For all other types of cookies we need your permission.
This site uses different types of cookies. Some cookies are placed by third party services that appear on our pages.
You can at any time change or withdraw your consent from the Cookie Declaration on our website.
Learn more about who we are, how you can contact us and how we process personal data in our Privacy Policy.
Please state your consent ID and date when you contact us regarding your consent.
- [x]
**Do not sell or share my personal information**
Deny Allow selection Customize Allow all
Join us Nov 12 for Durable Stories Vol. 1: Postgres, Postgres Everywhere
[](https://www.dbos.dev/blog/one-million-workflows#)
Products
DBOS Transact Open source durable execution libraryDBOS Conductor Control plane for agents & workflows
Resources
About DBOS Meet the team simplifying reliability.Videos Demos, deep dives, and more DBOS.Partners Explore the DBOS ecosystem.
Docs
Quickstart Launch your first workflow in minutes.Docs Step-by-step guides & real-world use cases.Example Applications Ready-to-run code to spark your project.
[](https://discord.com/invite/jsmC6pXGgX)
Explore
5K
dbos/dbos-transact-py Durable Execution Library - Pythondbos/dbos-transact-ts Durable Execution Library - TypeScriptdbos-inc/dbos-transact-go Durable Execution Library - Godbos/dbos-transact-rust Durable Execution Library - Rustdbos-inc/dbos-transact-java Durable Execution Library - Javadbos-inc/dbos-demo-apps Demo DBOS applicationsSee all repos
1M Durable Workflows per Second on Postgres
Peter Kraft
October 28, 2026
Postgres
Share this post
[](https://www.linkedin.com/shareArticle?mini=true&url=https://www.dbos.dev/blog/one-million-workflows&title=1M%20Durable%20Workflows%20per%20Second%20on%20Postgres)
[](https://twitter.com/intent/tweet?url=https://www.dbos.dev/blog/one-million-workflows&hashtags=dbos)
[](https://www.facebook.com/sharer/sharer.php?u=https://www.dbos.dev/blog/one-million-workflows)
[](mailto:?subject=Check%20out%20this%20article%20from%20DBOS&body=Check%20out%20this%20article%20from%20DBOS%3A%0Ahttps://www.dbos.dev/blog/one-million-workflows)
When building a durable workflow engine on Postgres, one of the most common questions we get is “does Postgres scale?” Now, there has been no shortage of blog posts on the subject (including ours), and at this point it’s beating a dead horse to say that a single Postgres database can get you remarkably far, handling tens of thousands of operations per second.
But what if you want to go past that? What if you have a workload that requires more than one database can handle, perhaps hundreds of thousands or millions of operations per second? Can Postgres scale then? In this blog post, we’ll show how we scaled Postgres-backed workflows to >1M workflow/sec.
1M Workflows/Sec
We’ll start with the good part: the results. By sharding durable workflows (using PgDog, but similar results are achievable with any sharding engine), we linearly scaled durable workflow throughput from 20K workflows/sec with one Postgres server to 1M workflows/sec with 50 Postgres servers:
Here’s the benchmark design:
- We scale the number of shards from 1 to 50. Each shard is a db.m7i.4xlarge RDS Postgres with 16 cores and 64 GB of RAM.
- We scale worker processes across several client hosts. We launch 48 worker processes per shard, for a total of 2400 worker processes at 50 shards. All worker processes are identical. Each worker process connects to a PgDog router on its host.
- Worker processes enqueue 20K workflows/shard/sec, up to 1M workflows/sec at 50 shards. We chose this number because 20K workflows/sec is the maximum sustainable throughput of a single shard. The benchmark uses simple no-op workflows with no steps.
- Worker processes enqueue all workflows into a single durable queue. Each workflow has a uniformly random shard key, so workflows are evenly distributed across shards. All worker processes dequeue and execute workflows from this single queue.
- All benchmark code is open-source on GitHub.
How Sharding Works
Because Postgres is a single-node database, the only way to scale to >1M workflows/sec (well beyond the capacity of a single server, however large), is to shard it. Sharding is a technique for horizontally scaling a single-node database by distributing work across many database servers, called “shards”. There are many popular sharding engines: we used PgDog because it’s lightweight and open-source; other Postgres engines include Citus, Neki, and Multigres, and maybe the most popular sharding engine overall is Vitess for MySQL.
Sharding engines are powerful because they distribute work across many servers but present users with the abstraction that they’re working with a single database. To make this work, users select a “shard key” column for each of their tables. The sharding engine partitions each table by its shard key, creating an instance of each table on each shard and assigning data items to shards based on their shard keys. Data items with the same shard keys are always assigned to the same shard, while data items with different shard keys could be stored on different shards. At a high level, sharding looks like this:
The core of a sharding engine is a **router** that parses queries and distributes them across the appropriate shards. If the query only needs data from a single shard (for example a SELECT query that specifies a single shard key value), that’s easy: the engine routes the query to that single shard and serves its response. If the query needs data from many shards (for example, an aggregation that does not specify a shard key), that’s harder: the engine must construct a distributed query plan that retrieves data from all those shards then combines their responses.
To make the performance of a sharded workload scale linearly, we must ensure each query can be served by a single shard. Otherwise, coordination between shards limits performance. Thus, to shard workflows, we add a `shard_key` column both to the main `workflow_status` table and to all dependent tables (such as the `workflow_steps` describing each workflows’ steps). This way, all information about a single workflow is stored on a single shard, and all operations done on a single workflow (such as enqueueing it, recording a step checkpoint, or sending a message to it) can be served by a single shard. Similarly, we dequeue workflows by shard: instead of dequeuing workflows from all shards at once, each process iterates through all shards, dequeueing workflows from one shard at a time.
Metastability and Feedback Loops
When we initially tested sharded workflows, we found that past a certain point (roughly 8-16 shards, or ~200K workflows/sec), performance was metastable: it would initially appear to scale linearly, but if any shard was perturbed in any way (for example, a large VACUUM briefly slowed down the shard), cluster-wide performance would collapse and never recover.
The reason for this metastability was a negative feedback loop in dequeueing workflows. The query for finding workflows to dequeue looks like the SQL below. Each process periodically polls each queue and shard, finding a number (determined by flow control) of eligible enqueued workflows on that queue and shard and dequeuing them. To minimize contention between workers, the query uses a locking clause (`FOR UPDATE SKIP LOCKED`), locking the workflows it selects and selecting the first unlocked workflows in the queue.
!Image 23: Dequeue SQL statement using FOR UPDATE SKIP LOCKED
The issue with this query is that it is susceptible to a negative feedback loop at a high process count. Let’s say some event (such as a large VACUUM) causes a shard to momentarily slow down, so that each dequeue query takes longer. Because each dequeue query takes longer, more processes concurrently attempt to dequeue from that shard. The problem is that if more processes try to dequeue from a shard, queries on that shard take even longer (due to lock contention, earlier queries hold locks so newer queries have to search ahead to find unlocked workflows). This loop feeds back on itself until millions of rows are locked by queries taking several seconds each, collapsing performance.
To fix this issue (and others like it), we had to stop the negative feedback loop by limiting concurrent accesses to a single shard. We did this using Postgres advisory locks. Each shard has a small number of “dequeue locks”. To dequeue from the shard, a process has to obtain one of the locks. If it can’t obtain a lock, it doesn’t wait, but rather moves on to the next shard and tries to dequeue from it instead. This design prevents a single shard from being overwhelmed, breaking the feedback loop and allowing stable scaling to 1M workflows/sec.
Learn More
If you like building scalable, reliable systems, we’d love to hear from you. At DBOS, our goal is to make Postgres-backed durable execution as simple and performant as possible. Check it out:
- Quickstart:https://docs.dbos.dev/quickstart
- GitHub:https://github.com/dbos-inc
- Discord community: https://discord.gg/eMUHrvbu67
Insights
Recent articles
The latest in durable execution, AI workflows &more.
[](https://www.dbos.dev/blog/whats-new-in-dbos-october-2026)
Product news
Oct 29, 2026
What's New in DBOS - October 2026
Workflow rewind, safer distributed execution, DBOS Transact Enterprise, and scaling to 1M+ workflows per second on sharded Postgres!
Qian Li
[](https://www.dbos.dev/blog/applications-are-dead-long-live-workflows)
How To
Oct 9, 2026
Applications are Dead. Long Live Workflows
Applications should be replaced by workflows as the unit of middleware management
Mike Stonebraker
[](https://www.dbos.dev/blog/whats-new-in-dbos-september-2026)
Product news
Sep 29, 2026
What's New in DBOS - September 2026
DBOS for Rust, autoscaling control, faster data retention, new integrations, and more!
Qian Li
Build your reliable backend.
Effortlessly.
Get started with DBOS
Use the open source DBOS Transact library, free forever. Pair it with DBOS Pro for premium tooling and support.
Products
DBOS TransactDBOS ConductorPricing PlansDBOS vs. Temporal
Customer Stories
Bristol Myers SquibbYutoriDosuAll stories
Developers
DocsQuickstart GuideExamplesTutorialsLLMs.txt
Company
Contact SalesBlogAbout UsOur TeamPartnersCareers
© 2026 DBOS, Inc.
Privacy PolicyTerms of ServiceOpen Patent PledgeCookies Settings
[](https://discord.com/invite/jsmC6pXGgX)[](https://github.com/dbos-inc)[](https://twitter.com/DBOS_Inc)[](https://www.linkedin.com/company/dbos-inc/)[](https://www.youtube.com/@DBOS-Inc)