Steady throughput
Start N workflow instances. Each runs 3 sequential steps, and each step is an activity that sleeps 50 ms in an external worker and returns. Measure completed workflows per second and end-to-end latency per workflow.
Benchmarks
Methodology first. Numbers when they are reproducible.
We publish the benchmark design and harness before any results, so anyone can check that the test is fair and run it themselves. This page has no numbers yet, on purpose.
Status: pending, run in progress
No results are published yet. Any Orch8 performance number you see elsewhere that does not link back to a run documented here is not ours.
Methodology
Crash recovery
Start the same N × 3-step load. While instances are mid-flight, kill the engine process with SIGKILL (no graceful shutdown), then restart it. Measure time from restart until every instance completes, and count activity executions beyond the expected 3 × N.
Same hardware
Every engine runs on the same machine with the same database version, the same worker language and concurrency, and default durability settings. The hardware, OS, database, and engine versions are published with the results.
- • The workload is deliberately boring: 3 steps and a fixed 50 ms activity. That isolates orchestration overhead (claiming, persisting, scheduling) from handler cost.
- • Every engine gets a warm-up period, and each configuration is run several times. We report the median run and publish all raw runs.
- • Recovery is judged on correctness first. An engine that drains faster but repeats completed activities is reported as such, not ranked higher.
- • Competitor setups follow their official self-hosting guides at the versions listed with the results. Corrections from maintainers are welcome and will be re-run.
Results
| Metric | Orch8 (PostgreSQL) | Comparison engines |
|---|---|---|
| Completed workflows per second (steady state) | Pending, run in progress | Pending, run in progress |
| End-to-end latency per workflow, p50 and p99 | Pending, run in progress | Pending, run in progress |
| Time to drain all instances after a crash and restart | Pending, run in progress | Pending, run in progress |
| Extra activity executions after the crash (above 3 × N) | Pending, run in progress | Pending, run in progress |
| Peak engine memory (RSS) | Pending, run in progress | Pending, run in progress |
The harness
Load comes from the engine repository's loadgen/ package, a continuous workflow load generator with light, steady, and stress presets. The benchmark procedure is being written up in docs/BENCHMARKS.md, which will be published with the results.
The load generator does not send API-key or tenant headers. Run it only against an isolated local engine and a disposable database, never against a shared or production environment.
terminal
# terminal 1: isolated local engine (never shared/production)
ORCH8_HTTP_ADDR=127.0.0.1:18080 \
ORCH8_API_KEY="" \
ORCH8_REQUIRE_TENANT_HEADER=false \
ORCH8_ALLOW_NO_TENANT_ISOLATION=1 \
orch8-server --config orch8.toml --insecure
# terminal 2: from the engine repo root
cd loadgen && npm install
npm run steady # or: light, stressDistributed
Distributed: control node plus executors
The single-node benchmark above deliberately excludes horizontally scaled deployments. Self-hosted multi-node setups, where one control node and several executor nodes share the same PostgreSQL, need their own measurements. This section defines them before any run.
Status: results pending
No distributed result files exist in the engine repository yet, and the automated harness does not start multi-node stacks. The procedure below is manual and reproducible; numbers will be published only with their raw files, as for the single-node runs.
Scale-out
Run the steady workload against one control node with 1 and then 3 executors on the same host and database. Measure completed workflows per second and latency at each size. The database is shared, so the curve shows where PostgreSQL becomes the limit.
Executor loss
While instances are mid-flight, SIGKILL one executor. Measure the time until its claimed instances complete on the surviving executors, and count activity executions beyond 3 × N. Execution is at-least-once, so duplicates are reported, not hidden.
Graceful drain
Drain one executor instead of killing it. Measure how long the drain takes and confirm that every in-flight instance either finishes or is recoverable without duplicate activities.
| Metric | 1 executor | 3 executors |
|---|---|---|
| Completed workflows per second (steady state) | Results pending | Results pending |
| End-to-end latency per workflow, p50 and p99 | Results pending | Results pending |
| Time to finish reclaimed instances after an executor SIGKILL | Results pending | Results pending |
| Extra activity executions after executor loss (above 3 × N) | Results pending | Results pending |
| Time to drain one executor gracefully | Results pending | Results pending |
Run it yourself
Use a disposable database and loopback addresses only. Every node points at the same PostgreSQL; the control node serves the API and the executors run the scheduler. Record the host, core count, PostgreSQL version, and engine version with your results.
terminal
# shared, disposable PostgreSQL 16
docker run -d --name orch8-bench-pg -e POSTGRES_PASSWORD=bench -p 55432:5432 postgres:16
export ORCH8_DATABASE_URL=postgres://postgres:bench@127.0.0.1:55432/postgres
export ORCH8_ENCRYPTION_KEY=$(openssl rand -hex 32)
export ORCH8_API_KEY="" ORCH8_REQUIRE_TENANT_HEADER=false ORCH8_ALLOW_NO_TENANT_ISOLATION=1
# control node: API only, no scheduler
ORCH8_NODE_ROLE=control ORCH8_HTTP_ADDR=127.0.0.1:18080 ORCH8_GRPC_ADDR=127.0.0.1:50051 \
orch8-server --config orch8.toml --insecure &
# three executors: scheduler only, health endpoint on its own port
for i in 1 2 3; do
ORCH8_NODE_ROLE=executor ORCH8_HTTP_ADDR=127.0.0.1:1808$i ORCH8_GRPC_ADDR=127.0.0.1:5005$i \
orch8-server --config orch8.toml --insecure &
done
# load against the control node, from the engine repo root
cd loadgen && npm install
LOADGEN_SERVER=http://127.0.0.1:18080 npm run steady
# executor loss: SIGKILL one executor mid-run and time the recovery
kill -9 <pid-of-one-executor>Instances held by a killed executor are reclaimed after stale_instance_threshold_secs (default 300 s) and worker tasks after worker_reaper_stale_secs (default 60 s). Report the values you ran with; lowering them shortens recovery and increases the chance of duplicate activity executions.
While you wait
Correctness under failure matters more than raw throughput. You can check that part today on your own machine.
Verify the technical details
The documentation is the source of truth for current behavior.