Benchmarks

Methodology first. Numbers when they are reproducible.

We publish the benchmark design and harness before any results, so anyone can check that the test is fair and run it themselves. This page has no numbers yet, on purpose.

Status: pending, run in progress

No results are published yet. Any Orch8 performance number you see elsewhere that does not link back to a run documented here is not ours.

Methodology

Steady throughput

Start N workflow instances. Each runs 3 sequential steps, and each step is an activity that sleeps 50 ms in an external worker and returns. Measure completed workflows per second and end-to-end latency per workflow.

Crash recovery

Start the same N × 3-step load. While instances are mid-flight, kill the engine process with SIGKILL (no graceful shutdown), then restart it. Measure time from restart until every instance completes, and count activity executions beyond the expected 3 × N.

Same hardware

Every engine runs on the same machine with the same database version, the same worker language and concurrency, and default durability settings. The hardware, OS, database, and engine versions are published with the results.

• The workload is deliberately boring: 3 steps and a fixed 50 ms activity. That isolates orchestration overhead (claiming, persisting, scheduling) from handler cost.
• Every engine gets a warm-up period, and each configuration is run several times. We report the median run and publish all raw runs.
• Recovery is judged on correctness first. An engine that drains faster but repeats completed activities is reported as such, not ranked higher.
• Competitor setups follow their official self-hosting guides at the versions listed with the results. Corrections from maintainers are welcome and will be re-run.

Results

Results table. Every cell is pending until a reproducible run is published.
Metric	Orch8 (PostgreSQL)	Comparison engines
Completed workflows per second (steady state)	Pending, run in progress	Pending, run in progress
End-to-end latency per workflow, p50 and p99	Pending, run in progress	Pending, run in progress
Time to drain all instances after a crash and restart	Pending, run in progress	Pending, run in progress
Extra activity executions after the crash (above 3 × N)	Pending, run in progress	Pending, run in progress
Peak engine memory (RSS)	Pending, run in progress	Pending, run in progress

The harness

Load comes from the engine repository's loadgen/ package, a continuous workflow load generator with light, steady, and stress presets. The benchmark procedure is being written up in docs/BENCHMARKS.md, which will be published with the results.

The load generator does not send API-key or tenant headers. Run it only against an isolated local engine and a disposable database, never against a shared or production environment.

terminal

1# terminal 1: isolated local engine (never shared/production)
2ORCH8_HTTP_ADDR=127.0.0.1:18080 \
3ORCH8_API_KEY="" \
4ORCH8_REQUIRE_TENANT_HEADER=false \
5ORCH8_ALLOW_NO_TENANT_ISOLATION=1 \
6orch8-server --config orch8.toml --insecure
7
8# terminal 2: from the engine repo root
9cd loadgen && npm install
10npm run steady   # or: light, stress

While you wait

Correctness under failure matters more than raw throughput. You can check that part today on your own machine.

Crash-recovery demo →Engine comparisons →

Verify the technical details

The documentation is the source of truth for current behavior.

Read the documentation →