Resource Pools & Rotation

Distribute work across multiple resources — API keys, email accounts, proxy servers, or any rate-limited service. Built-in rotation strategies, daily caps, and warmup ramps ensure even distribution without overloading any single resource.

Why Resource Pools

Many workflows interact with rate-limited external services. Without rotation, a single API key or mail server becomes a bottleneck:

●Email deliverability — rotate across SMTP accounts to avoid sender reputation damage
●API rate limits — distribute calls across multiple keys to stay under per-key quotas
●Proxy rotation — cycle through IPs to avoid blocks during scraping
●Load balancing — spread traffic across servers with weight-based distribution
●New resource onboarding — warmup ramps prevent new accounts from hitting limits on day one

Failure story: one key starts returning 429

Mark the resource unhealthy or exhausted so selection moves to eligible peers. Preserve the provider idempotency key during workflow retries. Do not immediately recycle the failing credential: wait for its rate window or health probe, then re-enable it deliberately.

Core Concepts

Concept	Description
Pool	A named collection of resources with a rotation strategy. Tenant-scoped.
Resource	A single unit in a pool (e.g., one API key, one mail server). Has a unique `resource_key`.
Strategy	How the next resource is selected: round_robin, weighted, or random.
Daily Cap	Maximum assignments per day per resource. Resets at midnight. 0 = unlimited.
Warmup	Gradual daily cap increase over N days for newly added resources.
Assignment	Result of requesting a resource: Assigned(key), Exhausted, or Empty.

Rotation Strategies

round_robin — Default

Assigns resources in order, cycling through all enabled resources with capacity. Tracks position via round_robin_index. Skips resources that have hit their daily cap.

weighted

Assigns proportional to each resource's weight. A resource with weight 3 receives ~3x more assignments than weight 1. Respects daily caps — exhausted resources are excluded from the weighted selection.

random

Random uniform selection among enabled resources with remaining capacity. Useful when distribution doesn't need to be perfectly even.

Assignment results: When all resources hit their daily cap, assignment returns Exhausted. If no resources are enabled, it returns Empty. Workflows should handle these cases (retry later or fail).

Warmup Ramps

New email accounts or freshly provisioned API keys often have lower limits initially. Warmup ramps gradually increase the daily cap from a starting value to the full cap over a configurable number of days.

How It Works

The effective daily cap is linearly interpolated:

1effective_cap = warmup_start_cap + (daily_cap - warmup_start_cap) * days_active / warmup_days
2
3Example: warmup_start_cap=10, daily_cap=100, warmup_days=10
4
5Day 0:  cap = 10
6Day 3:  cap = 10 + (90 * 3 / 10) = 37
7Day 5:  cap = 10 + (90 * 5 / 10) = 55
8Day 10: cap = 100 (warmup complete)
9Day 15: cap = 100 (stays at full)

Configuration

Field	Type	Description
warmup_start	date (YYYY-MM-DD)	When the warmup period begins
warmup_days	u32	Number of days to ramp from start cap to full cap. 0 = no warmup.
warmup_start_cap	u32	Starting daily cap on day 0 of warmup

Pool API

Create Pool

JSON

1POST /pools
2
3{
4  "tenant_id": "tenant-1",
5  "name": "Email Senders",
6  "strategy": "round_robin"
7}
8
9Response (201):
10{
11  "id": "550e8400-...",
12  "tenant_id": "tenant-1",
13  "name": "Email Senders",
14  "strategy": "round_robin",
15  "round_robin_index": 0,
16  "created_at": "2024-01-15T10:00:00Z",
17  "updated_at": "2024-01-15T10:00:00Z"
18}

List Pools

JSON

1GET /pools?tenant_id=tenant-1

Add Resource

JSON

1POST /pools/{pool_id}/resources
2
3{
4  "resource_key": "smtp-account-1",
5  "name": "Primary SMTP",
6  "weight": 2,
7  "daily_cap": 500,
8  "warmup_start": "2024-01-15",
9  "warmup_days": 14,
10  "warmup_start_cap": 50
11}
12
13Response (201):
14{
15  "id": "...",
16  "pool_id": "...",
17  "resource_key": "smtp-account-1",
18  "name": "Primary SMTP",
19  "weight": 2,
20  "enabled": true,
21  "daily_cap": 500,
22  "daily_usage": 0,
23  "daily_usage_date": null,
24  "warmup_start": "2024-01-15",
25  "warmup_days": 14,
26  "warmup_start_cap": 50,
27  "created_at": "2024-01-15T10:00:00Z"
28}

Update Resource

JSON

1PUT /pools/{pool_id}/resources/{resource_id}
2
3{
4  "enabled": false,
5  "daily_cap": 1000
6}

Delete Resource

JSON

1DELETE /pools/{pool_id}/resources/{resource_id}
2
3Response: 204 No Content

List Resources

JSON

1GET /pools/{pool_id}/resources
2
3Response: Array of PoolResource objects with current daily_usage

Validation: resource_key andname must be 1-255 characters. weight must be at least 1.

Using Pools in Workflows

Workflows request a resource from a pool before executing rate-limited steps. The assigned resource_key is used to look up the corresponding credential or configuration.

Example: Email Rotation

JSON

1[
2  {
3    "id": "get-sender",
4    "type": "pool_assign",
5    "params": {
6      "pool_id": "email-senders-pool-id"
7    }
8  },
9  {
10    "id": "send-email",
11    "type": "http_request",
12    "params": {
13      "url": "https://api.sendgrid.com/v3/mail/send",
14      "headers": {
15        "Authorization": "Bearer credentials://{{steps.get-sender.output.resource_key}}"
16      },
17      "body": {
18        "from": "{{steps.get-sender.output.resource_key}}@example.com",
19        "to": "{{context.recipient}}",
20        "subject": "Hello"
21      }
22    }
23  }
24]

Handling Exhaustion

JSON

1{
2  "id": "get-api-key",
3  "type": "pool_assign",
4  "params": { "pool_id": "openai-keys-pool" },
5  "retry": {
6    "max_attempts": 3,
7    "backoff": "exponential",
8    "initial_delay_ms": 60000
9  }
10}

When all resources are exhausted, the step fails with a transient error. A retry policy with delay gives resources time to reset at midnight.

Production Patterns

Multi-Provider LLM Pool

Create a pool with weighted resources: OpenAI (weight: 3, cap: 10000), Anthropic (weight: 2, cap: 5000), Groq (weight: 1, cap: 2000). The engine distributes calls proportionally while respecting per-provider rate limits.

SMTP Warmup

Add new mail accounts with warmup_start_cap: 20,daily_cap: 500, warmup_days: 30. Over a month, sending volume gradually increases, building sender reputation without triggering spam filters.

Graceful Degradation

Disable a resource via PATCH enabled: false when it's experiencing issues. The pool automatically routes to remaining resources. Re-enable when resolved — no workflow changes needed.

Capacity Monitoring

Poll GET /pools/{id}/resources to checkdaily_usage vs daily_cap across resources. Alert when total remaining capacity drops below threshold.

Previous← Signals & LLM Usage NextStreaming & Approvals →

Was this helpful?