Polling looks simple until several traders, endpoints and worker replicas all run on the same second. A production scheduler must limit total throughput, spread requests over time and recover without creating a retry storm.
Budget requests before scheduling
Fomo API allows five requests per second per API key. Reserve capacity for retries and operator requests. If you target four scheduled requests per second, a 50-trader swap poll takes at least 12.5 seconds before other work.
Assign intervals by freshness:
| Resource | Typical interval |
|---|---|
| Swaps | 5–15 seconds |
| Balances | 15–60 seconds |
| Ranking | 5–15 minutes |
| Leaderboard | 30–60 minutes |
| Profile | 6–24 hours |
Use a queue, not nested timers
Place jobs in one queue ordered by nextRunAt. A worker takes only the number permitted by a shared rate limiter. After completion, calculate the next run and add a small random offset.
nextRunAt = completedAt + interval + random(0, jitter)
Jitter prevents every process from waking simultaneously after deploy or restart.
Respect server instructions
On 429, read Retry-After and delay at least that long. On transient gateway failures, exponential backoff is appropriate:
delay = min(base × 2^attempt, maximum) + jitter
Cap attempts and total elapsed time. Do not retry 401, 403, 404 or 422 as though they were transient.
Coordinate replicas
An in-memory limiter protects one process only. Multiple workers sharing a key need a distributed token bucket, a single queue consumer group or key-specific partitions. Add a database lease so two workers cannot poll the same trader and endpoint concurrently.
Detect silent failure
Track both the last successful request and the newest processed event. Alert when either checkpoint is too old. Include the reason a job is delayed: rate budget, backoff, lease contention or downstream storage failure.
The goal is not maximum request volume. It is predictable freshness without dropped events, duplicate work or unfair API traffic.