Caching and rate limits for football APIs

The core pattern is simple. A single server-side job polls each feed on a schedule and writes the result into a cache, such as a database table, a key-value store or a file. Your web pages read only from that cache. A thousand readers looking at the same table cost you one request, not a thousand.

This also protects your key. The browser never sees it, because the browser never talks to the API.

Keep the cache shared across all your web servers. If each server polls independently, you multiply requests by the number of servers without gaining freshness.

Match each feed's refresh interval to how often it changes. Season fixtures change slowly, so a few times a day suffices. Standings change after matches, so refresh after the final whistle. Squads change mostly during transfer windows. Historical seasons barely change at all. Media can be cached for days.

On match days, add targeted refreshes for the competitions you feature, but avoid tight polling loops. Without a live service, nothing is gained by refreshing every few seconds.

Estimate daily volume first: multiply distinct requests per refresh by refreshes per day, then allow for detail requests, successful refreshes and other workers. The midnight UTC reset has no rollover. A 6,000-successful-request day on Starter creates 1,000 extra requests, costing €2.00 on a metered subscription. A quiet day tomorrow does not cancel today's overage. Failed or denied requests are not billed, but repeated failures still need investigation.

If the total is too high, lengthen intervals for slow-changing feeds first. That usually saves far more than shaving seconds off match-day polling.

FootballAPI.net applies a shared account safety limit of 100 requests per second, across the account's keys. Spread requests instead of firing every job at the top of the minute. A small queue with bounded concurrency prevents accidental bursts. Creating another key does not create another account allowance or another independent safety budget.

When you receive a 429 response, stop and wait. If a Retry-After header is present, honour it exactly. Retry with exponential backoff and a cap, and never retry in a tight loop, which turns a short limit into a long outage.

When a refresh fails, keep serving the last good copy. Mark it with its fetch time so readers can see how current it is. This stale-while-revalidate behaviour means a short upstream problem is invisible to most readers, while a longer one is clearly signposted.

Never replace a good cached value with an empty or error response. Validate each new response before writing it to the cache.

Build cache keys from the feed identifier and its parameters, for example the feed name with the league identifier and season. Normalise parameter order so equivalent requests share a key. Include a version in the key if you change how you process responses, so old and new formats do not collide.

Track request counts per feed, error rates by status code and cache age per key. Alert when a cache grows older than you expect for that feed. The usage page in your dashboard shows daily totals and recent requests, including denied attempts, which helps you spot runaway jobs before they consume your allowance.

Review the numbers weekly at first, then monthly once things settle.

A site covers two competitions. Fixtures refresh three times a day, standings twice a day plus after the last match, squads weekly, and media monthly. A single worker runs these jobs through a queue with modest concurrency. Pages read from a shared database. On a busy Saturday, traffic rises tenfold while request volume rises only slightly, because the extra readers all hit the cache.

When the upstream source briefly returns errors, the worker backs off, readers see cached data with its timestamp, and the next successful refresh updates everything.

Media behaves differently from JSON. Crests and photos change rarely, so long browser cache lifetimes are appropriate. Because FootballAPI.net serves images through its own media route with caching, your pages can reference those URLs directly. Avoid cache-busting query strings that defeat caching, and set explicit dimensions so pages render while images arrive.

If you build a content delivery layer of your own, respect the display terms: keep images on your product and do not republish them as a separate collection.

Use separate keys for development, staging and production to make rotation and request diagnosis easier, but remember that keys on the same account share daily usage. An unattended development loop can consume the same allowance as production. Give each environment an explicit polling budget, and use saved, redacted fixtures for tests that do not require fresh data.

After a deployment or a cache reset, warm the cache deliberately with a paced job rather than letting the first readers trigger misses. Pacing keeps you inside per-second limits, and warm caches mean the first visitors get the same fast pages as everyone else.