Module 4 · Lesson 3 of 6
Stateless servers and sessions
Why a login breaks on server two, and how to fix it.
The second API server went live at 09:55. At 10:02 support gets the first message: "I logged in, picked my seats, pressed Pay, and it asked me to log in again." Nothing is broken on either server. The bug is that each one remembers different things, and the load balancer doesn't know or care which one you talked to last.
The bug: memory that lives on one server
On the single server, logging in worked like this. The fan sent their phone number and OTP; the server checked it, created a session ({ userId: 812, expires: … }), stored it in a JavaScript Map in its own memory, and sent back a cookie with the session id. Every later request carried the cookie, and the server looked the id up in its map.
With two servers behind a round-robin balancer, the login request lands on API 1, which stores the session in its memory. The "pick seats" request lands on API 2, which has never heard of that session id. To API 2, the fan is a stranger.
Sessions are the most common case, but the same trap catches anything a server keeps for itself:
- Uploaded files saved to the server's local disk: the poster exists on API 1, so API 2 returns a 404 for it.
- In-memory caches of data that changes: each server has its own copy, so they disagree after an update.
- Rate-limit counters kept in memory: a scraper gets four times the limit because each server counts separately.
- Scheduled jobs started with
setIntervalin the server process: with four servers, the "send reminders" job runs four times. - WebSocket connections: a seat update that happens on API 1 never reaches fans connected to API 2 (module 6).
The fix: stateless servers
A stateless server keeps nothing between requests that another server would need. Everything a request depends on either comes with the request or lives in a shared store every server can reach. Any server can handle any request, so the balancer can send traffic anywhere, and you can add, remove or restart servers freely.
"Stateless" doesn't mean the system has no state. Gatepass is full of state. It means the state has moved out of the API servers into places built to hold it: the database, Redis, object storage. The servers become interchangeable workers.
Stateless servers don't remove state. They move it somewhere every server can reach, so any server can answer any request.
Three ways to fix sessions
1. Sticky sessions. Configure the balancer to send each fan back to the same server, usually by setting its own cookie. It's a config change, no code. But it only hides the problem: when that server crashes or is replaced in a deploy, its fans are logged out. Load also becomes uneven, because a server stays stuck with whoever it got first. Fine as a stopgap, a poor foundation.
2. A shared session store. Keep sessions in Redis instead of process memory. The cookie still holds a random session id; every server looks it up in the same Redis (GET session:s_9f2, about 0.5 ms). Logging out means deleting the key, so it takes effect instantly everywhere. This is the most common choice and the one Gatepass uses.
3. Signed tokens. Put the session data itself in the cookie or Authorization header, signed by the server so it can't be forged, like a JWT: { userId: 812, exp: 1735689600 } plus a signature. Any server holding the key can verify it without looking anything up. No session store at all. The catch is revocation: you can't easily "log out" a token that is valid for another hour, so tokens are kept short-lived (5–15 minutes) and paired with a refresh token that is checked against a store.
Check yourself
Gatepass uses sticky sessions with sessions in each server's memory. During the sale you deploy a fix, replacing servers one at a time. What do fans experience?
Moving everything else out
Apply the same thinking to each kind of local state:
- Files go to object storage like S3, never the server's disk (lesson 4.5).
- Caches of shared data move to Redis. A small per-server in-memory cache is fine for data that's identical everywhere and rarely changes, like a city list with a short TTL, but never for anything that must be consistent.
- Rate-limit counters move to Redis, shared by all servers (module 8).
- Scheduled jobs run in one dedicated worker, or use a lock so only one server runs each job (module 7).
- Configuration comes from environment variables or a config service, the same on every server.
A good test: could you kill any API server at any moment, start a fresh one from the same image, and have no user notice? If yes, your servers are stateless.
Your turn
Audit this single-server Gatepass code for state that breaks with two servers: sessions in a Map, posters saved to ./uploads, a setInterval that emails reminders every hour, and an in-memory cache of seat counts. For each, what goes wrong and where should it live instead?
Key takeaways
- With more than one server, anything a server keeps in its own memory or disk is invisible to the others.
- A stateless server keeps nothing between requests that another server would need. Any server can serve any request.
- Sticky sessions hide the problem until a server dies. A shared session store (like Redis) or short-lived signed tokens solve it.
- Move files to object storage, shared caches and counters to Redis, and scheduled jobs to a single worker.
- If you can kill any server at any time without users noticing, your servers are stateless.
Finished reading?
Mark it complete to tick it off in the course.