Files, object storage and CDNs

Every Gatepass event has a poster, every venue has a seat-map image, and every booking produces a PDF ticket. On the single server, all of these went into an ./uploads folder on its disk. Now there are four interchangeable servers, any of which can be replaced at any moment. Files need a home that isn't any of them.

Why files don't belong on API servers

A file on an API server's local disk breaks every promise of the previous lessons:

• Other servers can't see it. The organiser uploads a poster through API 1; a fan's request for it lands on API 3 and gets a 404.
• It dies with the server. Stateless servers are replaced on every deploy and every autoscaling event. Their disks go with them.
• It doesn't scale. A 400 KB poster served 2 million times is 800 GB of traffic flowing through your API servers, using CPU and bandwidth they need for bookings.

You could put files in the database as blobs. Databases can store them, but it's a poor fit: it bloats backups, fills the database's memory cache with image bytes instead of hot rows, and makes your most expensive, hardest-to-scale component serve your cheapest kind of data.

Object storage

Object storage (Amazon S3, Google Cloud Storage, Cloudflare R2, Azure Blob Storage, or MinIO if you host it yourself) is a service built for exactly this. Its model is deliberately simple:

• You store objects (any bytes, from a few bytes to terabytes) under a key like posters/evt_8812/3f9a1c.jpg, inside a bucket.
• You can PUT an object, GET it, DELETE it and list keys by prefix. You can't edit part of an object; you replace the whole thing.
• Storage is effectively unlimited, very durable (copies across several data centres, typically quoted at "eleven nines" of durability) and cheap, roughly $0.02 per GB per month.
• Every server, worker and CDN can reach the same object over HTTP.

The trade-off is that it's not a filesystem or a database. There are no queries, no transactions, and a request takes tens of milliseconds, not microseconds. That's fine, because you don't query posters. You fetch them by name.

The shape: database for metadata, bucket for bytes

The database still knows about every file; it just doesn't hold the bytes. The events table gets a poster_key column: posters/evt_8812/3f9a1c.jpg. The page renders an image URL from that key. The bytes live in the bucket.

Uploads: don't stream through your servers

The naive upload sends the file to your API server, which then forwards it to the bucket. Your server spends time and memory holding a 20 MB video for the length of a slow mobile upload. Do that for 500 organisers at once and your API servers are busy being a pipe.

The better pattern is a pre-signed URL:

1. The organiser's browser asks your API: "I want to upload a poster for event 8812."
2. Your API checks they're allowed to, picks the key (posters/evt_8812/<random>.jpg), and asks the storage SDK for a URL that permits one PUT to that key, valid for 5 minutes. No network call is needed; it's a signature.
3. The browser uploads the file directly to the bucket using that URL.
4. The browser tells your API the upload finished; your API records the key in the database. (Or the bucket sends an event when the object lands.)

Your servers only handle two tiny JSON requests. The bytes never touch them. Downloads of private files work the same way: a signed GET URL that expires, so a ticket PDF can be fetched only by its owner and only for a while.

Serving: put a CDN in front

Object storage can serve files directly, but it's in one region and charges for every byte out. A CDN in front (lesson 3.7) fixes both. The poster's key includes a content hash, so it's cached with max-age=31536000, immutable and each edge fetches it from the bucket once. Two million fans, a few hundred bucket reads.

For private files, like ticket PDFs, use short-lived signed URLs and Cache-Control: private. They're read rarely, so they don't need the CDN.

Other things you get

• Lifecycle rules. "Move PDFs older than 90 days to the cheap archive tier; delete abandoned uploads after a day." Configured once on the bucket.
• Versioning. Keep previous versions of overwritten or deleted objects, which protects you from a buggy cleanup script.
• Events. "When an object lands in uploads/raw/, start a job." That's how image resizing works: the organiser uploads one large poster, a worker writes 400 px, 800 px and 1600 px versions back to the bucket.

Check yourself

Organisers upload 50 MB promo videos. Uploads are slowing down the booking API. What's the best fix?

AAdd more API serversBStore videos in Postgres insteadCHave browsers upload directly to object storage with pre-signed URLsDCompress the videos on the API server before storing them

Your turn

Design how Gatepass stores and serves three kinds of file: event posters, seat-map images and ticket PDFs. For each, give the key format, who can read it, and how it's served.

01

02

03

Compare with an expertWrite at least 2 first.

Key takeaways

• Files on an API server's disk break statelessness: other servers can't see them and they vanish on replacement.
• Object storage stores bytes under keys, durably and cheaply, reachable by every server. It isn't a database or a filesystem.
• Keep metadata (the key, owner, size) in the database and bytes in the bucket.
• Upload and download with pre-signed URLs so bytes skip your API servers.
• Put a CDN in front of public files, with content-hashed keys so they can be cached forever.