Your uploads shouldn’t touch your server

We joined a government identity project and the first thing we did was read the cloud bill. It was eight times larger than it needed to be, for three separate reasons. This post is about the third one, because it’s the one almost everyone gets wrong and the one that scales worst.

Every application in that system was an upload. Photographs of faces, photographs of documents, biometrics captured in the field. And every one of those files was travelling through the backend on its way to storage.

What routing through the backend actually costs

The pattern looks harmless. A file arrives at your server, middleware buffers it, your code validates it, and then you push it to object storage:

app.post('/upload', upload.single('file'), async (req, res) => {
  await s3.putObject({
    Bucket: BUCKET,
    Key: key(req),
    Body: req.file.buffer,
  })
  res.json({ ok: true })
})

Three things happen that you don’t see on a small deployment.

Every byte occupies your instance’s memory. A 4 MB photograph is 4 MB of RAM for the duration of the request. Fifty concurrent uploads is 200 MB doing nothing but waiting. You size the instance for peak upload concurrency rather than for the work your application actually does.

Every byte crosses the network twice. Client to server, then server to storage. You pay for transit that had no reason to exist, and you pay for it again in latency, because the client’s upload isn’t finished until your server’s upload is finished.

Your throughput ceiling is your instance, not your storage. Object storage will absorb far more concurrent writes than any single application server. By putting the server in the middle you’ve replaced a service designed for this with one that isn’t.

None of this shows up as an error. It shows up as a larger instance, a slower upload, and a bill you assume is just what things cost.

The alternative

The client uploads directly to storage. Your server never sees the bytes — it only decides whether the upload may happen, and signs a URL that permits exactly that one thing.

// server: authorize, then sign
app.post('/upload-url', requireAuth, async (req, res) => {
  const key = `applications/${req.user.id}/${randomUUID()}`

  const url = await getSignedUrl(
    s3,
    new PutObjectCommand({
      Bucket: BUCKET,
      Key: key,
      ContentType: req.body.contentType,
    }),
    { expiresIn: 300 },
  )

  res.json({ url, key })
})

// client: upload straight to storage
const { url, key } = await api.post('/upload-url', {
  contentType: file.type,
})

await fetch(url, {
  method: 'PUT',
  body: file,
  headers: { 'Content-Type': file.type },
})

await api.post('/applications', { photoKey: key })

The server has done three small things: confirmed who is asking, decided where the file may go, and set a short expiry. It has moved zero bytes.

The parts people get wrong

Validation. You’ve lost the ability to inspect the file before it lands, so validate what you can before signing — content type, declared size, and who is asking — and validate the rest after the object exists. In practice that means a step that reads the stored object and rejects it if it isn’t what it claimed to be.

Nobody told the server the upload finished. The client tells you, and clients lie or disappear. Either confirm the object exists before treating the record as complete, or subscribe to storage events and let the record complete itself.

Orphans. Signed URLs that were used but never confirmed leave objects nobody references. A lifecycle rule that expires unreferenced objects in a prefix costs nothing and saves you finding out about them a year later.

Expiry. Five minutes is usually right. Long enough for a slow connection, short enough that a leaked URL is worthless by the time anyone finds it.

Scope the permission narrowly. Sign for one key, one method, one content type. A signed URL is a capability — treat it as one.

What it changed

On that project the change was one of three, and the three together took the monthly bill from $24,000 to $2,000. This one mattered most because of what the product was: in a system where every application is an upload, the upload path is the application.

The instances got smaller because they no longer had to hold files. The uploads got faster because they stopped making two trips. And the ceiling moved from something we ran to something Amazon runs.

When routing through the server is still right

Not never. If you must transform the file before it’s stored — resize, transcode, strip metadata, scan for malware before it exists anywhere — then the bytes have to reach something that can do that work.

But that’s an argument for a worker that pulls from storage after the fact, not for putting your API server in the path of every byte. And it’s worth checking whether you actually need it, or whether you’re doing it because the tutorial did.

This came out of building a national ID and passport issuance system, where the same review also found over-provisioning and infinite loops. Read the case study →