AWS – S3: Buckets, Policies and Static Sites

S3 stores objects and hands them back by key. Almost everything else in AWS leans on it — CloudFront serves from it, Athena queries it, CloudTrail and Firehose write into it, and every static site ends up there. This site is a few thousand HTML files in a bucket.

It is not a filesystem

There are no directories. A key is a single string that happens to contain slashes, and the console draws folders by grouping on them. Practical consequences:

• Renaming a "folder" means copying every object and deleting the originals
• An empty prefix does not exist — nothing to hold it
• Listing is by prefix, and listing a bucket with millions of keys is paginated and slow
• There is no partial write. An object is replaced whole, atomically

Reads are strongly consistent: a successful PutObject is immediately visible to the next GetObject. That has been true since 2020 and a lot of older advice about "eventual consistency" is now obsolete.

Storage classes

Those minimum durations are the part that bites. Move an object to Standard-IA and delete it a week later and you are billed for the full 30 days anyway, so a lifecycle rule that transitions short-lived objects can cost more than leaving them in Standard. For anything genuinely unpredictable, Intelligent-Tiering does this for you and is usually the right default.

aws s3api put-bucket-lifecycle-configuration --bucket my-logs --lifecycle-configuration '{
  "Rules": [{
    "ID": "archive-then-expire",
    "Status": "Enabled",
    "Filter": { "Prefix": "logs/" },
    "Transitions": [{ "Days": 30, "StorageClass": "STANDARD_IA" }],
    "Expiration": { "Days": 365 }
  }]
}'

Versioning is an undo button with a bill

With versioning on, overwriting keeps the old version and deleting writes a delete marker rather than removing anything. It is the best protection against a bad deploy or a mistaken rm, and it is why your storage cost can grow while the object count looks flat.

Two things follow. Versioning cannot be switched off once enabled — only suspended, which stops new versions and keeps existing ones. And you need a lifecycle rule for NoncurrentVersionExpiration, or old versions accumulate forever.

Block Public Access should stay on

The old way to host a static site was to make a bucket public and enable website hosting. Do not do that. Public buckets are the single most common cause of AWS data exposure, and the modern pattern is better in every respect.

Keep Block Public Access enabled and serve through CloudFront with Origin Access Control. The bucket stays entirely private; CloudFront is the only thing that can read it, and you get HTTPS, a real certificate, caching and edge locations as a side effect.

This is the actual policy on the bucket serving this site:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "AllowCloudFrontOAC",
    "Effect": "Allow",
    "Principal": { "Service": "cloudfront.amazonaws.com" },
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::lovemesomecoding.com/*",
    "Condition": {
      "StringEquals": {
        "AWS:SourceArn": "arn:aws:cloudfront::111122223333:distribution/E30YUPLP37MY9U"
      }
    }
  }]
}

Without the SourceArn condition that would read "any CloudFront distribution anywhere may read this bucket", which is a mistake people ship regularly.

Two flags that have cost this site real time

aws s3 sync skips unchanged files, so metadata never updates. Changing Cache-Control in a deploy script does nothing to objects already in the bucket — same size, same timestamp, skipped. Non-fingerprinted files have to be uploaded with cp --recursive instead.

--metadata-directive REPLACE without --content-type rewrites every object to binary/octet-stream. The directive replaces all metadata, content type included, and browsers then download your HTML instead of rendering it.

aws s3 cp ./out s3://lovemesomecoding.com --recursive \
  --exclude "_next/static/*" \
  --cache-control "public, max-age=0, must-revalidate" \
  --content-type "text/html"

Getting files in and out without a server

A presigned URL grants time-limited access to one object using your credentials, so a browser can upload or download directly. Your application never proxies the bytes:

aws s3 presign s3://my-bucket/report.pdf --expires-in 3600

For uploads, prefer a presigned POST, which lets you constrain content type and maximum size — a presigned PUT accepts whatever the client sends.

Encryption and access logs

Every object is encrypted at rest by default with SSE-S3, at no cost and with nothing to configure. Switch to SSE-KMS when you need an audit trail of who decrypted what, or a key you can revoke — but note KMS charges per request, so a bucket serving millions of objects can run up a meaningful KMS bill. S3 Bucket Keys cut that substantially and are worth enabling if you go this route.

For "who did what to this bucket", CloudTrail data events are the answer rather than S3 server access logs — they are structured, near real-time, and queryable. Both cost money at volume, and neither is on by default.

Costs that surprise people

Storage is rarely the interesting line. Data transfer out to the internet usually is, and serving through CloudFront is cheaper than serving from S3 directly as well as faster. Request counts matter at scale — millions of small GETs add up.

And incomplete multipart uploads are invisible: a failed large upload leaves parts that bill as storage and appear in no normal listing. Add a lifecycle rule to abort them after 7 days and forget about it.