Stop S3 storage bill shock: migration to low-cost storage

Here is an uncomfortable truth about S3 storage: the part everyone quotes, the price per gigabyte stored, is almost never the part that hurts. Storing 50 GB of user uploads costs about as much as a sandwich. The line items that turn a quiet month into a surprise invoice are the ones nobody models on a napkin: data transfer out, request charges, forgotten object versions, half-finished multipart uploads and the NAT gateway silently billing you for traffic to your own bucket.

If you are a solo founder with a Dockerized app, a handful of beta users and at least one recent bill that made your stomach drop, this post is for you. We will take apart where S3 storage money goes, decide honestly whether you should leave Amazon S3 at all, and then walk through a migration to predictable S3-compatible storage without rewriting your app.

Where S3 storage bills actually come from

Amazon S3 pricing has several dimensions. At the time of writing, S3 Standard in us-east-1 is roughly $0.023 per GB-month for storage. Always check the current AWS pricing page for your region, because the numbers change and vary by region. What matters more than the exact figures is the shape of the bill.

1. Data transfer out (the big one)

Every byte served from S3 to the public internet is billed as egress, after a monthly free allowance. If your app serves images, videos, PDFs or downloads directly from a bucket, egress scales with traffic, not with how much you store.

This is the classic bill-shock pattern: a post goes semi-viral, a scraper hammers your public bucket, or someone hotlinks your assets from a busy forum. Your stored data did not change, but your bill did.

2. Request charges

S3 bills per request: PUT, COPY, POST and LIST cost more than GET. Individually they cost fractions of a cent. At scale they add up, especially from:

• thumbnails generated on every page load instead of cached,
• a sync job that runs LIST across a big prefix every minute,
• health checks or backup tools that list buckets obsessively.

3. Things you stored and forgot

• Noncurrent versions. Turn on versioning (a good idea) without a lifecycle rule (a bad idea), and every overwrite keeps the old copy forever. You pay for all of them.
• Incomplete multipart uploads. Failed large uploads leave invisible parts behind. They do not show up in a normal ls, but they are billed as storage.
• Old backups in the same bucket as live assets.

4. The network path around S3

If your app runs on AWS in a private subnet and reaches S3 through a NAT gateway, you may be paying NAT data processing charges on top of S3 itself. A free S3 gateway endpoint fixes this, but only if you know to set it up.

S3 storage is cheap. S3 traffic, requests and forgotten data are where the surprises live. Before you migrate anywhere, find out which of these is actually on your bill.

Step 0: audit your current bill

Do not migrate on vibes. Spend 30 minutes finding the real cost drivers.

In AWS Cost Explorer, filter by service S3 and group by Usage type. Look for usage types containing:

• DataTransfer-Out-Bytes: internet egress,
• Requests-Tier1 / Requests-Tier2: PUT/LIST and GET requests,
• TimedStorage-ByteHrs: actual storage.

Then measure what is really in your buckets:

# Total size and object count per bucket
aws s3 ls s3://my-app-uploads --recursive --summarize --human-readable | tail -2

# Incomplete multipart uploads (invisible storage you pay for)
aws s3api list-multipart-uploads --bucket my-app-uploads

# Are old versions piling up?
aws s3api list-object-versions --bucket my-app-uploads \
  --query 'length(Versions[?IsLatest==`false`])'

Two quick wins you can apply today, even if you never migrate:

{
  "Rules": [
    {
      "ID": "abort-incomplete-mpu",
      "Status": "Enabled",
      "Filter": {},
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    },
    {
      "ID": "expire-old-versions",
      "Status": "Enabled",
      "Filter": {},
      "NoncurrentVersionExpiration": { "NoncurrentDays": 30 }
    }
  ]
}

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-app-uploads \
  --lifecycle-configuration file://lifecycle.json

If egress is your problem, putting a CDN in front of the bucket and making the bucket private is the other big lever. That is also exactly where "just use S3" starts turning into "now I am configuring CloudFront, origin access control and cache policies."

Should you leave Amazon S3 at all?

Being contrarian about S3 costs does not mean S3 is bad. It is one of the most reliable services on the internet. Be honest about which camp you are in.

Stay on Amazon S3 if:

• your app already lives deep in AWS (Lambda triggers on uploads, Athena queries, IAM roles everywhere),
• you rely on features like S3 Object Lock for compliance, Glacier-class archival or S3 event notifications,
• you have terabytes of data and a team that already knows how to tune it.

Consider moving if:

• you are a small team whose app runs outside AWS, so every file served crosses the internet and is billed as egress,
• you have three or four vendors (app host, database host, AWS for storage, a CDN) and each has its own bill and failure mode,
• you cannot predict next month's bill within a reasonable margin,
• your storage is "user uploads plus static assets," not a data lake.

Most pre-seed SaaS apps fit the second list. You use S3 as a place to put avatars, attachments and exports, through a standard SDK. That use case only needs the S3 API, not Amazon specifically.

Amazon S3 vs Granite.so for a small Dockerized app

S3 compatibility is what makes migration realistic: your code talks the S3 API, and you point it at a different endpoint. Here is how the two options compare for the founder profile above.

Two honest caveats:

1. "S3-compatible" is never 100%. Core operations such as PutObject, GetObject, DeleteObject, ListObjectsV2, multipart uploads and presigned URLs are the parts virtually every app uses, and they are what compatible providers implement. Niche features (Object Lock, S3 Select, event notifications, some bucket policy conditions) vary by provider. If you depend on one, verify it before migrating.
2. Predictable does not mean unlimited. Usage beyond included credits is still usage. The difference is that storage, CDN, app and database show up on one bill you can read, instead of four dashboards you have to reconcile. Check the current Granite.so pricing page for exact storage and bandwidth terms.

The migration: step by step

The plan below works for any S3-compatible target. The examples assume a Next.js or Node app with Postgres, but the pattern is the same in Python, Go or Ruby.

Step 1: make your storage endpoint configurable

If your code hardcodes AWS, fix that first. Everything should come from environment variables:

S3_ENDPOINT=https://<your-storage-endpoint>
S3_REGION=auto
S3_BUCKET=my-app-uploads
S3_ACCESS_KEY_ID=...
S3_SECRET_ACCESS_KEY=...
PUBLIC_ASSET_BASE_URL=https://<your-cdn-domain>

With the AWS SDK for JavaScript v3:

import { S3Client } from "@aws-sdk/client-s3";

export const s3 = new S3Client({
  region: process.env.S3_REGION ?? "us-east-1",
  endpoint: process.env.S3_ENDPOINT, // undefined = real AWS
  forcePathStyle: true,              // many S3-compatible providers expect this
  credentials: {
    accessKeyId: process.env.S3_ACCESS_KEY_ID!,
    secretAccessKey: process.env.S3_SECRET_ACCESS_KEY!,
  },
});

Use whatever region and addressing style your provider documents. This one change is 80% of the migration. Ship it while still pointing at AWS, and confirm nothing breaks.

Step 2: stop storing full URLs in your database

If your uploads table stores https://my-app-uploads.s3.amazonaws.com/avatars/42.png, every URL has to be rewritten on migration. Store the object key (avatars/42.png) and build the URL at render time:

export const assetUrl = (key: string) =>
  `${process.env.PUBLIC_ASSET_BASE_URL}/${key}`;

If you already have full URLs stored, a one-time SQL update handles it:

UPDATE uploads
SET file_key = regexp_replace(file_url, '^https://[^/]+/', '')
WHERE file_key IS NULL;

Step 3: copy the data with rclone

rclone is the standard tool for bucket-to-bucket copies between S3 providers. You can configure both sides with environment variables, so no config file needs to be committed anywhere:

# Source: Amazon S3
export RCLONE_CONFIG_AWS_TYPE=s3
export RCLONE_CONFIG_AWS_PROVIDER=AWS
export RCLONE_CONFIG_AWS_ACCESS_KEY_ID=AKIA...
export RCLONE_CONFIG_AWS_SECRET_ACCESS_KEY=...
export RCLONE_CONFIG_AWS_REGION=us-east-1

# Destination: any S3-compatible storage
export RCLONE_CONFIG_DEST_TYPE=s3
export RCLONE_CONFIG_DEST_PROVIDER=Other
export RCLONE_CONFIG_DEST_ACCESS_KEY_ID=...
export RCLONE_CONFIG_DEST_SECRET_ACCESS_KEY=...
export RCLONE_CONFIG_DEST_ENDPOINT=https://<your-storage-endpoint>

# Dry run first
rclone copy aws:my-app-uploads dest:my-app-uploads --dry-run

# Real copy, with parallelism and progress
rclone copy aws:my-app-uploads dest:my-app-uploads \
  --transfers 16 --checkers 32 --progress

Things to know before you press enter:

• You will pay AWS egress once for the data you move out. For a few dozen GB that is small. For large datasets, AWS has announced a process for waiving egress fees when customers move data off AWS. It goes through AWS support, so check the current terms.
• rclone copy never deletes anything at the destination. rclone sync does. Use copy until you are sure.
• Metadata such as Content-Type and Cache-Control is preserved by rclone for S3-to-S3 copies. Spot-check a few objects afterwards anyway.

Step 4: verify

rclone check aws:my-app-uploads dest:my-app-uploads --one-way
rclone size aws:my-app-uploads
rclone size dest:my-app-uploads

Object counts and total sizes should match. rclone check compares hashes where both providers expose them.

Step 5: cut over with a safety net

The low-drama cutover for a small app:

1. Run a final rclone copy to catch files uploaded since the first pass.
2. Switch the S3_* environment variables in your app to the new endpoint and redeploy.
3. Run one more rclone copy right after the deploy to pick up anything uploaded during the switch.
4. For a week, add a read fallback: if an object is missing at the new provider, fetch it from the old one and log it. If the log stays empty, you are done.
5. Make the old bucket read-only, wait a billing cycle, and only then delete it.

Do not delete the old bucket the same day you migrate. One billing cycle of storage costs almost nothing. Losing a customer's files costs you the customer.

Doing it on Granite.so

On Granite.so, the storage side of the steps above is short: create S3-compatible storage alongside your app, copy the endpoint and credentials into your app's environment variables, and serve public files through the built-in CDN instead of setting up a separate one. Your app deploys from Git or a Docker image with auto-builds on push, and your Postgres runs as a managed database alongside it. Code that already uses @aws-sdk/client-s3, boto3 or any other standard S3 library keeps working after the endpoint change from step 1.

For a pre-seed app with a beta cohort, the Hobby plan ($9/month, including $15 of monthly usage credits) usually covers app, database and storage while you validate. Move to Advanced ($39/month) when real traffic arrives.

The checklist

• Audit Cost Explorer by usage type: is it storage, egress or requests?
• Add lifecycle rules for incomplete uploads and old versions today
• Move storage config into environment variables
• Store object keys, not full URLs
• rclone copy, then rclone check
• Cut over, copy again, add a read fallback
• Keep the old bucket read-only for one billing cycle

S3 storage bill shock is almost always a visibility problem: many small meters running in places you do not look. Whether you stay on Amazon S3 with better lifecycle rules or move to a platform where files, app and database share one bill, the goal is the same: open next month's invoice and see the number you expected. If you want the consolidated route, you can start on Granite.so and point your existing S3 client at it this afternoon.