Count the dashboards you opened last week to keep your MVP alive. The app host. The database provider. The AWS console, just for one S3 bucket, with an IAM policy you copied from Stack Overflow in month one. Maybe a CDN on top. Four invoices, four status pages, four places a 2 AM incident can start, and not one of them knows the others exist.
You do not have a scaling problem yet. You have a vendor sprawl problem. The cheapest piece to consolidate is file storage, because the S3 API has become the de facto standard. Any S3 compatible storage service speaks the same protocol your code already uses, so you can move files next to your app and database without rewriting a single upload handler.
This guide is a set of commands for doing exactly that: audit, abstract, move and cut over, with code for Node.js and Python.
Understand what "S3 compatible" actually buys you
An S3 compatible object storage service implements Amazon's S3 REST API: the same endpoints, the same Signature V4 request signing, the same XML responses. In practice this means:
- Your SDK does not change.
@aws-sdk/client-s3,boto3, Go'saws-sdk-go-v2, Ruby'saws-sdk-s3, Laravel's Flysystem and Active Storage all accept a custom endpoint. - Your tools do not change.
aws s3,rclone,s3cmdand most backup tools work against any compatible endpoint. - Your data model does not change. Buckets, keys, prefixes and metadata behave the same.
What it does not guarantee is feature parity with Amazon S3. The core that almost every SaaS app uses is universally supported:
PutObject,GetObject,HeadObject,DeleteObjectListObjectsV2- multipart uploads for large files
- presigned URLs for direct browser uploads and downloads
The edges vary between providers: Object Lock, S3 Select, event notifications, storage-class transitions, detailed bucket-policy conditions, SSE-KMS with customer keys. If your app depends on any of these, check the provider's docs before you commit.
Rule of thumb: if your storage code only puts, gets, lists and signs URLs, you can move to any S3 compatible storage in an afternoon. If you have Lambda functions reacting to bucket events, you are not moving storage, you are moving architecture.
Command 1: audit what your app really uses
Before touching anything, list every S3 call in the codebase:
# Node / TypeScript
grep -rnE "Command\(|getSignedUrl|createPresignedPost" src/ | grep -iE "s3|object"
# Python
grep -rnE "\.(put_object|get_object|upload_file|upload_fileobj|download_file|generate_presigned_url|generate_presigned_post|list_objects_v2|delete_object)\(" .Then answer three questions:
- Which operations? Anything outside the core list above goes on a "verify" list.
- Where do URLs come from? Are you storing full
https://bucket.s3.amazonaws.com/...URLs in the database, or just object keys? - Who serves the files? Direct from the bucket, through a CDN, or proxied through your app?
For a typical MVP the answer is: put, get, delete, presigned uploads, full URLs stored in Postgres, files served straight from the bucket. All of that moves cleanly, and the next commands fix the URL issue on the way.
Command 2: put the endpoint behind environment variables
Hardcoded AWS assumptions are the only real lock-in in most codebases. Remove them.
# .env (never commit real values)
S3_ENDPOINT=https://<storage-endpoint>
S3_REGION=auto
S3_BUCKET=myapp-prod
S3_ACCESS_KEY_ID=...
S3_SECRET_ACCESS_KEY=...
PUBLIC_FILES_BASE_URL=https://<cdn-domain>Use whichever region value and URL style your provider documents. Many S3 compatible services accept any region string and prefer path-style URLs.
Node.js (AWS SDK v3)
// lib/storage.ts
import { S3Client, PutObjectCommand, DeleteObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
export const s3 = new S3Client({
region: process.env.S3_REGION ?? "us-east-1",
endpoint: process.env.S3_ENDPOINT,
forcePathStyle: true,
credentials: {
accessKeyId: process.env.S3_ACCESS_KEY_ID!,
secretAccessKey: process.env.S3_SECRET_ACCESS_KEY!,
},
});
const Bucket = process.env.S3_BUCKET!;
export async function presignUpload(key: string, contentType: string) {
return getSignedUrl(
s3,
new PutObjectCommand({ Bucket, Key: key, ContentType: contentType }),
{ expiresIn: 300 }, // 5 minutes
);
}
export async function deleteFile(key: string) {
await s3.send(new DeleteObjectCommand({ Bucket, Key: key }));
}
export const publicUrl = (key: string) =>
`${process.env.PUBLIC_FILES_BASE_URL}/${encodeURI(key)}`;Python (boto3)
# storage.py
import os
import boto3
from botocore.config import Config
s3 = boto3.client(
"s3",
endpoint_url=os.environ["S3_ENDPOINT"],
region_name=os.environ.get("S3_REGION", "us-east-1"),
aws_access_key_id=os.environ["S3_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["S3_SECRET_ACCESS_KEY"],
config=Config(signature_version="s3v4", s3={"addressing_style": "path"}),
)
BUCKET = os.environ["S3_BUCKET"]
def presign_upload(key: str, content_type: str) -> str:
return s3.generate_presigned_url(
"put_object",
Params={"Bucket": BUCKET, "Key": key, "ContentType": content_type},
ExpiresIn=300,
)
def public_url(key: str) -> str:
return f"{os.environ['PUBLIC_FILES_BASE_URL']}/{key}"For Django, django-storages with the S3 backend accepts the same values (endpoint_url, addressing_style) in its STORAGES options.
Deploy this change while still pointing at your current provider. If nothing breaks, you have removed the lock-in, and the actual move becomes a config change.
Command 3: store keys, not URLs
If your database contains full URLs, every one of them embeds the old vendor's hostname. Add a key column and backfill it:
ALTER TABLE attachments ADD COLUMN file_key text;
UPDATE attachments
SET file_key = regexp_replace(file_url, '^https?://[^/]+/(myapp-prod/)?', '')
WHERE file_key IS NULL;The optional (myapp-prod/)? group handles path-style URLs that include the bucket name. Check a sample of rows before dropping the old column. From now on, generate URLs at read time with publicUrl(key). The next time you change CDN domains or providers, you change one environment variable.
Command 4: upload directly from the browser
If your uploads currently stream through your app container (multer, request.files, a FileField form), you are paying for that bandwidth and memory twice, and a 200 MB video upload can take down a small instance. Presigned URLs let the browser send files straight to storage:
// app/api/uploads/route.ts (Next.js route handler)
import { randomUUID } from "node:crypto";
import { presignUpload } from "@/lib/storage";
export async function POST(req: Request) {
const { contentType } = await req.json();
if (!["image/png", "image/jpeg", "application/pdf"].includes(contentType)) {
return Response.json({ error: "unsupported type" }, { status: 400 });
}
const key = `uploads/${randomUUID()}`;
const url = await presignUpload(key, contentType);
return Response.json({ key, url });
}// client
const { key, url } = await fetch("/api/uploads", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ contentType: file.type }),
}).then((r) => r.json());
await fetch(url, { method: "PUT", headers: { "Content-Type": file.type }, body: file });
// then save `key` to your database via your normal APITwo things to get right:
- CORS. The bucket must allow
PUTfrom your app's origin. Configure it via the S3PutBucketCorsAPI or the provider's dashboard, depending on what your provider supports. - Validation. The presigned URL only authorizes one key and one content type for five minutes. Enforce file-size limits in your API and verify the object with
HeadObjectbefore you trust it.
Command 5: copy the data and cut over
With the code abstracted, the move is mechanical:
# Point rclone at old and new providers using env-based remotes
export RCLONE_CONFIG_OLD_TYPE=s3 RCLONE_CONFIG_OLD_PROVIDER=AWS
export RCLONE_CONFIG_OLD_ACCESS_KEY_ID=... RCLONE_CONFIG_OLD_SECRET_ACCESS_KEY=...
export RCLONE_CONFIG_OLD_REGION=us-east-1
export RCLONE_CONFIG_NEW_TYPE=s3 RCLONE_CONFIG_NEW_PROVIDER=Other
export RCLONE_CONFIG_NEW_ACCESS_KEY_ID=... RCLONE_CONFIG_NEW_SECRET_ACCESS_KEY=...
export RCLONE_CONFIG_NEW_ENDPOINT=https://<storage-endpoint>
rclone copy old:myapp-prod new:myapp-prod --transfers 16 --progress
rclone check old:myapp-prod new:myapp-prod --one-wayThen:
- Swap the
S3_*andPUBLIC_FILES_BASE_URLvariables and redeploy. - Run
rclone copyagain to catch uploads made during the switch. - Keep the old bucket read-only for one billing cycle before deleting it.
A quick smoke test from your laptop with the AWS CLI, which works against any compatible endpoint:
aws s3 ls s3://myapp-prod/uploads/ --endpoint-url "$S3_ENDPOINT" | headCommand 6: give dev and staging their own buckets
Consolidation is also a chance to fix a classic MVP sin: dev and prod sharing a bucket. Create myapp-dev and myapp-staging next to myapp-prod, give each environment its own credentials, and never let a local .env hold production keys.
If you prefer a fully local setup, several open-source S3 compatible servers can run in Docker Compose for development. Check the current licensing and distribution status of whichever one you pick, since several projects in this space have changed terms recently. Whatever you run locally, the code from command 2 should not need a single if (dev) branch. Only the environment variables change.
Compare: Amazon S3 vs an integrated platform
Amazon S3 is excellent at what it does. The question for a solo founder is not "which object store is better" but "how many moving parts do I want to own."
| Concern | Amazon S3 + separate app host | Granite.so |
|---|---|---|
| Vendors to manage | Storage, app host, DB host, often a CDN | One platform |
| CDN | Separate setup (for example CloudFront) | Built in |
| Access control | IAM users, policies, bucket policies | Storage credentials managed alongside your app |
| Pricing shape | Per GB stored, per request and per GB transferred, on its own bill | Part of a plan: Hobby $9/mo with $15 usage credits, Advanced $39/mo |
| Feature depth | Deepest in the market | Focused on app storage: uploads, assets, exports |
| Lock-in | Low for core API, high if you use AWS-only features | Low, since it is standard S3 API |
Where Amazon S3 genuinely wins: huge datasets, compliance features like Object Lock, event-driven pipelines, or an AWS-native team. Where it loses for a pre-seed MVP: the IAM learning curve, a separate CDN to configure, and a bill that moves with traffic in ways that are hard to predict when your app runs outside AWS. Check current pricing on both sides, because it changes.
How this looks on Granite.so
On Granite.so, consolidation is the default rather than a project. You deploy your app from Git or a Docker image with auto-builds on push, attach a managed database, and create S3 compatible storage in the same place. Files are served through the built-in CDN, so you do not configure a distribution separately. Your existing S3Client or boto3 code from command 2 works once you set the endpoint and credentials as environment variables on the app.
For a stack that has to stay under about $80/month, the budget is straightforward: Hobby at $9/month (with $15 of monthly usage credits) for the beta, Advanced at $39/month once paying users arrive. That leaves headroom for your domain, email provider and error tracking.
The consolidation checklist
- Grep every S3 call and flag non-core operations
- Move endpoint, bucket and keys into environment variables
- Store object keys instead of full URLs
- Switch large uploads to presigned URLs
rclone copy, thenrclone check, then cut over- Separate buckets and credentials per environment
- Keep the old bucket read-only for one billing cycle
Every vendor you remove is one less login, one less invoice and one less status page to refresh at 2 AM. S3 compatible storage is the easiest place to start because the API contract does the hard work for you. If you want files, app and database under one roof, try Granite.so and point your existing S3 client at it.