Turning off Lighthouse cut our cluster bill 62%

Killing a hosted product is usually described as a business decision. The part nobody writes down is the afternoon afterward, when someone has to go turn the actual thing off. This is that afternoon.

What was still running

Lighthouse's hosted SaaS — the API, the web app, the Neo4j graph store behind it — had been live for 53 days on a dedicated Kubernetes namespace. Deleting the deployments was the easy part:

kubectl -n lighthouse delete deployment lighthouse-api lighthouse-web
kubectl -n lighthouse delete statefulset neo4j

Three commands, and the public product was gone. The harder part was everything the product left behind that keeps costing money after the pods stop.

The orphaned disks

Block storage in Kubernetes doesn't clean up after itself by design — the reclaim policy on our storage class is Retain, which means deleting a PersistentVolumeClaim leaves the underlying disk alive on purpose, in case someone needs the data back. That's the right default. It also means two disks were still being billed for after the pods that used them were long gone:

• A 20GB volume backing the Neo4j graph data — the actual Lighthouse dead now.
• A 10GB volume from an even older FalkorDB deployment, already Released and unclaimed before we touched anything.

Thirty gigabytes, roughly $3/month, doing nothing. We checked what was actually in the corpus database first — 99,999 indexed chunks, two real user accounts (one of them ours, tier pro, no active subscription), a handful of API keys — confirmed there was nothing worth keeping, and deleted both the Kubernetes objects and the underlying DigitalOcean volumes.

The node pool was sized for a product that's gone

The bigger number was the compute underneath everything, not just Lighthouse. Three s-2vcpu-8gb-amd nodes, running at roughly 58% CPU and 25% memory requested — sized for Ship, Lighthouse's hosted stack, and a public-facing site, back when all three were live at once. With Lighthouse's workloads gone, that headroom was mostly empty.

DigitalOcean Kubernetes doesn't let you resize a node pool's droplet type in place — you stand up a new pool at the size you want, migrate the workloads over, and delete the old one. The sequence:

1. Create a new pool, s-2vcpu-4gb, count 2. (The AMD variant we wanted, s-2vcpu-4gb-amd, came back "region has insufficient capacity" — the plain Intel variant was available, and $4/month cheaper besides.)
2. Wait for both new nodes to report Ready.
3. Cordon all three old nodes so nothing new schedules there.
4. Drain them one at a time — kubectl drain --ignore-daemonsets --delete-emptydir-data — checking pod health between each one. Single-replica services (ingress, the marketing site) took a few seconds of rescheduling; anything running two replicas stayed up the whole time.
5. Delete the old node pool once everything was confirmed healthy on the new one.

Nobody noticed. That's the actual bar for this kind of change — not that it worked, but that it was boring enough nobody had to ask.

The number

Three s-2vcpu-8gb-amd nodes at $42/month each is $126/month. Two s-2vcpu-4gb nodes at $24/month each is $48/month. Add the $3/month from the orphaned disks, and total infrastructure spend on that cluster went from roughly $129/month to $48/month — a 62% cut, with the workload it's actually running today comfortably inside the new headroom.

The honest caveat: memory on the new pool runs hotter than we'd like — 65–80% of requests, not the 40–50% we'd planned for. If a workload grows, the fallback is one command, --count 3 on the same cheap pool, before reaching for anything bigger. Right-sizing isn't a one-time event; it's a number you keep checking after you've deleted the thing that made the old size necessary.