Scaling Node.js for a Growing Startup: What Actually Breaks First

Most Node.js apps don’t fall over because of traffic. They fall over because of one slow database query, a memory leak nobody watched, or a process that only uses one CPU core. I’ve worked on backend systems for startups around New York, and the failure pattern is almost always the same. Here’s what actually breaks first and how to fix it before you start throwing money at bigger servers.

Node runs on one thread, and that’s your first wall

A single Node process uses one CPU core for your JavaScript. If you’re on an 8-core box and only running one process, you’re wasting seven cores. This is the cheapest, fastest win you have, and a lot of teams skip it because they assume the framework handles it.

The built-in cluster module forks workers that share the same port. Each worker gets its own event loop and memory space.

import cluster from 'node:cluster';
import os from 'node:os';
import http from 'node:http';

const numCPUs = os.availableParallelism();

if (cluster.isPrimary) {
 console.log(`Primary ${process.pid} running`);

 for (let i = 0; i < numCPUs; i++) {
 cluster.fork();
 }

 cluster.on('exit', (worker, code, signal) => {
 console.error(`Worker ${worker.process.pid} died (${signal || code}). Restarting.`);
 cluster.fork();
 });
} else {
 http.createServer((req, res) => {
 res.writeHead(200);
 res.end('hellon');
 }).listen(3000);

 console.log(`Worker ${process.pid} started`);
}

In practice you rarely write this by hand anymore. PM2 in cluster mode, or a container orchestrator that scales replicas, does the same job. The point is: one process per core, and never let a crashed worker stay dead.

The database is almost always the real bottleneck

Adding Node workers helps until every worker hammers your Postgres or MySQL instance with new connections. Postgres defaults to 100 max connections. Spin up 8 workers with a naive pool of 20 each and you’ve asked for 160. The database starts refusing connections and your app looks “slow” when it’s actually getting rejected.

Two things fix this. First, size your pool correctly. A pool of 10 per worker across 8 workers is 80, which fits. Second, for anything beyond a couple of instances, put PgBouncer in front of Postgres and let it multiplex thousands of app connections onto a small number of real ones.

import { Pool } from 'pg';

const pool = new Pool({
 connectionString: process.env.DATABASE_URL,
 max: 10, // per worker, not per app
 idleTimeoutMillis: 30000,
 connectionTimeoutMillis: 5000,
});

// Always release. Leaked clients are a slow-motion outage.
export async function getUser(id) {
 const client = await pool.connect();
 try {
 const { rows } = await client.query(
 'SELECT id, email FROM users WHERE id = $1',
 [id]
 );
 return rows[0];
 } finally {
 client.release();
 }
}

Then look at the queries themselves. A missing index on a foreign key column turns a 2ms lookup into a 400ms sequential scan once the table grows past a few hundred thousand rows. EXPLAIN ANALYZE is not optional. If you’re not reading query plans, you’re guessing.

Don’t do CPU-heavy work in the request path

Anything that blocks the event loop blocks every request that worker is handling. Image resizing, PDF generation, big JSON transforms, password hashing with a high cost factor, CSV parsing of a 50MB upload. All of it stalls the whole process.

Move that work off the main thread:

• worker_threads for CPU-bound JavaScript in the same process.
• A job queue (BullMQ on Redis is the common choice) for anything that doesn’t need to finish before you respond.
• Native addons or a sidecar service for genuinely heavy lifting like video transcoding.

import { Worker } from 'node:worker_threads';

function resizeImage(buffer) {
 return new Promise((resolve, reject) => {
 const worker = new Worker(new URL('./resize-worker.js', import.meta.url), {
 workerData: buffer,
 });
 worker.once('message', resolve);
 worker.once('error', reject);
 });
}

// The event loop stays free while the worker chews on the image.
app.post('/avatar', async (req, res) => {
 const resized = await resizeImage(req.body);
 await saveAvatar(req.user.id, resized);
 res.json({ ok: true });
});

Caching: the boring version that works

Every startup eventually adds Redis and then uses it wrong. Two rules cover most of it. Cache reads that are expensive and tolerate slight staleness. Never cache anything you’ll immediately need to invalidate on write unless you have a plan for that invalidation.

For a lot of endpoints, an in-process LRU cache with a 30-second TTL removes 90% of the database load and adds zero infrastructure. Reach for Redis when you have multiple instances that need to share state, or when you need the cache to survive a deploy.

Observability before optimization

You can’t fix what you can’t see. At minimum, before you scale anything, get these in place:

• Structured logs with request IDs so you can trace a single request across services. Pino is fast and cheap.
• Metrics on event loop lag, memory (heap used vs RSS), and active database connections. Event loop lag climbing is the earliest sign of trouble.
• APM or tracing so you know which span is slow instead of guessing. OpenTelemetry is the safe bet in 2026 since it isn’t tied to one vendor.
• Error tracking with alerts that actually page someone.

import pino from 'pino';
import { monitorEventLoopDelay } from 'node:perf_hooks';

const log = pino({ level: process.env.LOG_LEVEL || 'info' });

const h = monitorEventLoopDelay({ resolution: 20 });
h.enable();

setInterval(() => {
 const lagMs = h.mean / 1e6;
 if (lagMs > 50) {
 log.warn({ lagMs }, 'event loop lag high');
 }
}, 10000);

Memory leaks will find you

Node apps that run for weeks tend to grow. Common causes: unbounded arrays used as caches, listeners added but never removed, closures holding references to big objects, and global state that accumulates. If RSS climbs steadily and never drops after garbage collection, you have a leak. Take a heap snapshot, compare two snapshots an hour apart, and look at what grew. Node’s built-in inspector and Chrome DevTools handle this fine.

A restart-every-N-hours cron is a band-aid. It buys you time to find the actual leak, but treat it as temporary.

When to stop optimizing and start splitting

Microservices are usually the wrong first move. A startup with five engineers maintaining twelve services spends most of its time on plumbing instead of product. But there’s a real threshold: when one part of the system has wildly different scaling needs (say, a webhook receiver getting hammered) or a different language genuinely fits the job, splitting that piece out makes sense.

Until then, a well-structured monolith with clear module boundaries scales further than most people expect. I’ve seen single Node services handle serious traffic on a handful of instances when the database and event loop were handled properly. If you’re weighing whether to rebuild or refactor, this is the kind of thing I help teams work through as part of custom software engineering, and the honest answer is usually refactor.

A reasonable order of operations

1. Add metrics and logging. Know what’s slow before you touch it.
2. Fix the slow queries and add the missing indexes.
3. Right-size the connection pool and add PgBouncer if you have multiple instances.
4. Run one process per core, with automatic restarts.
5. Move CPU-heavy work to workers or a queue.
6. Add caching where it’s clearly justified.
7. Only then add instances or bigger machines.

Scaling Node.js isn’t mysterious. It’s mostly about not blocking the one thread you have, not drowning your database, and being able to see what’s happening. Do those three things and you’ll get a lot further than you’d think on modest hardware.

FAQ

Do I need Kubernetes to scale a Node.js app?

No. A handful of instances behind a load balancer, with PM2 or Docker restarts, handles most startup traffic. Kubernetes earns its complexity when you have many services and a team that can maintain it.

How many database connections should my Node app use?

Start with 10 per process and multiply by your process count, keeping the total under your database’s connection limit. Add PgBouncer when you outgrow that, rather than raising the limit.

Is clustering or worker_threads the right fix for high CPU?

They solve different problems. Clustering gives you more event loops to handle more concurrent requests. Worker threads keep a single request from blocking the event loop. Use clustering for throughput, worker threads for heavy computation.