Crashes & Process Lifecycle

"Software modules should be fail-fast: they either do the right thing, or they detect the fault, signal failure, and stop." — Jim Gray, Why Do Computers Stop and What Can Be Done About It? (1985)

Part VI built a service that works. Part VII is about what it does when something goes wrong, and the first question is the one most codebases answer by accident: how does this process die, and what does it take with it?

Gray's sentence is forty years old and it is still the argument. A module that detects a fault and stops is repairable, because something outside it can restart it into a known state. A module that detects a fault and carries on has become unpredictable, and nothing outside it can tell. Node gives you both options with roughly equal ergonomics, and the more dangerous one looks like diligence — you catch the error, log it, keep serving. That is the decision this chapter is about.

Everything here was measured against Node 24.8.0 by spawning real child processes and reading their exit codes, because exit codes are the entire vocabulary your process has for talking to whatever supervises it. Some of it corrected me, and one thing I could not reproduce on this platform at all, which I say where it comes up rather than quietly asserting the folklore.

Part V's An Express App That Scales already covered draining in-flight HTTP connections on shutdown, so that ground is not re-covered. This chapter is about the process: the four ways it ends, what keeps it alive when you want it gone, why process.exit() loses work you thought was finished, and why a crash loop is sometimes the correct behaviour rather than a bug.

A crash is a decision

A Node process ends in one of four ways, and they differ in what they lose.

The event loop empties. No timers, no sockets, no pending work, so there is nothing left to do and the process exits 0. This is the only ending that loses nothing.

An exception reaches the top. Node prints the stack to stderr and exits 1, measured.

A promise rejects with nobody listening. Since Node 15 this is fatal too: measured at exit 1, and the timer I scheduled afterwards never ran. Older habits assume a warning here, and --unhandled-rejections=warn restores that — measured at exit 0 with the rest of the program continuing — which is a flag worth recognising in a legacy service because it converts a crash into a silent gap.

Somebody calls process.exit(n). Immediate, unconditional, and §4 is about what that costs.

The distinction that matters operationally is between exit 0 and everything else, because that is all a supervisor sees. Kubernetes, systemd and a process manager all make the same decision from the same one byte: zero means finished, non-zero means failed, restart according to policy. Your exit code is the only sentence your process gets to say about its own death — which is why process.exit(0) in an error path is worse than a crash: it reports success.

That last pair is the practical point. A service that validates its configuration at boot (ch2) and exits 1 gets restarted, fails again, and enters a crash loop that is visible — an alert, a status, a rollback. The same service exiting 0 is reported as a clean shutdown and quietly stays down.

The loop empties, or something holds it open

A Node process exits when it has nothing left to do, and "nothing left to do" is precise: no referenced handles and no pending requests in libuv's bookkeeping. Anything that registers a handle — a timer, a listening server, an open socket, a file watcher — keeps the process alive whether or not it will ever do anything again.

This is measurable directly. process.getActiveResourcesInfo() reports what is currently holding the loop open: measured as [] on an idle script and ["Timeout"] once an interval exists. When a service refuses to exit after shutdown, that call names the culprit, and it is usually a setInterval for metrics or a database pool nobody closed.

The escape hatch is unref(), which tells Node to stop counting a handle when deciding whether to stay alive. Measured: an interval firing every 50ms, unref'd, did not keep the process running past a 10ms timer — the process exited and the interval simply stopped existing.

The distinction in that last line is worth holding onto. unref() does not stop a timer; it removes the timer's vote on whether the process should live. A metrics interval that is unref'd still fires on a busy server and simply stops mattering during shutdown, which is exactly what you want — whereas clearInterval during shutdown is also correct and more explicit.

Catching the exception is not recovery

Here is what an uncaughtException handler actually does, measured. With a handler installed, a thrown error printed the handler's message and the process exited 0 — and in the variant where I also scheduled a timer, the timer still ran afterwards. The process does not limp on in a degraded mode. It continues completely normally, from a stack that was abandoned partway through whatever it was doing.

That is the problem, and it is not theoretical. The abandoned frame might have been halfway through the unit of work from ch3 — a transaction opened, a connection checked out of the pool, an invariant temporarily broken between two writes. None of that unwinds. The connection is not released, the transaction is not rolled back, and the in-memory object is left in a state your code has no branch for.

One more measured detail, because it is a genuinely surprising exit code: throwing inside the uncaughtException handler exits 7, not 1. Node reserves that for a failure in the internal exception handler, so a logging call that itself throws produces an exit code most runbooks do not mention.

The legitimate use of the handler is narrow and worth stating precisely: log the error, flush what you can, and exit non-zero. It is a last-chance diagnostic hook, not a recovery mechanism, and the deciding argument is that the code which would perform the recovery is the same code that just proved it does not understand the current state.

process.exit() abandons work in flight

This is where I was most surprised by my own measurements, and where I have to report a failure to reproduce.

The folklore is that process.exit() truncates buffered stdout. I could not demonstrate that on Windows: 20,000 lines written to a pipe immediately before process.exit(0) arrived complete, every time. The reason is platform-specific — Node's stdout is synchronous to pipes on Windows and to files and TTYs on POSIX, but asynchronous to pipes on POSIX — so the classic truncation is real on Linux, which is where your service runs, and simply cannot be shown from here. I am telling you rather than presenting folklore as measurement.

What I could measure is sharper, and portable. fs.writeFile('a.txt', …, callback) followed immediately by process.exit(0) left the file on disk — and the callback never ran. The side effect happened; your program never learned that it had. That is worse than losing the write, because retry logic and logging both stay silent while the effect persists. With process.exitCode = 0 instead, the file was written and the callback ran.

The same shape kills a shutdown routine. A flush that awaits a 20ms timer, with process.exit(0) queued on the next tick, printed nothing and wrote nothing — the continuation after the await was simply abandoned. Letting the loop drain instead ran both.

So the rule is short: set process.exitCode and let the loop end. Reserve process.exit() for the case where you have decided that abandoning in-flight work is better than waiting — a hard timeout backstop during shutdown, which §5 builds — and never as the ordinary way to finish.

The shutdown sequence, and the crash loop

Putting it together, an orderly stop has a fixed order, and each step exists because of a specific failure.

Stop accepting new work first. Closing the listener before draining means the queue is finite; drain first and you are chasing a moving target. Then let in-flight work finish — the connection draining Part V ch5 built. Then release resources: the pool from ch3, the session store from ch4, timers. Then set exitCode and let the loop empty, so every callback in §4 gets to run. And back the whole thing with a hard timeout that calls process.exit(1) if the sequence stalls, because a shutdown that hangs is the one outcome worse than an abrupt one.

Two guards make it survive contact with operators. A second signal must be ignored rather than starting a parallel shutdown. And the timeout must be shorter than whatever the platform will wait before sending SIGKILL, or the platform decides for you — Kubernetes defaults to thirty seconds, so a twenty-five second budget leaves room to exit on your own terms.

Note the exit code on the forced path. A shutdown that had to abandon work did not complete cleanly, and reporting 1 is honest — it tells the supervisor, and the graph, that this stop lost something.

Common Mistakes

1. Catching uncaughtException to keep serving. Measured, the process continues normally from an abandoned stack, so later requests read corrupted state and nothing throws again.

2. A handler that does not exit. It is a diagnostic hook, not recovery; log, flush against a timeout, and exit non-zero.

3. Assuming an unhandled rejection is a warning. Since Node 15 it is fatal — measured at exit 1, with the code scheduled after it never running.

4. process.exit(0) on an error path. The supervisor believes you, leaves the service down, and reports a clean shutdown.

5. process.exit() instead of exitCode. Measured: a writeFile landed on disk while its callback never ran, so the effect happened invisibly.

6. Awaiting a flush with no timeout. A broken log sink turns a crash into a hang, and a hung process sometimes still passes health checks.

7. Async work in an exit handler. It is synchronous-only; a setTimeout inside one never fires, measured.

8. Not guarding against a second signal. A repeat SIGTERM starts a parallel shutdown that closes resources the first is still using.

9. A shutdown budget longer than the platform's. Kubernetes sends SIGKILL after thirty seconds by default, so exceeding it forfeits the orderly exit.

10. Treating a crash loop as the bug. With invalid configuration it is the correct, visible outcome; silencing it produces a service that is up and wrong.

Interview Corner

Hands-on

Summary

• A crash is a decision. A module that stops on a fault is repairable; one that continues is unpredictable, and nothing outside it can tell.
• A Node process ends four ways — the loop empties, an exception escapes, a rejection goes unhandled, or somebody calls exit — and they differ in what they lose.
• Unhandled rejections are fatal since Node 15, measured at exit 1, with anything scheduled after them never running.
• The exit code is the whole conversation with your supervisor, which is why process.exit(0) on an error path is worse than crashing: it reports success.
• Referenced handles keep the process alive, and process.getActiveResourcesInfo() names them — measured as [] idle and ["Timeout"] with an interval.
• unref() removes a handle's vote, not the work: the timer still fires while the process happens to be alive.
• Catching uncaughtException continues normally from an abandoned stack, measured at exit 0 with a later timer still running, leaving transactions open and invariants broken.
• Throwing inside that handler exits 7, a code most runbooks never mention.
• process.exit() abandons the callback queue: measured, a writeFile reached disk while its callback never ran, so the effect happened invisibly.
• Shut down in order — stop accepting, drain, release, set exitCode, let the loop end — with a hard timeout shorter than the platform's grace period.

In the next chapter, Logging & Observability, we make the failures in this chapter legible: structured logs that survive being searched, the request correlation that turns scattered lines into one story, and the honest limits of an ecosystem with no consensus on any of it.