Exit Code 0 Lied: The Silent Node.js Bug That Hit sarvam-cli Three Times

Reading Time: 9 minutes

11 min read

A Node.js readline silent exit happens when rl.question() never fires its callback. The interface closes first, the awaited promise stays pending forever, and the event loop drains. As a result, the process exits cleanly with status 0 — mid-prompt, no error, no stack trace. I hit this in three separate places in sarvam-cli, an MIT-licensed agentic coding CLI. The fix is eight lines. The interesting part is what each call site should do when input ends, and why “decline” is the only safe answer at an approval prompt.

Table of Contents

TL;DR

  • rl.question() never fires its callback if the readline interface closes first. Ctrl+D, Ctrl+C, and exhausted piped stdin all close it.
  • A pending promise is not a running task. Node finds nothing scheduled, drains the loop, and exits with status 0 — so the failure presents as success.
  • The fix is to resolve to null on close, not to reject. End of input is normal, not exceptional.
  • What null means differs per call site: exit at a REPL prompt, decline at a consent gate, abort without writing in a config wizard.
  • At an approval prompt, === "y" fails closed and !== "n" fails open. Same line count, opposite blast radius.
  • Test the interactive path in a pseudo-terminal. Piping only exercises the non-TTY code path, and your users are on the other one.

The Symptom: a % at the End of a Terminal Paste

Someone sent me a session transcript from sarvam-cli. The last four lines:

❯ /model
Current model: sarvam-105b
Available: sarvam-105b
model> %

That trailing % is zsh telling you the previous command produced output with no final newline. Specifically, it only appears when zsh has regained control — which means the process exited. While sitting at a prompt. Having printed model> and then simply stopped existing.

No error. No traceback. Nothing in the logs.

Why My First Diagnosis Was Wrong

My first theory was wrong, and it is worth saying so. I assumed stdin contention. The code attached a raw process.stdin.on("data") listener for a Ctrl+O keybinding while a readline interface was consuming the same stream. Two readers, one pipe — a classic. I wrote it up confidently.

Then I reproduced it before fixing it, and the theory collapsed. The /model flow completed perfectly. Additionally, the line buffer survived a mid-line keypress intact. Whatever killed the process, it was not stdin contention.

What Actually Causes the Node.js readline Silent Exit

Here is the code every Node CLI writes to get an async prompt:

const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
const ask = (q) => new Promise((resolve) => rl.question(q, (a) => resolve(a)));

const name = await ask("Your name: ");

rl.question() registers a one-shot callback for the next line of input. That is the whole contract. Consequently, there is exactly one way for it to never be called: the interface closes before a line arrives.

Three ordinary things close it:

  • Ctrl+D — EOF at the terminal
  • Ctrl+C — readline’s default SIGINT behaviour closes the interface
  • Piped stdin running out of linesprintf 'a\nb\n' | sarvam when the CLI asks five questions

When any of those happen, the callback is dropped. The promise attached to it stays pending. await never returns.

Now the part that turns a bug into a silent bug. A pending promise is not a running task. Node does not wait on it, because nothing is scheduled — no timer, no socket, no handle. The event loop finds itself with nothing to do, and does the correct thing:

It exits. Status 0.

From the outside, your program reported success. From the inside, it never finished the line it was on.

$ printf '\n\n\n' | sarvam --init
  sarvam init
  ----------------

Provider [a/b] (default: a): API key: $ echo $?
0

Success. Nothing written.

Why Exit Code 0 Is the Real Damage

A crash is loud. You get a stack trace, a non-zero status, and CI goes red. This is the opposite. It is a false success, and false success is the failure mode that survives longest in production, because nothing is watching for it.

That gap between “the process succeeded” and “the product worked” is exactly the product quality vs code quality split — no test suite in this repo was wrong, and every one of them passed.

Concretely, in my case:

sarvam --init && echo "configured" && deploy.sh

--init exits 0 having written no config file. However, the && chain proceeds anyway. Then deploy.sh runs against a machine that was never configured. The one signal a shell script has for “did this work” was lying.

Three Call Sites, One Copy-Pasted Habit

Once I understood the mechanism, I grepped for the shape rather than the symptom — rl.question wrapped in a new Promise. It appeared three times.

1. The main REPL prompt. Ctrl+D exited silently. In practice, this is the one everyone would eventually notice and shrug at, because “Ctrl+D quits” looks like intended behaviour when the process disappears.

2. A nested sub-prompt. The one in the transcript. Identical cause, more visible, because it left a half-drawn prompt on screen.

3. The --init configuration wizard. The worst of the three, and the one nobody would have found by hand. Specifically, it only misbehaves under piped stdin — which is exactly how CI and setup scripts drive it.

Same eight lines, copy-pasted, three times. That is the honest reason it appeared three times: not three bugs, one habit.

The Fix: Resolve to a Sentinel When the Interface Closes

Resolve to null when the interface closes, so every caller gets a value instead of hanging:

let closed = false;
rl.on("close", () => {
  closed = true;
});

const ask = (q: string): Promise<string | null> =>
  new Promise((resolve) => {
    if (closed) return resolve(null);

    let answered = false;
    const onClose = () => {
      if (!answered) {
        answered = true;
        resolve(null);
      }
    };

    rl.once("close", onClose);
    rl.question(q, (a) => {
      answered = true;
      rl.removeListener("close", onClose);
      resolve(a);
    });
  });

Three details matter more than they look:

  • resolve(null), not reject(). End of input is not exceptional. It is the normal way a pipe finishes and the normal way a user quits. Rejecting forces every call site into a try/catch and tempts people to swallow it.
  • The answered guard. Without it, a close event firing after a legitimate answer double-resolves. That is harmless with promises, but it hides ordering bugs.
  • removeListener on the success path. A long REPL session asks hundreds of questions. Without this you accumulate a close listener per prompt, and Node starts warning you about a leak around 11.

What null Should Mean at Each Call Site

This is where a mechanical fix becomes a design decision. null means “input ended.” What you should do about that differs at every call site. Furthermore, getting it wrong at one of them is a security bug.

At the main prompt — exit cleanly.

const input = await ask("❯ ");
if (input === null) break;   // fall through to the normal shutdown path

At an approval prompt — decline. Always.

const ans = await ask(`▸ ${tool}: ${summary} [y/N] `);
if (ans === null) return false;  // stdin closed — never assume consent
return ans.toLowerCase().trim() === "y";

This is the one that matters. sarvam-cli is an agentic assistant that can run shell commands and write files, gated behind a [y/N] prompt. Therefore, “input ended” must never be read as approval.

Note how easily it goes the other way. Had the original been written as a denial check:

return ans.toLowerCase().trim() !== "n";   // looks equivalent. isn't.

…then an empty or absent answer becomes approval, and a closed stdin auto-approves every pending action. Same number of lines, opposite blast radius. Fail-closed is not a philosophy here. It is a comparison operator — and it is the most concrete example of cybersecurity architecture principles I have shipped in one line of code.

Why a Partial Write Is Worse Than No Write

In the config wizard — abort without writing.

const answers: string[] = [];
for (const q of questions) {
  const a = await ask(q);
  if (a === null) {
    rl.close();
    console.error("\n  init aborted — input ended before every question was answered.");
    console.error(`  Nothing written to ${CONFIG_PATH}.\n`);
    return null;   // caller exits non-zero
  }
  answers.push(a);
}

The tempting alternative is to save whatever you collected. Don’t. In this codebase a partial config with an empty apiKey field is worse than no file at all, because of a second bug it interacts with:

// The config file wins over the environment...
const apiKey = file.apiKey ?? process.env.API_KEY ?? "";

?? only falls through on null/undefined. An empty string is a defined value. As a result, a config file containing "apiKey": "" permanently shadows the environment variable. Export API_KEY all you like — the CLI reports “no API key found” forever, and the file causing it looks empty and harmless.

A partial write turns a clean failure into a persistent one. No write beats a bad write.

On the caller side, actually surface it:

if (args.init) {
  const created = await initConfigInteractive();
  process.exit(created ? 0 : 1);
}

Proving It: Testing the Interactive Path in a pty

You cannot test this properly with a pipe. readline behaves differently when stdin is not a TTY, so piping only exercises one of the two paths — and the interactive path is the one your users are on.

The one-liner smoke test is what I would add to CI first:

$ printf '\n\n\n' | sarvam --init; echo "exit=$?"
  init aborted — input ended before every question was answered.
  Nothing written to /home/vamshi/.sarvam/config.json.
exit=1          # ← was 0 before the fix. Non-zero is the whole point.

For the interactive path, drive a pseudo-terminal. Additionally, this needs no new dependencies — it is Python’s stdlib:

#!/usr/bin/env python3
"""Send Ctrl+D at a prompt and assert the CLI exits like it means it."""
import os, pty, subprocess, time

master, slave = pty.openpty()
p = subprocess.Popen(["sarvam"], stdin=slave, stdout=slave, stderr=slave, close_fds=True)
os.close(slave)

time.sleep(1.0)            # let it draw its prompt
os.write(master, b"\x04")  # Ctrl+D  (use b"\x03" for Ctrl+C)
time.sleep(1.0)

print("exit:", p.wait(timeout=5))

Running it against the fixed build:

$ python3 test_eof.py
exit: 0         # ← clean shutdown, farewell line printed
                #   before the fix this also printed 0 — but with no farewell

That distinction is the whole test. Together, the exit code and the farewell line tell you which of the two happened. This is what continuous security validation looks like at CLI scale: assert the observable behaviour, not just the status.

Quick Reference

Expected behaviour after the fix:

Input Exit code Behaviour
exit / quit 0 Normal shutdown
Ctrl+D (EOF) 0 Clean shutdown, farewell printed
Ctrl+C (SIGINT) 130 Clean shutdown, distinguishable from success
Piped stdin, too few lines 1 Aborts, writes nothing
Approval prompt, stdin closed Returns false — action declined

Use 130 for SIGINT. That is the 128 + signal convention, and the only way a wrapping script can tell “the user interrupted this” from “this finished.” However, it needs an explicit handler, because readline’s default is a silent close:

let interrupted = false;
rl.on("SIGINT", () => {
  interrupted = true;
  rl.close();
});
// …after the loop:
if (interrupted) process.exitCode = 130;

Beyond Node: Any Callback With a Path That Never Runs

The specific API is Node’s. However, the pattern is everywhere: an async primitive whose completion callback has a path that never runs.

Any time you wrap a callback API in a promise, ask the same question — what are all the ways this callback might not be called? Closed streams, cancelled requests, timed-out sockets, aborted signals. In every one of those cases, a bare new Promise(resolve => api(cb)) becomes a permanent hang. Moreover, in an event-loop runtime, a permanent hang looks exactly like a clean exit.

The tell is a process that ends without printing whatever it normally prints on the way out. If your CLI has a farewell line, a summary, or a flush, its absence is your signal — not the exit code, which is lying.

CISSP Domain Mapping

Domain Name Relevance
3 Security Architecture and Engineering Secure defaults and fail-closed design. When the system loses the ability to obtain consent, it must assume consent was refused. Note how narrowly it was avoided: === "y" fails closed, !== "n" fails open, and code review rarely catches the difference.
8 Software Development Security Error handling at trust boundaries. Silent failure is the anti-pattern — a system that cannot distinguish “succeeded” from “never ran” cannot be reasoned about, and every consumer downstream inherits the ambiguity.
7 Security Operations Exit codes are an operational interface. 0 means a shell && chain proceeds. Returning 0 from a function that did nothing is, in automation terms, a false negative on an integrity check.

Key Takeaways

  1. rl.question() never fires if the interface closes first. Ctrl+D, Ctrl+C, and exhausted piped stdin all close it.
  2. A pending promise is not a running task. Node exits cleanly when the loop empties, so the failure presents as success.
  3. Resolve to a sentinel, don’t reject. End of input is normal, not exceptional.
  4. Decide what “input ended” means per call site. Exit at a prompt, decline at a consent gate, abort at a wizard.
  5. Never assume consent from absent input. Write === "y", never !== "n".
  6. A partial write can be worse than no write — especially where an empty string is a meaningful, shadowing value.
  7. Test the interactive path in a pty. A pipe tests the other code path entirely.
  8. Reproduce before you fix. My confident first diagnosis was wrong, and only a reproduction attempt caught it before it became a wasted refactor.

Try sarvam-cli

The CLI in this post is sarvam-cli — an MIT-licensed, open-source agentic coding assistant powered by Sarvam AI. It reads, writes, and edits files and runs shell commands in your project, with your approval before any side effect. That approval gate is exactly the one discussed above, which is why the fail-closed behaviour mattered enough to write up.

git clone https://github.com/indic-ai-contribs/sarvam-cli.git
cd sarvam-cli
npm install
npm run build
npm link

sarvam --init     # exits non-zero now if you don't finish the wizard

The fixes described here shipped in v0.2.9 and v0.2.10. Issues and pull requests are welcome — particularly from anyone who has fought the same class of bug in their own CLI. If the project is useful to you, a star on the sarvam-cli GitHub repo genuinely helps it reach more Indian-language AI developers.

Get the next deep-dive in your inbox when it publishes → subscribe to linuxcent.com

EKS 1.33 Upgrade Blocker: Fixing Dead Nodes & NetworkManager on Rocky Linux

Reading Time: 5 minutes

The EKS 1.33+ NetworkManager Trap: A Complete systemd-networkd Migration Guide for Rocky & Alma Linux

TL;DR:

  • The Blocker: Upgrading to EKS 1.33+ is breaking worker nodes, especially on free community distributions like Rocky Linux and AlmaLinux. Boot times are spiking past 6 minutes, and nodes are failing to get IPs.
  • The Root Cause: AWS is deprecating NetworkManager in favor of systemd-networkd. However, ripping out NetworkManager can leave stale VPC IPs in /etc/resolv.conf. Combined with the systemd-resolved stub listener (127.0.0.53) and a few configuration missteps, it causes a total internal DNS collapse where CoreDNS pods crash and burn.
  • The Subtext: AWS is pushing this modern networking standard hard. Subtly, this acts as a major drawback for Rocky/Alma AMIs, silently steering frustrated engineers toward Amazon Linux 2023 (AL2023) as the “easy” way out.
  • The “Super Hack”: Automate the clean removal of NetworkManager, bypass the DNS stub listener by symlinking /etc/resolv.conf directly to the systemd uplink, and enforce strict state validation during the AMI build.

If you’ve been in the DevOps and SRE space long enough, you know that vendor upgrades rarely go exactly as planned. But lately, if you are running enterprise Linux distributions like Rocky Linux or AlmaLinux on AWS EKS, you might have noticed the ground silently shifting beneath your feet.

With the push to EKS 1.33+, AWS is mandating a shift toward modern, cloud-native networking standards. Specifically, they are phasing out the legacy NetworkManager in favor of systemd-networkd.

While this makes sense on paper, the transition for community distributions has been incredibly painful. AWS support couldn’t resolve our issues, and my SRE team had practically given up, officially halting our EKS upgrade process. It’s hard not to notice that this massive, undocumented friction in Rocky Linux and AlmaLinux conveniently positions AWS’s own Amazon Linux 2023 (AL2023) as the path of least resistance.

I’m hoping the incredible maintainers at free distributions like Rocky Linux and AlmaLinux take note of this architectural shift. But until the official AMIs catch up, we have to fix it ourselves. Here is the exact breakdown of the cascading failure that brought our clusters to their knees, and the “super hack” script we used to fix it.

The Investigation: A Cascading SRE Failure

When our EKS 1.33+ worker nodes started booting with 6+ minute latencies or outright failing to join the cluster, I pulled apart our Rocky Linux AMIs to monitor the network startup sequence. What I found was a classic cascading failure of services, stale data, and human error.

Step 1: The Race Condition

Initially, the problem was a violent tug-of-war. NetworkManager was not correctly disabled by default, and cloud-init was still trying to invoke it. This conflicted directly with systemd-networkd, paralyzing the network stack during boot. To fix this, we initially disabled the NetworkManager service and removed it from cloud-init.

Step 2: The Stale Data Landmine

Here is where the trap snapped shut. Because NetworkManager was historically the primary service responsible for dynamically generating and updating /etc/resolv.conf, completely disabling it stopped that file from being updated.

When we baked the new AMI via Packer, /etc/resolv.conf was orphaned and preserved the old configuration—specifically, a stale .2 VPC IP address from the temporary subnet where the AMI build ran.

Step 3: The Human Element

We’ve all been there: during a stressful outage, wires get crossed. While troubleshooting the dead nodes, one of our SREs mistakenly stopped the systemd-resolved service entirely, thinking it was conflicting with something else.

Step 4: Total DNS Collapse

When the new AMI booted up and joined the EKS node group, the environment was a disaster zone:

  1. NetworkManager was dead (intentional).
  2. systemd-resolved was stopped (accidental).
  3. /etc/resolv.conf contained a dead, stale IP address from a completely different subnet.

When kubelet started, it dutifully read the host’s broken /etc/resolv.conf and passed it up to CoreDNS. CoreDNS attempted to route traffic to the stale IP, failed, and started crash-looping. Internal DNS resolution (pod.namespace.svc.cluster.local) totally collapsed. The cluster was dead in the water.

Flowchart showing the cascading DNS failure in EKS worker nodes
The perfect storm: How stale data and disabled services led to a total CoreDNS collapse.

Linux Internals: How systemd Manages DNS (And Why CoreDNS Breaks)

To understand how to permanently fix this, we need to look at how systemd actually handles DNS under the hood. When using systemd-networkd, resolv.conf management is handled through a strict partnership with systemd-resolved.

Architecture diagram of systemd-networkd and systemd-resolved D-Bus communication
How systemd collects network data and the critical symlink choice that dictates EKS DNS health.

Here is how the flow works: systemd-networkd collects network and DNS information (from DHCP, Router Advertisements, or static configs) and pushes it to systemd-resolved via D-Bus. To manage your DNS resolution effectively, you must configure the /etc/resolv.conf symbolic link to match your desired mode of operation. You have three choices:

1. The “Recommended” Local DNS Stub (The EKS Killer)

By default, systemd recommends using systemd-resolved as a local DNS cache and manager, providing features like DNS-over-TLS and mDNS.

  • The Symlink: ln -sf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf
  • Contents: Points to 127.0.0.53 as the only nameserver.
  • The Problem: This is a disaster for Kubernetes. If Kubelet passes 127.0.0.53 to CoreDNS, CoreDNS queries its own loopback interface inside the pod network namespace, blackholing all cluster DNS.

2. Direct Uplink DNS (The “Super Hack” Solution)

This mode bypasses the local stub entirely. The system lists the actual upstream DNS servers (e.g., your AWS VPC nameservers) discovered by systemd-networkd directly in the file.

  • The Symlink: ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
  • Contents: Lists all actual VPC DNS servers currently known to systemd-resolved.
  • The Benefit: CoreDNS gets the real AWS VPC nameservers, allowing it to route external queries correctly while managing internal cluster resolution perfectly.

3. Static Configuration (Manual)

If you want to manage DNS manually without systemd modifying the file, you break the symlink and create a regular file (rm /etc/resolv.conf). While systemd-networkd still receives DNS info from DHCP, it won’t touch this file. (Not ideal for dynamic cloud environments).


The Solution: A Surgical systemd Cutover

Knowing the internals, the path forward is clear. We needed to not only remove the legacy stack but explicitly rewire the DNS resolution to the Direct Uplink to prevent the stale data trap and bypass the notorious 127.0.0.53 stub listener.

Here is the exact state we achieved:

  1. Lock down cloud-init so it stops triggering legacy network services.
  2. Completely mask NetworkManager to ensure it never wakes up.
  3. Ensure systemd-resolved is enabled and running, but with the DNSStubListener explicitly disabled (DNSStubListener=no) so it doesn’t conflict with anything.
  4. Destroy the stale /etc/resolv.conf and create a symlink to the Direct Uplink (ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf).
  5. Reconfigure and restart systemd-networkd.

Pro-Tip for Debugging: To ensure systemd-networkd is successfully pushing DNS info to the resolver, verify your .network files in /etc/systemd/network/. Ensure UseDNS=yes (which is the default) is set in the [DHCPv4] section. You can always run resolvectl status to see exactly which DNS servers are currently assigned to each interface over D-Bus!

The Automation: Production AMI Prep Script

Manual hacks are great for debugging, but SRE is about repeatable automation. We’ve open-sourced the eks-production-ami-prep.sh script to handle this cutover automatically during your Packer or Image Builder pipeline. It standardizes the cutover, wipes out the stale data, and includes a strict validation suite.


The Results

By actively taking control of the systemd stack and ensuring /etc/resolv.conf was dynamically linked rather than statically abandoned, we completely unblocked our EKS 1.33+ upgrade.

More impressively, our system bootup time dropped from a crippling 6+ minutes down to under 2 minutes. We shouldn’t have to abandon fantastic, free enterprise distributions just because a cloud provider shifts their networking paradigm. If your team is struggling with AWS EKS upgrades on Rocky Linux or AlmaLinux, integrate this automation into your pipeline and get your clusters back in the fast lane.