WardtermSSH client
EN
Download

client_loop: send disconnect: Broken pipe: keep SSH from dropping

The network connection under your SSH session died, and ssh found out when it tried to send something. The usual cause is a router or firewall forgetting an idle connection. Add ServerAliveInterval 60 to ~/.ssh/config so the session is never idle, and use tmux for work that must survive a real network change.

Updated . The Broken pipe message was reproduced with OpenSSH 10.3 on macOS through a local proxy that reset the TCP connection.

The message

client_loop: send disconnect: Broken pipe

In our test, a TCP reset under a running session printed this:

Read from remote host server: Connection reset by peer
client_loop: send disconnect: Broken pipe

Older clients print the same problem in another form:

packet_write_wait: Connection to 203.0.113.10 port 22: Broken pipe

Broken pipe means ssh wrote to a connection that no longer exists. It does not tell you who ended it, and it has nothing to do with your keys or password.

Causes, in order of likelihood

1. A router or firewall dropped an idle connection

Home routers, office firewalls and cloud NAT gateways keep a table of open connections and remove the idle ones, often after a few minutes. Your session looks fine until you press a key. The clue is that the session dies only after you leave it alone, and at a fairly regular time. A short test is to set the fix below on one connection and leave it idle for longer than it usually survives.

2. Your computer slept or changed network

Closing a laptop, or moving from Wi-Fi to a phone hotspot, ends the connection. When your address changes, no keepalive can save it. The clue is that it happens right after you open the laptop or move.

3. The server ended it

A reboot, a crash, or an idle timeout set by the administrator all look similar from your side. Check the server's uptime and log, and look for idle settings:

uptime
sudo journalctl -u ssh --since "1 hour ago"
sudo sshd -T | grep -iE '^(clientalive|channeltimeout|unusedconnectiontimeout)'

A session ended on purpose by the server usually prints Connection to server closed by remote host. instead of Broken pipe.

The fix on the client: ServerAliveInterval

Put this in ~/.ssh/config:

Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3

To try it once without changing any file:

ssh -o ServerAliveInterval=60 user@server

After 60 seconds of silence from the server, ssh sends a small message through the encrypted connection and expects a reply. It does two things:

  • The traffic keeps the connection in the router's table.
  • If the replies stop, ssh gives up after ServerAliveCountMax attempts, about 3 minutes here, instead of hanging.

The default interval is 0, meaning off, and the default count is 3. If your connection is removed sooner than a minute, lower the interval. When the limit is reached, this is what you see:

Timeout, server 203.0.113.10 not responding.

That message means the keepalive is doing its job: the network really stopped answering.

The same on the server: ClientAliveInterval

If you run the server and want this for everyone, add a file such as /etc/ssh/sshd_config.d/keepalive.conf:

ClientAliveInterval 60
ClientAliveCountMax 3

Then check and reload:

sudo sshd -t && sudo systemctl reload ssh

The unit is called sshd on some systems. Besides keeping connections alive, this also clears out sessions from clients that have vanished. When we froze a client, the server logged this and closed the session:

Timeout, client not responding from user user 203.0.113.20 port 40896

TCPKeepAlive is not the same thing

TCPKeepAlive is on by default on both sides, and turning it on again rarely helps. There are two differences:

  • It is sent by the operating system on its own schedule. On Linux the default wait is 2 hours, far too long to keep a router's table entry alive.
  • It is outside the encrypted channel, so it can be spoofed. Server alive messages cannot.

The manual also notes that TCP keepalive makes connections die if a route is down for a short while. If you would rather ride out brief outages, you can turn it off and rely on ServerAliveInterval alone.

When to use tmux instead

Keepalives stop a healthy connection from being dropped for being idle. They cannot save a session when the network actually goes away. If the connection ends, the shell on the server and what runs in it normally end too.

For long jobs, or if you move between networks, run the work inside tmux on the server:

tmux new -A -s work

Run the same command after you reconnect, and you are back in the same session, with the job still running. See Keep SSH sessions alive with tmux. Use both: keepalives for the everyday drops, tmux for what must not be lost.

If you change networks often, a Mosh client goes further: the session follows your new address instead of ending.

Questions

What does client_loop: send disconnect: Broken pipe mean?

The network connection under your SSH session ended, and ssh noticed when it tried to send data. It has nothing to do with your keys or password.

What is a good ServerAliveInterval value?

60 seconds works with most routers and firewalls. If idle sessions die in less than a minute, use 30. With ServerAliveCountMax 3, a dead connection ends after about three intervals.

Does ServerAliveInterval keep a session alive when I change network?

No. It stops idle drops on a working connection. When your address changes, the connection ends, so run long work in tmux on the server or use Mosh.

Sources

More in Error messages

From the blog

All guides