SSH port forwarding persists after a client hard reboot because the server lacks a mechanism to detect the disconnected peer, leaving the socket open and the forwarded port bound. To resolve this, you must configure TCP keepalives on both the client and server sides or implement application-level heartbeat logic within the forwarded service itself.
Understanding the Persistence Mechanism
When you establish an SSH tunnel using flags like
-L or
-R, the server creates a listening socket that forwards traffic to your local machine. Under normal circumstances, when you close the terminal or disconnect gracefully, the SSH daemon (
sshd) receives a signal, closes the connection, and releases the port. However, a hard reboot of the client machine bypasses this graceful shutdown sequence. The operating system does not send a FIN packet or a RST packet to the server; it simply ceases to exist on the network.
From the perspective of the Linux kernel on the server, the connection appears active. The TCP state machine sees no error because no packets have been exchanged recently to indicate failure. Consequently, the port remains in a LISTEN or ESTABLISHED state, consuming resources and potentially blocking other services from binding to that same port. This behavior is not a bug but a standard characteristic of persistent TCP connections without explicit termination signals.
Configuring Client-Side Keepalives
The most effective way to prevent orphaned connections is to force the SSH client to send periodic keepalive packets. These packets act as heartbeats, ensuring that if the remote end disappears, the connection is detected and closed quickly. You can achieve this by modifying your SSH configuration file located at
~/.ssh/config.
Add the following directives to the host block or globally:
Host *
ServerAliveInterval 15
ServerAliveCountMax 3
The
ServerAliveInterval setting instructs the client to send a request through the encrypted channel every 15 seconds. If the server does not respond within three consecutive intervals (controlled by
ServerAliveCountMax), the client assumes the connection is dead and terminates it. When the client process exits cleanly due to timeout, the associated SSH tunnel ports are also released on the server side. This method requires no changes to the server configuration and works across all modern SSH clients.
For more aggressive detection, you might consider enabling TCP keepalives directly via the operating system rather than the SSH protocol. This involves setting
TCPKeepAlive yes in your
~/.ssh/config. While this sends OS-level keepalives, they are less reliable for detecting application-layer disconnections compared to the SSH-specific
ServerAliveInterval.
Adjusting Server-Side Timeouts
If you cannot modify the client configuration, you can adjust the server's behavior to drop idle connections sooner. Edit the
/etc/ssh/sshd_config file on the remote server. Look for the
ClientAliveInterval and
ClientAliveCountMax directives.
Set these values to enforce stricter timeouts:
ClientAliveInterval 60
ClientAliveCountMax 2
This configuration tells
sshd to send a keepalive message to the client every 60 seconds. If two responses are missed, the server will terminate the session. This effectively cleans up zombie tunnels caused by hard reboots. After changing these settings, restart the SSH service using
sudo systemctl restart sshd for the changes to take effect. Note that this affects all SSH sessions on the server, so ensure the interval is long enough to avoid disrupting legitimate long-running administrative tasks.
Implementing Application-Level Heartbeats
In some cases, even with TCP keepalives, the port may remain bound if the underlying application listening on the forwarded port does not handle connection drops correctly. For example, if you are forwarding a database port or a web service, the application itself might hold the socket open. In such scenarios, configuring the application to detect broken pipes or implement its own heartbeat mechanism is necessary.
Additionally, consider using tools like
autossh which wrap the SSH command and automatically restart the tunnel if the connection fails. This provides a layer of resilience against transient network issues but does not inherently solve the cleanup problem on the server side unless the parent process is monitored.
For related issues regarding connectivity stability, see
Troubleshooting SSH Connection Issues: Intermittent Failures with Correct Password. Another common issue involves delays during connection establishment, which can sometimes be confused with persistence issues. Refer to
How to troubleshoot a long delay when connecting over SSH? for diagnostics on latency problems.
Verifying and Cleaning Up Orphaned Ports
To verify if a port is still held by an SSH process after a client reboot, use the
netstat or
ss command on the server. Run
ss -tlnp | grep :PORT_NUMBER to check the status of the specific port. If the state is
ESTAB or
LISTEN and the process ID belongs to
sshd, the tunnel is still active.
You can manually kill the process if necessary, but this is a temporary fix. Identifying the PID with
ps aux | grep sshd allows you to terminate the specific session using
kill -9 PID. However, relying on manual intervention is not scalable. Implementing the keepalive settings described above ensures automatic cleanup, reducing the need for administrative overhead.
Frequently Asked Questions
Why doesn't SSH automatically close the port when the client crashes?
SSH relies on TCP protocols which do not immediately detect a crash unless keepalive packets are configured. Without explicit heartbeat settings, the server assumes the connection is valid until data exchange resumes or a timeout occurs.
Can I set different keepalive intervals for different hosts?
Yes, you can specify unique settings for each host in your
~/.ssh/config file by creating separate
Host blocks with their respective
ServerAliveInterval values.
What happens if I set ServerAliveInterval too low?
Setting the interval too low (e.g., under 5 seconds) can generate excessive network traffic and increase CPU usage on both client and server, potentially leading to false positives if the network is slightly congested.
Does TCPKeepAlive work better than ServerAliveInterval?
ServerAliveInterval is generally preferred because it operates within the SSH protocol layer, providing more accurate detection of session health compared to OS-level TCP keepalives which may be ignored by firewalls or NAT devices.
How do I check if my SSH config changes were applied?
You can test the configuration by running
ssh -G hostname to dump the effective configuration parameters for a specific host and verify that
serveraliveinterval matches your intended value.