Automatically Restart SSH Tunnel for Seamless Connectivity in Coffee Shops and Beyond

Automatically restarting an SSH tunnel requires a watcher script that distinguishes between the tunnel process and itself. By using unique port bindings or specific command signatures, you can prevent infinite restart loops while ensuring seamless database connectivity over unstable networks like public Wi-Fi.

The Challenge of Unstable Connections

When working in environments with fluctuating internet access, such as coffee shops or mobile hotspots, maintaining a persistent SSH tunnel to a remote database is critical. Standard approaches often involve a simple loop that checks if the connection is alive and restarts it if not. However, this naive implementation frequently fails because the monitoring script inadvertently matches its own process ID when scanning for active tunnels. This leads to an infinite loop where the script detects itself, triggers a restart, spawns a new instance, and repeats until system resources are exhausted. To solve this, we must implement a robust detection mechanism that isolates the actual tunnel process from the watchdog utility. This ensures that only genuine disconnections trigger a reconnection attempt, preserving system stability and network efficiency.

Designing a Robust Watcher Script

The core of the solution lies in writing a shell script that manages the lifecycle of the SSH tunnel. The script should perform three main actions: start the tunnel, monitor its status, and restart it only when necessary. To avoid matching itself, the script must use a unique identifier or check for specific arguments that distinguish the tunnel from the watcher. Here is a production-ready example using Bash. This script assumes you are forwarding a local port to a remote database port.

#!/bin/bash

# Configuration
LOCAL_PORT=5432
REMOTE_HOST="remote-db.example.com"
REMOTE_PORT=5432
SSH_KEY="/path/to/private/key"
LOG_FILE="/var/log/ssh-tunnel.log"

# Function to log messages
log() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}

# Function to start the tunnel
start_tunnel() {
    # Use -f to background immediately after authentication
    # Use -N to indicate no remote command execution
    # Use -L for local port forwarding
    ssh -f -N -o ServerAliveInterval=60 -o ServerAliveCountMax=3 \
        -i "$SSH_KEY" \
        -L "${LOCAL_PORT}:${REMOTE_HOST}:${REMOTE_PORT}" \
        "user@${REMOTE_HOST}"
    
    if [ $? -eq 0 ]; then
        log "Tunnel started successfully on port ${LOCAL_PORT}"
    else
        log "Failed to start tunnel"
        exit 1
    fi
}

# Main loop
while true; do
    # Check if the tunnel is running by looking at the listening port
    # We use lsof or ss to check if the LOCAL_PORT is in LISTEN state
    # This avoids grepping for 'ssh' which might match the watcher itself
    
    if ! ss -tln | grep -q ":${LOCAL_PORT}"; then
        log "Tunnel down. Restarting..."
        start_tunnel
    else
        log "Tunnel is active."
    fi
    
    # Wait before checking again to avoid excessive CPU usage
    sleep 10
done

This approach uses ss (or netstat) to verify if the local port is open. Since the watcher script does not open the port, it will not interfere with the check. This eliminates the risk of self-matching entirely.

Ensuring Stability with SSH Options

Beyond the watcher logic, configuring the SSH client itself plays a vital role in maintaining connectivity. The -o ServerAliveInterval=60 option sends a keep-alive message every 60 seconds through the encrypted channel. If the server does not respond within ServerAliveCountMax attempts, SSH terminates the connection gracefully. This allows the watcher script to detect the drop quickly rather than waiting for a timeout. Additionally, using -f backgrounds the process after authentication, which is essential for daemons. Without it, the SSH process would hold the terminal open, preventing the script from continuing its loop. Always ensure that your SSH key has appropriate permissions (chmod 600) to prevent security warnings that could interrupt automation. For more details on managing SSH connections securely, see Closing SSH Connection with Keygen. Understanding how keys are managed helps in setting up the automated environment without manual intervention.

Running as a System Service

While a simple bash loop works for testing, deploying this as a systemd service ensures it survives reboots and provides better logging integration. Create a unit file at /etc/systemd/system/ssh-tunnel.service:

[Unit]
Description=Auto-Restart SSH Tunnel
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/ssh-tunnel-watcher.sh
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Enable and start the service with systemctl enable --now ssh-tunnel.service. This method handles the complexity of process management, allowing you to focus on connectivity rather than debugging scripts. If you are integrating this into a containerized environment, refer to Access Remote SSH Tunnel Inside Docker Container for NAT Traversal for insights on handling network namespaces and port conflicts in isolated environments.

Troubleshooting Common Issues

Even with a robust script, issues may arise. First, verify that the firewall on both local and remote machines allows traffic on the specified ports. Second, check the log file for any authentication errors, which often occur if the SSH key expires or permissions change. Finally, ensure that the ss command is available in your environment; on minimal systems, you may need to install the iproute2 package.

Frequently Asked Questions

Why does my SSH tunnel keep disconnecting?

Unstable network connections are the primary cause, especially on mobile data or public Wi-Fi. Using ServerAliveInterval helps detect dead links faster, but physical layer interruptions will still break the session.

Can I use this script with multiple tunnels?

Yes, but you must modify the script to handle different local ports or create separate instances. Ensure each instance uses a unique PID file or port check to avoid interference between tunnels.

How do I debug why the tunnel isn't starting?

Check the system logs using journalctl -u ssh-tunnel.service. Look for authentication failures or permission denied errors, which are common when SSH keys are misconfigured.

Is it safe to run this script as root?

It is generally safer to run it under a dedicated user account with limited privileges. Running as root exposes your system to higher risks if the script or SSH configuration is compromised.

What happens if the remote server goes offline permanently?

The script will continuously attempt to reconnect based on the RestartSec interval. You may want to add a maximum retry counter to stop exhausting resources if the target is unreachable.