Change SSH config in server rescue mode

When a Linux server becomes inaccessible over SSH due to misconfigured settings, a broken update, or file system issues, booting into rescue mode often becomes the only viable recovery path. It’s a process many sysadmins eventually face, usually under pressure — and without documentation in hand. The SSH daemon may be misbehaving, or root login might be denied due to a change you can’t reverse remotely. This is where the old but gold rescue mode + chroot method becomes vital.

Changing SSH configuration in rescue mode isn’t hard by itself, but it requires precise steps and a clear understanding of your system’s mount structure and chroot behavior. Even slight missteps (like forgetting to bind /run) can silently break services inside the chroot, leaving you with a server that boots but doesn’t serve.

Let’s walk through what it really takes to safely edit your SSH config from rescue mode, not just the commands, but also the why behind each step, plus some of the traps you might not expect until it's too late.

Why Rescue Mode Matters When SSH Fails

Modern Linux systems especially those using systemd tend to be quite rigid with boot processes. A failed fstab entry, a broken network stack, or a misconfigured sshd_config can lock you out. If you’ve disabled PasswordAuthentication but forgot to deploy public keys properly, or if your PermitRootLogin setting is off and your only user lacks sudo, remote recovery is impossible.

In these cases, providers like Hetzner, OVH, or DigitalOcean often offer a rescue mode a minimal live environment (usually Debian-based) that boots over the network and drops you into a root shell. From here, you can mount the root partition of your actual system and chroot into it. This lets you operate almost as if the server had booted normally, including modifying configs, inspecting logs, and re-enabling SSH access.

The Rescue Process, Step by Step

Once you’ve activated rescue mode through your provider panel and connected via SSH:

This ensures old SSH host keys don't conflict if the rescue system uses different ones than your original OS.

Next, mount the root filesystem. Replace /dev/sda1 with your actual root partition if it differs:

Most users stop here and attempt to chroot, only to run into issues with systemctl or nano not behaving correctly. To avoid that, bind the needed virtual filesystems first:

Now chroot into the system:

From this point, you're inside your original OS environment.

Editing the SSH Configuration Safely

With the chroot active, navigate to the SSH config:

The following lines are often the culprits behind remote lockouts:

Each of these plays a specific role:

• PasswordAuthentication yes allows login with a password. If disabled without valid keys, you're locked out.
• PubkeyAuthentication yes ensures SSH keys are accepted.
• PermitRootLogin yes is essential if you're relying on root login directly — which, for servers without sudo-capable users, is often necessary.
• UsePAM yes is required for password logins to work in many setups; disabling it breaks more than people expect.

After saving changes, you should also verify your key files still exist and are readable:

Ensure there's no disk space issue or permissions misconfiguration wiping out .ssh or its contents. It's surprisingly common on full filesystems or during broken cloud-init deployments.

Before You Exit Chroot

Many people forget this step, but it’s important to cleanly exit and unmount everything. Do this in reverse:

If you don’t unmount cleanly, the rescue OS might mark the filesystem as dirty or leave systemd hanging on ghost files during reboot. That could slow down or even prevent a clean boot.

Why Your Changes Might Not Stick

Sometimes, even after correctly editing sshd_config, the changes don’t take effect. Common causes:

SELinux contexts: If your distro uses SELinux (e.g., CentOS, RHEL, Fedora), changes to /etc/ssh/sshd_config might be ignored if the file context is wrong. Run restorecon -v /etc/ssh/sshd_config before exiting chroot.

Broken init system: If systemctl fails inside chroot, don't worry. SSH will start on next real boot.

Missing host keys: Check /etc/ssh/ for files like ssh_host_rsa_key. If these are missing, SSH won't start. You can regenerate them with ssh-keygen -A inside chroot.

SSH bound to wrong interface: Some users modify the ListenAddress directive, which might break SSH after a network reconfiguration. Unless needed, keep it commented.

Tangible Scenarios Where This Matters

• A junior admin disables password login before properly configuring public keys across all nodes.
• Cloud-init overwrites SSH keys on next boot because NoCloud metadata is misconfigured.
• A server update replaces /etc/ssh/sshd_config with a vendor default and disables root login.
• In some RAID or LVM setups, rescue mode only boots the first disk; you have to manually activate the array or volume group before mounting.
• Some rescue systems don't include nano, forcing the use of vi or downloading editors manually.

Final words

Fixing SSH in rescue mode is one of those experiences you only need once to never forget. The difference between a 5-minute recovery and 3 hours of frustration often comes down to knowing that chroot alone isn’t enough, and that SSH config isn’t the only file involved in successful login.

Always test your access after changes. Always assume SSH can fail, and keep an alternate user with sudo, just in case. And if your provider allows boot snapshots, take one before major changes to SSH or networking.

Done right, rescue mode gives you a second chance without reinstalling. Misused, it can make things worse, especially if you blindly edit configs without understanding what each line does.

There's satisfaction in fixing a locked-down server without wiping it. But there's greater value in setting things up so that rescue mode is something you never need again.