The OpenSSH long password bug. Five checks to see if your servers and containers are exposed

OpenSSH released version 10.6p2 yesterday to work around a bug that lets someone log in to some Linux systems by typing a password over 256 characters long. Not a clever password. Just a very long one.

Most servers aren't affected. If yours runs Ubuntu, Debian or Red Hat, you can probably relax. But "probably" isn't much use when it's SSH, so here's how to check properly, including the Docker containers people tend to forget about.

What happened

The OpenSSH release notes say the bug affects "some Linux systems when built and run with PAM support disabled". The OpenSSH team considers the root cause to be outside sshd, but it can be exploited through it, so they've fixed it there.

The researcher's write-up at sshhole.com has the detail. On systems that use musl (a small alternative to the standard C library, used by Alpine Linux and a lot of embedded kit), the function that checks passwords fails on very long input and returns a single asterisk. An asterisk is also what Linux puts in the password field of an account that should never log in with a password. So the two match, and sshd lets you in.

The write-up says that on Alpine's official AWS and Azure images, the default alpine user has an asterisk for a password and passwordless admin rights. That's root.

The team at Dev Partners wrote a good plain-English explanation of why it happens this morning, and it's worth reading for the lesson about error handling in your own code. I'm sticking to the checks.

The Register reports there's "no indication that vulnerable systems have been exploited". I haven't seen a CVE number for it yet in the sources I've read.

Does it affect you?

All four of these have to be true:

• sshd is built against musl, not the usual glibc. In practice that means Alpine, OpenWrt or a custom embedded build.
• sshd runs without PAM. Alpine's openssh-server package doesn't use PAM by default.
• Password logins are switched on. On Alpine they are by default.
• At least one account has an asterisk as its password.

The write-up is clear that Debian, Ubuntu, Fedora and other glibc systems are fine. So the places to look are Alpine servers, Alpine-based Docker containers that someone has added SSH to, and routers or appliances you can SSH into.

Five checks

1. Is sshd using musl? On the server or inside the container:

ldd /usr/sbin/sshd | grep musl

If that prints nothing, you're on glibc and this bug isn't yours. You can stop here, though check 5 is still worth doing.

2. Is PAM off and are passwords on? Ask sshd for the settings it's actually using:

sudo sshd -T | grep -E 'usepam|passwordauthentication'

You want to see passwordauthentication no. If usepam is missing from the output altogether, treat that as PAM being off.

3. Which accounts have an asterisk?

sudo awk -F: '$2 == "*" {print $1}' /etc/shadow

On Alpine you'll likely see root and several system accounts. Any of those, and password logins turned on, is the risky combination.

4. Do any of your containers run sshd? On a Docker host:

for c in $(docker ps --format '{{.Names}}'); do
  echo "== $c"
  docker exec "$c" sh -c 'grep PRETTY_NAME /etc/os-release; pgrep -l sshd'
done

Most Alpine images don't run SSH at all, so you'll usually get just the name. Anything showing Alpine and an sshd process goes back to checks 1 to 3.

5. Check from outside. From your own machine, ask the server which login methods it offers:

ssh -o PreferredAuthentications=none -o BatchMode=yes youruser@yourserver

It will refuse you, and the error lists the methods. If you see password or keyboard-interactive in the brackets, password logins are on. This works on any server, Alpine or not, and it's a quick way to spot one you forgot about.

What to do this week

Turn password logins off. This is the fix that works today, on any version. Add one line in its own file:

echo "PasswordAuthentication no" | sudo tee /etc/ssh/sshd_config.d/00-no-passwords.conf

Make sure your SSH key works first, keep your current session open, then restart sshd (rc-service sshd restart on Alpine) and test a new login before you log out.

Update OpenSSH when you can. The fix is in 10.6p2. When I looked at the Alpine package index, only the edge branch showed 10.6_p2. The stable branches (3.22 to 3.24) had new openssh builds dated the same day, but I couldn't confirm from the index whether they carry the fix. So don't rely on the version number alone. Turn passwords off as well.

Rebuild containers, don't patch them. If an image needs a fix, change the Dockerfile and redeploy. A fix made inside a running container disappears on the next deploy.

Ask whether the container needs SSH at all. Usually docker exec does the same job with nothing listening on the network.

The bigger point

Password SSH is one of those settings that's on until somebody turns it off. Plenty of servers are fine because nobody ever needed it, not because anyone decided. If an Alpine container with SSH in it was set up by a developer who's since moved on, the team at Dev Partners take on that kind of inherited setup and can tidy it up.

For the servers themselves, this is exactly what a health check covers. I log in, see what's listening and how it's set up, and tell you in plain English what to change.