SSH Keys: How They Work and How to Use Them Safely

#What SSH protects

SSH (Secure Shell) is the protocol most people use to log into Linux servers, lab virtual machines and Git hosts. It opens a shell over an encrypted channel, and the same channel can copy files or forward ports. It has three jobs. It encrypts the traffic so nobody on the network path can read it. It proves to the server who you are. It proves to you who the server is.

Keys do the second job without sending a password across the network. A password can be guessed or phished. A key pair works differently. The server stores only the public half. To log in, your client signs a challenge with the private half, and the server checks the signature. The private key never leaves your machine.

SSH does not protect you in some cases, and those cases matter as much as the protocol itself. If your own computer is compromised, malware running as your user can read the private key file or ask your agent to sign things. If the private key is stolen and its passphrase is weak, an attacker can try to guess it offline. If you accept a server's identity without checking it, you may be talking to someone who is not the server at all.

#Two kinds of keys

SSH uses two different kinds of keys, and people mix them up often.

Host keys identify the server. Each SSH server has one or more, usually stored as /etc/ssh/ssh_host_*_key. The private part stays on the server. The public part is presented to every client that connects. The first time your client sees it, the client saves it in known_hosts. On later connections, a changed host key triggers a warning.

User keys identify you. You generate the pair on your own machine. The private key stays with you. The public key goes on the server, in ~/.ssh/authorized_keys for the account you want to use.

The difference has practical consequences. Copying your private key to a server gives that server, and anyone who can read it, the material needed to act as you everywhere the key is trusted. Ignoring a host key warning means you may send your password or your commands to an impostor.

#Why Ed25519 with a passphrase

OpenSSH supports several key types. RSA still works, but new RSA keys should be at least 3072 bits, and the public key lines get long. Ed25519 keys are short, fast to verify and standardized for SSH in RFC 8709. OpenSSH 9.5 and later make Ed25519 the default for ssh-keygen. Ed25519 is the right default for a lab key unless a system you need to reach only accepts an older type.

A passphrase encrypts the private key file on disk. Without one, anyone who copies the file can use the key. With one, they need the file and the passphrase. That protects against the most common accidents: a backup sent to the wrong place, a key left in a shared folder, a file synced to a cloud drive by mistake. A passphrase does not help once the key is loaded into an agent and malware runs as you. Keep the passphrase long, store it in a password manager, and keep a backup of the key file somewhere safe.

#How a connection works

RFC 4253 describes the transport layer. At a high level, a connection goes through these steps:

1. Your client opens a TCP connection to the server, usually on port 22.
2. Both sides send the algorithms they support and agree on a key exchange. The exchange produces shared secrets, and from that point all traffic is encrypted.
3. The server sends its host key and a signature over the exchange. Your client checks that host key against known_hosts. If the host is new, the client shows the fingerprint and asks you to confirm it.
4. You authenticate. The client offers a public key. If the server finds it in authorized_keys, it sends a challenge. Your client signs the challenge with the private key, and the server checks the signature.

Notice where trust comes from. Step 3 is about the server. Step 4 is about you. A fingerprint check that you skip in step 3 leaves the rest of the handshake working for whoever answered the call.

#Fingerprints

A fingerprint is a short hash of a public key. OpenSSH shows SHA256 fingerprints by default. Two people can compare a fingerprint to confirm they hold the same public key, without reading the whole key aloud.

The commands below were run on a Windows 11 machine with OpenSSH 9.9p2 (ssh -V), in Git Bash. The key was a scratch copy created for this observation, with an empty passphrase, so the output could be shown here. Your real key will have a different fingerprint.

ssh-keygen -t ed25519 -C 'lab-key' -f ./lab_ed25519 -N ''
ssh-keygen -lf ./lab_ed25519.pub

Observed output:

256 SHA256:2BAyZTrHADdiG1FNwfCwb/fC5vSVZPwfGnZjRaaExc4 lab-key (ED25519)

ssh-keygen -lvf prints the same fingerprint with a random-art picture. The picture is easier to notice when something changes, but the fingerprint text is what you compare.

#Hands-on lab

You need a virtual machine or cloud instance that you own and control, with an OpenSSH server running and an account that can use sudo. A local VM in VirtualBox or Hyper-V works. So does a small cloud instance you pay for and can delete. Do not run these steps against a machine that belongs to someone else.

#Step 1: generate the key

ssh-keygen -t ed25519 -C 'lab-key' -f ~/.ssh/lab_ed25519

When prompted, enter a long passphrase. The command creates two files: lab_ed25519 (private) and lab_ed25519.pub (public). Only the .pub file is meant to leave your machine.

#Step 2: check the fingerprint

ssh-keygen -lf ~/.ssh/lab_ed25519.pub

Write this fingerprint down, or note the first few characters. You will compare it again later.

#Step 3: lock down the private key file

chmod 600 ~/.ssh/lab_ed25519

On Linux and macOS, ssh refuses a private key that other users can read. On Windows, OpenSSH checks file ACLs rather than the Unix mode bits. On this Windows machine, chmod 600 ran without an error, but ls -l still showed -rw-r--r-- afterward. Do not assume the mode bits changed there. If ssh complains about permissions on Windows, inspect the file with icacls and remove access for other accounts.

#Step 4: install the public key on the VM

ssh-copy-id -i ~/.ssh/lab_ed25519.pub youruser@your-lab-vm

This is the one time you type the account password on purpose. The command appends the public key to ~/.ssh/authorized_keys on the VM.

The first connection to the VM also shows you a host key fingerprint. Compare it with the real one. On the VM, run:

ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub

If the two fingerprints match, type yes. If they do not match, stop. Find out why before you continue. A mismatch can mean you reinstalled the VM, or that someone sits between you and it.

This step was not run here, because no VM was available on this machine. Commands in steps 4 to 6 are shown without output for that reason. You can check each one on your own VM.

#Step 5: log in with the key

ssh -i ~/.ssh/lab_ed25519 youruser@your-lab-vm

Your client asks for the key passphrase, not the account password. If you get a shell, the key works. Keep this session open for the next step.

#Step 6: turn off password login

On the VM:

sudo sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo sshd -t && sudo systemctl restart ssh

The sed line replaces the PasswordAuthentication line, whether or not it is commented out. sshd -t checks the configuration syntax and stops the chain if it fails. systemctl restart ssh applies the change. On Fedora and other RHEL-family systems the service is called sshd, so use that name.

Three things commonly go wrong here:

• The line is overridden. On many Debian and Ubuntu systems, /etc/ssh/sshd_config includes files from /etc/ssh/sshd_config.d/, and the first value read for a keyword wins. Check the value that sshd actually uses:

sudo sshd -T | grep -i '^passwordauthentication'

After the change, the output should be passwordauthentication no. If it still says yes, find the file that sets it and fix it there.

• The line is missing. sed only edits a line that exists. If there is no PasswordAuthentication line, the default applies, and the sshd_config(5) man page says that default is yes. Add the line yourself:

echo 'PasswordAuthentication no' | sudo tee -a /etc/ssh/sshd_config

• You lock yourself out. Before you close the session you used in step 5, open a second terminal and log in with the key. Only close the first terminal after the second one works. If you lose access, use the VM console from your hypervisor or cloud dashboard to repair the configuration.

Then test from your own machine that password login is refused:

ssh -o PubkeyAuthentication=no -o PreferredAuthentications=password youruser@your-lab-vm

The connection should fail with a permission denied message. Run this check yourself on the VM you own. It was not run on this machine.

#The agent and known_hosts

#ssh-agent

Typing the passphrase on every connection gets tiring. ssh-agent keeps the decrypted key in memory for your session, so you type the passphrase once.

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/lab_ed25519
ssh-add -l

ssh-add -l lists the fingerprints the agent holds. On this Windows machine, ssh-add -l ran without an agent running and printed Could not open a connection to your authentication agent. with exit code 2. Windows uses the OpenSSH Authentication Agent service for this. Start that service from the Services console or from an administrator PowerShell. That is a change to your system, so do it yourself and only if you want the agent running.

An agent is convenient, and it is also a target. Anyone who can talk to the agent socket while you are logged in can ask it to sign things. For that reason, avoid agent forwarding (ssh -A) to hosts you do not fully trust. A root user on the remote host can use your forwarded agent. If you need to reach a private host through a bastion, use a jump host instead:

ssh -J youruser@bastion youruser@private-vm

#known_hosts

The client stores host keys in ~/.ssh/known_hosts, one line per host and key type. Lines can be hashed if you set HashKnownHosts yes, which hides host names from someone who reads the file.

To look up a host:

ssh-keygen -F your-lab-vm

To remove an entry after you rebuild a VM and get a new host key:

ssh-keygen -R your-lab-vm

Do this only when you know why the key changed. A changed host key is a warning, not a nuisance. Removing the old entry without checking the new fingerprint teaches you to ignore the next warning, and that next one may be real.

#Key points

• Never share the private key. Only the .pub file goes to servers, to Git hosts or to other people.
• Protect the private key with a passphrase, and keep a backup of the file.
• Verify the host fingerprint on first connect, and investigate any change later.

#Common mistakes

• Pasting the private key into a chat, issue, ticket or screenshot. Treat it as a password that cannot be reset. If it leaks, generate a new key, install it, and remove the old public key from authorized_keys.
• Installing the private key on the server instead of the public one. The server needs only the .pub line in authorized_keys.
• Permissions on the server. On Linux, ~/.ssh should be 700 and authorized_keys should be 600. Looser modes can make sshd ignore the key. Check your sshd logs when a key login fails.
• Disabling passwords before key login works. This is the most common lockout. Follow the order in step 6.
• One key for everything. Reusing one key across lab VMs, Git hosts and production systems means one leak opens everything. Use separate keys by purpose, such as lab_ed25519 for the lab and another key for GitHub.
• Accepting an unknown host key without checking. The first connection is the moment an impostor is most likely to get in. Compare the fingerprint before typing yes.
• Losing the passphrase. There is no recovery. Store it in a password manager.
• Duplicated lines in authorized_keys. Running ssh-copy-id twice is harmless, but appending a key by hand several times makes the file harder to audit. Remove keys you no longer use.

#How to go further

Once the lab works, a few steps make daily use safer and tidier.

• Per-host settings in ~/.ssh/config. Give each host an alias, set the IdentityFile, and add IdentitiesOnly yes so the client offers only the key you chose. This stops the client from trying every key in your agent against a server.
• Security keys. OpenSSH can create keys stored on a hardware security key, using ssh-keygen -t ed25519-sk. The private part then cannot be copied off the device. Check your OpenSSH version and your key's support before you rely on this.
• SSH certificates. Organizations with many servers often use a certificate authority instead of distributing keys to every host. The concept is worth learning even if you do not need it yet.
• Read the source documents. The ssh-keygen and sshd_config man pages describe every option you will meet. RFC 4253 explains the transport layer, and RFC 8709 covers Ed25519 in SSH.

If file modes and ownership are still new to you, read the earlier article on Linux files and permissions first. The next article covers isolated Docker networks for labs, which you can use to keep lab traffic away from your everyday network.

#Sources

• OpenBSD manual, ssh-keygen(1): https://man.openbsd.org/ssh-keygen
• OpenBSD manual, sshd_config(5): https://man.openbsd.org/sshd_config
• OpenSSH 9.5 release notes: https://www.openssh.com/txt/release-9.5
• RFC 4253, The Secure Shell (SSH) Transport Layer Protocol: https://www.rfc-editor.org/rfc/rfc4253
• RFC 8709, Ed25519 and Ed448 Public Key Algorithms for the Secure Shell (SSH) Protocol: https://www.rfc-editor.org/rfc/rfc8709