TL;DR — SSH key management is one of the most overlooked security surfaces in modern infrastructure. This guide covers key types, agent configuration, certificate-based authentication, rotation strategies, and hardening patterns that production teams use to keep their systems resilient against credential compromise.

SSH remains the backbone of remote server administration, CI/CD pipelines, and inter-service communication across cloud and on-premise environments. Yet despite its ubiquity, SSH key management is frequently treated as a “set it and forget it” problem — keys are generated once, copied to authorized_keys files, and never revisited. That attitude creates a quietly dangerous attack surface. A single leaked private key can grant an attacker persistent, passwordless access to every system where the corresponding public key is authorized.

This article dives deep into OpenSSH key management: how to generate stronger keys, manage them with agents, enforce least-privilege access, rotate credentials at scale, and adopt certificate-based authentication to replace the fragile authorized_keys model entirely.

Understanding SSH Key Types and When to Use Each

OpenSSH supports several public-key algorithms, each with different security and performance characteristics. Choosing the right one matters.

  • Ed25519 — The modern default. Based on Edwards-curve Digital Signature Algorithm, it offers equivalent security to RSA-3072 with smaller key sizes and faster signing. Key sizes are fixed at 256 bits, eliminating the ambiguity of parameter selection.
  • RSA — The legacy workhorse. While still widely supported, RSA keys below 2048 bits are considered insecure. NIST and the OpenSSH project recommend a minimum of 3072 bits for new deployments, with 4096 bits for high-security environments.
  • ECDSA — Elliptic Curve Digital Signature Algorithm using NIST curves. It offers strong security at small key sizes but has drawn scrutiny due to its NIST P-256 curve, which some security practitioners distrust after revelations about potential backdoors.

Generating an Ed25519 key pair is straightforward:

ssh-keygen -t ed25519 -C "deploy@infra-prod" -f ~/.ssh/id_ed25519_infra -a 100

The -a flag sets the number of KDF rounds for passphrase protection, making brute-force attacks against encrypted keys significantly more expensive. A value of 100 is a reasonable starting point; increase it for high-value credentials.

The ssh-agent Pattern and Its Pitfalls

ssh-agent is the most common mechanism for managing decrypted private keys in memory. It solves a real problem: you shouldn’t store passphrase-less private keys on disk, but you also shouldn’t type a passphrase every time you run git push or ansible-playbook.

The standard workflow:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519_infra

Once added, the agent holds the decrypted key in memory and responds to signature requests from the SSH client. The private key never touches disk in plaintext form.

However, ssh-agent has well-documented pitfalls in production environments:

  • Agent forwarding (ForwardAgent yes) propagates your local agent to remote hosts. A compromised intermediate server can use your forwarded agent to authenticate to any other host where your key is authorized. This is one of the most common lateral-movement vectors in breach scenarios.
  • Persistent agents on developer workstations mean that if the machine is unattended or compromised, an attacker inherits all loaded keys.
  • No revocation granularity. Removing a key from the agent doesn’t retroactively invalidate its use on servers where it’s already authorized.

For production automation, consider using ssh-agent in short-lived containers or CI runners where the agent lifecycle is tied to the job. Avoid agent forwarding in favor of direct SSH access or bastion host patterns.

SSH Config as a Security Control

The ~/.ssh/config file is an underappreciated security tool. It lets you enforce per-host settings, specify identities, and disable dangerous defaults — all without remembering flags on every command.

A hardened configuration might look like this:

Host *.prod.example.com
    User deploy
    IdentityFile ~/.ssh/id_ed25519_prod
    IdentitiesOnly yes
    ServerAliveInterval 60
    ServerAliveCountMax 3
    ProxyJump bastion.prod.example.com

Host *.staging.example.com
    User deploy
    IdentityFile ~/.ssh/id_ed25519_staging
    IdentitiesOnly yes
    StrictHostKeyChecking accept-new

Host *
    AddKeysToAgent yes
    IgnoreUnknown UseKeychain
    UseKeychain yes
    ForwardAgent no
    PermitLocalCommand no

Key observations:

  • IdentitiesOnly yes prevents the SSH client from trying every key on disk, which avoids accidental authentication with the wrong credential and reduces information leakage to servers.
  • ForwardAgent no as a global default ensures agent forwarding is opt-in, not the norm.
  • ProxyJump replaces the older ProxyCommand pattern for bastion hosts, providing a cleaner and more auditable configuration.

Certificate-Based Authentication: Replacing authorized_keys

The authorized_keys model is fundamentally fragile at scale. Each public key must be distributed to every host where the corresponding user needs access. Revocation requires editing files on every host. Auditing who has access to what requires cross-referencing multiple files across multiple machines.

SSH Certificate Authorities (CAs) solve this by signing user public keys with a CA key, producing a certificate that contains the user identity, validity period, and enforced constraints. Servers trust the CA key (configured via TrustedUserCAKeys in sshd_config) and accept any certificate signed by it.

Setting Up an SSH CA

Generate the CA key:

ssh-keygen -t ed25519 -f ~/.ssh/ca_user_key -C "SSH CA for user authentication"

Sign a user’s public key with an expiration:

ssh-keygen -s ~/.ssh/ca_user_key -I "deploy-user" -n deploy -V +8h ~/.ssh/id_ed25519_deploy.pub

This produces id_ed25519_deploy-cert.pub, which the user presents to servers. The -V +8h flag sets an 8-hour validity window, enforcing automatic expiration without any manual revocation.

Server-Side Configuration

On each server, add to /etc/ssh/sshd_config:

TrustedUserCAKeys /etc/ssh/ca_user_key.pub
AuthorizedPrincipalsFile /etc/ssh/auth_principals/%u

The AuthorizedPrincipalsFile directive maps certificate principals to system users, enforcing that a certificate signed for the deploy principal can only log in as the deploy user. This replaces the need to maintain per-host authorized_keys files entirely.

Benefits at Scale

  • Automatic expiration. Short-lived certificates eliminate the need for complex revocation lists. A certificate that expires in 8 hours cannot be used after that window, regardless of whether it was stolen.
  • Centralized auditing. The CA logs every signing event, providing a complete audit trail of who received what credentials and when.
  • Principle enforcement. Principals restrict which accounts a certificate can access, implementing least-privilege at the protocol level.
  • Revocation on demand. Even before expiration, you can revoke a certificate by adding its serial number to a revocation list configured via RevokedKeys in sshd_config.

Companies like Netflix and Dropbox have published detailed accounts of their SSH CA migrations, describing how certificate-based authentication replaced fragile key distribution at thousands of servers [1].

Key Rotation Strategies for Production

Key rotation is the practice of periodically replacing cryptographic keys to limit the window of exposure if a key is compromised. In SSH infrastructure, this means rotating both host keys and user keys on a defined schedule.

Host Key Rotation

SSH host keys authenticate the server to the client, preventing man-in-the-middle attacks. Rotating host keys requires careful coordination because clients cache host keys in ~/.ssh/known_hosts.

A practical rotation process:

  1. Generate new host keys on the server.
  2. Distribute the new host key fingerprints through a trusted channel (configuration management, internal documentation).
  3. Update the known_hosts entries on all client machines via automation.
  4. Restart sshd with the new keys.
  5. Monitor for authentication failures.

Tools like ssh-keyscan can help automate bulk collection:

ssh-keyscan -t ed25519 server1.prod.example.com server2.prod.example.com >> /etc/ssh/trusted_hosts

User Key Rotation

User key rotation is more complex because it involves distributing new keys to developers and service accounts while revoking old ones. In a certificate-based model, rotation is trivial: issue new certificates with updated validity periods and let old ones expire.

In a traditional authorized_keys model, rotation requires:

  • Generating new key pairs for each user.
  • Updating authorized_keys on every affected host.
  • Communicating the transition to users.
  • Removing old keys after a grace period.

This is where configuration management tools like Ansible, Chef, or Puppet become essential. An Ansible playbook can update authorized_keys across an entire fleet in a single run:

- name: Deploy SSH authorized keys
  authorized_key:
    user: "{{ item.user }}"
    key: "{{ item.key }}"
    state: present
  loop: "{{ ssh_users }}"

Hardening sshd: Beyond Key Management

Strong keys and proper certificate management mean little if the SSH daemon itself is misconfigured. Several sshd_config options provide critical hardening:

  • PasswordAuthentication no — Disables password-based authentication entirely, forcing all access through cryptographic credentials.
  • PermitRootLogin no or prohibit-password — Prevents direct root login, forcing attackers to compromise a non-privileged account first.
  • MaxAuthTries 3 — Limits authentication attempts per connection, slowing brute-force attacks.
  • AllowUsers / AllowGroups — Restricts SSH access to explicitly permitted users or groups.
  • PubkeyAuthentication yes — Explicitly enables public key authentication (it’s the default, but stating it makes intent clear).

A hardened sshd_config excerpt:

PasswordAuthentication no
PermitRootLogin no
MaxAuthTries 3
PubkeyAuthentication yes
ChallengeResponseAuthentication no
UsePAM yes
X11Forwarding no
AllowTcpForwarding no
AllowAgentForwarding no

Disabling AllowTcpForwarding and AllowAgentForwarding prevents SSH from being used as a proxy channel, which is a common technique attackers use to pivot through a compromised host.

Patterns in Production: Real-World Considerations

Ephemeral Keys in CI/CD

Modern CI/CD platforms like GitHub Actions, GitLab CI, and CircleCI support SSH key injection as masked secrets. The best practice is to generate a throwaway key pair for each pipeline run, sign it with a short-lived CA certificate, and revoke it immediately after the job completes. This eliminates the risk of a static CI key being leaked from a logs repository or a compromised runner.

Bastion Hosts and Jump Servers

A bastion host acts as a single, hardened entry point. Developers SSH into the bastion, then from the bastion to internal hosts. The bastion should enforce MFA (via pam_google_authenticator or similar), log all sessions, and have agent forwarding disabled. Internal hosts should only accept connections from the bastion’s IP range.

Infrastructure as Code for SSH Configuration

Managing SSH configuration manually across hundreds of servers is a recipe for drift. Tools like ssh-audit can scan for misconfigurations, while infrastructure-as-code platforms ensure that sshd_config changes are version-controlled, reviewed, and applied consistently.

Key Takeaways

  • Prefer Ed25519 keys over RSA for new deployments — they offer stronger security with smaller size and faster operations.
  • Adopt SSH Certificate Authorities to replace the authorized_keys model, gaining automatic expiration, centralized auditing, and principle enforcement.
  • Disable agent forwarding globally and only enable it when absolutely necessary — it is a top lateral-movement vector in real-world breaches.
  • Rotate keys on a defined schedule, and use short-lived certificates to make rotation automatic rather than manual.
  • Harden sshd_config with PasswordAuthentication no, PermitRootLogin no, and restricted forwarding options as baseline controls.
  • Treat SSH keys as secrets — store them in a secrets manager, encrypt them at rest, and never commit private keys to version control.

Further Reading