What Are SSH Keys? A Complete Guide to SSH Key Authentication

SSH keys are cryptographic key pairs used to authenticate to remote servers without passwords. Instead of typing a password (which can be guessed, phished, or brute-forced), you prove your identity by demonstrating possession of a private key that matches a public key the server already trusts.

Every DevOps engineer, system administrator, and developer uses SSH keys daily — for server access, Git operations, CI/CD pipelines, and automated deployments. Yet most organizations have no inventory of their SSH keys, no rotation schedule, and no offboarding process when engineers leave.

---

How SSH Keys Work

An SSH key pair consists of:

• Private key (~/.ssh/id_ed25519) — stays on your machine, never transmitted
• Public key (~/.ssh/id_ed25519.pub) — deployed to servers you want to access

The Authentication Flow

1. You connect: ssh user@server.example.com
2. Server sends a random challenge
3. Your SSH client signs the challenge with your private key
4. Server verifies the signature using your public key (from authorized_keys)
5. Signature valid → you're authenticated. No password crossed the network.

The private key never leaves your machine. The server never sees it. An eavesdropper on the network sees only the signature (useless for future authentication because each challenge is unique).

---

Generating SSH Keys

Ed25519 (Recommended)

ssh-keygen -t ed25519 -C "yourname@company.com"
# Produces:
#   ~/.ssh/id_ed25519      (private key — PROTECT THIS)
#   ~/.ssh/id_ed25519.pub  (public key — deploy to servers)

# With passphrase (encrypts private key at rest):
ssh-keygen -t ed25519 -C "yourname@company.com" -N "strong-passphrase"

Why Ed25519:

• 128-bit security (same as RSA-3072)
• 32-byte keys (tiny compared to RSA’s 256+ bytes)
• Fast signing and verification
• Deterministic (no random nonce — immune to nonce-reuse attacks)
• Constant-time implementation (resistant to timing side channels)

RSA-4096 (Legacy Compatibility)

ssh-keygen -t rsa -b 4096 -C "yourname@company.com"
# Only use if connecting to systems that don't support Ed25519
# (very old OpenSSH < 6.5, some embedded systems)

FIDO2/U2F Hardware Key (Strongest)

ssh-keygen -t ed25519-sk -C "yourname@company.com"
# -sk = security key (YubiKey, etc.)
# Private key material lives IN the hardware token
# Requires physical touch for every authentication
# Cannot be stolen via software compromise

---

Deploying Public Keys

To a Single Server

# Automated (copies public key to server's authorized_keys)
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server.example.com

# Manual
cat ~/.ssh/id_ed25519.pub | ssh user@server "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

At Scale (Configuration Management)

# Ansible — deploy keys from central source of truth
- name: Deploy authorized SSH keys
  authorized_key:
    user: deploy
    key: "{{ lookup('file', 'keys/' + item + '.pub') }}"
    state: present
    exclusive: yes  # Remove any key NOT in this list
  loop: "{{ approved_users }}"

Disabling Password Authentication (Hardening)

# /etc/ssh/sshd_config
PasswordAuthentication no       # No passwords — keys only
PubkeyAuthentication yes        # Enable key auth
PermitRootLogin prohibit-password  # Root can only use keys
ChallengeResponseAuthentication no

After this, brute-force attacks against SSH are impossible — there’s no password to guess.

---

SSH Key Management Challenges

The Scale Problem

Each engineer has 1-3 keys deployed to 10-50 servers. For a 100-person team:

• ~5,000-15,000 authorized_keys entries across all servers
• No central inventory (keys scattered in files on individual servers)
• No expiry (SSH keys work forever unless manually removed)
• No offboarding automation (engineer leaves → keys persist)

The Visibility Problem

# How many SSH keys are on this server?
wc -l ~/.ssh/authorized_keys
# 47 keys. Who do they belong to? When were they added? Are they all still needed?
# Often: nobody knows.

The Rotation Problem

SSH keys have no built-in expiry. A key generated in 2020 works identically in 2030. Without active rotation:

• Compromised keys grant access indefinitely
• Algorithm upgrades never happen (RSA-1024 keys from 2015 still active)
• Compliance frameworks flag “no rotation evidence”

---

SSH Key Best Practices

1. Always Use a Passphrase

# Passphrase encrypts the private key file at rest
ssh-keygen -t ed25519 -N "my-strong-passphrase"

# Use ssh-agent to avoid retyping (caches decrypted key in memory)
eval $(ssh-agent)
ssh-add ~/.ssh/id_ed25519  # Enter passphrase once
# Subsequent SSH connections use cached key — no passphrase prompt

Without a passphrase: anyone who reads the file has your identity. With a passphrase: they also need to crack the encryption.

2. One Key Per Purpose

~/.ssh/id_ed25519_work     → work servers
~/.ssh/id_ed25519_github   → GitHub
~/.ssh/id_ed25519_personal → personal servers

# ~/.ssh/config
Host github.com
  IdentityFile ~/.ssh/id_ed25519_github

Host *.work.example.com
  IdentityFile ~/.ssh/id_ed25519_work
  User deploy

If one key is compromised, only that context is affected.

3. Never Share Private Keys

❌ Emailing a private key to a colleague
❌ Storing a private key in a shared drive
❌ Committing a private key to Git
❌ Using the same key across multiple people ("team key")

✅ Each person generates their own key pair
✅ Only public keys are shared/deployed
✅ Private keys never leave the owner's device

4. Rotate Annually

# Generate new key
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_2026 -C "user@company 2026"

# Deploy new public key to all servers
ssh-copy-id -i ~/.ssh/id_ed25519_2026.pub user@server

# Verify new key works
ssh -i ~/.ssh/id_ed25519_2026 user@server "echo OK"

# Remove old key from servers (after confirming new key works everywhere)
# Update ~/.ssh/config to use new key

5. Consider SSH Certificates (The Future)

SSH certificates add what keys lack: expiry dates and centralized trust.

# CA signs your public key with an 8-hour validity
ssh-keygen -s /path/to/ca_key -I "alice" -n "deploy,ubuntu" -V "+8h" ~/.ssh/id_ed25519.pub

# Server trusts the CA (one line in sshd_config)
TrustedUserCAKeys /etc/ssh/ca.pub

# After 8 hours: certificate expires, access ends automatically
# No authorized_keys to manage. No keys to remove on offboarding.

---

SSH Keys in CI/CD

GitHub Actions

- name: Deploy via SSH
  env:
    SSH_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
  run: |
    mkdir -p ~/.ssh
    echo "$SSH_KEY" > ~/.ssh/id_ed25519
    chmod 600 ~/.ssh/id_ed25519
    ssh -o StrictHostKeyChecking=no deploy@server "cd /app && git pull"

Security concern: The private key exists as a CI/CD secret. If the pipeline is compromised, the key is exposed. Better: use short-lived SSH certificates issued per-pipeline-run via Vault or Smallstep.

GitLab CI

deploy:
  script:
    - eval $(ssh-agent)
    - echo "$SSH_PRIVATE_KEY" | ssh-add -
    - ssh deploy@server "docker-compose pull && docker-compose up -d"

---

FAQ

Q: SSH keys vs passwords — which is more secure? A: SSH keys, overwhelmingly. A password can be guessed (brute force), phished, or intercepted. An Ed25519 key has 128-bit security — brute-forcing it would take longer than the age of the universe. Plus, the private key never crosses the network.

Q: What if I lose my private key? A: Generate a new key pair and deploy the new public key to all servers. The old key (if it still exists somewhere) should be removed from all authorized_keys files. There’s no way to recover a lost private key.

Q: Can someone hack my server if they have my public key? A: No. The public key is… public. It’s designed to be shared. Only the private key can authenticate. Having someone’s public key gives you zero access.

Q: How do I know which keys are on a server?

# List all authorized keys with fingerprints
ssh-keygen -lf ~/.ssh/authorized_keys
# Shows: key size, fingerprint, comment (usually email/name), algorithm

Q: Should I use ssh-agent forwarding? A: Avoid it. Agent forwarding exposes your SSH agent to the remote host. If that host is compromised, the attacker can use your agent to access other servers. Use ProxyJump instead:

# Instead of: ssh -A bastion, then ssh target
# Use: ssh -J bastion target (routes through bastion without exposing agent)
ssh -J bastion.example.com target-server.internal