SSH Key Authentication and ssh-agent

1. Public-key authentication proves possession

A user key pair contains a private key that must remain controlled by the user or automation boundary and a public key that may be distributed to servers. During authentication, the server challenges the client to prove possession of the private key for an authorized public key. The private key is not uploaded to the server.

Authorization commonly lives in ~/.ssh/authorized_keys, but centralized directories, SSH certificates, identity-aware gateways, and configuration-management systems can provide the same decision at larger scale. Possession of a key is not the complete policy: account state, source restrictions, forced commands, certificates, multi-factor requirements, and server configuration can also participate.

sequenceDiagram
  participant U as User or automation
  participant A as ssh-agent or private-key provider
  participant C as SSH client
  participant S as SSH server
  U->>C: Request login
  C->>S: Offer public-key identity
  S-->>C: Authentication challenge
  C->>A: Request signature
  A-->>C: Signature, private key stays protected
  C->>S: Signed proof
  S->>S: Match authorized key and policy
  S-->>C: Authentication success or failure

2. Generate keys for an explicit identity and purpose

Ed25519 keys are compact and widely used in current OpenSSH deployments. RSA may remain necessary for compatibility with older systems or external policy, but key size and accepted signature algorithms must be evaluated separately. Security-key-backed types such as ed25519-sk or ecdsa-sk can require a physical authenticator and optionally user verification.

# Human identity with a clear filename and comment
ssh-keygen -t ed25519 \
  -a 64 \
  -C 'abolfazl@workstation:devops-admin' \
  -f "$HOME/.ssh/id_ed25519_devops_admin"

# Inspect public-key fingerprint and randomart
ssh-keygen -lf "$HOME/.ssh/id_ed25519_devops_admin.pub" -E sha256
ssh-keygen -lvf "$HOME/.ssh/id_ed25519_devops_admin.pub" -E sha256

# Inspect the public key derived from a private key
ssh-keygen -y -f "$HOME/.ssh/id_ed25519_devops_admin" | \
  ssh-keygen -lf - -E sha256

# Hardware-backed key, when a supported authenticator is present
# ssh-keygen -t ed25519-sk -O verify-required -f "$HOME/.ssh/id_ed25519_sk_admin"

The -a option increases key-derivation rounds for the private-key passphrase format. A passphrase protects a copied key file, but it does not help after the unlocked key is exposed in a compromised process or agent. Hardware-backed keys can reduce private-key extraction risk, though account recovery and token inventory become operational requirements.

3. Permissions and ownership are part of authentication

OpenSSH can reject insecure authorization files because another user might modify them. The home directory, .ssh directory, authorized_keys, and parent ownership all matter. Avoid “fixing” the problem with world-writable permissions.

# On the remote account, create a controlled SSH directory
install -d -m 700 "$HOME/.ssh"
touch "$HOME/.ssh/authorized_keys"
chmod 600 "$HOME/.ssh/authorized_keys"

# Verify path ownership and permissions component by component
namei -l "$HOME/.ssh/authorized_keys" 2>/dev/null || true
stat -c '%A %a %U:%G %n' "$HOME" "$HOME/.ssh" "$HOME/.ssh/authorized_keys"

# Install a public key through an existing authenticated channel
ssh-copy-id -i "$HOME/.ssh/id_ed25519_devops_admin.pub" user@server.example.net

# Test the exact identity without falling back to unrelated keys
ssh -o IdentitiesOnly=yes \
  -i "$HOME/.ssh/id_ed25519_devops_admin" \
  user@server.example.net

ssh-copy-id reduces quoting and permission mistakes but still depends on a trusted authenticated connection. Configuration management is preferable when provisioning many servers because it creates reviewable, repeatable authorization state.

4. Authorized keys can carry restrictions

An authorized_keys line can restrict a key by source address, force a command, disable forwarding, or attach environment and certificate-authority semantics. These controls are useful for backup, deployment, and monitoring identities that should not receive an unrestricted shell.

# Conceptual restricted key examples; substitute a real public key
# from="192.0.2.0/24",restrict ssh-ed25519 AAAA... backup-client
# command="/usr/local/sbin/deploy-receive",restrict ssh-ed25519 AAAA... deploy-job
# cert-authority,principals="ops,release" ssh-ed25519 AAAA... user-ca

# Inspect authorization file without printing private material
awk '
  /^[[:space:]]*#/ || /^[[:space:]]*$/ {next}
  {print NR ":" $1, $2, $NF}
' "$HOME/.ssh/authorized_keys" 2>/dev/null || true

5. ssh-agent holds usable identities for client processes

An agent lets clients request signatures without repeatedly reading private-key files. It does not make every loaded key safe. Any process able to reach the agent socket under the same security context may ask it to sign authentication challenges, subject to confirmation and lifetime controls.

# Start an agent for the current shell
# eval "$(ssh-agent -s)"

# List loaded keys
ssh-add -l 2>/dev/null || true

# Add a key for one hour
ssh-add -t 1h "$HOME/.ssh/id_ed25519_devops_admin"

# Require confirmation for each use when supported by the environment
ssh-add -c "$HOME/.ssh/id_ed25519_devops_admin"

# Remove one identity or all identities
ssh-add -d "$HOME/.ssh/id_ed25519_devops_admin.pub"
# ssh-add -D

# Verify the agent endpoint
printf 'SSH_AUTH_SOCK=%s\n' "${SSH_AUTH_SOCK:-not-set}"
ssh-add -l 2>&1 || true

Agent forwarding exposes access to the local agent through the remote host. The private key is not copied, but a compromised remote host can attempt to use the forwarded agent while the session exists. Prefer jump hosts with ProxyJump, destination-specific identities, or short-lived certificates instead of broad agent forwarding.

6. Human keys and automation keys require different controls

7. Hands-on lab: exercise a disposable key and isolated agent

The lab creates an unencrypted key only inside a disposable course directory so it can run non-interactively. Do not reuse it for any real account. Production human keys should use a passphrase or hardware-backed protection.

lab="$HOME/devops-academy/linux/chapter13/lesson02"
rm -rf "$lab"
install -d -m 700 "$lab"
cd "$lab"

ssh-keygen -q -t ed25519 -a 32 -N '' \
  -C 'DISPOSABLE-DEVOPS-ACADEMY-LAB' \
  -f lab_identity
chmod 600 lab_identity
chmod 644 lab_identity.pub

ssh-keygen -lf lab_identity.pub -E sha256 > fingerprint.txt
ssh-keygen -y -f lab_identity > derived-public-key.txt
cmp -s \
  <(awk '{print $1, $2}' lab_identity.pub) \
  <(awk '{print $1, $2}' derived-public-key.txt)

ssh-agent sh -c '
  ssh-add -t 5m ./lab_identity >/dev/null
  ssh-add -l -E sha256 > agent-list.txt
  ssh-add -D >/dev/null
'

stat -c '%A %a %U:%G %n' lab_identity lab_identity.pub > permissions.txt
sha256sum fingerprint.txt derived-public-key.txt agent-list.txt permissions.txt > evidence.sha256

8. Common key-authentication mistakes

9. Knowledge check

10. Summary

SSH user keys are authorization instruments with a lifecycle: generate them for a named purpose, protect private material, distribute only public material, constrain authorization, test exact identities, monitor use, and revoke cleanly. ssh-agent improves usability but creates a signing boundary that must be limited by lifetime, confirmation, endpoint security, and careful avoidance of unnecessary forwarding.

11. Further reading

• ssh-keygen(1), ssh-agent(1), and ssh-add(1).
• ssh-copy-id(1).
• OpenSSH authorized-keys format and options.