Secure File Transfer with scp, sftp, and rsync

1. Transfer requirements determine the tool

All three tools can use SSH transport, but they provide different operating models. scp is convenient for direct copies. Modern OpenSSH scp uses the SFTP protocol by default while retaining familiar source/destination syntax. sftp exposes a file-transfer protocol with interactive and batch commands. rsync compares file trees, transfers changed data, and can preserve selected metadata efficiently.

flowchart TB
  A["What is the transfer requirement?"] --> B{"One direct copy?"}
  B -->|Yes| C["scp"]
  B -->|No| D{"Interactive or scripted remote file operations?"}
  D -->|Yes| E["sftp batch or interactive mode"]
  D -->|No| F{"Tree synchronization, resume, filters, or mirroring?"}
  F -->|Yes| G["rsync over SSH"]
  C --> H["Verify destination content and metadata"]
  E --> H
  G --> H
  H --> I["Record evidence and cleanup"]

2. scp provides concise copy semantics

# Local file to a remote directory
scp ./release.tar.gz app-prod:/srv/releases/

# Remote file to the current local directory
scp app-prod:/var/log/app/health.json ./

# Recursive directory copy
scp -r ./static/ app-prod:/srv/www/

# Preserve modification times and modes
scp -p ./release.tar.gz app-prod:/srv/releases/

# Limit bandwidth in Kbit/s and enable compression for compressible data
scp -l 20000 -C ./large-text-dump.sql app-prod:/srv/staging/

# Modern scp uses SFTP by default. -O requests the legacy SCP protocol
# only when an old server requires it and the risk is understood.
# scp -O ./file legacy-host:/tmp/

Remote operands use [user@]host:path. Quote shell metacharacters deliberately because local and remote path interpretation can differ by tool and protocol mode. Test unusual names, spaces, wildcards, and leading hyphens in a disposable directory before automating them.

3. sftp supports controlled remote file operations

SFTP is a protocol carried through SSH, not “FTP with encryption.” It supports operations such as listing, upload, download, rename, create directory, remove, and permission changes according to server capabilities and policy. Batch mode makes operations reviewable and fails when a command fails unless the command is prefixed to ignore errors.

# Interactive session
sftp app-prod

# Example interactive commands:
# pwd
# lpwd
# ls -la
# lls -la
# put release.tar.gz /srv/releases/release.tar.gz.part
# rename /srv/releases/release.tar.gz.part /srv/releases/release.tar.gz
# get /var/log/app/health.json ./health.json
# bye

# Batch file for repeatable deployment
cat > deploy.sftp <<'EOF'
-mkdir /srv/releases/incoming
put release.tar.gz /srv/releases/incoming/release.tar.gz.part
rename /srv/releases/incoming/release.tar.gz.part /srv/releases/incoming/release.tar.gz
ls -l /srv/releases/incoming/release.tar.gz
EOF

sftp -b deploy.sftp app-prod

Upload to a temporary name and rename only after successful completion when consumers watch the destination directory. Rename behavior and atomicity depend on server and filesystem context, so validate the target platform.

4. rsync synchronizes trees and selected attributes

Rsync uses a quick check based primarily on size and modification time unless options request other behavior. Over a remote shell, rsync normally starts a remote rsync process through SSH, so rsync must be installed on both ends. Archive mode -a is a bundle of options; it is not “preserve absolutely everything.” ACLs, extended attributes, hard links, sparse files, numeric IDs, and ownership may need additional options and privileges.

# Always preview a meaningful synchronization first
rsync -aivn --delete-delay \
  --exclude '.git/' \
  ./site/ app-prod:/srv/www/site/

# Apply after reviewing the itemized dry run
rsync -aiv --delete-delay \
  --exclude '.git/' \
  ./site/ app-prod:/srv/www/site/

# Resume partial large transfers in a controlled directory
rsync -aiv --partial --partial-dir=.rsync-partial \
  ./artifacts/ app-prod:/srv/artifacts/

# Preserve ACLs and xattrs when both endpoints and privileges support them
rsync -aivAX ./tree/ app-prod:/srv/tree/

# Use a specific SSH profile or option set
rsync -aiv -e 'ssh -o IdentitiesOnly=yes' \
  ./reports/ app-prod:/srv/reports/

5. Trailing slashes change directory semantics

With rsync, source and source/ are not equivalent. A source directory without a trailing slash transfers the directory itself into the destination. A trailing slash transfers the directory’s contents. This single character is one of the most common causes of unexpected nested paths.

mkdir -p demo/source/sub demo/dest-a demo/dest-b
printf 'alpha\n' > demo/source/file.txt
printf 'beta\n' > demo/source/sub/nested.txt

# Creates demo/dest-a/source/...
rsync -a demo/source demo/dest-a/

# Creates contents directly under demo/dest-b/...
rsync -a demo/source/ demo/dest-b/

find demo -type f -printf '%P\n' | sort

Destination trailing slashes can also affect interpretation when the target does not exist. Write test cases for the exact production path layout instead of relying on memory.

6. Verification must match the risk

A zero exit status proves the tool completed according to its own checks, not that the business outcome is correct. Verify file count, expected paths, sizes, checksums where appropriate, ownership, permissions, ACLs, extended attributes, symlink targets, and application readability.

# Build a source manifest before transfer
find ./release -type f -print0 | sort -z | xargs -0 sha256sum > source.sha256

# On the destination, create the same relative manifest
ssh app-prod '
  cd /srv/releases/current &&
  find . -type f -print0 | sort -z | xargs -0 sha256sum
' > destination.sha256

# Compare manifests after normalizing source path prefixes if needed
diff -u source.sha256 destination.sha256

# Metadata-oriented inspection
find ./release -printf '%M %u:%g %s %TY-%Tm-%TdT%TH:%TM:%TS %P\n' | sort > source-metadata.txt
ssh app-prod "find /srv/releases/current -printf '%M %u:%g %s %TY-%Tm-%TdT%TH:%TM:%TS %P\\n' | sort" \
  > destination-metadata.txt

Checksumming every file increases I/O and may not be necessary for routine synchronization over authenticated transport, but it is useful for release artifacts, evidence preservation, or suspected corruption. Choose verification proportional to consequences.

7. Hands-on lab: model a safe local rsync release

This lab uses local directories to teach the same planning and verification steps without requiring a remote server.

lab="$HOME/devops-academy/linux/chapter13/lesson04"
rm -rf "$lab"
mkdir -p "$lab/source/assets" "$lab/destination" "$lab/backups"
cd "$lab"

printf 'version=1\n' > source/app.conf
printf 'console.log("v1");\n' > source/assets/app.js
printf 'remove me\n' > destination/obsolete.txt

# Capture the proposed changes; do not apply yet
rsync -aivn --delete-delay --backup --backup-dir="$lab/backups" \
  source/ destination/ > dry-run.txt
cat dry-run.txt

# Apply the reviewed plan
rsync -aiv --delete-delay --backup --backup-dir="$lab/backups" \
  source/ destination/ > applied.txt

# Verify content and demonstrate where deleted/replaced files were retained
find source -type f -print0 | sort -z | xargs -0 sha256sum > source.sha256
find destination -type f -print0 | sort -z | xargs -0 sha256sum > destination.sha256
sed 's#  source/#  destination/#' source.sha256 > expected-destination.sha256
diff -u expected-destination.sha256 destination.sha256

test -f backups/obsolete.txt
find . -maxdepth 3 -type f -printf '%P\n' | sort > inventory.txt
sha256sum dry-run.txt applied.txt source.sha256 destination.sha256 inventory.txt > evidence.sha256

8. Common secure-transfer mistakes

9. Knowledge check

10. Summary

Secure transfer requires more than encrypted transport. Choose scp for direct copies, SFTP for file-operation workflows, and rsync for controlled synchronization. Make paths and trailing slashes explicit, preview destructive actions, preserve only required metadata, use temporary names where consumers observe files, and verify content and operational behavior after transfer.

11. Further reading

• OpenSSH scp(1) manual and sftp(1) manual.
• Official rsync(1) manual.
• rrsync(1) for restricted rsync-over-SSH accounts.