rsync (Incremental Backup)

# -----------------------------------------------
#  Basic rsync syntax
# -----------------------------------------------

# rsync [options] source destination
#   → Transfers only changed files from source to destination
#   → Works for both local-to-local and SSH remote transfers

# -----------------------------------------------
#  Commonly used options
# -----------------------------------------------

# -a (--archive)
#   → Archive mode. Equivalent to -rlptgoD.
#     Enables recursive copy, symbolic link preservation,
#     permission preservation, timestamp preservation,
#     and owner preservation all at once.
#   Example: rsync -a /src/ /dst/

# -v (--verbose)
#   → Displays the names of files being transferred.

# -z (--compress)
#   → Compresses data during transfer. Useful for network backups.

# -n (--dry-run)
#   → Simulates the transfer without actually copying anything.
#   Example: rsync -avn /src/ user@remote:/dst/

# -e (--rsh)
#   → Specifies the remote shell to use.
#   Example: -e "ssh -p 2222"   → Use SSH on port 2222.

# --delete
#   → Deletes files from the destination that no longer exist in the source (full mirroring).
#   Example: rsync -az --delete /src/ user@remote:/dst/

# --exclude={pattern}
#   → Excludes files matching the specified pattern.
#   Example: --exclude="*.log"

# --exclude-from={file}
#   → Reads exclude patterns from a file.
#   Example: --exclude-from=/home/okabe/exclude_list.txt

# --backup
#   → Saves existing files that would be overwritten as backups.

# --backup-dir={directory}
#   → Specifies the directory where --backup saves the old files.

# --progress
#   → Displays the transfer progress for each file.

# --stats
#   → Displays transfer statistics (total bytes, speed, etc.) after completion.

# --link-dest={directory}
#   → Hard-links files that are identical to the previous backup.
#   → Only changed files are physically copied, making it ideal for incremental backups.

Option	Description
-a（--archive）	Archive mode. Preserves recursion, permissions, timestamps, ownership, and more in one flag. The standard option for backups.
-v（--verbose）	Displays the names of files being transferred. Use -vv for even more detail.
-z（--compress）	Compresses data in real time during transfer. Reduces backup time over slow network connections.
-n（--dry-run）	Simulates the sync without transferring anything. Combine with -v to preview the list of changes.
-e（--rsh）	Specifies the remote shell to use. Useful for changing the SSH port or specifying a key file.
--delete	Removes files from the destination that no longer exist in the source. Use when you need a complete mirror.
--exclude={pattern}	Excludes files and directories matching the given pattern from the sync. Wildcards are supported.
--exclude-from={file}	Reads exclude patterns from a file. Makes it easier to manage a large number of exclusions.
--backup	Saves files at the destination that would be overwritten under a different name. Use together with --backup-dir.
--backup-dir={directory}	Specifies the directory where --backup stores the old files.
--progress	Displays real-time transfer progress (bytes transferred, speed, estimated time) for each file.
--stats	Displays statistics after the transfer completes, including total files transferred, bytes sent, and transfer speed.
--link-dest={directory}	Hard-links files that are identical to those in the specified directory, physically copying only changed files. Ideal for saving disk space with incremental backups.

Incremental backup to a remote server

# -----------------------------------------------
#  Incremental backup from local to remote over SSH
# -----------------------------------------------

# Syncs okabe's home directory to /backup/okabe/ on the remote server (mayuri-backup).
# --delete removes files from the remote that no longer exist locally.
# It is safer to first run a dry run with -n to preview what will be transferred.

# First, check the diff with --dry-run
rsync -avzn --delete \
    /home/okabe/ \
    mayuri@mayuri-backup:/backup/okabe/

# Once you have verified the output, remove -n to run the actual transfer
rsync -avz --delete \
    /home/okabe/ \
    mayuri@mayuri-backup:/backup/okabe/

# If SSH is running on port 2222, specify it with the -e option
rsync -avz --delete \
    -e "ssh -p 2222 -i /home/okabe/.ssh/id_ed25519" \
    /home/okabe/ \
    mayuri@mayuri-backup:/backup/okabe/

Run the following command:

$ rsync -avz --delete /home/okabe/ mayuri@mayuri-backup:/backup/okabe/
sending incremental file list
./
.bashrc
Documents/steins_gate_notes.txt
Documents/divergence_memo.txt
Pictures/lab_members.jpg

sent 1,234,567 bytes  received 4,321 bytes  567,890.00 bytes/sec
total size is 9,876,543  speedup is 7.98

Incremental backups with --link-dest

# -----------------------------------------------
#  Incremental backup using --link-dest (hard-link method)
# -----------------------------------------------

# Backs up to a directory named after today's date.
# By passing the previous backup to --link-dest,
# unchanged files are stored as hard links, saving significant disk space.

TODAY=$(date +%Y-%m-%d)
YESTERDAY=$(date -d yesterday +%Y-%m-%d)
BACKUP_ROOT=/backup/kurisu

# Create today's backup, using yesterday's as the hard-link base
rsync -az --delete \
    --link-dest="${BACKUP_ROOT}/${YESTERDAY}" \
    /home/kurisu/ \
    "${BACKUP_ROOT}/${TODAY}/"

Run the following command:

$ ls /backup/kurisu/
2025-03-23  2025-03-24  2025-03-25
$ du -sh /backup/kurisu/2025-03-25
4.2M /backup/kurisu/2025-03-25

Narrowing the backup scope with an exclude list

# -----------------------------------------------
#  Selective backup using an exclude list file
# -----------------------------------------------

# Syncs mayuri's working directory, but excludes
# cache files and temporary files from the backup.

# Create an exclude list file (one pattern per line)
cat <<'EOF' > /home/mayuri/rsync_exclude.txt
.cache/
.npm/
node_modules/
*.tmp
*.log
*.swp
EOF

# Run the backup while referencing the exclude list
rsync -avz \
    --exclude-from=/home/mayuri/rsync_exclude.txt \
    /home/mayuri/workspace/ \
    hashida@lab-server:/backup/mayuri/workspace/

Run the following command:

$ rsync -avz --exclude-from=/home/mayuri/rsync_exclude.txt \
> /home/mayuri/workspace/ hashida@lab-server:/backup/mayuri/workspace/
sending incremental file list
project_alpha/main.py
project_alpha/config.json
project_beta/README.txt

sent 87,654 bytes  received 1,234 bytes  35,555.20 bytes/sec
total size is 4,567,890  speedup is 51.56

Automating daily backups with crontab

# -----------------------------------------------
#  Register with crontab to run an automatic daily backup
# -----------------------------------------------

# Create a backup script at /usr/local/bin/daily_backup.sh

#!/bin/bash

TODAY=$(date +%Y-%m-%d)
YESTERDAY=$(date -d yesterday +%Y-%m-%d)
BACKUP_ROOT=/backup/lab-members
LOG=/var/log/rsync_backup.log
REMOTE_USER=rukako
REMOTE_HOST=divergence-storage
REMOTE_PORT=22

# Log the start time
echo "=== Backup started: $(date) ===" >> "${LOG}"

# Back up all users under /home in one pass.
# Use --link-dest to reference yesterday's backup for hard linking.
rsync -az --delete \
    --link-dest="${BACKUP_ROOT}/${YESTERDAY}" \
    -e "ssh -p ${REMOTE_PORT} -i /root/.ssh/id_ed25519" \
    /home/ \
    "${REMOTE_USER}@${REMOTE_HOST}:${BACKUP_ROOT}/${TODAY}/" \
    >> "${LOG}" 2>&1

# Log the exit status
if [ $? -eq 0 ]; then
    echo "Backup succeeded: ${TODAY}" >> "${LOG}"
else
    echo "Backup FAILED: ${TODAY}" >> "${LOG}"
fi

# Remove backup directories older than 30 days to save disk space
find "${BACKUP_ROOT}" -maxdepth 1 -type d -mtime +30 -exec rm -rf {} +

echo "=== Backup finished: $(date) ===" >> "${LOG}"

The same logic can also be written as:

# -----------------------------------------------
#  Grant execute permission to the script and register it in crontab
# -----------------------------------------------

# Grant execute permission to the script
chmod +x /usr/local/bin/daily_backup.sh

# Edit crontab to schedule it to run every day at 3:00 AM
crontab -e

Run the following command:

# Add the following line to crontab (runs every day at 03:00)
0 3 * * * /usr/local/bin/daily_backup.sh

rsync uses a delta-transfer algorithm to transfer only the changed parts of files. The first run copies all files, but subsequent runs transfer only modified portions, significantly reducing network bandwidth usage and backup time. For remote backups over SSH, managing connection settings in ssh_config keeps the -e option concise. Incremental backups using --link-dest make each generation appear as a complete snapshot while physically copying only changed files, resulting in high storage efficiency — a strategy widely adopted for production server backups. For scheduled execution, registering the script in crontab is the standard approach. Getting into the habit of previewing changes with --dry-run (-n) before running the actual transfer helps prevent unintended file deletions or overwrites.