EN

Linux and networking

Incremental Backup with rsync and cron

Back up one Linux server to another with rsync and cron: dedicated, restricted SSH key, hard-link snapshots, rotation and a restore test.

8 min read, updated on 18 November 2026

On this page

This guide explains how to back up a server every night to a second server with rsync and cron. Each backup looks like a full, dated copy, but unchanged files are not stored again: they are linked to the previous backup with hard links.

The backed-up server uses a dedicated SSH key, limited to writing into a single folder. Even if this server is compromised, the key cannot open a shell on the backup server, nor read back or delete older backups through rsync.

Prerequisites

  • Two servers running Debian 12 or 13, or Ubuntu 22.04 or 24.04, with sudo access
  • The source server, the one to back up, called web01 in this guide
  • The backup server, with enough disk space, ideally in another location
  • A file system that supports extended attributes on the backup server, such as ext4 or XFS

Replace BACKUP_SERVER_IP and SOURCE_SERVER_IP with the addresses of your servers.

Port to open

Only the backup server receives a connection: port 22/tcp, or your SSH port, must be reachable from the source server's address. With UFW, on the backup server:

Terminal
sudo ufw allow from SOURCE_SERVER_IP to any port 22 proto tcp

Keep the rule that lets you administer this server yourself. Our guide Secure your server details the SSH configuration.

Prepare the backup server

  1. Install the packages. python3 is required by the rrsync script, which ships with rsync:

    Terminal
    sudo apt update
    sudo apt install -y rsync python3 cron
  2. Create a dedicated account, with no usable password, and the destination folder:

    Terminal
    sudo useradd --create-home --home-dir /srv/rsyncbackup --shell /bin/sh rsyncbackup
    sudo install -d -o rsyncbackup -g rsyncbackup -m 700 /srv/rsyncbackup/.ssh /srv/rsyncbackup/web01

Prepare the source server

  1. Install the packages:

    Terminal
    sudo apt update
    sudo apt install -y rsync cron openssh-client
  2. Create a key reserved for the backup. It has no passphrase, because cron must be able to use it on its own: this is why it will be heavily restricted in the next step.

    Terminal
    sudo install -d -m 700 /root/.ssh
    sudo ssh-keygen -t ed25519 -N '' -C 'backup-rsync@web01' -f /root/.ssh/backup_rsync
    sudo cat /root/.ssh/backup_rsync.pub

Restrict the key on the backup server

  1. Open the authorized keys file:

    Terminal
    sudo nano /srv/rsyncbackup/.ssh/authorized_keys
  2. Add a single line, with your public key at the end:

    Text
    from="SOURCE_SERVER_IP",restrict,command="/usr/bin/rrsync -wo -no-del /srv/rsyncbackup/web01" ssh-ed25519 AAAA...your_public_key backup-rsync@web01
    Element Purpose
    from= Accepts this key from the source server only
    restrict Disables the terminal and SSH forwarding
    command= Forces rrsync in place of any other command
    -wo Allows write-only access, with no reading back
    -no-del Rejects rsync's deletion options
  3. Apply the permissions:

    Terminal
    sudo chown rsyncbackup:rsyncbackup /srv/rsyncbackup/.ssh/authorized_keys
    sudo chmod 600 /srv/rsyncbackup/.ssh/authorized_keys
  4. Display the server fingerprint to compare it in the next step:

    Terminal
    sudo ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub

Check the connection

On the source server, record the backup server's key, then compare its fingerprint with the previous one:

Terminal
ssh-keyscan -t ed25519 BACKUP_SERVER_IP | sudo tee -a /root/.ssh/known_hosts
sudo ssh-keygen -lf /root/.ssh/known_hosts

If the fingerprints differ, delete the added line and find the cause before continuing. Then test the key:

Terminal
sudo ssh -i /root/.ssh/backup_rsync -o IdentitiesOnly=yes rsyncbackup@BACKUP_SERVER_IP

The expected response is an rrsync error containing Not invoked via sshd. It proves that the key is accepted and that it does not give a shell.

Create the backup script

On the source server:

  1. Create the exclusion list:

    Terminal
    sudo nano /etc/backup-rsync.exclude
    Text
    /root/.ssh/backup_rsync*
    /root/.ssh/restore_rsync*
    /root/.cache/
    /home/*/.cache/
    /var/www/*/cache/
    *.tmp
  2. Create the script, filling in the backup server's address and the folders to copy:

    Terminal
    sudo nano /usr/local/sbin/backup-rsync.sh
    Terminal
    #!/bin/bash
    set -uo pipefail
    
    BACKUP_HOST="BACKUP_SERVER_IP"
    BACKUP_USER="rsyncbackup"
    SSH_PORT=22
    SSH_KEY="/root/.ssh/backup_rsync"
    SOURCES=(/etc /home /root /var/www /var/backups)
    EXCLUDES="/etc/backup-rsync.exclude"
    STATE_DIR="/var/lib/backup-rsync"
    SNAPSHOT="$(date +%Y-%m-%d_%H%M%S)"
    
    exec 9>/run/lock/backup-rsync.lock
    if ! flock -n 9; then
      echo "$(date -Is) backup already running"
      exit 0
    fi
    
    mkdir -p "$STATE_DIR"
    EXISTING=()
    for path in "${SOURCES[@]}"; do
      [ -e "$path" ] && EXISTING+=("$path")
    done
    
    SSH_CMD="ssh -i $SSH_KEY -p $SSH_PORT -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o ConnectTimeout=20"
    OPTIONS=(-aHAX --numeric-ids --relative --timeout=600 --stats
      --exclude-from="$EXCLUDES" -M--fake-super -e "$SSH_CMD")
    
    if [ -s "$STATE_DIR/last" ]; then
      OPTIONS+=("--link-dest=/$(cat "$STATE_DIR/last")")
    fi
    
    echo "$(date -Is) starting $SNAPSHOT"
    rsync "${OPTIONS[@]}" "${EXISTING[@]}" "$BACKUP_USER@$BACKUP_HOST:$SNAPSHOT/"
    status=$?
    
    # Code 24 means some files vanished during the copy.
    if [ "$status" -ne 0 ] && [ "$status" -ne 24 ]; then
      echo "$(date -Is) failed with rsync code $status"
      exit "$status"
    fi
    
    MARKER="$(mktemp)"
    rsync -e "$SSH_CMD" "$MARKER" "$BACKUP_USER@$BACKUP_HOST:$SNAPSHOT/.backup-complete"
    status=$?
    rm -f "$MARKER"
    if [ "$status" -ne 0 ]; then
      echo "$(date -Is) could not write completion marker, rsync code $status"
      exit "$status"
    fi
    
    echo "$SNAPSHOT" > "$STATE_DIR/last"
    echo "$(date -Is) finished $SNAPSHOT"
  3. Make it executable and run the first backup. It copies everything and can take a while:

    Terminal
    sudo chmod 700 /usr/local/sbin/backup-rsync.sh
    sudo /usr/local/sbin/backup-rsync.sh
  4. Wait a minute, then run it again to create a second snapshot:

    Terminal
    sudo /usr/local/sbin/backup-rsync.sh
Option Purpose
-a Preserves the directory tree, timestamps, permissions and symbolic links
-H, -A, -X Preserve hard links, ACLs and extended attributes
--numeric-ids Preserves numeric user and group IDs
--relative Reproduces full paths in the snapshot
--link-dest Links unchanged files to the previous snapshot
-M--fake-super Stores owners and permissions in extended attributes, without being root at the destination

Check the snapshots

On the backup server:

Terminal
sudo ls -la /srv/rsyncbackup/web01
sudo sh -c 'du -sh /srv/rsyncbackup/web01/*'
sudo sh -c 'stat -c "%h %n" /srv/rsyncbackup/web01/*/etc/hostname'

du counts each shared file only once: the second snapshot should therefore look much smaller than the first. stat must show a link count of at least 2 for an unchanged file. Each completed snapshot contains a .backup-complete file.

Schedule with cron

On the source server:

  1. Create the scheduled task:

    Terminal
    sudo nano /etc/cron.d/backup-rsync
    Text
    SHELL=/bin/bash
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    15 3 * * * root /usr/local/sbin/backup-rsync.sh >> /var/log/backup-rsync.log 2>&1

    This file must be owned by root and end with a newline.

  2. Limit the size of the log:

    Terminal
    sudo nano /etc/logrotate.d/backup-rsync
    Text
    /var/log/backup-rsync.log {
        weekly
        rotate 8
        compress
        missingok
        notifempty
    }
  3. Check that cron is active, then check the log the next day:

    Terminal
    sudo systemctl is-active cron
    sudo tail -n 30 /var/log/backup-rsync.log
    sudo cat /var/lib/backup-rsync/last

Delete old snapshots

Rotation runs on the backup server. The source server therefore has no way to erase the history.

  1. Create the script:

    Terminal
    sudo nano /usr/local/bin/backup-rsync-prune
    Terminal
    #!/bin/bash
    set -euo pipefail
    
    ROOT="/srv/rsyncbackup/$1"
    KEEP="${2:-14}"
    cd "$ROOT"
    
    mapfile -t COMPLETE < <(find . -mindepth 2 -maxdepth 2 -type f -name .backup-complete -printf '%h\n' | sed 's|^\./||' | sort)
    COUNT="${#COMPLETE[@]}"
    [ "$COUNT" -gt "$KEEP" ] || exit 0
    
    OLDEST_KEPT="${COMPLETE[COUNT-KEEP]}"
    for dir in 20*/; do
      snapshot="${dir%/}"
      if [[ "$snapshot" < "$OLDEST_KEPT" ]]; then
        rm -rf -- "${ROOT:?}/$snapshot"
      fi
    done

    It keeps the last 14 completed snapshots and removes everything older, including interrupted snapshots. Deleting a snapshot does not remove files still used by the following ones.

  2. Schedule it:

    Terminal
    sudo chmod 755 /usr/local/bin/backup-rsync-prune
    echo '45 5 * * * rsyncbackup /usr/local/bin/backup-rsync-prune web01 14' | sudo tee /etc/cron.d/backup-rsync-prune
  3. To be alerted if backups stop, monitor this check, which fails if no snapshot has completed in the last 26 hours:

    Terminal
    sudo find /srv/rsyncbackup/web01 -mindepth 2 -maxdepth 2 -name .backup-complete -mmin -1560 | grep -q . && echo "OK" || echo "NO RECENT BACKUP"

Test the restore

Do this test right away, then regularly. Since the backup key cannot read the data, create a second key, read-only, reserved for restores.

  1. On the source server, create this key:

    Terminal
    sudo ssh-keygen -t ed25519 -N '' -C 'restore-rsync@web01' -f /root/.ssh/restore_rsync
    sudo cat /root/.ssh/restore_rsync.pub
  2. On the backup server, add this line to /srv/rsyncbackup/.ssh/authorized_keys:

    Text
    from="SOURCE_SERVER_IP",restrict,command="/usr/bin/rrsync -ro /srv/rsyncbackup/web01" ssh-ed25519 AAAA...restore_key restore-rsync@web01
  3. On the source server, list the snapshots:

    Terminal
    BACKUP_HOST="BACKUP_SERVER_IP"
    SSH_RESTORE="ssh -i /root/.ssh/restore_rsync -p 22 -o BatchMode=yes -o IdentitiesOnly=yes"
    sudo rsync --list-only -e "$SSH_RESTORE" "rsyncbackup@$BACKUP_HOST:/"
  4. Restore /etc into a temporary folder, from the latest completed snapshot:

    Terminal
    SNAPSHOT="$(sudo cat /var/lib/backup-rsync/last)"
    sudo mkdir -p /var/tmp/rsync-restore
    sudo rsync -aHAX --numeric-ids -M--fake-super -e "$SSH_RESTORE" "rsyncbackup@$BACKUP_HOST:$SNAPSHOT/etc/" /var/tmp/rsync-restore/etc/
  5. Compare the content, owner and permissions:

    Terminal
    sudo diff -r /etc /var/tmp/rsync-restore/etc | head -n 20
    sudo stat -c '%U:%G %a %n' /etc/shadow /var/tmp/rsync-restore/etc/shadow

    Only files modified since the backup should differ, and the two stat lines must show the same owner and permissions.

  6. Delete the test copy, which contains sensitive data:

    Terminal
    sudo rm -rf /var/tmp/rsync-restore
  7. If you have nothing to restore, remove the restore-rsync key line on the backup server. You can add it again when needed.

Restore after an incident

Pick from the list a snapshot containing .backup-complete and put its name in SNAPSHOT. First simulate the operation with --dry-run and check the list, because it can overwrite files:

Terminal
sudo rsync -aHAX --numeric-ids -M--fake-super --dry-run --itemize-changes -e "$SSH_RESTORE" "rsyncbackup@$BACKUP_HOST:$SNAPSHOT/var/www/" /var/www/

If the result looks right, run the command again without --dry-run. If the source server is lost, prepare the new server, generate a restore key, add it on the backup server with the new address in from=, then restore your folders one by one.

Update

rsync is updated with the system, on both servers:

Terminal
sudo apt update
sudo apt upgrade

After an update, check the log of the next backup. If you replace the backup server, its host key changes: update /root/.ssh/known_hosts on the source server after verifying the new fingerprint.

Troubleshooting

Permission denied (publickey)

On the backup server, check the public key, the address in from= and the permissions. The SSH log often shows the cause:

Terminal
sudo ls -ld /srv/rsyncbackup /srv/rsyncbackup/.ssh
sudo ls -l /srv/rsyncbackup/.ssh/authorized_keys
sudo journalctl -u ssh -n 30 --no-pager

The home folder must not be writable by the group or other users.

Host key verification failed.

The backup server's key is missing from /root/.ssh/known_hosts or has changed. Redo the verification step, without disabling the check.

python3: No such file or directory

rrsync cannot start. Install python3 on the backup server.

rrsync rejects paths containing ... The --link-dest path must start with /, as in the script: it is then relative to the allowed folder.

reading from write-only server is not allowed

You are trying to read with the backup key. Use the restore_rsync restore key.

Another instance of rrsync is already accessing this directory.

A backup is already running, or an old connection has not ended. Check with pgrep -a rsync on both servers.

Code 23: some files/attrs were not transferred

Some files could not be copied. The log lists them, just before this message. The script treats this snapshot as failed: fix the cause and run it again.

Operation not supported (95) on attributes

The backup server's file system does not support the extended attributes used by --fake-super. Put the backups on an ext4 or XFS volume.

Code 255 or 12

The SSH connection failed or was cut. Run the connection test again and check the network, the firewall and the disk space with our guide Check disk space.

All files are copied again on every run

Check /var/lib/backup-rsync/last: it must contain the name of an existing snapshot. Also check that this snapshot contains the expected files and that the destination file system supports hard links.

References