rclone sync

Make source and dest identical, modifying destination only.

Synopsis

Sync the source to the destination, changing the destination only. Doesn't transfer files that are identical on source and destination, testing by size and modification time or MD5SUM. Destination is updated to match source, including deleting files if necessary (except duplicate objects, see below). If you don't want to delete files from destination, use the copy command instead.

Important: Since this can cause data loss, test first with the --dry-run or the --interactive/-i flag.

rclone sync --interactive SOURCE remote:DESTINATION

Note that files in the destination won't be deleted if there were any errors at any point. Duplicate objects (files with the same name, on those providers that support it) are also not yet handled.

It is always the contents of the directory that is synced, not the directory itself. So when source:path is a directory, it's the contents of source:path that are copied, not the directory name and contents. See extended explanation in the copy command if unsure.

If dest:path doesn't exist, it is created and the source:path contents go there.

It is not possible to sync overlapping remotes. However, you may exclude the destination from the sync with a filter rule or by putting an exclude-if-present file inside the destination directory and sync to a destination that is inside the source directory.

Rclone will sync the modification times of files and directories if the backend supports it. If metadata syncing is required then use the --metadata flag.

Note that the modification time and metadata for the root directory will not be synced. See https://github.com/rclone/rclone/issues/7652 for more info.

Note: Use the -P/--progress flag to view real-time transfer statistics

Note: Use the rclone dedupe command to deal with "Duplicate object/directory found in source/destination - ignoring" errors. See this forum post for more info.

Logger Flags

The --differ, --missing-on-dst, --missing-on-src, --match and --error flags write paths, one per line, to the file name (or stdout if it is -) supplied. What they write is described in the help below. For example --differ will write all paths which are present on both the source and destination but different.

The --combined flag will write a file (or stdout) which contains all file paths with a symbol and then a space and then the path to tell you what happened to it. These are reminiscent of diff files.

= path means path was found in source and destination and was identical
- path means path was missing on the source, so only in the destination
+ path means path was missing on the destination, so only in the source
* path means path was present in source and destination but different.
! path means there was an error reading or hashing the source or dest.

The --dest-after flag writes a list file using the same format flags as lsf (including customizable options for hash, modtime, etc.) Conceptually it is similar to rsync's --itemize-changes, but not identical -- it should output an accurate list of what will be on the destination after the sync.

Note that these logger flags have a few limitations, and certain scenarios are not currently supported:

--max-duration / CutoffModeHard
--compare-dest / --copy-dest
server-side moves of an entire dir at once
High-level retries, because there would be duplicates (use --retries 1 to disable)
Possibly some unusual error scenarios

Note also that each file is logged during the sync, as opposed to after, so it is most useful as a predictor of what SHOULD happen to each file (which may or may not match what actually DID.)

rclone sync source:path dest:path [flags]

Options

      --absolute                Put a leading / in front of path names
      --combined string         Make a combined report of changes to this file
      --create-empty-src-dirs   Create empty source dirs on destination after sync
      --csv                     Output in CSV format
      --dest-after string       Report all files that exist on the dest post-sync
      --differ string           Report all non-matching files to this file
  -d, --dir-slash               Append a slash to directory names (default true)
      --dirs-only               Only list directories
      --error string            Report all files with errors (hashing or reading) to this file
      --files-only              Only list files (default true)
  -F, --format string           Output format - see lsf help for details (default "p")
      --hash h                  Use this hash when h is used in the format MD5|SHA-1|DropboxHash (default "md5")
  -h, --help                    help for sync
      --match string            Report all matching files to this file
      --missing-on-dst string   Report all files missing from the destination to this file
      --missing-on-src string   Report all files missing from the source to this file
  -s, --separator string        Separator for the items in the format (default ";")
  -t, --timeformat string       Specify a custom time format, or 'max' for max precision supported by remote (default: 2006-01-02 15:04:05)

Options shared with other commands are described next. See the global flags page for global options not listed here.

Copy Options

Flags for anything which can copy a file

      --check-first                                 Do all the checks before starting transfers
  -c, --checksum                                    Check for changes with size & checksum (if available, or fallback to size only)
      --compare-dest stringArray                    Include additional server-side paths during comparison
      --copy-dest stringArray                       Implies --compare-dest but also copies files from paths into destination
      --cutoff-mode HARD|SOFT|CAUTIOUS              Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default HARD)
      --ignore-case-sync                            Ignore case when synchronizing
      --ignore-checksum                             Skip post copy check of checksums
      --ignore-existing                             Skip all files that exist on destination
      --ignore-size                                 Ignore size when skipping use modtime or checksum
  -I, --ignore-times                                Don't skip items that match size and time - transfer all unconditionally
      --immutable                                   Do not modify files, fail if existing files have been modified
      --inplace                                     Download directly to destination file instead of atomic download to temp/rename
      --max-backlog int                             Maximum number of objects in sync or check backlog (default 10000)
      --max-duration Duration                       Maximum duration rclone will transfer data for (default 0s)
      --max-transfer SizeSuffix                     Maximum size of data to transfer (default off)
  -M, --metadata                                    If set, preserve metadata when copying objects
      --modify-window Duration                      Max time diff to be considered the same (default 1ns)
      --multi-thread-chunk-size SizeSuffix          Chunk size for multi-thread downloads / uploads, if not set by filesystem (default 64Mi)
      --multi-thread-cutoff SizeSuffix              Use multi-thread downloads for files above this size (default 256Mi)
      --multi-thread-streams int                    Number of streams to use for multi-thread downloads (default 4)
      --multi-thread-write-buffer-size SizeSuffix   In memory buffer size for writing when in multi-thread mode (default 128Ki)
      --no-check-dest                               Don't check the destination, copy regardless
      --no-traverse                                 Don't traverse destination file system on copy
      --no-update-dir-modtime                       Don't update directory modification times
      --no-update-modtime                           Don't update destination modtime if files identical
      --order-by string                             Instructions on how to order the transfers, e.g. 'size,descending'
      --partial-suffix string                       Add partial-suffix to temporary file name when --inplace is not used (default ".partial")
      --refresh-times                               Refresh the modtime of remote files
      --server-side-across-configs                  Allow server-side operations (e.g. copy) to work across different configs
      --size-only                                   Skip based on size only, not modtime or checksum
      --streaming-upload-cutoff SizeSuffix          Cutoff for switching to chunked upload if file size is unknown, upload starts after reaching cutoff or when file ends (default 100Ki)
  -u, --update                                      Skip files that are newer on the destination

Sync Options

Flags used for sync commands

      --backup-dir string               Make backups into hierarchy based in DIR
      --delete-after                    When synchronizing, delete files on destination after transferring (default)
      --delete-before                   When synchronizing, delete files on destination before transferring
      --delete-during                   When synchronizing, delete files during transfer
      --fix-case                        Force rename of case insensitive dest to match source
      --ignore-errors                   Delete even if there are I/O errors
      --max-delete int                  When synchronizing, limit the number of deletes (default -1)
      --max-delete-size SizeSuffix      When synchronizing, limit the total size of deletes (default off)
      --suffix string                   Suffix to add to changed files
      --suffix-keep-extension           Preserve the extension when using --suffix
      --track-renames                   When synchronizing, track file renames and do a server-side move if possible
      --track-renames-strategy string   Strategies to use when synchronizing using track-renames hash|modtime|leaf (default "hash")

Important Options

Important flags useful for most commands

  -n, --dry-run         Do a trial run with no permanent changes
  -i, --interactive     Enable interactive mode
  -v, --verbose count   Print lots more stuff (repeat for more)

Flags for filtering directory listings

      --delete-excluded                     Delete files on dest excluded from sync
      --exclude stringArray                 Exclude files matching pattern
      --exclude-from stringArray            Read file exclude patterns from file (use - to read from stdin)
      --exclude-if-present stringArray      Exclude directories if filename is present
      --files-from stringArray              Read list of source-file names from file (use - to read from stdin)
      --files-from-raw stringArray          Read list of source-file names from file without any processing of lines (use - to read from stdin)
  -f, --filter stringArray                  Add a file filtering rule
      --filter-from stringArray             Read file filtering patterns from a file (use - to read from stdin)
      --ignore-case                         Ignore case in filters (case insensitive)
      --include stringArray                 Include files matching pattern
      --include-from stringArray            Read file include patterns from file (use - to read from stdin)
      --max-age Duration                    Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off)
      --max-depth int                       If set limits the recursion depth to this (default -1)
      --max-size SizeSuffix                 Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off)
      --metadata-exclude stringArray        Exclude metadatas matching pattern
      --metadata-exclude-from stringArray   Read metadata exclude patterns from file (use - to read from stdin)
      --metadata-filter stringArray         Add a metadata filtering rule
      --metadata-filter-from stringArray    Read metadata filtering patterns from a file (use - to read from stdin)
      --metadata-include stringArray        Include metadatas matching pattern
      --metadata-include-from stringArray   Read metadata include patterns from file (use - to read from stdin)
      --min-age Duration                    Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off)
      --min-size SizeSuffix                 Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off)

Listing Options

Flags for listing directories

      --default-time Time   Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z)
      --fast-list           Use recursive list if available; uses more memory but fewer transactions