Skip to main content
AWS Command Line Reference

AWS CLI S3 Sync — Complete CLI Reference, Syntax and Examples

6 min read

AWS CLI s3 sync recursively copies new and updated files from a source to a destination, comparing size and last-modified timestamps. It is used for one-way synchronization between local directories and S3 buckets or between two S3 buckets.

aws s3 sync <source> <destination> [--delete] [--dryrun] [--exclude "<value>"] [--include "<value>"] [--grants ...] [--sse <value>] [--source-region <value>] [--region <value>]

Syntax

aws s3 sync <source> <destination> [--delete] [--dryrun] [--quiet]
  [--exclude "<value>"] [--include "<value>"] [--acl <value>]
  [--follow-symlinks | --no-follow-symlinks] [--no-guess-mime-type]
  [--sse <value>] [--sse-c <value>] [--sse-c-key <value>]
  [--sse-kms-key-id <value>] [--grants ...] [--cache-control <value>]
  [--content-disposition <value>] [--content-encoding <value>]
  [--content-language <value>] [--content-type <value>]
  [--expires <value>] [--metadata KeyName1=string,KeyName2=string]
  [--storage-class <value>] [--website-redirect-location <value>]
  [--source-region <value>] [--region <value>]
  [--endpoint-url <value>] [--cli-connect-timeout <value>]
  [--cli-read-timeout <value>]

Tested on Amazon Linux 2 with AWS CLI v2.34.60.

Options and Flags

Flag Type Default Description
--delete Boolean false Delete files in destination that are not present in source.
--dryrun Boolean false Display operations that would be performed, without executing them.
--exclude String (none) Exclude files matching pattern (e.g., "*.jpg").
--include String “*” (implicit) Include files matching pattern after an exclusion.
--quiet Boolean false Suppress all non‑error output.
--acl String (bucket policy) Canned ACL: private, public-read, aws-exec-read, etc.
--grants String (none) Grant specific permissions: Permission=Grantee_Type=Grantee_ID.
--sse String (none) Server-side encryption: AES256 or aws:kms.
--sse-kms-key-id String (KMS default) Customer master key (CMK) ID for SSE-KMS.
--cache-control String (none) Cache‑Control header for uploaded objects.
--metadata Map (none) Metadata key-value pairs (Key1=val1,Key2=val2).
--storage-class String STANDARD Storage class: STANDARD_IA, INTELLIGENT_TIERING, GLACIER, etc.
--source-region String (same as dest) Region of the source bucket (required for cross‑region sync).
--region String (from config) Region of the destination bucket.
See also  AWS CLI iam list-roles: Syntax, Flags, Examples, and FAQ

Usage Examples

# Sync local directory to S3, deleting remote files not present locally
aws s3 sync . s3://amzn-s3-demo-bucket --delete --exclude "*.map" --cache-control "max-age=3600"

Deploys a static website. Files with .map extension are skipped; object metadata sets a one‑hour cache TTL. The --delete flag removes obsolete objects from the bucket, mirroring local state exactly.

# Dry-run cross-region bucket sync without making changes
aws s3 sync s3://my-us-west-2-bucket s3://my-us-east-1-bucket --source-region us-west-2 --region us-east-1 --dryrun --quiet

Validates the operation before execution. Useful for backup workflows where you want to see which objects will be transferred (and possibly deleted) before committing the sync.

# Sync all objects except Logs/, keep KMS encryption
aws s3 sync s3://amzn-s3-demo-bucket s3://amzn-s3-demo-bucket2 --exclude "Logs/*" --sse aws:kms --sse-kms-key-id alias/replica-key --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers

Replicates a bucket while excluding the Logs/ prefix. Uses SSE-KMS with a specific CMK and makes replicated objects publicly readable (via grant).

# Sync using access point ARN
aws s3 sync . s3://arn:aws:s3:us-west-2:123456789012:accesspoint/myaccesspoint/

Demonstrates syncing to an S3 access point instead of a bucket ARN.

# Sync from S3 to local with exclude pattern
aws s3 sync s3://amzn-s3-demo-bucket/ . --exclude "*another/*"

Syncs all files from bucket prefix to current directory, skipping files under another/.

Troubleshooting & Common Errors

Error Message Root Cause Resolution Command
fatal error: An error occurred (AccessDenied) when calling the ListObjectsV2 operation IAM user lacks s3:ListBucket permission on the source bucket. aws s3 ls s3://source-bucket --no-sign-request (test public access) or attach policy:
{"Effect":"Allow","Action":"s3:ListBucket","Resource":"arn:aws:s3:::source-bucket"}
fatal error: An error occurred (SignatureDoesNotMatch) when calling the PutObject operation Clock skew > 5 minutes between client and AWS API. sudo ntpdate pool.ntp.org or sudo chronyc -a makestep
fatal error: An error occurred (BucketRegionError) when calling the HeadBucket operation Source region not specified and default region differs. Add --source-region <actual-region> to sync command.
fatal error: An error occurred (InvalidArgument) when calling the PutObject operation: KMS key not found SSE‑KMS key ID/alias does not exist in the destination region. aws kms describe-key --key-id alias/replica-key --region us-east-1 to verify.
See also  aws ec2 describe-instances: Syntax, Filters & Troubleshooting

Performance Considerations

The aws s3 sync command sequentially lists source objects and transfers each file one at a time, which can limit throughput on large transfers. Tuning focuses on network timeouts and operating‑system buffers, as the CLI does not expose built‑in parallelism or batch‑size flags for sync.

  • Timeouts — The flags --cli-connect-timeout and --cli-read-timeout control the time (in seconds) the CLI waits for a TCP connection or a read operation. Defaults are 60 and 60 seconds respectively. For high‑latency or lossy links, increase these values to reduce premature failures. Example: aws s3 sync . s3://my-bucket --cli-connect-timeout 120 --cli-read-timeout 180.
  • TCP and OS buffering — The AWS CLI relies on the kernel’s TCP stack. Tune kernel parameters such as net.core.rmem_default, net.core.wmem_default, and net.ipv4.tcp_rmem/tcp_wmem (see Linux kernel documentation). Larger buffers help on high‑bandwidth‑delay‑product links. MTU should be set at the network interface level (e.g., ip link set mtu 9000 for jumbo frames).
  • Parallelism — aws s3 sync itself does not offer parallelism flags. To increase concurrency, split the source directory manually and run multiple sync processes, or use tools like s5cmd (vendor: Peak6). The AWS CLI’s max_concurrent_requests setting (configurable in ~/.aws/config under [default] s3 = max_concurrent_requests = 20) applies to cp and mv but is not used by sync — sync runs operations sequentially.
# Increase timeouts for sync over high-latency connections
aws s3 sync ./local s3://my-bucket --cli-connect-timeout 120 --cli-read-timeout 180

Frequently Asked Questions

What is the difference between --delete and --exact-timestamps flags in aws s3 sync?

The --delete flag removes destination files that do not exist in the source. The --exact-timestamps flag forces comparison by modification time in addition to size (by default, sync compares both size and last-modified time). Without --exact-timestamps, sync uses size and modification time; with it, timestamps must match exactly. Combine both to replicate source exactly, including deletions and timestamps.

aws s3 sync s3://src-bucket/ s3://dst-bucket/ --delete --exact-timestamps

When should I use the --exclude flag with aws s3 sync?

Use --exclude to skip specific files or patterns during sync. The flag accepts glob patterns; multiple excludes can be chained. Use --include after --exclude to re-include subsets. Example: skip all files except .txt files:

aws s3 sync . s3://mybucket/ --exclude "*" --include "*.txt"

How do I fix “upload failed: … Access Denied” error during aws s3 sync?

Verify IAM permissions: s3:ListBucket and s3:GetObject on source, s3:PutObject on destination. This error is common with cross-account sync or restrictive bucket policies. Use aws sts get-caller-identity to confirm credentials. Example policy validation:

aws s3api put-bucket-policy --bucket dest-bucket --policy file://policy.json

Does aws s3 sync work with S3-compatible storage like MinIO?

Yes, using --endpoint-url. Specify endpoint and region explicitly. Example sync to MinIO:

aws s3 sync /data s3://mybucket/ --endpoint-url http://minio:9000 --region us-east-1

Ensure credentials are configured for the third-party service.

See also  aws ec2 describe-instances: Syntax, Filters & Troubleshooting

What is the fastest way to sync a large directory to S3 with aws s3 sync?

Because sync runs sequentially, consider splitting the directory and running multiple sync processes in parallel. Increase timeouts for large files. Disable output with --quiet and exclude unnecessary files. Example optimized command:

aws s3 sync /data s3://mybucket/ --quiet --exclude "*.tmp"

For files larger than 5 GB, multipart upload is used automatically.