litestream/docs/PROVIDER_COMPATIBILITY.md

Storage Provider Compatibility Guide

This document details S3-compatible storage provider compatibility with Litestream, including known limitations, required configuration, and tested configurations.

Overview

Litestream uses the AWS SDK v2 for S3-compatible storage backends. While most providers implement the S3 API, there are important differences in behavior that can affect Litestream's operation.

Provider-Specific Configuration

AWS S3 (Default)

Status: Fully supported (primary target)

replicas:
  - url: s3://bucket-name/path
    region: us-east-1

Notes:

• No special configuration required
• All features fully supported
• Checksum validation enabled by default

Cloudflare R2

Status: Supported with default configuration

Known Limitations:

• Strict concurrent upload limit (2-3 concurrent uploads max)
• Does not support aws-chunked content encoding
• Does not support request/response checksums

Configuration:

replicas:
  - url: s3://bucket-name/path?endpoint=https://ACCOUNT_ID.r2.cloudflarestorage.com
    access-key-id: your-access-key-id
    secret-access-key: your-secret-access-key

Automatic Defaults (applied when R2 endpoint detected):

• concurrency=2 - Limits concurrent multipart upload parts
• Checksums disabled automatically
• Force path style enabled

Important: The endpoint must use https:// scheme for R2 detection to work.

Related issues: #948, #947, #940, #941

Backblaze B2 (S3-Compatible API)

Status: Supported with configuration

Known Limitations:

• Requires signed payloads for all requests
• Specific authentication endpoint required

Configuration:

replicas:
  - url: s3://bucket-name/path?endpoint=https://s3.REGION.backblazeb2.com&sign-payload=true&force-path-style=true
    access-key-id: your-key-id
    secret-access-key: your-application-key

Required Settings:

• sign-payload=true - Required for B2 authentication
• force-path-style=true - Required for bucket access
• Endpoint format: https://s3.REGION.backblazeb2.com

Related issues: #918, #894

DigitalOcean Spaces

Status: Supported with configuration

Known Limitations:

• Does not support aws-chunked content encoding
• Signature requirements differ from AWS

Configuration:

replicas:
  - url: s3://bucket-name/path?endpoint=https://REGION.digitaloceanspaces.com&force-path-style=false
    access-key-id: your-spaces-key
    secret-access-key: your-spaces-secret

Notes:

• Use virtual-hosted style paths (force-path-style=false)
• Checksum features disabled automatically for custom endpoints

Related issues: #943

MinIO

Status: Fully supported

Configuration:

replicas:
  - url: s3://bucket-name/path?endpoint=https://your-minio-server:9000&force-path-style=true
    access-key-id: your-access-key
    secret-access-key: your-secret-key

Notes:

• Works well with default settings
• Force path style recommended for single-server deployments

Scaleway Object Storage

Status: Supported with configuration

Known Limitations:

• MissingContentLength errors with streaming uploads
• Requires Content-Length header

Configuration:

replicas:
  - url: s3://bucket-name/path?endpoint=https://s3.REGION.scw.cloud&force-path-style=true
    access-key-id: your-access-key
    secret-access-key: your-secret-key

Related issues: #912

Hetzner Object Storage

Status: Supported with configuration

Known Limitations:

• InvalidArgument errors with default AWS SDK settings
• Does not support aws-chunked content encoding

Configuration:

replicas:
  - url: s3://bucket-name/path?endpoint=https://REGION.your-objectstorage.com&force-path-style=true
    access-key-id: your-access-key
    secret-access-key: your-secret-key

Filebase

Status: Supported with configuration

Known Limitations:

• Authentication failures with default SDK settings after SDK v2 migration

Configuration:

replicas:
  - url: s3://bucket-name/path?endpoint=https://s3.filebase.com&force-path-style=true
    access-key-id: your-access-key
    secret-access-key: your-secret-key

Tigris

Status: Supported with configuration

Configuration:

replicas:
  - url: s3://bucket-name/path?endpoint=https://fly.storage.tigris.dev&force-path-style=true
    access-key-id: your-access-key
    secret-access-key: your-secret-key

Wasabi

Status: Supported

Configuration:

replicas:
  - url: s3://bucket-name/path?endpoint=https://s3.REGION.wasabisys.com
    access-key-id: your-access-key
    secret-access-key: your-secret-key

Google Cloud Storage (GCS)

Status: Fully supported (native client)

replicas:
  - url: gcs://bucket-name/path

Authentication:

• Uses Application Default Credentials
• Set GOOGLE_APPLICATION_CREDENTIALS environment variable
• Or use workload identity on GCP

Azure Blob Storage (ABS)

Status: Fully supported (native client)

replicas:
  - url: abs://container-name/path
    account-name: your-account-name
    account-key: your-account-key

Alternative Authentication:

• Connection string: AZURE_STORAGE_CONNECTION_STRING
• Managed identity on Azure

Alibaba Cloud OSS

Status: Supported (native client)

replicas:
  - url: oss://bucket-name/path?endpoint=oss-REGION.aliyuncs.com
    access-key-id: your-access-key-id
    access-key-secret: your-access-key-secret

SFTP

Status: Supported

replicas:
  - url: sftp://hostname/path
    user: username
    password: password  # or use key-path

Configuration Reference

S3 Query Parameters

Parameter	Description	Default
endpoint	Custom S3 endpoint URL	AWS S3
region	AWS region	Auto-detected
force-path-style	Use path-style URLs	false (auto for custom endpoints)
sign-payload	Sign request payloads	false
skip-verify	Skip TLS verification	false
concurrency	Multipart upload concurrency	5 (2 for R2)
part-size	Multipart upload part size	5MB
sse-algorithm	Server-side encryption	None
sse-kms-key-id	KMS key for encryption	None
sse-customer-key	Customer-provided encryption key	None

Provider Detection

Litestream automatically detects certain providers and applies appropriate defaults:

Provider	Detection Pattern	Applied Settings
Cloudflare R2	*.r2.cloudflarestorage.com	concurrency=2, checksums disabled
Backblaze B2	*.backblazeb2.com	sign-payload=true, force-path-style=true
DigitalOcean	*.digitaloceanspaces.com	force-path-style=false
Scaleway	*.scw.cloud	force-path-style=true
Filebase	s3.filebase.com	force-path-style=true
Tigris	*.tigris.dev	force-path-style=true
MinIO	minio in hostname	force-path-style=true

Troubleshooting

Common Errors

InvalidArgument: Unsupported content encoding: aws-chunked

• Provider doesn't support AWS SDK v2 chunked encoding
• Use a custom endpoint with automatic checksum disabling
• Or explicitly disable checksums

SignatureDoesNotMatch

• Try sign-payload=true in the URL
• Verify credentials are correct
• Check endpoint URL format

MissingContentLength

• Provider requires Content-Length header
• This is handled automatically for known providers

Too many concurrent uploads or timeout errors

• Reduce concurrency: ?concurrency=2
• Particularly important for Cloudflare R2

AccessDenied or authentication failures

• Verify credentials
• Check IAM/bucket permissions
• For B2, ensure sign-payload=true

Debug Mode

Enable verbose logging to diagnose issues:

LITESTREAM_DEBUG=1 litestream replicate ...

Or in configuration:

logging:
  level: debug

Testing Your Configuration

Test connectivity without starting replication:

# List any existing backups
litestream snapshots s3://bucket/path?endpoint=...

# Perform a test restore (requires existing backup)
litestream restore -o /tmp/test.db s3://bucket/path?endpoint=...

Version Compatibility

• Litestream v0.5.x: AWS SDK v2, improved provider compatibility
• Litestream v0.4.x: AWS SDK v1, different authentication handling
• Litestream v0.3.x: Legacy format, not compatible with v0.5.x restores

When upgrading from v0.3.x, be aware that v0.5.x uses a different backup format and cannot restore backups created by v0.3.x. See the upgrade guide for migration instructions.

Reporting Issues

When reporting provider compatibility issues, please include:

1. Provider name and region
2. Litestream version (litestream version)
3. Full error message
4. Configuration (with credentials redacted)
5. Whether the issue is with replication, restore, or both

File issues at: GitHub Issues