A powerful command line utility to walk an Amazon S3 hierarchy. Think of it as the find command but specifically designed for Amazon S3.
Github Release page provides ready-to-use binaries for:
- Windows (x86_64)
- Linux (x86_64 and ARM)
- macOS (x86_64 and ARM)
Binaries for both architectures allow you to run s3find natively on Intel-based and ARM-based machines (like Apple M1/M2/M3/M4, AWS Graviton, and Raspberry Pi).
Requirements: Rust and Cargo
# Build
cargo build --release
# Install from local source
cargo install --path .
# Install latest from git
cargo install --git https://github.com/AnderEnder/s3find-rs
# Install from crates.io
cargo install s3find# Fast unit and module tests
cargo test --lib -- --nocapture
# Full default test suite (without opt-in LocalStack integration tests)
cargo test --all -- --nocapture
# LocalStack integration tests (opt-in; requires Docker or Podman socket)
cargo test --features localstack-tests --test localstack_integration -- --nocapture
# Lint
cargo clippy --all-targets --all-features -- -D warnings- Listing failures and subcommand failures return a non-zero exit code instead of succeeding with partial output.
downloadskips existing destination files and continues with the remaining objects unless--forceis set.- With tag filters enabled, result order stays aligned with S3 traversal order, and
--limitapplies to that ordered match set. --summarizeprints normal summary statistics and, when tag filters are used, tag fetch statistics includingexcludedobjects.
s3find [OPTIONS] <s3-path> [COMMAND]Where:
<s3-path>is formatted ass3://bucket/path[OPTIONS]are filters and controls[COMMAND]is the action to perform on matched objects
s3find supports multiple AWS authentication methods in the following priority:
- Command-line credentials (
--aws-access-keyand--aws-secret-key) - Environment variables (
AWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEY) - AWS profile credentials file (configurable via
AWS_PROFILEandAWS_SHARED_CREDENTIALS_FILE) - AWS instance IAM profile
- AWS container IAM profile
s3find supports self-hosted and third-party S3-compatible services such as MinIO, Ceph, and others. Use the following options to connect to these services:
--endpoint-url <URL>- Custom S3 endpoint URL--force-path-style- Use path-style bucket addressing (required for most non-AWS S3 services)
s3find 's3://mybucket/path' \
--endpoint-url 'http://localhost:9000' \
--force-path-style \
--aws-access-key 'minioadmin' \
--aws-secret-key 'minioadmin' \
--aws-region 'us-east-1' \
lss3find 's3://mybucket/data' \
--endpoint-url 'https://ceph.example.com' \
--force-path-style \
--aws-access-key 'your-access-key' \
--aws-secret-key 'your-secret-key' \
--name '*.log' \
printUse s3find --help for the top-level reference and s3find <path> <command> --help for subcommand syntax. Commonly used commands:
s3find --help
s3find s3://example-bucket/path print --help
s3find s3://example-bucket/path copy --helpUse the --name option with glob patterns to match objects:
# Find all objects in a path
s3find 's3://example-bucket/example-path' --name '*' ls
# Find objects with specific extension
s3find 's3://example-bucket/example-path' --name '*.json' lsThe print command supports different output formats:
# Default format
s3find 's3://example-bucket/example-path' --name '*' print
# Text format
s3find 's3://example-bucket/example-path' --name '*' print --format text
# JSON format
s3find 's3://example-bucket/example-path' --name '*' print --format json
# CSV format
s3find 's3://example-bucket/example-path' --name '*' print --format csvUse --iname for case-insensitive glob pattern matching:
s3find 's3://example-bucket/example-path' --iname '*data*' lsUse --regex for regular expression pattern matching:
# Find objects ending with a number
s3find 's3://example-bucket/example-path' --regex '[0-9]$' print# Exact match - files exactly 0 bytes
s3find 's3://example-bucket/example-path' --bytes-size 0 print
# Strictly larger than 10 megabytes
s3find 's3://example-bucket/example-path' --bytes-size +10M print
# Strictly smaller than 10 kilobytes
s3find 's3://example-bucket/example-path' --bytes-size -10k print# Files modified in the last 10 seconds
s3find 's3://example-bucket/example-path' --mtime -10s print
# Files modified more than 10 minutes ago
s3find 's3://example-bucket/example-path' --mtime +10m print
# Files modified in the last 10 hours
s3find 's3://example-bucket/example-path' --mtime -10h printFilter objects by their storage class:
# Find objects in STANDARD storage class
s3find 's3://example-bucket/example-path' --storage-class STANDARD print
# Find objects in GLACIER storage class
s3find 's3://example-bucket/example-path' --storage-class GLACIER printFilter objects based on their S3 tags using --tag for key-value matching or --tag-exists for key presence:
# Find objects with a specific tag key and value
s3find 's3://example-bucket/example-path' --tag 'environment=production' ls
# Find objects with multiple tags (AND logic - all must match)
s3find 's3://example-bucket/example-path' --tag 'environment=production' --tag 'team=data' ls
# Find objects that have a specific tag key (any value)
s3find 's3://example-bucket/example-path' --tag-exists 'owner' ls
# Combine tag filters with other filters
s3find 's3://example-bucket/example-path' --name '*.log' --tag 'retention=long-term' ls
# Control concurrency for tag fetching (default: 50)
s3find 's3://example-bucket/example-path' --tag 'env=prod' --tag-concurrency 100 lsRequired IAM Permissions:
Tag filtering requires the s3:GetObjectTagging permission on the objects being searched.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObjectTagging"
],
"Resource": [
"arn:aws:s3:::example-bucket",
"arn:aws:s3:::example-bucket/*"
]
}
]
}Combine filters to create more specific queries:
# Files between 10 and 20 bytes
s3find 's3://example-bucket/example-path' --bytes-size +10 --bytes-size -20 print
# Combine different filter types
s3find 's3://example-bucket/example-path' --bytes-size +10M --name '*backup*' prints3find 's3://example-bucket/example-path' --name '*.tmp' delete# List objects
s3find 's3://example-bucket/example-path' --name '*' ls
# List objects with their tags
s3find 's3://example-bucket/example-path' --name '*' lstagss3find 's3://example-bucket/example-path' --name '*' exec --utility 'echo {}'s3find 's3://example-bucket/example-path' --name '*.pdf' download ./downloads# Copy files to another location
s3find 's3://example-bucket/example-path' --name '*.dat' copy 's3://example-bucket/example-path2'
# Move files to another location
s3find 's3://example-bucket/example-path' --name '*.dat' move 's3://example-bucket/example-path2'
# Flatten keys while copying
s3find 's3://example-bucket/example-path' --name '*.dat' copy --flat 's3://example-bucket/example-path2's3find 's3://example-bucket/example-path' --name '*archive*' tags 'key:value' 'env:staging's3find 's3://example-bucket/example-path' --name '*.public.html' publicControl the number of results and request behavior:
# Limit to first 10 matching objects
s3find 's3://example-bucket/example-path' --name '*' --limit 10 ls
# Control page size for S3 API requests
s3find 's3://example-bucket/example-path' --name '*' --number 100 lsLimit how deep s3find descends into the object hierarchy:
# Only objects at the bucket root level (no subdirectories)
s3find 's3://example-bucket/' --maxdepth 0 ls
# Objects up to one subdirectory level deep
s3find 's3://example-bucket/' --maxdepth 1 ls
# Objects up to two levels deep
s3find 's3://example-bucket/data/' --maxdepth 2 lsThe --maxdepth option uses S3's delimiter-based traversal for efficient server-side filtering, avoiding the need to fetch objects beyond the specified depth.
List all versions of objects in versioned buckets:
# List all versions of all objects
s3find 's3://example-bucket/' --all-versions ls
# List all versions matching a pattern
s3find 's3://example-bucket/' --all-versions --name '*.log' ls
# Print all versions in JSON format
s3find 's3://example-bucket/' --all-versions print --format jsonWhen --all-versions is enabled:
- Uses the S3 ListObjectVersions API instead of ListObjectsV2
- Shows all versions of each object, not just the current version
- Includes delete markers (shown with size 0)
- Each entry displays the version ID alongside the key (e.g.,
file.txt?versionId=abc123); the underlying S3 key remains unchanged - The latest version is marked with
(latest) - Delete markers are marked with
(delete marker)
Version-aware operations: When using --all-versions, operations work on specific versions:
delete- Deletes specific object versions (not just the current version)copy/move- Copies or moves specific versions to the destinationdownload- Downloads specific versions of objects
Note: Delete markers are automatically skipped for operations that don't support them (copy, move, download, tags, restore, change-storage, public) since they have no content.
Note: --all-versions is not compatible with --maxdepth. If both are specified, --all-versions takes precedence and --maxdepth is ignored.
Tag filtering requires an individual GetObjectTagging API call for each object that passes the other filters. To optimize performance and cost:
-
Apply other filters first: Use
--name,--mtime,--bytes-size, or--storage-classfilters to reduce the number of objects before tag filtering is applied. -
Use
--limit: When testing or exploring, use--limitto cap the number of objects processed. -
Adjust concurrency: Use
--tag-concurrencyto tune parallel API calls (default: 50). Higher values increase throughput but may cause throttling. -
Cost awareness:
GetObjectTaggingrequests incur S3 API charges that vary by region. For large buckets with millions of objects, apply filters to reduce the number of tag fetch operations. Refer to the AWS S3 pricing documentation for current rates.
Example: Optimized tag filtering
# Bad: Fetches tags for ALL objects (expensive for large buckets)
s3find 's3://large-bucket/' --tag 'env=prod' ls
# Good: Apply cheap filters first, then tag filter
s3find 's3://large-bucket/' --name '*.log' --mtime -7d --tag 'env=prod' ls
# Good: Use limit when exploring
s3find 's3://large-bucket/' --tag 'env=prod' --limit 100 lsWhen --summarize is enabled, tag fetch statistics are displayed showing success, failed, throttled, access denied, and excluded counts.
For more information, see the GitHub repository.