Skip to content

Add S3 Express Append benchmark#405

Merged
harshavardhana merged 5 commits intominio:masterfrom
klauspost:add-express-append
Jul 6, 2025
Merged

Add S3 Express Append benchmark#405
harshavardhana merged 5 commits intominio:masterfrom
klauspost:add-express-append

Conversation

@klauspost
Copy link
Collaborator

@klauspost klauspost commented Jun 26, 2025 •
edited
Loading

Each thread uploads one object and appends to it.

With master AiStor:

image

@klauspost
Add S3 Express Append benchmark
ff79e14 
Each thread uploads one object and appends to it.
@klauspost klauspost requested a review from harshavardhana June 26, 2025 16:17
@klauspost
Copy link
Collaborator Author

klauspost commented Jul 1, 2025

APPEND (S3 Express)

Benchmarks S3 Express One Zone Append Object operations.

WARP will upload --obj.size objects for each --concurrent and append up to 10,000 parts to these.
Each append operation will be one part and the size of each part will be --part.size - a new object will be created when the part limit is reached.

If no --checksum is specified, the CRC64NVME checksum will be used. The checksum type must support full object checksums (CRC32, CRC32C, CRC64NVME).

Example:

λ warp append -duration=1m -obj.size=1MB
╭─────────────────────────────────╮
│ WARP S3 Benchmark Tool by MinIO │
╰─────────────────────────────────╯

Benchmarking: Press 'q' to abort benchmark and print partial results...

 λ ████████████████████████████████████████████████████████████████████████░  99%

Reqs: 4997, Errs:0, Objs:4997, Bytes: 4765.5MiB
 -    APPEND Average: 84 Obj/s, 80.4MiB/s; Current 88 Obj/s, 84.4MiB/s, 280.7 ms/req


Report: APPEND. Concurrency: 20. Ran: 58s
 * Average: 80.15 MiB/s, 84.04 obj/s
 * Reqs: Avg: 234.6ms, 50%: 203.9ms, 90%: 354.1ms, 99%: 711.3ms, Fastest: 58.3ms, Slowest: 1213.9ms, StdDev: 109.5ms

Throughput, split into 58 x 1s:
 * Fastest: 123.8MiB/s, 129.80 obj/s
 * 50% Median: 80.1MiB/s, 83.97 obj/s
 * Slowest: 23.6MiB/s, 24.74 obj/s

The "obj/s" indicates the number of append operations per second.

@harshavardhana harshavardhana merged commit 87e1d1d into minio:master Jul 6, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@harshavardhana harshavardhana harshavardhana approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@klauspost @harshavardhana