Skip to content

Proof of Concept: Use IO.copy_stream to serve files#2703

Closed
casperisfine wants to merge 1 commit intopuma:masterfrom
casperisfine:io-copy-stream
Closed

Proof of Concept: Use IO.copy_stream to serve files#2703
casperisfine wants to merge 1 commit intopuma:masterfrom
casperisfine:io-copy-stream

Conversation

@casperisfine
Copy link
Copy Markdown
Contributor

Ref: https://puma/puma/issues/2697

$ benchmarks/wrk/big_response.sh
Puma starting in single mode...
* Puma version: 5.5.0 (ruby 3.0.2-p107) ("Zawgyi")
*  Min threads: 4
*  Max threads: 4
*  Environment: development
*          PID: 17879
* Listening on http://0.0.0.0:9292
Use Ctrl-C to stop
Running 1m test @ http://localhost:9292
  2 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.37ms    5.89ms  48.28ms   94.46%
    Req/Sec     0.88k   148.97     1.07k    82.08%
  Latency Distribution
     50%    2.21ms
     75%    2.78ms
     90%    4.09ms
     99%   35.75ms
  105651 requests in 1.00m, 108.24GB read
Requests/sec:   1758.39
Transfer/sec:      1.80GB
- Gracefully stopping, waiting for requests to finish
$ benchmarks/wrk/big_file.sh
Puma starting in single mode...
* Puma version: 5.5.0 (ruby 3.0.2-p107) ("Zawgyi")
*  Min threads: 4
*  Max threads: 4
*  Environment: development
*          PID: 18034
* Listening on http://0.0.0.0:9292
Use Ctrl-C to stop
Running 1m test @ http://localhost:9292
  2 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.06ms    1.09ms  20.98ms   97.94%
    Req/Sec     1.85k   150.69     2.03k    89.92%
  Latency Distribution
     50%    0.94ms
     75%    1.03ms
     90%    1.21ms
     99%    4.91ms
  221380 requests in 1.00m, 226.81GB read
Requests/sec:   3689.18
Transfer/sec:      3.78GB
- Gracefully stopping, waiting for requests to finish

Description

This is just a quick and dirty proof of concept, it would need a bit more work to properly handle ranges and such.

@MSP-Greg
Copy link
Copy Markdown
Member

@casperisfine

Thanks for the PR.

  1. Not sure about the changes to ext/puma_http11/http11_parser.c, is that just ragel changes? Maybe another PR?
  2. request.rb has the line if !chunked && content_length && res_body.is_a?(::File), maybe res_body.is_a?(::IO)?

Ref: https://puma/issues/2697

```
$ benchmarks/wrk/big_response.sh
Puma starting in single mode...
* Puma version: 5.5.0 (ruby 3.0.2-p107) ("Zawgyi")
*  Min threads: 4
*  Max threads: 4
*  Environment: development
*          PID: 17879
* Listening on http://0.0.0.0:9292
Use Ctrl-C to stop
Running 1m test @ http://localhost:9292
  2 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.37ms    5.89ms  48.28ms   94.46%
    Req/Sec     0.88k   148.97     1.07k    82.08%
  Latency Distribution
     50%    2.21ms
     75%    2.78ms
     90%    4.09ms
     99%   35.75ms
  105651 requests in 1.00m, 108.24GB read
Requests/sec:   1758.39
Transfer/sec:      1.80GB
- Gracefully stopping, waiting for requests to finish
```

```
$ benchmarks/wrk/big_file.sh
Puma starting in single mode...
* Puma version: 5.5.0 (ruby 3.0.2-p107) ("Zawgyi")
*  Min threads: 4
*  Max threads: 4
*  Environment: development
*          PID: 18034
* Listening on http://0.0.0.0:9292
Use Ctrl-C to stop
Running 1m test @ http://localhost:9292
  2 threads and 4 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.06ms    1.09ms  20.98ms   97.94%
    Req/Sec     1.85k   150.69     2.03k    89.92%
  Latency Distribution
     50%    0.94ms
     75%    1.03ms
     90%    1.21ms
     99%    4.91ms
  221380 requests in 1.00m, 226.81GB read
Requests/sec:   3689.18
Transfer/sec:      3.78GB
- Gracefully stopping, waiting for requests to finish
```
@casperisfine
Copy link
Copy Markdown
Contributor Author

Thanks for the PR.

Welcome. I'll see if I find the time to turn it in a production ready one at some point, but mostly wanted to showcase the potential of IO.copy_stream in the context of https://puma/puma/issues/2697.

Not sure about the changes to ext/puma_http11/http11_parser.c, is that just ragel changes? Maybe another PR?

No clue either, I must admit I didn't pay too much attention to the diff (because PoC). I removed it now.

request.rb has the line if !chunked && content_length && res_body.is_a?(::File), maybe res_body.is_a?(::IO)?

Yes, it could. Now in many cases IO.copy_stream might not be able to use an optimized mechanism if res_body is not a true file (e.g. with sendfile(2) the source fd can't be a socket), but worst case it will fallback to doing read(write()) in C, which will save in allocations etc, so will always be faster anyway.

@nateberkopec
Copy link
Copy Markdown
Member

Fixed by #2923

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature waiting-for-changes Waiting on changes from the requestor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants