Use new request parser by anuragsoni · Pull Request #819 · mirage/ocaml-cohttp

anuragsoni · 2022-01-07T04:06:43Z

Hello,

I'll preface this by saying that this is far from being ready, and I've had to comment/break a lot of things to get a minimal server example working.

Background

I've been working on a HTTP request (and chunked body) parser and prototype for a HTTP 1.1 server and I decided to use the Cohttp types as the base layer for my explorations. The server experiments in shuttle have shown promise, but instead of adding yet another server library I'd like to explore if it's possible to integrate some of that work directly into Cohttp. I'm opening this really early in the process to see if an approach similar to what I'm proposing here will fit well with the Cohttp maintainer's vision (and to hear if there are suggestions on improving the IO layer even further).

I am starting with cohttp-lwt-unix first as that's the cohttp backend I use at work, and I wanted to try my work on a non async backend to validate that the work in shuttle can be ported to non async libraries as well. All of the new code in this diff is a direct port of implementations from the shuttle library.

I am using the test payload referenced in #328 to measure progress.

Disclaimer: All of these test runs were made on my laptop, and if someone has a better hardware setup for such benchmarking then I'd appreciate getting some more measurements from there.

Library Versions: Cohttp (the version in the current trunk as of January 6 2022), LWT (5.5.0), OCaml (4.13.1)

I ran an initial set of tests using wrk2 using 1000 active connections and two different request rates (80_000, and 100_000). The results from the runs on my laptop can be seen at https://gist.github.com/anuragsoni/eb095a7f4c3aec1f4ba65b9866d9f9d3 . The summary from that gist is:

Cohttp_lwt_unix from the current trunk had trouble getting more than 60,000 requests per second and had average latencies around 4-7 seconds with tail latencies > 10 seconds at times
Cohttp_lwt_unix with the new request parser and still using Lwt_io via conduit was able to comfortable serve 80_000 RPS with average latency around 2ms and tail latencies around 7ms. When attempting to server 100_000 RPS the average latencies degraded to 1.2 seconds with tail latencies of 2 seconds
As an experiment I tried to run cohttp-lwt-unix server that doesn't use conduit + Lwt_io for the read path, and instead just uses Lwt_unix.file_descr paired with the buffer that's part of this PR. With that the server was able to handle 100_000. RPS with average latencies of 2ms with tail latencies of 7.6ms - link here
Httpaf_lwt_unix has been used as the benchmark here and it can easily server 100_000 rps with an average latency of 2.5ms with a tail of 9ms

Apologies for wall of text, but I wanted to start a discussion even if I only have a partially working solution so far, as I'd like to get some insights into whether the proposed implementation here makes sense as a starting point, and I'm also hoping to learn how best to test/benchmark such work so we can have an accurate picture of any potential improvements.

Thanks,
Anurag

rgrinberg · 2022-01-07T04:22:41Z

As an experiment I tried to run cohttp-lwt-unix server that doesn't use conduit + Lwt_io for the read path, and instead just uses Lwt_unix.file_descr paired with the buffer that's part of this PR. With that the server was able to handle 100_000. RPS with average latencies of 2ms with tail latencies of 7.6ms - link here

If I understand correctly, the slow down is because Lwt_io allocates an additional bigarray?

cohttp/src/s.ml

rgrinberg · 2022-01-07T04:23:53Z

Httpaf_lwt_unix has been used as the benchmark here and it can easily server 100_000 rps with an average latency of 2.5ms with a tail of 9ms

So what was the limit for Httpaf here for reference? In other words, how much further can it push past 100k with decent latency

anuragsoni · 2022-01-07T04:25:00Z

http-parser/src/http_parser.mli

+  ?pos:int -> ?len:int -> string -> (Http.Request.t * int, error) result
+
+val parse_chunk_length :
+  ?pos:int -> ?len:int -> string -> (int * int, error) result


I have used int for Chunk lengths as the initial work that led to this parser made an assumption that we'll always run on 64bit systems. With that assumption the int would be enough to represent the largest chunk size that's allowed by the parser.

anuragsoni · 2022-01-07T04:25:50Z

http-parser/src/http_parser.mli

+val parse_chunk :
+  ?pos:int ->
+  ?len:int ->
+  string ->
+  chunk_kind ->
+  (chunk_parser_result * int, error) result


My intention was to not have to use parse_chunk_length manually, and instead go via the parse_chunk which helps orchestrate threading through the chunk body state

I'll also note that I don't make use of any of the chunked parsing code in this PR and I'm limiting this diff to just the request (upto the headers) parsing for now

anuragsoni · 2022-01-07T04:27:25Z

cohttp-lwt-unix/src/input_channel.ml

@@ -0,0 +1,24 @@
+open Lwt.Infix
+
+type t = { buf : Bytebuffer.t; chan : Lwt_io.input_channel }


Adding a buffer layer alongside Lwt_io as it helps in driving the core IO layer that needs access to the internal view within the buffer to feed data to the new parser.

Before this is considered ready, we might want a way to grow the buffer too I suppose (with an upper bound beyond which its considered an error maybe?)

Could you expand on this a bit more. When does the buffer need to grow?

I don't think it needs to grow as such as long we pick a value that we feel is reasonable to parse the request + headers. But I noticed that the body decoding implementation in cohttp references that it'd like to read up to a max of 32k bytes at a time when reading chunked bodies

ocaml-cohttp/cohttp/src/transfer_io.ml

Line 52 in efe0599

(* read between 0 and 32Kbytes of a chunk *)

So I figured if we want to start with a smaller buffer than that, we'll need to grow it if we still want to eventually read 32k bytes when we get to bodies. Or we could just pick a large enough buffer to start with and never touch it again.

32K sounds like a reasonable maximum. I suppose we should determine the default buffer size experimentally and let the user customize it if need be.

For reference, I went back through the history to find the PR that first introduced the limit: #330 Looks like at that stage it was arbitrary. I think it would be nice to make it configurable, something similar to Lwt_io.set_default_buffer_size io_buf

rgrinberg · 2022-01-07T04:28:58Z

Is the intention to make http-parser a public API? If that's the case, a package is appropriate. But if it's not, we need to find a new place for it to live (either in http or cohttp).

anuragsoni · 2022-01-07T04:31:55Z

So what was the limit for Httpaf here for reference? In other words, how much further can it push past 100k with decent latency

On my laptop its able to go up to 125k with decent latency. With a run of 150k rps it degraded into average latency 2.4 seconds

anuragsoni · 2022-01-07T04:43:54Z

Is the intention to make http-parser a public API? If that's the case, a package is appropriate. But if it's not, we need to find a new place for it to live (either in http or cohttp).

I am not fully sure about this. I think I'd be fine with both a public package, or something that lives within either http or cohttp.

anuragsoni · 2022-01-07T05:09:09Z

examples/bench/lwt_unix_server.ml

@@ -0,0 +1,43 @@
+open Cohttp_lwt_unix
+
+let text =


This is the lwt server I'm using to benchmark cohttp

rgrinberg · 2022-01-07T05:45:17Z

I am not fully sure about this. I think I'd be fine with both a public package, or something that lives within either http or cohttp.

Do you think the current API is stable? If not, then we should probably include it in Http under a Private module. Once it stabilizes, we can promote it into a separate package, library, etc.

rgrinberg · 2022-01-07T06:08:34Z

cohttp-async/src/io.ml

        return ())

+  let refill _ = failwith "Not implemented yet"
+  let with_input_buffer _ = failwith "not implemented yet"


I'm thinking about how to implement this, and I think it would actually make things easier if Byte_buffer was Bigstring based or at least functorized on the buffer type. Reason being is that I don't see how to implement these functions in an efficient manner for Async's Reader for example. I'd imagine we want to use read_one_chunk_at_a_time to implement with_input_buffer. I understand how bigstrings don't are a wash performance wise, but it helps keep compatibility with existing implementations. Perhaps we could functorize Byte_buffer to make it possible to continue experimenting in Shuttle/Fiber without normal Bytes.

I agree but if we want to keep it simple we could check how big of a performace hit would be to just use bigstring

Moving to bigstrings will also mean moving the parser to bigstrings. Otherwise we pay the price of allocating more strings?

I was thinking of implementing the async backend in a similar manner as the lwt where there is a byte buffer paired with the reader.

That said, now that I think about it there is going to be some breaking changes in the sense that people using the expert response in servers will need to adapt to the switch to the new input type in the IO backends.

Moving to bigstrings will also mean moving the parser to bigstrings. Otherwise we pay the price of allocating more strings?

Yes but that is very localized to the Source module, no? In any case it was just a curiosity, and as you mentioned you also already have an idea to make this a non-issue

That said, now that I think about it there is going to be some breaking changes in the sense that people using the expert response in servers will need to adapt to the switch to the new input type in the IO backends.

That is the last of the problems. We are allowed to break things in cohttp now, as long as we can justify it. Imho the speedup and latency numbers shown here would justify the breakage

string is the way to go in OCaml 5.0, since it's both non-moving and allocated via malloc (all large ocaml objects are mallocated in multicore)

We can probably deal with this in another manner by pushing the buffer to the core cohttp library, and then the refill becomes a way for the IO layers to fill the buffer managed by cohttp. That will let us keep the IO types for the IO layers the same as they are today, while still benefiting from the buffering we need to drive the parser.

I'm not a huge fan of moving a buffer to the core library as that removes one benefit for libraries like shuttle that could otherwise drive the whole cohttp server machinery with just the 1 buffer that's used by its channel implementation.

That will let us keep the IO types for the IO layers the same as they are today, while still benefiting from the buffering we need to drive the parser.

Note that we don't necessarily need to retain the same IO types. For example, if we could just create Reader.t/Writer.t on the fly when the user provides Expert that would be good enough. Though I'm not sure if it's at all possible.

I'm not a huge fan of moving a buffer to the core library as that removes one benefit for libraries like shuttle that could otherwise drive the whole cohttp server machinery with just the 1 buffer that's used by its channel implementation.

Yeah, it's not worth it. If we must break the current api and make Reader/Writer impossible to use, then so be it.

if we could just create Reader.t/Writer.t on the fly when the user provides Expert that would be good enough.

This should be doable. There will be some awkwardness for sure, but we can definitely get something working. We only need to recreate the reader since I haven't switched the writer in the existing implementation. pseudo code for how I'd do this in async

let reader_of_in_channel t = let unconsumed_payload = Bytebuffer.unconsumed_data t.buf in let reader_pipe = Reader.pipe t.reader in Unix.pipe ~info:(Info.create ...) >>| fun (`Reader rd, `Writer wr) -> let new_reader = Reader.create rd in let writer = Writer.create wr in Writer.write writer unconsumed_payload; don't_wait_for ( Writer.transfer writer reader_pipe ~stop:(Reader.close_finished new_reader) (fun s -> Writer.write writer s) >>= fun () -> Writer.close writer ); new_reader

anuragsoni · 2022-01-07T16:49:31Z

cohttp-async/src/bytebuffer.ml

@@ -0,0 +1,87 @@
+open Core


I need to think about how to remove some of this duplication as this is very similar to the module proposed for the lwt backend

What are the differences preventing you from making this generic? It seems like you just need a function over a monad + reader.

no difference. This can be moved to something that's instantiated using the IO functor (like we do for other modules like request/response)

anuragsoni · 2022-01-07T16:50:13Z

I pushed an async update as well. But unlike the lwt backend I see no difference in perf between the old implementation and the new implementation 😞 I'll spend more time on this to try and understand what's happening.

anuragsoni · 2022-01-07T18:20:42Z

I reworked the async server loop and now I can see improvement when compared to the existing cohttp implementation. With the older cohttp-async server when I attempt to target 80_000 RPS with 1000 active clients I have an average latency of 8 seconds with tails of 12 seconds. With the new implementation the average latency drops to 2.3 seconds with tails of 3.5 seconds. This is still quite a way behind the lwt implementation.

That said, I think on the async backend the current limiting factor might be the conduit setup. I wrote another async backend for cohttp using shuttle https://github.com/anuragsoni/shuttle/blob/b296ac77f590735ea56d94ef19ae503251dcd317/shuttle_cohttp/shuttle_cohttp.ml and with that backend the average latency when serving 80_000 RPS is 6.9ms with tails of 16ms . These numbers are closer to what the new lwt backend gets here.

All of the async tests were run on macOS where it uses the select backend (the lwt tests were with libev so it probably picks up kqueue on macOS?), and the async backend might perform a little better when run on linux where it has access to epoll.

mseri · 2022-01-07T20:19:53Z

I think you can use the libuv backend now on macos. It is experimental but in my tests it has been working pretty reliably

anuragsoni · 2022-01-07T20:22:31Z

I think you can use the libuv backend now on macos. It is experimental but in my tests it has been working pretty reliably

I will try that! That said the libev backend for lwt seems to be working well so I'm not expecting much difference between that and libuv since the current state of libuv support in lwt is mostly for polling file descriptors for IO readiness only (same as libev). I'd be more interested in seeing test runs of async on a linux system so we can try the epoll backend for it :)

cohttp/test/test_request.ml

examples/bench/async_server.ml

rgrinberg · 2022-01-07T21:02:16Z

http/src/http.ml

+                Source.advance source 1;
+                stop := true)
+              else (
+                stop := true;


I think we can make stop := true outside of the if.

anuragsoni · 2022-01-07T21:09:58Z

@rgrinberg @mseri I should've asked this before I rebased. How would you like me to sync with new changes from the main branch? I'm happy to rebase but if merge commits will make it easier to review new changes I can switch to that for new updates.

rgrinberg · 2022-01-07T21:11:42Z

Rebase as much as you want. Rudi.

…

On Jan 7, 2022, 2:10 PM -0700, Anurag Soni ***@***.***>, wrote: @rgrinberg @mseri I should've asked this before I rebased. How would you like me to sync with new changes from the main branch? I'm happy to rebase but if merge commits will make it easier to review new changes I can switch to that for new updates. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

mseri · 2022-01-07T21:24:32Z

I agree, go wild

mseri · 2022-01-08T11:43:05Z

Is this expected?

(cd _build/default/cohttp-lwt-unix/test && ./test_sanity.exe)
[DEBUG][application]: Cohttp debugging output is active
[ERROR][cohttp.lwt.server]: Unhandled exception: Lwt_io.Channel_closed("input")
.....E.
==============================================================================
Error: Cohttp Server Test:5:expert response

(Failure "Test produced 1 log messages at level >= warn")
------------------------------------------------------------------------------
Ran: 7 tests in: 0.04 seconds.
FAILED: Cases: 7 Tried: 7 Errors: 1 Failures: 0 Skip:  0 Todo: 0 Timeouts: 0.

I think it happens when a consumer reads from a channel that has already been exhausted or that has been closed for other reasons (timeouts?)

mseri · 2022-01-08T11:47:36Z

When expert mode was introduced this was handled by fee808a (See the commentary on #647 (comment))

cohttp-lwt-unix/src/bytebuffer.ml

anuragsoni · 2022-01-08T17:16:13Z

The CI failure on ocaml-ci is because of a test for http parser. I was using int to represent chunk length, but one of the test uses a value that is too large for int when running on 32 bit systems.

rgrinberg · 2022-01-08T20:57:54Z

Do you think it's acceptable to change to int64?

anuragsoni · 2022-01-09T02:10:19Z

Do you think it's acceptable to change to int64?

Yes. Using int64 for chunk length should be fine.

anuragsoni · 2022-01-09T04:28:10Z

I factored out the buffer implementation from cohttp-lwt-unix, and I now have the mirage backend compiling with the new parser implementation. I believe that with this last commit, unless I've missed it, I don't have anything else that's still commented/left unfinished, so I guess this PR can be considered not a draft.

The mirage backend is the piece I'm least confident about as I don't have much experience with it, and I don't see any existing tests for it within the repository.

For the Async and lwt-unix backend I think I have an idea about how we can still keep the same public api by ensuring that the callback needed for the Expert response still uses Reader.t or Lwt_io.t (see comment #819 (comment)), but I'm not sure how to achieve the same for the mirage backend.

* Hand written http request/response parser * Far more efficient IO layer

rgrinberg · 2022-01-09T08:33:51Z

Thanks for rewriting half of cohttp! I'll merge it now and we can solve the remaining issues in separate PR's.

@samoht @dinosaure @hannesm could you have a look/optimize the mirage backend? The mirage should also be able to reap the performance benefits.

anuragsoni · 2022-01-09T17:35:30Z

I did a quick test with cohttp mirage today. I used the macOS backend as I don't have an easy access to a linux machine at the moment.

wrk2 -t2 -c1000 -R 80000 --latency "http://localhost:8080" -d 30s

wrk bench against cohttp-mirage 5.0.0

Initialised 2 threads in 0 ms.
Running 30s test @ http://localhost:8080
  2 threads and 1000 connections
  Thread calibration: mean lat.: 1707.276ms, rate sampling interval: 6586ms
  Thread calibration: mean lat.: 1707.088ms, rate sampling interval: 6586ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.78s     1.79s   11.26s    58.17%
    Req/Sec    25.87k     6.93    25.88k    25.00%
  Latency Distribution (HdrHistogram - Recorded Latency)
 50.000%    6.78s 
 75.000%    8.33s 
 90.000%    9.26s 
 99.000%    9.95s 
 99.900%   10.74s 
 99.990%   11.04s 
 99.999%   11.21s 
100.000%   11.27s 

  Detailed Percentile spectrum:
       Value   Percentile   TotalCount 1/(1-Percentile)

    3583.999     0.000000            1         1.00
    4313.087     0.100000        91013         1.11
    4927.487     0.200000       181366         1.25
    5550.079     0.300000       271885         1.43
    6164.479     0.400000       362114         1.67
    6782.975     0.500000       452952         2.00
    7090.175     0.550000       498130         2.22
    7397.375     0.600000       543526         2.50
    7708.671     0.650000       588484         2.86
    8015.871     0.700000       633877         3.33
    8327.167     0.750000       679194         4.00
    8478.719     0.775000       701774         4.44
    8634.367     0.800000       724806         5.00
    8790.015     0.825000       747399         5.71
    8945.663     0.850000       769649         6.67
    9101.311     0.875000       793039         8.00
    9175.039     0.887500       803666         8.89
    9256.959     0.900000       815683        10.00
    9330.687     0.912500       826238        11.43
    9404.415     0.925000       837298        13.33
    9486.335     0.937500       849214        16.00
    9527.295     0.943750       855257        17.78
    9560.063     0.950000       860116        20.00
    9601.023     0.956250       866107        22.86
    9641.983     0.962500       871998        26.67
    9682.943     0.968750       877757        32.00
    9699.327     0.971875       880092        35.56
    9723.903     0.975000       883610        40.00
    9740.287     0.978125       885882        45.71
    9764.863     0.981250       889004        53.33
    9789.439     0.984375       891641        64.00
    9805.823     0.985938       892816        71.11
    9838.591     0.987500       893844        80.00
    9912.319     0.989062       895295        91.43
    9986.047     0.990625       896624       106.67
   10067.967     0.992188       898031       128.00
   10117.119     0.992969       898780       142.22
   10166.271     0.993750       899456       160.00
   10223.615     0.994531       900185       182.86
   10280.959     0.995313       900847       213.33
   10346.495     0.996094       901569       256.00
   10379.263     0.996484       901925       284.44
   10420.223     0.996875       902305       320.00
   10461.183     0.997266       902621       365.71
   10518.527     0.997656       903012       426.67
   10567.679     0.998047       903342       512.00
   10600.447     0.998242       903542       568.89
   10625.023     0.998437       903676       640.00
   10665.983     0.998633       903878       731.43
   10698.751     0.998828       904043       853.33
   10739.711     0.999023       904213      1024.00
   10764.287     0.999121       904318      1137.78
   10788.863     0.999219       904397      1280.00
   10813.439     0.999316       904487      1462.86
   10838.015     0.999414       904574      1706.67
   10862.591     0.999512       904657      2048.00
   10878.975     0.999561       904711      2275.56
   10895.359     0.999609       904756      2560.00
   10903.551     0.999658       904783      2925.71
   10919.935     0.999707       904823      3413.33
   10944.511     0.999756       904879      4096.00
   10952.703     0.999780       904895      4551.11
   10969.087     0.999805       904929      5120.00
   10977.279     0.999829       904945      5851.43
   10985.471     0.999854       904958      6826.67
   11018.239     0.999878       904981      8192.00
   11034.623     0.999890       904994      9102.22
   11051.007     0.999902       905005     10240.00
   11067.391     0.999915       905013     11702.86
   11083.775     0.999927       905027     13653.33
   11100.159     0.999939       905035     16384.00
   11116.543     0.999945       905042     18204.44
   11124.735     0.999951       905047     20480.00
   11132.927     0.999957       905050     23405.71
   11149.311     0.999963       905056     27306.67
   11165.695     0.999969       905063     32768.00
   11173.887     0.999973       905068     36408.89
   11173.887     0.999976       905068     40960.00
   11182.079     0.999979       905071     46811.43
   11190.271     0.999982       905074     54613.33
   11198.463     0.999985       905077     65536.00
   11198.463     0.999986       905077     72817.78
   11198.463     0.999988       905077     81920.00
   11206.655     0.999989       905079     93622.86
   11214.847     0.999991       905080    109226.67
   11231.231     0.999992       905082    131072.00
   11231.231     0.999993       905082    145635.56
   11239.423     0.999994       905083    163840.00
   11247.615     0.999995       905084    187245.71
   11247.615     0.999995       905084    218453.33
   11255.807     0.999996       905086    262144.00
   11255.807     0.999997       905086    291271.11
   11255.807     0.999997       905086    327680.00
   11255.807     0.999997       905086    374491.43
   11255.807     0.999998       905086    436906.67
   11263.999     0.999998       905087    524288.00
   11263.999     0.999998       905087    582542.22
   11263.999     0.999998       905087    655360.00
   11263.999     0.999999       905087    748982.86
   11263.999     0.999999       905087    873813.33
   11272.191     0.999999       905088   1048576.00
   11272.191     1.000000       905088          inf
#[Mean    =     6784.695, StdDeviation   =     1792.427]
#[Max     =    11264.000, Total count    =       905088]
#[Buckets =           27, SubBuckets     =         2048]
----------------------------------------------------------
  1493513 requests in 30.00s, 2.91GB read
  Socket errors: connect 0, read 4294, write 12, timeout 0
Requests/sec:  49781.00
Transfer/sec:     99.41MB

wrk bench against cohttp-mirage pinned to commit 06d2127 (the new parser)

Initialised 2 threads in 0 ms.
Running 30s test @ http://localhost:8080
  2 threads and 1000 connections
  Thread calibration: mean lat.: 841.765ms, rate sampling interval: 3358ms
  Thread calibration: mean lat.: 841.993ms, rate sampling interval: 3358ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.50s   934.19ms   5.17s    57.79%
    Req/Sec    32.60k    44.57    32.66k    40.00%
  Latency Distribution (HdrHistogram - Recorded Latency)
 50.000%    3.50s 
 75.000%    4.31s 
 90.000%    4.80s 
 99.000%    5.10s 
 99.900%    5.14s 
 99.990%    5.16s 
 99.999%    5.17s 
100.000%    5.17s 

  Detailed Percentile spectrum:
       Value   Percentile   TotalCount 1/(1-Percentile)

    1847.295     0.000000            1         1.00
    2217.983     0.100000       114641         1.11
    2535.423     0.200000       228475         1.25
    2854.911     0.300000       342500         1.43
    3178.495     0.400000       456837         1.67
    3500.031     0.500000       570689         2.00
    3661.823     0.550000       627566         2.22
    3823.615     0.600000       684763         2.50
    3987.455     0.650000       742185         2.86
    4151.295     0.700000       799156         3.33
    4313.087     0.750000       855709         4.00
    4395.007     0.775000       884270         4.44
    4476.927     0.800000       913599         5.00
    4558.847     0.825000       942403         5.71
    4640.767     0.850000       971157         6.67
    4722.687     0.875000       999184         8.00
    4763.647     0.887500      1013301         8.89
    4804.607     0.900000      1027709        10.00
    4845.567     0.912500      1041969        11.43
    4886.527     0.925000      1055969        13.33
    4927.487     0.937500      1070119        16.00
    4947.967     0.943750      1077293        17.78
    4968.447     0.950000      1084506        20.00
    4988.927     0.956250      1091739        22.86
    5009.407     0.962500      1098939        26.67
    5029.887     0.968750      1106206        32.00
    5038.079     0.971875      1109132        35.56
    5050.367     0.975000      1113496        40.00
    5058.559     0.978125      1116363        45.71
    5070.847     0.981250      1120601        53.33
    5079.039     0.984375      1123380        64.00
    5087.231     0.985938      1126084        71.11
    5091.327     0.987500      1127401        80.00
    5095.423     0.989062      1128677        91.43
    5103.615     0.990625      1131101       106.67
    5107.711     0.992188      1132259       128.00
    5111.807     0.992969      1133367       142.22
    5115.903     0.993750      1134435       160.00
    5119.999     0.994531      1135452       182.86
    5124.095     0.995313      1136386       213.33
    5128.191     0.996094      1137288       256.00
    5128.191     0.996484      1137288       284.44
    5132.287     0.996875      1138087       320.00
    5132.287     0.997266      1138087       365.71
    5136.383     0.997656      1138792       426.67
    5136.383     0.998047      1138792       512.00
    5140.479     0.998242      1139407       568.89
    5140.479     0.998437      1139407       640.00
    5140.479     0.998633      1139407       731.43
    5144.575     0.998828      1139916       853.33
    5144.575     0.999023      1139916      1024.00
    5144.575     0.999121      1139916      1137.78
    5148.671     0.999219      1140293      1280.00
    5148.671     0.999316      1140293      1462.86
    5148.671     0.999414      1140293      1706.67
    5152.767     0.999512      1140550      2048.00
    5152.767     0.999561      1140550      2275.56
    5152.767     0.999609      1140550      2560.00
    5152.767     0.999658      1140550      2925.71
    5152.767     0.999707      1140550      3413.33
    5156.863     0.999756      1140712      4096.00
    5156.863     0.999780      1140712      4551.11
    5156.863     0.999805      1140712      5120.00
    5156.863     0.999829      1140712      5851.43
    5156.863     0.999854      1140712      6826.67
    5160.959     0.999878      1140802      8192.00
    5160.959     0.999890      1140802      9102.22
    5160.959     0.999902      1140802     10240.00
    5160.959     0.999915      1140802     11702.86
    5160.959     0.999927      1140802     13653.33
    5165.055     0.999939      1140854     16384.00
    5165.055     0.999945      1140854     18204.44
    5165.055     0.999951      1140854     20480.00
    5165.055     0.999957      1140854     23405.71
    5165.055     0.999963      1140854     27306.67
    5165.055     0.999969      1140854     32768.00
    5165.055     0.999973      1140854     36408.89
    5165.055     0.999976      1140854     40960.00
    5169.151     0.999979      1140877     46811.43
    5169.151     0.999982      1140877     54613.33
    5169.151     0.999985      1140877     65536.00
    5169.151     0.999986      1140877     72817.78
    5169.151     0.999988      1140877     81920.00
    5169.151     0.999989      1140877     93622.86
    5169.151     0.999991      1140877    109226.67
    5169.151     0.999992      1140877    131072.00
    5169.151     0.999993      1140877    145635.56
    5169.151     0.999994      1140877    163840.00
    5169.151     0.999995      1140877    187245.71
    5169.151     0.999995      1140877    218453.33
    5169.151     0.999996      1140877    262144.00
    5169.151     0.999997      1140877    291271.11
    5169.151     0.999997      1140877    327680.00
    5169.151     0.999997      1140877    374491.43
    5169.151     0.999998      1140877    436906.67
    5169.151     0.999998      1140877    524288.00
    5173.247     0.999998      1140879    582542.22
    5173.247     1.000000      1140879          inf
#[Mean    =     3503.824, StdDeviation   =      934.188]
#[Max     =     5169.152, Total count    =      1140879]
#[Buckets =           27, SubBuckets     =         2048]
----------------------------------------------------------
  1890298 requests in 30.00s, 3.69GB read
Requests/sec:  63009.22
Transfer/sec:    125.83MB

* We only have a new parser for requests for now * We only touched the read path with mirage#819

…p-mirage, cohttp-lwt, cohttp-lwt-unix, cohttp-lwt-jsoo, cohttp-eio, cohttp-curl, cohttp-curl-lwt, cohttp-curl-async, cohttp-bench and cohttp-async (6.0.0~alpha0) CHANGES: - cohttp-eio: ensure "Host" header is the first header in http client requests (bikallem mirage/ocaml-cohttp#939) - cohttp-eio: add TE header in client. Check TE header is server (bikallem mirage/ocaml-cohttp#941) - cohttp-eio: add User-Agent header to request from Client (bikallem mirage/ocaml-cohttp#940) - cohttp-eio: add Content-Length header to request/response (bikallem mirage/ocaml-cohttp#929) - cohttp-eio: add cohttp-eio client api - Cohttp_eio.Client (bikallem mirage/ocaml-cohttp#879) - http: add requires_content_length function for requests and responses (bikallem mirage/ocaml-cohttp#879) - cohttp-eio: use Eio.Buf_write and improve server API (talex5 mirage/ocaml-cohttp#887) - cohttp-eio: update to Eio 0.3 (talex5 mirage/ocaml-cohttp#886) - cohttp-eio: convert to Eio.Buf_read (talex5 mirage/ocaml-cohttp#882) - cohttp lwt client: Connection cache and explicit pipelining (madroach mirage/ocaml-cohttp#853) - http: add Http.Request.make and simplify Http.Response.make (bikallem mseri mirage/ocaml-cohttp#878) - http: add pretty printer functions (bikallem mirage/ocaml-cohttp#880) - New eio based client and server on top of the http library (bikallem mirage/ocaml-cohttp#857) - New curl based clients (rgrinberg mirage/ocaml-cohttp#813) + cohttp-curl-lwt for an Lwt backend + cohttp-curl-async for an Async backend - Completely new Parsing layers for servers (anuragsoni mirage/ocaml-cohttp#819) + Cohttp now uses an optimized parser for requests. + The new parser produces much less temporary buffers during read operations in servers. - Faster header comparison (gasche mirage/ocaml-cohttp#818) - Introduce http package containing common signatures and structures useful for compatibility with cohttp - and no dependencies (rgrinberg mirage/ocaml-cohttp#812) - async(server): allow reading number of active connections (anuragsoni mirage/ocaml-cohttp#809) - Various internal refactors (rgrinberg, mseri, mirage/ocaml-cohttp#802, mirage/ocaml-cohttp#812, mirage/ocaml-cohttp#820, mirage/ocaml-cohttp#800, mirage/ocaml-cohttp#799, mirage/ocaml-cohttp#797) - http (all cohttp server backends): Consider the connection header in response in addition to the request when deciding on whether to keep a connection alive (anuragsoni, mirage/ocaml-cohttp#843) + The user provided Response can contain a connection header. That header will also be considered in addition to the connection header in requests when deciding whether to use keep-alive. This allows a handler to decide to close a connection even if the client requested a keep-alive in the request. - async(server): allow creating a server without using conduit (anuragsoni mirage/ocaml-cohttp#839) + Add `Cohttp_async.Server.Expert.create` and `Cohttp_async.Server.Expert.create_with_response_action`that can be used to create a server without going through Conduit. This allows creating an async TCP server using the Tcp module from `Async_unix` and lets the user have more control over how the `Reader.t` and `Writer.t` are created. - http(header): faster `to_lines` and `to_frames` implementation (mseri mirage/ocaml-cohttp#847) - cohttp(cookies): use case-insensitive comparison to check for `set-cookies` (mseri mirage/ocaml-cohttp#858) - New lwt based server implementation: cohttp-server-lwt-unix + This new implementation does not depend on conduit and has a simpler and more flexible API - async: Adapt cohttp-curl-async to work with core_unix. - *Breaking changes* + refactor: move opam metadata to dune-project (rgrinberg mirage/ocaml-cohttp#811) + refactor: deprecate Cohttp_async.Io (rgrinberg mirage/ocaml-cohttp#807) + fix: move more internals to Private (rgrinberg mirage/ocaml-cohttp#806) + fix: deprecate transfer encoding field (rgrinberg mirage/ocaml-cohttp#805) + refactor: deprecate Cohttp_async.Body_raw (rgrinberg mirage/ocaml-cohttp#804) + fix: deprecate more aliases (rgrinberg mirage/ocaml-cohttp#803) + refactor: deprecate connection value(rgrinberg mirage/ocaml-cohttp#798) + refactor: deprecate using attributes (rgrinberg mirage/ocaml-cohttp#796) + cleanup: remove cohttp-{curl,server}-async (rgrinberg mirage/ocaml-cohttp#904) + cleanup: remove cohttp-{curl,server,proxy}-lwt (rgrinberg mirage/ocaml-cohttp#904) + fix: all parsers now follow the spec and require `\r\n` endings. Previously, the `\r` was optional. (rgrinberg, mirage/ocaml-cohttp#921) - `cohttp-lwt-jsoo`: do not instantiate `XMLHttpRequest` object on boot (mefyl mirage/ocaml-cohttp#922)

@art-w

CHANGES: - cohttp-lwt-unix: Add http/https proxy support for client requests (@art-w @MisterDA, mirage/ocaml-cohttp#1080) - cohttp-mirage: Support conduit 8.0.0 (@hannesm, mirage/ocaml-cohttp#1104) ## v6.0.0 (2024-11-21) - bump minimum dune version to 3.8 (@avsm) - cohttp-eio: Use system authenticator in example. - http, cohttp: remove the scheme field from requests. This means that [Request.uri] no longer returns the same URI as was to create the request with [Request.make] (@rgrinberg 1086) - cohttp-eio: Remove unused `Client_intf` module (talex5 mirage/ocaml-cohttp#1081) - cohttp-eio: Make server response type abstract and allow streaming in cohttp-eio (talex5 mirage/ocaml-cohttp#1024) - cohttp-{lwt,eio}: server: add connection header to response if not present (ushitora-anqou mirage/ocaml-cohttp#1025) - cohttp-curl: Curl no longer prepends the first HTTP request header to the output. (jonahbeckford mirage/ocaml-cohttp#1030, mirage/ocaml-cohttp#987) - cohttp-eio: client: use permissive argument type for make_generic - cohttp-eio: Improve error handling in example server (talex5 mirage/ocaml-cohttp#1023) - cohttp-eio: Don't blow up `Server.callback` on client disconnections. (mefyl mirage/ocaml-cohttp#1015) - http: Fix assertion in `Source.to_string_trim` when `pos <> 0` (mefyl mirage/ocaml-cohttp#1017) - cohttp: `Cohttp.Request.make_for_client` no longer allows setting both `~chunked:true` and `~body_length`. - cohttp-lwt-unix: Don't blow up when certificates are not available and no-network requests are made. (akuhlens mirage/ocaml-cohttp#1027) + Makes `cohttp-lwt.S.default_ctx` lazy. ## v6.0.0~beta2 (2024-01-05) - cohttp-eio: Don't blow up `Server.run` on client disconnections. (mefyl mirage/ocaml-cohttp#1011) - cohttp-eio: Match body encoding with headers. (mefyl mirage/ocaml-cohttp#1012) - cohttp-lwt: Preserve extended `Server.S.IO` signature. (mefyl mirage/ocaml-cohttp#1013) ## v6.0.0~beta1 (2023-10-27) - cohttp-eio: move new Cohttp.{Client,Server} modules under Cohttp.Generic (mseri mirage/ocaml-cohttp#1003) - cohttp-eio: Add Client.make_generic and HTTPS support. (talex5 mirage/ocaml-cohttp#1002) - cohttp: move generic client and server signatures to cohttp and use them across all packages. (mefyl mirage/ocaml-cohttp#984) - cohttp-eio: Complete rewrite to follow common interfaces and behaviors. (mefyl mirage/ocaml-cohttp#984) ## v6.0.0~alpha2 (2023-08-08) - cohttp-lwt: Do not leak exceptions to `Lwt.async_exception_hook`. (mefyl mirage/ocaml-cohttp#992, mirage/ocaml-cohttp#995) - http.header, cohttp, cohttp-eio: remove "first" and "move_to_first" and the special treatment of the "host" header (mseri mirage/ocaml-cohttp#988, mirage/ocaml-cohttp#986) - http.header: introduce "iter_ord" to guarantee iteration following the order of the entries in the headers (mseri mirage/ocaml-cohttp#986) - do not omit mandatory null Content-Length headers (mefyl mirage/ocaml-cohttp#985) - cohttp-async, cohttp-curl-async: compatibility with core/async v0.16.0 (mseri, dkalinichenko-js mirage/ocaml-cohttp#976) - cohttp-lwt server: call conn_closed before drainig the body of response on error (pirbo mirage/ocaml-cohttp#982) - cohttp-eio: Relax socket interface requirement on `Server.connection_handler`. (mefyl mirage/ocaml-cohttp#983) ## v6.0.0~alpha1 (2023-04-28) - cohttp,cohttp-async server: correctly close broken streams (reported by Stéphane Glondu, fix by samhot and anuragsoni) - cohttp-eio: remove unused code from tests to work with Eio 0.8 (talex5 mirage/ocaml-cohttp#967) - Upgrade dune to v3.0 (bikallem mirage/ocaml-cohttp#947) - cohttp-eio: allow client to optionally configure request pipelining (bikallem mirage/ocaml-cohttp#949) - cohttp-eio: update to Eio 0.7 (talex5 mirage/ocaml-cohttp#952) - cohttp-eio: update examples to use eio 0.7 primitives (bikallem mirage/ocaml-cohttp#957) - cohttp-eio: generate Date header in responses (bikallem mirage/ocaml-cohttp#955) - cohttp-eio: further improve Cohttp_eio.Client ergonomics (bikallem #?) - cohttp-eio: server api improvements (bikallem mirage/ocaml-cohttp#962) ## v6.0.0~alpha0 (2022-10-24) - cohttp-eio: ensure "Host" header is the first header in http client requests (bikallem mirage/ocaml-cohttp#939) - cohttp-eio: add TE header in client. Check TE header is server (bikallem mirage/ocaml-cohttp#941) - cohttp-eio: add User-Agent header to request from Client (bikallem mirage/ocaml-cohttp#940) - cohttp-eio: add Content-Length header to request/response (bikallem mirage/ocaml-cohttp#929) - cohttp-eio: add cohttp-eio client api - Cohttp_eio.Client (bikallem mirage/ocaml-cohttp#879) - http: add requires_content_length function for requests and responses (bikallem mirage/ocaml-cohttp#879) - cohttp-eio: use Eio.Buf_write and improve server API (talex5 mirage/ocaml-cohttp#887) - cohttp-eio: update to Eio 0.3 (talex5 mirage/ocaml-cohttp#886) - cohttp-eio: convert to Eio.Buf_read (talex5 mirage/ocaml-cohttp#882) - cohttp lwt client: Connection cache and explicit pipelining (madroach mirage/ocaml-cohttp#853) - http: add Http.Request.make and simplify Http.Response.make (bikallem mseri mirage/ocaml-cohttp#878) - http: add pretty printer functions (bikallem mirage/ocaml-cohttp#880) - New eio based client and server on top of the http library (bikallem mirage/ocaml-cohttp#857) - New curl based clients (rgrinberg mirage/ocaml-cohttp#813) + cohttp-curl-lwt for an Lwt backend + cohttp-curl-async for an Async backend - Completely new Parsing layers for servers (anuragsoni mirage/ocaml-cohttp#819) + Cohttp now uses an optimized parser for requests. + The new parser produces much less temporary buffers during read operations in servers. - Faster header comparison (gasche mirage/ocaml-cohttp#818) - Introduce http package containing common signatures and structures useful for compatibility with cohttp - and no dependencies (rgrinberg mirage/ocaml-cohttp#812) - async(server): allow reading number of active connections (anuragsoni mirage/ocaml-cohttp#809) - Various internal refactors (rgrinberg, mseri, mirage/ocaml-cohttp#802, mirage/ocaml-cohttp#812, mirage/ocaml-cohttp#820, mirage/ocaml-cohttp#800, mirage/ocaml-cohttp#799, mirage/ocaml-cohttp#797) - http (all cohttp server backends): Consider the connection header in response in addition to the request when deciding on whether to keep a connection alive (anuragsoni, mirage/ocaml-cohttp#843) + The user provided Response can contain a connection header. That header will also be considered in addition to the connection header in requests when deciding whether to use keep-alive. This allows a handler to decide to close a connection even if the client requested a keep-alive in the request. - async(server): allow creating a server without using conduit (anuragsoni mirage/ocaml-cohttp#839) + Add `Cohttp_async.Server.Expert.create` and `Cohttp_async.Server.Expert.create_with_response_action`that can be used to create a server without going through Conduit. This allows creating an async TCP server using the Tcp module from `Async_unix` and lets the user have more control over how the `Reader.t` and `Writer.t` are created. - http(header): faster `to_lines` and `to_frames` implementation (mseri mirage/ocaml-cohttp#847) - cohttp(cookies): use case-insensitive comparison to check for `set-cookies` (mseri mirage/ocaml-cohttp#858) - New lwt based server implementation: cohttp-server-lwt-unix + This new implementation does not depend on conduit and has a simpler and more flexible API - async: Adapt cohttp-curl-async to work with core_unix. - *Breaking changes* + refactor: move opam metadata to dune-project (rgrinberg mirage/ocaml-cohttp#811) + refactor: deprecate Cohttp_async.Io (rgrinberg mirage/ocaml-cohttp#807) + fix: move more internals to Private (rgrinberg mirage/ocaml-cohttp#806) + fix: deprecate transfer encoding field (rgrinberg mirage/ocaml-cohttp#805) + refactor: deprecate Cohttp_async.Body_raw (rgrinberg mirage/ocaml-cohttp#804) + fix: deprecate more aliases (rgrinberg mirage/ocaml-cohttp#803) + refactor: deprecate connection value(rgrinberg mirage/ocaml-cohttp#798) + refactor: deprecate using attributes (rgrinberg mirage/ocaml-cohttp#796) + cleanup: remove cohttp-{curl,server}-async (rgrinberg mirage/ocaml-cohttp#904) + cleanup: remove cohttp-{curl,server,proxy}-lwt (rgrinberg mirage/ocaml-cohttp#904) + fix: all parsers now follow the spec and require `\r\n` endings. Previously, the `\r` was optional. (rgrinberg, mirage/ocaml-cohttp#921) - `cohttp-lwt-jsoo`: do not instantiate `XMLHttpRequest` object on boot (mefyl mirage/ocaml-cohttp#922) ## v5.0.0 (2021-12-15) - Cohttp.Header: new implementation (lyrm mirage/ocaml-cohttp#747) + New implementation of Header modules using an associative list instead of a map, with one major semantic change (function ```get```, see below), and some new functions (```clean_dup```, ```get_multi_concat```) + More Alcotest tests as well as fuzzing tests for this particular module. ### Purpose The new header implementation uses an associative list instead of a map to represent headers and is focused on predictability and intuitivity: except for some specific and documented functions, the headers are always kept in transmission order, which makes debugging easier and is also important for [RFC7230§3.2.2](https://tools.ietf.org/html/rfc7230#section-3.2.2) that states that multiple values of a header must be kept in order. Also, to get an intuitive function behaviour, no extra work to enforce RFCs is done by the basic functions. For example, RFC7230§3.2.2 requires that a sender does not send multiple values for a non list-value header. This particular rule could require the ```Header.add``` function to remove previous values of non-list-value headers, which means some changes of the headers would be out of control of the user. With the current implementation, an user has to actively call dedicated functions to enforce such RFCs (here ```Header.clean_dup```). ### Semantic changes Two functions have a semantic change : ```get``` and ```update```. #### get ```get``` was previously doing more than just returns the value associated to a key; it was also checking if the searched header could have multiple values: if not, the last value associated to the header was returned; otherwise, all the associated values were concatenated and returned. This semantics does not match the global idea behind the new header implementation, and would also be very inefficient. + The new ```get``` function only returns the last value associated to the searched header. + ```get_multi_concat``` function has been added to get a result similar to the previous ```get``` function. #### update ```update``` is a pretty new function (mirage/ocaml-cohttp#703) and changes are minor and related to ```get``` semantic changes. + ```update h k f``` is now modifying only the last occurrences of the header ```k``` instead of all its occurrences. + a new function ```update_all``` function has been added and work on all the occurrences of the updated header. ### New functions : + ```clean_dup``` enables the user to clean headers that follows the {{:https://tools.ietf.org/html/rfc7230#section-3.2.2} RFC7230§3.2.2} (no duplicate, except ```set-cookie```) + ```get_multi_concat``` has been added to get a result similar to the previous ```get``` function. - Cohttp.Header: performance improvement (mseri, anuragsoni mirage/ocaml-cohttp#778) **Breaking** the headers are no-longer lowercased when parsed, the headers key comparison is case insensitive instead. - cohttp-lwt-unix: Adopt ocaml-conduit 5.0.0 (smorimoto mirage/ocaml-cohttp#787) **Breaking** `Conduit_lwt_unix.connect`'s `ctx` param type changed from `ctx` to `ctx Lazy.t` - cohttp-mirage: fix deprecated fmt usage (tmcgilchrist mirage/ocaml-cohttp#783) - lwt_jsoo: Use logs for the warnings and document it (mseri mirage/ocaml-cohttp#776) - lwt: Use logs to warn users about leaked bodies and document it (mseri mirage/ocaml-cohttp#771) - lwt, lwt_unix: Improve use of logs and the documentation, fix bug in the Debug.enable_debug function (mseri mirage/ocaml-cohttp#772) - lwt_jsoo: Fix exception on connection errors in chrome (mefyl mirage/ocaml-cohttp#761) - lwt_jsoo: Fix `Lwt.wakeup_exn` `Invalid_arg` exception when a js stack overflow happens in the XHR completion handler (mefyl mirage/ocaml-cohttp#762). - lwt_jsoo: Add test suite (mefyl mirage/ocaml-cohttp#764). ## v4.0.0 (2021-03-24) - cohttp.response: fix malformed status header for custom status codes (mseri aalekseyev mirage/ocaml-cohttp#752) - remove dependency to base (samoht mirage/ocaml-cohttp#745) - add GitHub Actions workflow (smorimoto mirage/ocaml-cohttp#739) - `cohttp-lwt-jsoo`: Forward exceptions to caller when response is null (mefyl mirage/ocaml-cohttp#738) - Use implicit executable dependency for generate.exe (TheLortex mirage/ocaml-cohttp#735) - cohttp: fix chunked encoding of empty body (mefyl mirage/ocaml-cohttp#715) - cohttp-async: fix body not being uploaded with unchunked Async.Pipe (mefyl mirage/ocaml-cohttp#706) - cohttp-{async, lwt}: fix surprising behaviours of Body.is_empty (anuragsoni mirage/ocaml-cohttp#714 mirage/ocaml-cohttp#712 mirage/ocaml-cohttp#713) - refactoring of tests (mseri mirage/ocaml-cohttp#709, dinosaure mirage/ocaml-cohttp#692) - update documentation (dinosaure mirage/ocaml-cohttp#716, mseri mirage/ocaml-cohttp#720) - fix deadlock in logging (dinosaure mirage/ocaml-cohttp#722) - improve media type parsing (seliopou mirage/ocaml-cohttp#542, dinosaure mirage/ocaml-cohttp#725) - [reverted] breaking changes to client and server API to use conduit 3.0.0 (dinosaure mirage/ocaml-cohttp#692). However, as the design discussion did not reach consensus, these changes were reverted to preserve better compatibility with existing cohttp users. (samoht mirage/ocaml-cohttp#741) **Potentially breaking changes** - remove `wrapped false` from the codebase (rgrinberg mirage/ocaml-cohttp#734) - cohttp: add Uri.scheme to Request.t (brendanlong mirage/ocaml-cohttp#707) - cohttp: update HTTP codes (emillon mirage/ocaml-cohttp#711) - cohttp-lwt-jsoo: rename Cohttp_lwt_xhr to Cohttp_lwt_jsoo for consistency (mseri mirage/ocaml-cohttp#717) - cohttp: fix transfer-encoding ordering in headers (mseri mirage/ocaml-cohttp#721) - lower-level support for long-running cohttp-async connections (brendanlong mirage/ocaml-cohttp#704) - add of_form and to_form functions to body (seliopou mirage/ocaml-cohttp#440, mseri mirage/ocaml-cohttp#723) - cohttp-lwt: partly inline read_response, fix body stream leak (madroach dinosaure mirage/ocaml-cohttp#696). Note: there is a new warning that may show up in your logs when bodies are leaked, see also [mirage/ocaml-cohttp#730](mirage/ocaml-cohttp#730). - add comparison functions for Request.t and Response.t via ppx_compare (msaffer-js dinosaure mirage/ocaml-cohttp#686) ## v3.0.0 - aborted ## v2.5.5 (2021-03-15) - `Cohttp_async.resolve_local_file`, `Cohttp_lwt.resolve_local_file` and `Cohttp_lwt_unix.resolve_file` are now the same code under the hood (`Cohttp.Path.resolve_local_file`). The old names have been preserved for compatibility, but will be marked as deprecated in the next release. This changes the behavior of `Cohttp_lwt_unix.resolve_file`: it now percent-decodes the paths and blocks escaping from the docroot correctly. This also fixes and tests the corner cases in these methods when the docroot is empty. (ewanmellor mirage/ocaml-cohttp#755) ## v2.5.4 (2020-07-21) - cohttp: a change in mirage/ocaml-cohttp#694 modified the semantics of Header.replace. The semantics change is reverted, and a new Header.update function is introduced, following the semantics of Map.update. (mirage/ocaml-cohttp#702 mseri) - cohttp: reimplement update to support compilers that are older than OCaml 4.06 (mirage/ocaml-cohttp#703 mseri) ## v2.5.3 (2020-06-27) - cohttp-async: adapt to async >= v0.14 (mirage/ocaml-cohttp#699 copy) ## v2.5.2 (2020-06-27) - cohttp, cohttp-async: correctly set host header for unix domain sockets, implement Unix domain socket support for cohttp-async (mirage/ocaml-cohttp#698 Leonidas-from-XIV) - cohttp: better body encoding management when creating request and response, and correction of Header.replace function (mirage/ocaml-cohttp#694 lyrm) ## v2.5.1 (2020-02-18) - cohttp-lwt: pass ctx through HEAD client requests (mirage/ocaml-cohttp#689 hannesm) ## v2.5.0 (2019-12-17) - cohttp-async: support async v0.13.0 (mirage/ocaml-cohttp#680 copy) - cohttp-lwt-jsoo: support js_of_ocaml 3.5.0 and higher (avsm) ## v2.4.0 (2019-11-08) - mirage: adapt to new mirage interfaces: mirage-flow 2.0.0, mirage-channel 4.0.0, mirage-kv 3.0.0 (mirage/ocaml-cohttp#678 hannesm) - async: use Pipe.singleton instead of Pipe.of_list as it is more efficient (mirage/ocaml-cohttp#677 smuenzel-js) ## v2.3.0 (2019-08-18) - use conduit-mirage instead of mirage-conduit, which was renamed upstream in conduit. The minimum OCaml version supported for conduit-mirage is now OCaml 4.07 and higher. (mirage/ocaml-cohttp#672 avsm) - remove deprecation warnings in OCaml 4.08.0 using stdlib-shims (mirage/ocaml-cohttp#672 avsm) - async: do not read body if none is present (mirage/ocaml-cohttp#671 emillon) ## v2.2.0 (2019-07-20) - Previously if the client closed the connection while cohttp was handling a request, the server would crash (by default, unless the user overrode that using `on_exn` or changing Lwt's async exception handler). Now, cohttp will just log this at `info` level and continue. Exceptions produced by user code are logged as errors, while other exceptions generated by cohttp call back to the conduit exception handler, as before. (mirage/ocaml-cohttp#669 talex5) ## v2.1.3 (2019-07-12) - support uri.3.0.0 that has optional sexp support (mirage/ocaml-cohttp#668 avsm) - use re.1.9.0 api to remove deprecation warnings (mirage/ocaml-cohttp#664 vbmithr) ## v2.1.2 (2019-04-09) - cohttp: handle empty cookie components gracefully without raising an exception (mirage/ocaml-cohttp#663 martinslota) ## v2.1.1 (2019-04-05) - `cohttp-mirage`: remove dependency on the `result` module (mirage/ocaml-cohttp#662 hannesm) - Support Async v0.12.0 and higher (mirage/ocaml-cohttp#661 copy) ## v2.1.0 (2019-03-01) - `cohttp-mirage`: update to the newest `Mirage_kv.RO` API ## v2.0.0 (2019-02-04) Compatibility breaking interface changes: Async: Expert response action no longer writes empty HTTP body (mirage/ocaml-cohttp#647 by andreas) In cohttp.0.99, a number of subpackages were turned into explicit opam packages to simplify dependency management. To aid migration, some compatibility shims were left in place so that the old findlib names would continue to work. They have now been removed as of this release. If you were still using them, then please rename them as follows: - `cohttp.lwt-core` -> `cohttp-lwt` - `cohttp.lwt` -> `cohttp-lwt-unix` - `cohttp.js` -> `cohttp-lwt-jsoo` - `cohttp.async` -> `cohttp-async` - `cohttp.top` -> `cohttp-top` Other changes and bugfixes: * Lwt, Mirage: Add log warnings for uncaught exceptions (mirage/ocaml-cohttp#592 by ansiwen) * Log invalid client input and do not catch out of memory exceptions (mirage/ocaml-cohttp#652 hannesm) * Port opam files to opam2 and add local synopsis and descriptions. * Lwt: Add Expert response action for servers (mirage/ocaml-cohttp#647 by andreas) * Use the namespaced `js_of_ocaml` interfaces from 3.3.0 onwards (mirage/ocaml-cohttp#654 avsm) * Use Base64 3.1.0 interfaces (mirage/ocaml-cohttp#655 avsm) * Clean up redundant conflicts in the `opam` files (avsm) ## v1.2.0 (2018-10-19) * Support more than a single chunk extension for RFC7320 compliance (mirage/ocaml-cohttp#618 by djs55) * Lwt-unix: add a `?backlog` argument to the serve function (samoht) * Use the uri.2.0.0 interfaces for sexpression generation of types (avsm) * Switch to `sexplib0` for a more lightweight s-expression library (mseri) * Minimum OCaml compiler version requirement is now 4.04.1 (mseri) * Add an example of using custom resolvers to the README (mseri) ## v1.1.1 (2018-08-13) * Update to be compatible with new async/core (mirage/ocaml-cohttp#607 by rgrinberg) * Remove use of deprecated `Lwt_logs` (mirage/ocaml-cohttp#609 by raphael-proust) * Do not rely on locale for printing qvalues (mirage/ocaml-cohttp#611 by vbmithr) * Ppx dependencies aren't just build time dependencies (mirage/ocaml-cohttp#625 by rgrinberg) ## v1.1.0 (2018-03-28) * Add an "expert mode" to hand off raw responses to a custom handler, which in turns makes protocols like Websockets easier (mirage/ocaml-cohttp#488 by msaffer). * Set the user-agent by default if one is not provided (mirage/ocaml-cohttp#586 by TheCBaH). * Fix typo in the `cohttp.js` META file. * Refresh use of the Re library to the latest version (mirage/ocaml-cohttp#602 by rgrinberg). * Rearrange the ppx dependencies to be more specific (mirage/ocaml-cohttp#596 by yomimono). * Explicitly depend on sexplib in the Async backend (mirage/ocaml-cohttp#605 by kevinqiu). ## v1.0.2 (2018-01-06) * Support Async v0.10.0 and OCaml 4.06.0 (mirage/ocaml-cohttp#588 via vbmithr) * Require `ppx_type_conv`>=v0.9.1` due to a bug with duplicate modules present in earlier versions. ## v1.0.1 (2018-01-03) * cohttp-mirage: expose the missing IO module (mirage/ocaml-cohttp#594, samoht) * cohttp-mirage: catch exceptions when closing channels in mirage client (mirage/ocaml-cohttp#589, ansiwen) ## v1.0.0 (2017-11-16) * opam: rename `mirage-http` to `cohttp`-mirage` (mirage/ocaml-cohttp#572) * cohttp-lwt{,-unix}: wrap the libraries in top-level modules (mirage/ocaml-cohttp#568) * opam: improve dependencies (mirage/ocaml-cohttp#574, mirage/ocaml-cohttp#566, mirage/ocaml-cohttp#575) * cohttp: add the convenience function `Header.is_empty` (mirage/ocaml-cohttp#576) * fix compatibility with OCaml 4.06 and `-safe-string` (mirage/ocaml-cohttp#580, mirage/ocaml-cohttp#581) ## v0.99.0 (2017-07-12) Port build to jbuilder, and break up OPAM packages into multiple independent packages instead of being optional dependencies against the main `cohttp` package. This makes it significantly easier to depend on precisely the libraries you need, but requires porting applications to use the new `ocamlfind` and `opam` scheme. The new package layout is: - `cohttp`: the main `Cohttp` module - `cohttp-lwt`: the portable Lwt implementation - `cohttp-lwt-unix`: the Lwt/Unix implementation - `cohttp-lwt-jsoo`: the js-of-ocaml JavaScript implementation - `cohttp-async`: the Jane Street Async implementation - `mirage-http`: the MirageOS compatible implementation - `cohttp-top`: a toplevel printer for the Cohttp types. In each of these packages, the `opam` and `ocamlfind` package names are now _the same_, so you will need to rename the former subpackages such as `cohttp.async` to `cohttp-async`. The implementation is otherwise the same, so no other code changes should be required. In return for these breaking changes to the packaging, it is now significantly easier to depend on a particular backend, also for us to rev the interfaces towards a stable 1.0 release. Jbuilder also builds the source tree around 4x faster than it did previously. A number of deprecation warnings have been added to the source tree as well to mark the interfaces that will be removed in 1.0. These are `Cohttp_lwt.{Client,Server,Net}`, and a `Cohttp_lwt.Body` alias was added to deprecate the direct use of `Cohttp_lwt_body`. This will let us unify the namespaces of all the packages to use a single top-level module for each package in the future. Most of the release and packaging work here was done by rgrinberg and avsm. ## 0.22.0 (2017-03-09) * Lwt: ensure conn_closed is closed once client goes away (mirage/ocaml-cohttp#528) * Use the Logs library for logging. (mirage/ocaml-cohttp#532) ## 0.21.1 (2017-02-18) * Remove -principal from type checking temporarily, to workaround a bug in the OCaml 4.03 type checker that causes compilation hangs (http://caml.inria.fr/mantis/view.php?id=7305). * Improve documentation in the `test_xhr.ml` js_of_ocaml test. * XHR: Allow setting withCredentials * Async: pass along ?ssl_config when connecting to Uri's (mirage/ocaml-cohttp#510) * Lwt: Add on ?on_exn to Server.create (mirage/ocaml-cohttp#518) * Add Header.to_frames ## 0.21.0 (2016-05-22) * Allow to request paths as strings (mirage/ocaml-cohttp#470, mirage/ocaml-cohttp#478) ## 0.20.2 (2016-04-04) * Update META version (mirage/ocaml-cohttp#473) * uri.services is only required by cohttp.async 0.20.1 (2016-04-01) * Switch cohttp to use ppx (mirage/ocaml-cohttp#457) * Lwt: Fix leak on HEAD client requests (mirage/ocaml-cohttp#467) 0.20.0 (2016-03-25) * Switch to pa_fields_conv and pa_sexp_conv for camlp4 extensions (mirage/ocaml-cohttp#465) * Compatibility with latest async (mirage/ocaml-cohttp#468) * Async: Add support for SSL parameters in client (mirage/ocaml-cohttp#466) * Lwt: ignore Sig.sigpipe under Windows (mirage/ocaml-cohttp#456) * Lwt: Fix FD leak (mirage/ocaml-cohttp#447) * Lwt: Log uncaught user exceptions * Async: Close non-persistent async connections (mirage/ocaml-cohttp#442) 0.19.3 (2015-09-28): * Support Async 113.00 by explicitly using the blocking Core `printf` (mirage/ocaml-cohttp#431) * cohttp_curl_async: add `-data-binary` to send POST data. (mirage/ocaml-cohttp#425) 0.19.2 (2015-08-20): * Improve Cohttp_async.Client error handling. When a Uri.t fails to resolve it is now included in the error. (mirage/ocaml-cohttp#420) 0.19.1 (2015-08-08): * Bring make_body_writer and write_header in Cohttp.S.Http_io. Needed by ocaml-git 0.19.0 (2015-08-05): Compatibility breaking interface changes: * Remove `read_form` from the `Request/Response/Header` interfaces as this should be done in `Body` handling instead (mirage/ocaml-cohttp#401). New features and bug fixes: * Remove `IO.write_line` as it was unused in any interfaces. * Do not use the `lwt` camlp4 extension. No observable external difference. * Do not return a code stacktrace in the default 500 handler. * Add `Cohttp.Header.compare` (mirage/ocaml-cohttp#411) * Fix typos in CLI documentation (mirage/ocaml-cohttp#413 via moonlightdrive) * Use the Lwt 2.5.0 buffer API. * `Cohttp_lwt.read_response` now has a non-optional `closefn` parameter (mirage/ocaml-cohttp#400). * Add a `Cohttp_lwt_s` module that contains all the Lwt module types in one convenient place (mirage/ocaml-cohttp#397). 0.18.3 (2015-07-12): * Allow `DELETE` requests to have request bodies (mirage/ocaml-cohttp#383). * Improve the Lwt client `callv` for HTTP/1.1 pipelined requests (mirage/ocaml-cohttp#379 via Török Edwin). 0.18.2 (2015-06-19): * Do not add content encoding for 204's (mirage/ocaml-cohttp#375) 0.18.1 (2015-06-05): * Remove trailing whitespace from headers (mirage/ocaml-cohttp#372) * Don't reverse order of list valued headers (mirage/ocaml-cohttp#372) 0.18.0 (2015-06-02): * Add Cohttp_async.Client.callv. Allows for making requests while reusing an HTTP connection (mirage/ocaml-cohttp#344) * Responses of status 1xx/204/304 have no bodies and cohttp should not attempt to read them (mirage/ocaml-cohttp#355) * Add top level printers. See cohttp.top findlib package (mirage/ocaml-cohttp#363) * Add `Header.to_string` (mirage/ocaml-cohttp#362) * Fix chunk truncation in chunked transfer encoding (mirage/ocaml-cohttp#360) Compatibility breaking interface changes: * Remove `Request`/`Response` modules outside of Cohttp pack (mirage/ocaml-cohttp#349) 0.17.2 (2015-05-24): * Remove dependency on the Lwt Camlp4 syntax extension (mirage/ocaml-cohttp#334). * Add `make github` target to push documentation to GitHub Pages (mirage/ocaml-cohttp#338 from Jyotsna Prakash). * Add Async integration tests and consolidate Lwt tests using the new framework (mirage/ocaml-cohttp#337). * Fix allocation of massive buffer when handling fixed size http bodies (mirage/ocaml-cohttp#345) 0.17.1 (2015-04-24): * [async] Limit buffer size to a maximum of 32K in the Async backend (mirage/ocaml-cohttp#330 from Stanislav Artemkin). * Add `Cohttp.Conf.version` with the library version number included. * Remove debug output from `cohttp-curl-async`. * Add the beginning of a `DESIGN.md` document to explain the library structure. 0.17.0 (2015-04-17): Compatibility breaking interface changes: * `CONNECT` and `TRACE` methods added to `Code`.Exhaustive matches will need updating. New features and bug fixes: * `Link` header parsing has been added as `Cohttp.Link`, `Header.get_links` and `Header.add_links` * `cohttp_server_*` now obeys `HEAD` requests and responds 405 to unknown methods * `Cohttp_async.Server.response` type is now exposed as a `response * body` pair * Failure to read a body in a pipelined response no longer terminates the stream * Fix `cohttp_curl_lwt -X HEAD` sending empty chunked body (mirage/ocaml-cohttp#313) * Fix a bug which left extra `\r\n` in buffer at end of chunked reads * Fix handling of request URI for query strings and `CONNECT` proxies (mirage/ocaml-cohttp#308, mirage/ocaml-cohttp#318) * Fix precedence of `Host` header when request-URI is absolute URI * Fix request URI path to be non-empty except for * requests (e.g. `OPTIONS *`) 0.16.1 (2015-04-09): New features and bug fixes: * Fix handling of request paths starting with multiple slashes (mirage/ocaml-cohttp#308) 0.16.0 (2015-03-23): Compatibility breaking interface changes: * Response.t and Request.t fields are no longer mutable * [lwt] Fix types in `post_form` to be a `string * string list` instead of a `Header.t` (mirage/ocaml-cohttp#257) * Simplify the `Net` signature which needs to be provided for Lwt servers to not be required. Only the Lwt client needs a `Net` functor argument to make outgoing connections. (mirage/ocaml-cohttp#274) * The `Request` and `Response` records are no longer mutable, so use functional updates instead via `Fieldslib.Field.fset Request.Fields.<field>`. (mirage/ocaml-cohttp#296) * `Request.has_body` does not permit a body to be set for methods that RFC7231 forbids from having one (`HEAD`, `GET` and `DELETE`). New features and bug fixes: * Fix linking problem caused by sub-libraries using cohttp modules outside the cohttp pack. * Added async client for S3. (mirage/ocaml-cohttp#304) * Fix String_io.read_line to trim '\r' from end of string (mirage/ocaml-cohttp#300) * Fix `cohttp-server-lwt` to correctly bind to a specific interface (mirage/ocaml-cohttp#298). * Add `Cohttp_async.request` to send raw, unmodified requests. * Supplying a `content-range` or `content-range` header in any client request will always override any other encoding preference (mirage/ocaml-cohttp#281). * Add a `cohttp-lwt-proxy` to act as an HTTP proxy. (mirage/ocaml-cohttp#248) * Extend `cohttp-server-async` file server to work with HTTPS (mirage/ocaml-cohttp#277). * Copy basic auth from `Uri.userinfo` into the Authorization header for HTTP requests. (mirage/ocaml-cohttp#255) * Install binaries via an OPAM `.install` file to ensure that they are reliably uninstalled. (mirage/ocaml-cohttp#252) * Use the `magic-mime` library to add a MIME type by probing filename during static serving in the Lwt/Async backends. (mirage/ocaml-cohttp#260) * Add `Cohttp.Header.add_opt_unless_exists` to set a header only if an override wasn't supplied, and to initialise a fresh Header value if none is present. * Do not override user-supplied headers in `post_form` or `redirect`. * `Request.make` does not inject a `transfer-encoding` header if there is no body present in the request (mirage/ocaml-cohttp#246). * `Server.respond` no longer overrides user-supplied headers that specify the `content-length` or `transfer-encoding` headers (mirage/ocaml-cohttp#268). * `cohttp_server_lwt` and `cohttp_server_async` now include sizes in directory listing titles * Add `Header.add_multi` to initialise a header structure with multiple fields more efficiently (mirage/ocaml-cohttp#272). * Expose `IO.ic` and `IO.oc` types for `Cohttp_async` (mirage/ocaml-cohttp#271). * Skip empty body chunks in `Transfer_io.write` (mirage/ocaml-cohttp#270). * With the Lwt backend, `read` hangs if trying to fetch more than `Sys.max_string_length` (which can be triggered on 32-bit platforms). Read only a maximum that fits into a string (mirage/ocaml-cohttp#282). * `cohttp-curl-lwt` now takes http method as parameter (mirage/ocaml-cohttp#288) * Fix installation of server binaries in OPAM metadata. (mirage/ocaml-cohttp#295) 0.15.2 (2015-02-15): * When transfer encoding is unknown, read until EOF when body size is unknown. (mirage/ocaml-cohttp#241) * Add some missing documentation to `Cohttp.S.IO` signature. (mirage/ocaml-cohttp#233) * Add `Cohttp.Header.mem` to check if a header exists. * Add `Cohttp.Conf` module to expose the library version number. (mirage/ocaml-cohttp#259) * Add `Cohttp.Header.add_unless_exists` to update a key if it doesn't already exist. (mirage/ocaml-cohttp#244) * Add `Cohttp.Header.get_location` to retrieve redirection information. (mirage/ocaml-cohttp#254) * [async] Clean up the `Net.lookup` function to use `Or_error.t` instead of raising. (mirage/ocaml-cohttp#247) * [tests] Add more tests for `content-range` handling. (mirage/ocaml-cohttp#249) 0.15.1 (2015-01-10): * Lwt 2.4.7 renamed `blit_bytes_string` to `blit_to_bytes`, so depend on the newer API now. (mirage/ocaml-cohttp#230) * Use `cmdliner` in all of the Lwt client and server binaries. This gives `cohttp-lwt-server` a nice Unix-like command-line interface now that can be viewed with the `--help` option. (mirage/ocaml-cohttp#218 via Runhang Li) * Improve `oasis` constraints and regenerate `opam` file (mirage/ocaml-cohttp#229 via Christophe Troestler). 0.15.0 (2014-12-24): Compatibility breaking interface changes: * Change `Cohttp_lwt_body.map` to use a non-labelled type to fit the Lwt style better (mirage/ocaml-cohttp#200). * Depend on Base64 version 2, which uses `B64` as the toplevel module name (mirage/ocaml-cohttp#220). New features and bug fixes: * Remove use of deprecated `Lwt_unix.run` and replace it with `Lwt_main.run`. Should be no observable external change (mirage/ocaml-cohttp#217). * Improve ocamldoc of `Cohttp.S` signature (mirage/ocaml-cohttp#221). 0.14.0 (2014-12-18): Compatibility breaking interface changes: * Simplify the Lwt server signature, so that manual construction of a `callback` is no longer required (mirage/ocaml-cohttp#210). Code that previous looked like: ``` let conn_closed (_,conn_id) () = <...> let config = { Server.callback; conn_closed } in ``` should now be: ``` let conn_closed (_,conn_id) = <...> let config = Server.make ~callback ~conn_closed () in ``` * Remove the `Cohttp.Base64` module in favour of the external `base64` library (which is now a new dependency). New features and bug fixes: * Lwt `respond_error` now defaults to an internal server error if no status code is specified (mirage/ocaml-cohttp#212). * Modernise the `opam` file using the OPAM 1.2 workflow (mirage/ocaml-cohttp#211). * Flush the response body to the network by default, rather than buffering by default. The `?flush` optional parameter can still be explicitly set to false if flushing is not desired (mirage/ocaml-cohttp#205). 0.13.0 (2014-12-05): Compatibility breaking interface changes: * Add sexp converters for Conduit contexts and `Lwt` client and server modules and module types. New features and bug fixes: * Can use the Conduit 0.7+ `CONDUIT_TLS=native` environment variable to make HTTPS requests using the pure OCaml TLS stack instead of depending on OpenSSL bindings. All of the installed binaries (client and server) can work in this mode. * Add `Cohttp_lwt_unix_debug` which lets libraries control the debugging output from Cohttp. Previously the only way to do this was to set the `COHTTP_DEBUG` environment variable at the program start. * Add `cohttp-curl-lwt` as a lightweight URI fetcher from the command-line. It uses the `cmdliner` as a new dependency. * Remove build dependency check on `lwt.ssl` for `cohttp.lwt`. This has been moved to conduit, so only `lwt.unix` is needed here now. 0.12.0 (2014-11-07): Compatibility breaking interface changes: * Rename `Cohttp.Auth.t` to `Cohttp.Auth.credential` and `Cohttp.Auth.req` to `Cohttp.Auth.challenge`. Also expose an `Other` variant to make it more extensible for unknown authentication types. The `Cohttp.Auth` functions using these types have also been renamed accordingly. * Rename `Cohttp.Transfer.encoding_to_string` to `string_of_encoding` for consistency with the rest of Cohttp's APIs. * The `has_body` function in the Request and Response modules now explicitly signals when the body size is unknown. * Move all the module type signatures into `Cohttp.S`. * If users have percent-encoded file names, their resolution is changed: `resolve_local_file` in `Cohttp_async` and `Cohttp_lwt` now always percent-decode paths (mirage/ocaml-cohttp#157) * Remove the `Cohttp_lwt.Server.server` type synonym to `t`. * When reading data from a HTTP body stream using the `Fixed` encoding, we need to maintain state (bytes remaining) so we know when to finish. The `Cohttp.Request` and `Cohttp.Response` interfaces now expose a `reader` and `writer` types to track this safely. * Add `is_empty` function to the `Cohttp.S.Body` module type. * Add `Strings` representation to `Cohttp.Body` to efficiently hold a list of body chunks. * Move flushing logic for HTTP bodies into the portable `Request` and `Response` modules instead of individual Lwt and Async backends. * Port module interfaces to the latest Conduit (0.6.0+) API. * Cohttp requires OCaml 4.01.0 or higher now. New features and bugfixes: * Add a `Cohttp_lwt_xhr` JavaScript backend that enables Cohttp logic to be mapped to `XMLHTTPRequest` in browsers via `js_of_ocaml` (via Andy Ray). * Add a `Cohttp.String_io` and `String_io_lwt` module that uses OCaml `string` or `Buffer.t` to read and write HTTP requests and responses instead of network connections. * `cohttp_server_lwt` and `cohttp_server_async` now return better errors (mirage/ocaml-cohttp#158) * `cohttp_server_lwt` and `cohttp_server_async` now serve indexes directly (mirage/ocaml-cohttp#162) * [lwt] Add `stop` thread to terminate a running server if it finishes (mirage/ocaml-cohttp#147). * Add `Cohttp.Connection.compare` to make ordering of connections possible. * Add `Body.map` and `Body.as_pipe` to work with HTTP bodies more easily. * Remove link-time dependency on camlp4 via META fixes (mirage/ocaml-cohttp#127). * Support HTTP methods and versions other than the standard ones. (mirage/ocaml-cohttp#142). * Improve `cohttp_server_lwt` and `cohttp_server_async` directory listings (mirage/ocaml-cohttp#158) * Fix `Cohttp_async.resolve_local_file` directory traversal vulnerability (mirage/ocaml-cohttp#158) * [async] In the Async server, do not close the Reader too early. * [async] Close file descriptors more eagerly in the HTTP client (mirage/ocaml-cohttp#167). * Reduce thread allocation by replacing `return <const>` with `return_none`, `return_unit` or `return_nil`. 0.11.2 (2014-04-21) * Fix build by add a missing build-deps in _oasis. 0.11.1 (2014-04-17): * Remove an errant async_ssl reference left in the _oasis file that is now handled by the Conduit library (mirage/ocaml-cohttp#116). * Add an Lwt-based SimpleHTTPServer equivalent as `cohttp-server-lwt` (mirage/ocaml-cohttp#108). * `Cohttp.Connection.t` now exposes sexp accessor functions (mirage/ocaml-cohttp#117). 0.11.0 (2014-04-01): * Remove dependency on `ocaml-re` in order to make library POSIX thread-safe. * Shift most of the connection handling logic out to a Conduit library that worries about which SSL library to use, and fails if SSL is not available. * Add Async-SSL support for both client and server (mirage/ocaml-cohttp#102). * Add Lwt-SSL support for the server side (the client side existed before). * Fix buggy Async chunked POST handling. 0.10.0 (2014-03-02): * Interface change: The `Request` and `Response` module types now explicitly signal `Eof` and `Invalid` (for errors), to help the backend distinguish them. * Interface change: Unify HTTP body handling across backends into a `Cohttp.Body` module. This is extended by Async/Lwt implementations with their specific ways of handling bodies (Pipes for Async, or Lwt_stream for Lwt). * [lwt] Interface change: HTTP client calls now raise Lwt exceptions rather than return an option type. This permits better error handling in Lwt. * [lwt] Interface change: The `Server` callback now always provides a `body` argument, since `Cohttp_lwt_body` now explicitly supports empty bodys. * Add `Cohttp.Header.is_keep_alive` to test if a connection should be reused. * [lwt] Respect the `keep-alive` header in the server request handling. * [async] Add a `Body` that takes a `Pipe` or a `string`, similarly to Lwt. * Install `cohttp-server` binary even if tests are disabled. * Begin an `examples` directory with some simple uses of the library. 0.9.16 (2014-01-30): * Add some module type equalities in `Cohttp_lwt_unix` so that `Cohttp_lwt_unix.Server.Request.IO.ic` can be equivalen to `Lwt_io.input_channel`. * Add sexp converters to most Cohttp types (mirage/ocaml-cohttp#83). * Improve Travis tests to cover more upstream users of Cohttp. * Refactor build flags to let the portable Lwt-core be built independently of Lwt.unix. 0.9.15 (2014-01-11): * Remove `Cohttp_mirage` libraries, which have now moved to `mirage/mirage-http-*` on GitHub. * Add an "HTTP only" `Cookie` attribute (mirage/ocaml-cohttp#69). * Fix parsing of cookies with `=` in the values (mirage/ocaml-cohttp#71). * Add `Max-age` support for cookies (mirage/ocaml-cohttp#70). * Make the `Response` record fields mutable to match the `Request` (mirage/ocaml-cohttp#67). * Fix compilation with Async 109.58.00 (mirage/ocaml-cohttp#77). * Make Header handling case-insensitive (by forcing lowercase) (mirage/ocaml-cohttp#75). * Remove the `>>` operator as it was unused and had incorrect precedence (mirage/ocaml-cohttp#79). 0.9.14 (2013-12-15): * Install a `cohttp-server` binary that serves local directory contents via a web server (mirage/ocaml-cohttp#54). * Add a `flush` function to the `IO` module type and implement in Lwt/Async/Mirage. * Add option `flush` support in the Async and Lwt responders (mirage/ocaml-cohttp#52). * Autogenerate HTTP codes from citricsquid's JSON representation of the HTTP RFCs. * Always set `TCP_NODELAY` for Lwt/Unix server sockets for low-latency responses (mirage/ocaml-cohttp#58). * Added a Server-Side Events test-case from the HTML5 Doctor. See `lib_test/README.md`. * Async.Server response now takes an optional `body` rather than a mandatory `body option` (mirage/ocaml-cohttp#62). * Regenerate build system using OASIS 0.4.0. 0.9.13 (2013-12-10): * The `cohttp.lwt-core` is now installed as an OS-independent Lwt library. * Add support for Mirage 1.0, via `cohttp.mirage-unix` and `cohttp.mirage-xen`. * Add a new `Cohttp.Connection` module to manage server's connections identifiers. * Share the same configuration type for the different server implementations. * Add `Accept_types` module to the `Cohttp` pack. 0.9.12 (2013-11-28): * Improve documentation for `Cohttp.Header`. * Expose Fieldslib setters and getters for most of the `Cohttp` types (mirage/ocaml-cohttp#38). * `Cohttp.Set_cookie.t` is no longer an abstract type to make it easier to update (mirage/ocaml-cohttp#38). * [Lwt] ignore SIGPIPE unconditionally if using the Lwt/Unix module (mirage/ocaml-cohttp#37). * Rename `Cookie` creation parameters for consistency (interface breaking, see mirage/ocaml-cohttp#44). * Fix transfer-length detection (regression from 0.9.11 in mirage/ocaml-cohttp#42). * Add Merin editor file (mirage/ocaml-cohttp#41). 0.9.11 (2013-10-27): * Request module: When sending a request, add the port information in the host header field if available. * Request module: When parsing a request, add scheme, host and port information in the uri. * TCP server: When creating the socket for the server, do not force PF_INET6 but take the sockaddr value. * Add HTTP OPTIONS method. * Use getaddrinfo instead of gethostbyname for DNS resolution. * Async: improve HTTP/1.0 support (mirage/ocaml-cohttp#35). * Build with debug symbols, binary annotations by default. * Add Travis CI test scripts. 0.9.10 (2013-06-21): * Add `set-cookie` header extraction functions for clients that read cookies. * Explicitly flush the debug output when the `COHTTP_DEBUG` env variable is set. * [async] Add client head/post/patch/delete methods. * [lwt] Client.head no longer returns a response body, just the metadata. * [lwt] Do not send chunked encoding headers with GET/DELETE requests that have no body. 0.9.9 (2013-06-12): * Disable the mirage executable test as it was building too aggressively and breaking builds. 0.9.8 (2013-05-24): * Lwt interface change: Rewrite Lwt backends to share code, and remove duplicate function calls from Uri. * Depend on `Uri` 1.3.8+ as it exposes the parameter query functions now removed from `Request`. * Do not depend on Cstruct in core library, as only Mirage needs it. * Remove `Cohttp_async.body` type alias and just use `string Pipe.Reader.t` for more explicit types. 0.9.7 (2013-05-10): * Attach a GC finaliser to the Lwt client to ensure that even an HTTP body isn't consumed, the socket will eventually be closed (mirage/ocaml-cohttp#11). * Add an Async.Server interface, and revise the Client interface to be more in line with Core standards. * Add 422 Unprocessable Entity code. * Refactor modules better across Lwt/Async, but incompatible with earlier releases for Async (Lwt is unchanged at present). * Add user agent string and User-Agent header helper function * The git history of this release is full of adventures in parameterised monads and refactoring, but this isn't in the actual release. Yet. 0.9.6 (2013-03-18): * Depend on Async (>= 109.12.00), which has an incompatible API with earlier versions. * Rearrange core library files for `obuild` support. 0.9.5 (2012-12-29): * Fix cookie parsing to retrieve the correct header. * Update to `mirage-net` 0.5.0 API (based on cstruct 0.6.0). 0.9.4 (2012-12-19): * Add Lwt `respond_redirect` and `respond_need_auth` helpers. * Add enough Basic authorization support to serve a password-protected website. * Fix Lwt file serving to not throw exception on trying to serve a directory. * Port Async interface to 108.07.00 or higher (incompatible with earlier versions). 0.9.3 (2012-10-27): * Add basic cookie support back to the portable library. * `Cohttp_lwt.Client.post_form` now uses non-chunked encoding for the POST instead of chunked. * Various improvements and tests for the pipelined Lwt Client.callv * If an Lwt callback does not consume a body, ensure it has been drained by the API to prevent future pipelines from stalls. * Fix handling of Lwt server non-empty POST bodies. * Map the `put` functions to HTTP PUT instead of POST. 0.9.2 (2012-09-20): * Add Request.get_param to extract a singleton key from queries. * Fix chunked encoding handling when short reads occur. * Install HTML documentation for all enabled drivers. * Use ocaml-uri-1.3.2 interface for query parsing. * Lwt: Add Server.respond_file and resolve_file for the Unix library to make it easier to serve static files. * Lwt: Server.respond_not_found takes an optional Uri.t now. 0.9.1 (2012-09-11): * Functorise for Async, Lwt_unix and Mirage. * Use URI and Re libraries to not need Str any more. * More robust parsing for various HTTP headers. 0.9.0 (2012-08-01): * Initial public release.

anuragsoni commented Jan 7, 2022

View reviewed changes

cohttp/src/s.ml Show resolved Hide resolved

anuragsoni commented Jan 7, 2022

View reviewed changes

rgrinberg reviewed Jan 7, 2022

View reviewed changes

anuragsoni commented Jan 7, 2022

View reviewed changes

mseri reviewed Jan 7, 2022

View reviewed changes

cohttp/test/test_request.ml Show resolved Hide resolved

rgrinberg reviewed Jan 7, 2022

View reviewed changes

examples/bench/async_server.ml Outdated Show resolved Hide resolved

rgrinberg reviewed Jan 7, 2022

View reviewed changes

mseri reviewed Jan 8, 2022

View reviewed changes

cohttp-lwt-unix/src/bytebuffer.ml Outdated Show resolved Hide resolved

anuragsoni marked this pull request as ready for review January 9, 2022 04:52

New HTTP request parser

c49431b

* Hand written http request/response parser * Far more efficient IO layer

rgrinberg merged commit 06d2127 into mirage:master Jan 9, 2022

anuragsoni deleted the new-parser branch January 9, 2022 17:35

anuragsoni mentioned this pull request Jan 9, 2022

Update Changes #826

Merged

talex5 mentioned this pull request Jan 10, 2022

Decide on cstruct vs bytes ocaml-multicore/eio#140

Open

2 tasks

mseri mentioned this pull request Jan 10, 2022

Conduit Free Servers #821

Open

smorimoto mentioned this pull request Jan 13, 2022

Latency benchmarking cohttp shows queueing #328

Open

rgrinberg mentioned this pull request Jan 15, 2022

Revert to Cohttp rgrinberg/opium#279

Open

bikallem pushed a commit to bikallem/ocaml-cohttp that referenced this pull request Jan 24, 2022

Update Changes

36503c4

* We only have a new parser for requests for now * We only touched the read path with mirage#819

mseri mentioned this pull request Oct 24, 2022

[new release] http, cohttp, cohttp-top, cohttp-server-lwt-unix, cohttp-mirage, cohttp-lwt, cohttp-lwt-unix, cohttp-lwt-jsoo, cohttp-eio, cohttp-curl, cohttp-curl-lwt, cohttp-curl-async and cohttp-async (6.0.0~alpha0) ocaml/opam-repository#22353

Merged

mseri mentioned this pull request Nov 14, 2022

memory consumption blowups in downstream projects using Lwt ocsigen/lwt#972

Open

samoht mentioned this pull request Mar 3, 2025

[new release] cohttp (14 packages) (6.1.0) ocaml/opam-repository#27537

Merged

		@@ -0,0 +1,24 @@
		open Lwt.Infix

		type t = { buf : Bytebuffer.t; chan : Lwt_io.input_channel }

Conversation

anuragsoni commented Jan 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

Uh oh!

rgrinberg commented Jan 7, 2022

Uh oh!

Uh oh!

rgrinberg commented Jan 7, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rgrinberg commented Jan 7, 2022

Uh oh!

anuragsoni commented Jan 7, 2022

Uh oh!

anuragsoni commented Jan 7, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rgrinberg commented Jan 7, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anuragsoni Jan 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anuragsoni commented Jan 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anuragsoni commented Jan 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mseri commented Jan 7, 2022

Uh oh!

anuragsoni commented Jan 7, 2022

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anuragsoni commented Jan 7, 2022 •

edited

Loading

anuragsoni Jan 7, 2022 •

edited

Loading

anuragsoni commented Jan 7, 2022 •

edited

Loading

anuragsoni commented Jan 7, 2022 •

edited

Loading

mseri commented Jan 7, 2022 •

edited

Loading

anuragsoni commented Jan 9, 2022 •

edited

Loading