This http-client code works when the content-length is “small” but starts
failing hard once it is a few kilobytes:
let response : Http.Response.t * Eio.Buf_read.t =
Cohttp_eio.Client.call
eio
?port
~headers:(Http.Header.of_list headers)
~host ~meth ?body resource
in
let body = Cohttp_eio.Client.read_fixed response inThe exception cannot be caught (it seems to be in the loop of the Luv backend):
Uncaught exception in run loop:
Exception: Invalid_argument("read_start: handle used after calling close!")
Raised at Stdlib.invalid_arg in file "stdlib.ml", line 30, characters 20-45
Called from Eio_luv.Low_level.Stream.read_into.(fun) in file "vendor/eio/lib_eio_luv/eio_luv.ml", line 512, characters 32-62
Called from Eio_luv.wakeup in file "vendor/eio/lib_eio_luv/eio_luv.ml", line 1179, characters 4-8
Called from Eio_luv.run2.(fun) in file "vendor/eio/lib_eio_luv/eio_luv.ml", line 1194, characters 10-40
Fatal error: exception Failure("Deadlock detected: no events scheduled but main function hasn't returned")
Raised at Stdlib.failwith in file "stdlib.ml", line 29, characters 17-33
Called from Stdlib__Fun.protect in file "fun.ml", line 33, characters 8-15
Re-raised at Stdlib__Fun.protect in file "fun.ml", line 38, characters 6-52
Called from Dune__exe__Backend_server_basics in file "test/backend_server_basics.ml", line 79, characters 2-1023
Looking at the code
client.ml:100:
- it seems that indeed
Eio.Buf_read.of_flow ~initial_size ~max_size:max_int conn is lazy enough
- so when
Cohttp_eio.Client.read_fixed gets called it still tries to pull from the socket (?)
- The doc of
Eio.Net.with_tcp_connect says: “[conn] is closed after [f] returns (if it isn't already closed by then).”
let buf_write conn =
let initial_size = 0x1000 in
Buf_write.with_flow ~initial_size:0x1000 conn (fun writer ->
let request = Http.Request.make ?meth ?version ~headers resource_path in
let request = Http.Request.add_te_trailers request in
write_request pipeline_requests request writer body;
let reader =
Eio.Buf_read.of_flow ~initial_size ~max_size:max_int conn
in
let response = response reader in
(response, reader))
in
match conn with
| None ->
let service =
match port with Some p -> string_of_int p | None -> "80"
in
Eio.Net.with_tcp_connect ~host ~service env#net (fun conn ->
Eio.traceln "using with_tcp_connect";
buf_write conn)
| Some conn ->
Eio.traceln "NOT using with_tcp_connect";
buf_write conn
I tried to replace read_fixed with this to see what happens:
let body =
let buf = snd response in
let res = Buffer.create 42 in
let rec go n =
try
Buffer.add_char res (Eio.Buf_read.any_char (* take_all *) buf);
if n % 20 = 0 then traceln "not done reading: %d" n;
go (n + 1)
with _ ->
traceln "done reading: %d" n;
Buffer.contents res
in
go 0
inIt fails after saying +not done reading: 3960
This http-client code works when the content-length is “small” but starts
failing hard once it is a few kilobytes:
The exception cannot be caught (it seems to be in the loop of the Luv backend):
Looking at the code
client.ml:100:
Eio.Buf_read.of_flow ~initial_size ~max_size:max_int connis lazy enoughCohttp_eio.Client.read_fixedgets called it still tries to pull from the socket (?)Eio.Net.with_tcp_connectsays: “[conn] is closed after [f] returns (if it isn't already closed by then).”I tried to replace
read_fixedwith this to see what happens:It fails after saying
+not done reading: 3960