I am trying to download a table of ~30GB from BigQuery to Arrow.

I am following the documentation example with the following changes:

• I have set the "max stream count" to 100.
• I am reading the streams in parallel

I run this from a VM in google cloud in the same region as my big query table however my download speed seems to be limited to "only" ~250MB/s and my CPU usage is ~20%. My VM is a e2-standard-8 (8 vCPUs, 32 GB memory)

Neither my network bandwidth or my CPUs seems to be a limit here because I can run three times the same program in parallel and get ~700MB download speed and ~60% CPU usage. However I cannot get this speed for a single request.

How can I go faster ? Is this 250MB/s a limit by BigQuery, a limit of the API, something that I am missing ?
What maximum speed should I expect when downloading BigQuery tables with this API ?