Error Content-Range header not present in partial response #664
Description
It looks like Content-Range header is not working.
https://github.com/gaul/s3proxy/blob/master/src/main/java/org/gaul/s3proxy/S3ProxyHandler.java#L1782
Got error when trying to read delta table via s3proxy. If I access swift directly, it works.
IOException: IO Error: Hit DeltaKernel FFI error (from: While trying to read from delta table: 's3://testdataset/'): Hit error: 7 (ParquetError) with message (Arrow error: External: Generic S3 error: Content-Range header not present in partial response)
debug log of s3proxy, it got "content-range: bytes 0-910152/910153" from swift.
[s3proxy] D 07-18 10:47:14.168 S3Proxy-Jetty-17 o.j.h.i.JavaUrlHttpCommandExecutorService:56 |::] Receiving response -464838638: HTTP/1.1 206 Partial Content
[s3proxy] D 07-18 10:47:14.168 S3Proxy-Jetty-17 jclouds.headers:56 |::] << HTTP/1.1 206 Partial Content
[s3proxy] D 07-18 10:47:14.168 S3Proxy-Jetty-17 jclouds.headers:56 |::] << date: Thu, 18 Jul 2024 10:47:13 GMT
[s3proxy] D 07-18 10:47:14.168 S3Proxy-Jetty-17 jclouds.headers:56 |::] << content-length: 910153
[s3proxy] D 07-18 10:47:14.169 S3Proxy-Jetty-17 jclouds.headers:56 |::] << last-modified: Wed, 17 Jul 2024 01:11:47 GMT
[s3proxy] D 07-18 10:47:14.169 S3Proxy-Jetty-17 jclouds.headers:56 |::] << x-openstack-request-id: txfdcc88656b4e4e46ad3dc-006698f2b1
[s3proxy] D 07-18 10:47:14.169 S3Proxy-Jetty-17 jclouds.headers:56 |::] << x-trans-id: txfdcc88656b4e4e46ad3dc-006698f2b1
[s3proxy] D 07-18 10:47:14.169 S3Proxy-Jetty-17 jclouds.headers:56 |::] << content-range: bytes 0-910152/910153
[s3proxy] D 07-18 10:47:14.169 S3Proxy-Jetty-17 jclouds.headers:56 |::] << etag: "f92ea2b80c7ecf328f7cc99fab58500f"
[s3proxy] D 07-18 10:47:14.170 S3Proxy-Jetty-17 jclouds.headers:56 |::] << content-type: binary/octet-stream
[s3proxy] D 07-18 10:47:14.170 S3Proxy-Jetty-17 jclouds.headers:56 |::] << x-amz-request-id: txfdcc88656b4e4e46ad3dc-006698f2b1
[s3proxy] D 07-18 10:47:14.170 S3Proxy-Jetty-17 jclouds.headers:56 |::] << x-amz-id-2: txfdcc88656b4e4e46ad3dc-006698f2b1
[s3proxy] D 07-18 10:47:14.170 S3Proxy-Jetty-17 jclouds.headers:56 |::] << Content-Type: binary/octet-stream
[s3proxy] D 07-18 10:47:14.170 S3Proxy-Jetty-17 jclouds.headers:56 |::] << Content-Length: 910153
import duckdb
access_key = "xxx"
secret_key = "xxx"
endpoint_url="s3proxy.s3proxy.svc.cluster.local"
conn = duckdb.connect()
conn.execute(f"""
CREATE SECRET s1 (
TYPE S3,
USE_SSL 'false',
URL_STYLE 'path',
ENDPOINT '{endpoint_url}',
KEY_ID '{access_key}',
SECRET '{secret_key}');
""")
total_rows_result = conn.execute("SELECT * FROM delta_scan('s3://testdataset') limit 1;")
total_rows = total_rows_result.fetchall()
print(f" {total_rows}")