-
Notifications
You must be signed in to change notification settings - Fork 270
Description
Context
I am running s3-proxy with encryption enabled, connected to an Azure Blob Storage account. All files are uploaded through the s3-proxy and have an .s3enc extension.
My client is trying to fetch the last few bytes of a file using an open-ended range request. The process is as follows:
- A HEAD request retrieves the total Content-Length of the file.
- A GET request fetches the last X bytes, using the range header.
For example, I am making the following request to get the last 4 bytes of a file:
curl http://localhost:80/encrypted_parquet_files/titanic.parquet -H "range: bytes=1073555-"
Problem
The server correctly returns the last 4 bytes in the body, but the response headers are incorrect. Specifically, the Content-Length header contains the total file size (1073559) rather than the length of the requested range (4).
Example
For an encrypted file:
Request:
curl --head http://localhost:80/encrypted_parquet_files/titanic.parquet # Content-Lenght: 1073559
curl http://localhost:80/encrypted_parquet_files/titanic.parquet -H "range: bytes=1073555-"
Response:
Content-Length: 1073559
For an unencrypted file, the same request works as expected, and the headers are correct:
Request:
curl http://localhost:80/unencrypted_parquet_files/titanic.parquet -H "range: bytes=1073555-"
Response:
Content-Range: bytes 1073555-1073558/1073559
Accept-Ranges: bytes
Content-Length: 4
Expected Behavior
For the encrypted file, the server should return headers like the unencrypted file, including the correct Content-Length (4) and Content-Range (e.g., bytes 1073555-1073558/1073559).
Actual Behavior
The server is incorrectly returning the full file size (1073559) in the Content-Length header for the encrypted file. This breaks the client because the server promised a larger payload, but the connection was closed after sending only 4 bytes.