Skip to content

Issue with websites sending invalid Content-Encoding headers #961

@mmeier86

Description

@mmeier86

This issue is related to FreshRSS/FreshRSS#8374

I came across an issue with blogs hosted on alternative S3 implementations (Ceph, in this case) and being uploaded with Hugo's deploy command. The effect of this setup is that files from that bucket get the Content-Encoding: aws-chunked header, which is then also send out when the file, in this case the index.xml containing the RSS feed, is requested by a client.

This leads to problems in FreshRSS, using SimplePie, as cURL only supports certain specific values for the Content-Encoding header. SimplePie currently already has a workaround for this issue:

simplepie/src/File.php

Lines 141 to 144 in 92bc715

if (curl_errno($fp) === CURLE_WRITE_ERROR || curl_errno($fp) === CURLE_BAD_CONTENT_ENCODING) {
curl_setopt($fp, CURLOPT_ENCODING, 'none');
$responseHeaders = curl_exec($fp);
}

The none value set in the current code does not seem to be officially supported by cURL, which only accepts "", which sends an Accept-Encoding header with all supported encodings, or a specific encoding, namely identity, deflate, gzip, br and zstd. With none set, the server might still send the Content-Encoding: aws-chunked invalid header, in which case curl errors out, as it doesn't know what to do with this encoding.

My proposal is to change the value from 'none' to null. This disables any attempt at decompressing the response body on cURL's side. At least in my tests in FreshRSS, this allowed SimplePie to successfully fetch the index.xml file and extract the RSS feed from it, even when the server sends the invalid Content-Encoding header.

I will provide a PR which makes the proposed change from 'none' to null in a moment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions