Skip to content

Force HTTPS for selected domains#676

Merged
mblaney merged 3 commits intosimplepie:masterfrom
FreshRSS:https_everywhere
Apr 12, 2021
Merged

Force HTTPS for selected domains#676
mblaney merged 3 commits intosimplepie:masterfrom
FreshRSS:https_everywhere

Conversation

@Alkarex
Copy link
Contributor

@Alkarex Alkarex commented Mar 28, 2021

Some feeds have embedded references using e.g. //youtube.com/ or http://youtube.com/ which are increasingly failing as HTTPS is required.

This PR allows setting a list of domains for which to force HTTPS with a new function SimplePie::set_https_domains([])

Downstream PR: FreshRSS/FreshRSS#1087
FreshRSS/FreshRSS#1098

Some feeds have embeded references using e.g. `//youtube.com/` or `http://youtube.com/` which are increasingly failing as HTTPS is required.

This PR allows setting a list of domains for which to force HTTPS with a new function `SimplePie::set_https_domains([])`

Downstream PR: FreshRSS/FreshRSS#1087
FreshRSS/FreshRSS#1098
* @see SimplePie_Misc::https_url()
* Example array('biz', 'example.com', 'example.org', 'www.example.net');
*/
public function set_https_domains($domains)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list of domains is transformed into a tree for fast lookup and to avoid ambiguities.

Alkarex added a commit to Alkarex/FreshRSS that referenced this pull request Mar 28, 2021
Alkarex added a commit to FreshRSS/FreshRSS that referenced this pull request Mar 28, 2021
@skyzyx
Copy link
Member

skyzyx commented Mar 28, 2021

Well I'm not opposed to this idea in general, I would hate to be on the hook for owning this list.

Is there anyway we could source a good list from somebody else who's already maintaining one? The EFF's HTTPS Everywhere extension comes to mind as a possible good source of data.

Assuming it's a JSON document, we could periodically fetch the list, transform it into a PHP lookup table, and ship a new release.

@skyzyx
Copy link
Member

skyzyx commented Mar 28, 2021

My mistake; I was on my phone and didn't see that this is a bring-your-own-list situation.

I think this is a good idea.

@Alkarex
Copy link
Contributor Author

Alkarex commented Mar 28, 2021

Not that it should be used, but just as an example, downstream, we use this list by default, customisable by end-users https://github.com/FreshRSS/FreshRSS/blob/edge/force-https.default.txt

Our two most important use-cases dailymotion.com, youtube.com are even set in the source-code at the moment but I will remove that if this PR is merged to keep the same code:

https://github.com/FreshRSS/FreshRSS/blob/634005de9a4b5e415ebf7c1106c769a0fbed5cfd/lib/SimplePie/SimplePie/Sanitize.php#L82

{
$value = $this->registry->call('Misc', 'absolutize_url', array($element->getAttribute($attribute), $this->base));
if ($value !== false)
$value = $this->https_url($value);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

absolutize_url can return false, I don't think you should check the url in that case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed 5bfec78

@mblaney mblaney merged commit d4b76ea into simplepie:master Apr 12, 2021
@Alkarex Alkarex deleted the https_everywhere branch April 12, 2021 09:01
Alkarex added a commit to FreshRSS/simplepie that referenced this pull request Apr 13, 2021
From simplepie#676
A parenthesis was misplaced
@Alkarex Alkarex mentioned this pull request Apr 13, 2021
mblaney pushed a commit that referenced this pull request Apr 14, 2021
From #676
A parenthesis was misplaced
Alkarex added a commit to Alkarex/FreshRSS that referenced this pull request Apr 17, 2021
Alkarex added a commit to FreshRSS/FreshRSS that referenced this pull request Apr 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants