Skip to content

Extension installation and updates via GUI#7712

Open
tryallthethings wants to merge 11 commits intoFreshRSS:edgefrom
tryallthethings:extension-updater
Open

Extension installation and updates via GUI#7712
tryallthethings wants to merge 11 commits intoFreshRSS:edgefrom
tryallthethings:extension-updater

Conversation

@tryallthethings
Copy link
Contributor

Feel free to make changes. The only issue I've found is that hitting the GitHub rate limit is an option when testing 😄

Closes

#2900
FreshRSS/Extensions/issues/323

Changes proposed in this pull request:

  • Added extension installation and update mechanism for individual extensions and bulk updates
  • Added helper functions to detect repository type, find version matching https://raw.githubusercontent.com/FreshRSS/Extensions/master/extensions.json, download release, find release notes, sanitize downloaded extension files
  • Added translations for update actions
  • change in base CSS to display button icon and text without wrapping (Plus icon and install button text will wrap by default since the column is too small)

How to test the feature manually:

  1. Replace files with the changes of this PR
  2. Go to Settings → Extensions
  3. Every known extension will have an "Install" or "Update" button in the newly added "Actions" column. Clicking it will install the extension or update it

Pull request checklist:

  • clear commit messages
  • code manually tested
  • unit tests written (optional if too hard)
  • documentation updated

Additional information can be found in the documentation.

…ions and bulk updates

+ Added helper functions to detect repository type, find version matching https://raw.githubusercontent.com/FreshRSS/Extensions/master/extensions.json, download release, find release notes, sanitize extension files
+ Added translations for update actions
* change in base CSS to display icon and text without wrapping (Plus icon and install will wrap by default since the column is too small)
* Some additional error handling
+ Added check and notification if GitHub rate limit is hit
Comment on lines +360 to +369
if ($info === null) {
$info = [
'name' => $entry,
'entrypoint' => $entry,
'version' => Minz_Request::paramString('version'),
'url' => Minz_Request::paramString('url'),
'method' => '',
'directory' => Minz_Request::paramString('directory'),
];
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if ($info === null) {
$info = [
'name' => $entry,
'entrypoint' => $entry,
'version' => Minz_Request::paramString('version'),
'url' => Minz_Request::paramString('url'),
'method' => '',
'directory' => Minz_Request::paramString('directory'),
];
}
if ($info === null) {
Minz_Error::error(404);
return;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extensions that are not on the list shouldn't be allowed to be installed (since such option isn't exposed via UI)

### Manual Installation
For extensions that are not in the community list, or if you prefer to install them manually, you can place the extension files directly on your server.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list isn't that safe really. Updates aren't vetted, only the initial addition:

https://github.com/FreshRSS/Extensions/blob/2ca8d82b0868014d6289cb25d9758913f68a6895/generate.php
https://github.com/FreshRSS/Extensions/blob/2ca8d82b0868014d6289cb25d9758913f68a6895/.github/workflows/generate.yml

But if there's no UI option to input a URL then of course code to handle it ranges somewhere between not making sense and being a security risk.

Copy link
Member

@Inverle Inverle Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list isn't that safe really. Updates aren't vetted, only the initial addition:

I'm aware, maybe something can be done about this?
(if it's the administrator installing the extension and not an XSS payload, then maybe an extra warning before installing that the extension could be potentially unsafe[1] would be good enough?)

[1]: only when the extension is outside of https://github.com/FreshRSS/Extensions

But if there's no UI option to input a URL then of course code to handle it ranges somewhere between not making sense and being a security risk.

It requires slightly more effort to get RCE if the malicious extension has to be on the list.
Or the attacker can just rely on some vulnerable extension that's already in the list if any.

Copy link
Member

@Frenzie Frenzie Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm aware, maybe something can be done about this?

Unfortunately I don't think so. That would result in too much automated PR noise that nobody wants to deal with.[1] The only other truly functional model is how it was before, namely that an actual human being who cares makes a PR. But in that case you miss out on knowing about most updates. So it's pretty much a choice between how it was and how it is. There's a similar discussion over here btw, in case anyone comes up with something workable.

Perhaps Copilot could provide AI slop reviews of the changes, but the added value of such a thing is likely minimal if not counterproductive by giving a false sense of security.

[1] Though at the moment it seems okay, I suppose.

Comment on lines +109 to +111
<input type="hidden" name="url" value="<?= htmlspecialchars($ext['url']) ?>" />
<input type="hidden" name="version" value="<?= htmlspecialchars($ext['version']) ?>" />
<input type="hidden" name="directory" value="<?= htmlspecialchars($ext['directory']) ?>" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<input type="hidden" name="url" value="<?= htmlspecialchars($ext['url']) ?>" />
<input type="hidden" name="version" value="<?= htmlspecialchars($ext['version']) ?>" />
<input type="hidden" name="directory" value="<?= htmlspecialchars($ext['directory']) ?>" />

We probably don't need this if we only allow installing extensions from the list.

Comment on lines +340 to +342
if (!FreshRSS_Auth::hasAccess('admin')) {
Minz_Error::error(403);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!FreshRSS_Auth::hasAccess('admin')) {
Minz_Error::error(403);
}
if (!FreshRSS_Auth::hasAccess('admin')) {
Minz_Error::error(403);
return;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has the same effect, but just for clarity

Comment on lines +841 to +918
/**
* Removes potentially dangerous or unnecessary files from an extension directory.
*
* This function recursively scans for files and directories matching a list of
* patterns (e.g., VCS directories, temp files, config files) and deletes them.
*
* @param string $path Extension directory path.
*/
private function sanitizeExtension(string $path): void {
$patterns = [
// VCS and CI
'.git',
'.gitignore',
'.gitattributes',
'.gitmodules',
'.gitlab-ci.yml',
'.travis.yml',
'appveyor.yml',
'.github',
'.circleci',

// Development artifacts
'tests',
'test',
'spec',
'node_modules',
'.idea',
'.vscode',
'*.sublime*',
'composer.phar',
'yarn.lock',
'package-lock.json',
'*.log',

// Config and sensitive files
'.env*',
'*.key',
'*.pem',
'*.p12',
'*.pfx',
'*.crt',
'id_rsa*',
'id_dsa*',
'*.secret',
'*.credentials',

// OS and temporary files
'*.swp',
'*.swo',
'*~',
'*.bak',
'*.tmp',
'.DS_Store',
'Thumbs.db',
];

if (!is_dir($path)) {
return;
}

$iterator = new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($path, RecursiveDirectoryIterator::SKIP_DOTS),
RecursiveIteratorIterator::CHILD_FIRST
);

/** @var SplFileInfo $file */
foreach ($iterator as $file) {
$filename = $file->getFilename();
foreach ($patterns as $pattern) {
if (fnmatch($pattern, $filename)) {
Minz_Log::debug('Sanitizing: removing `' . $file->getPathname() . '` matching pattern `' . $pattern . '`');
recursive_unlink($file->getPathname());
// Once unlinked, continue to the next file in the iterator.
continue 2;
}
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this is necessary, it probably won't break any extensions, but I don't see the point.
It could remove files needed by the extension in some cases.

I think it's a bad assumption to make that required source code isn't also contained within test, tests or perhaps spec. Those directory names are somewhat generic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As someone who cloned gigantic .git (several GB) folders for a simple script with 10 lines of code as well as received more API keys / passwords than I can count in my career, it felt like the right thing to do. The list is certainly not perfect (and I already broke an extension as I had the vendor folder on the list 😄) but it's a starting point. I would prefer a curated list of updates with vetted releases, but that requires some manpower. Very few of the plugins use the GitHub / GitLab releases at all, and I have to download the full source of the plugin instead of a "cleaned" release .ZIP.

Copy link
Member

@Inverle Inverle Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As someone who cloned gigantic .git (several GB) folders for a simple script with 10 lines of code

Fair enough

The list is certainly not perfect (and I already broke an extension as I had the vendor folder on the list 😄)

Which is why it doesn't make sense to include generic names like the previously mentioned test, tests, spec or vendor

edit: was it this one? https://github.com/kapdap/freshrss-extensions/tree/master/xExtension-ClickableLinks

And getting rid of entire file extensions also seems like a little too much

But I'm okay with this part:

	// VCS and CI
			'.git',
			'.gitignore',
			'.gitattributes',
			'.gitmodules',
			'.gitlab-ci.yml',
			'.travis.yml',
			'appveyor.yml',
			'.github',
			'.circleci',

and

                        'node_modules',
			'.idea',
			'.vscode',
			'composer.phar',
			'yarn.lock',
			'package-lock.json',
			'.DS_Store',
			'Thumbs.db',

We should keep the list as small as possible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do a shallow clone in Git btw.

Comment on lines +1010 to +1069
/**
* Updates all installed extensions that have a newer version available.
*
* This action is restricted to administrators and must be called via a POST request.
*/
public function updateallAction(): void {
if (!FreshRSS_Auth::hasAccess('admin')) {
Minz_Error::error(403);
return;
}

$url_redirect = ['c' => 'extension', 'a' => 'index'];

if (!Minz_Request::isPost()) {
Minz_Request::forward($url_redirect, true);
return;
}

Minz_Log::notice('Starting batch extension update process.');

// Build the installed extensions list like in indexAction
$installedExtensions = [];
foreach (Minz_ExtensionManager::listExtensions() as $ext) {
$installedExtensions[$ext->getEntrypoint()] = $ext->getVersion();
}

$available = $this->getAvailableExtensionList();
$updatedCount = 0;
$failedCount = 0;
$failedNames = [];

foreach ($available as $ext) {
$entrypoint = $ext['entrypoint'];
if (
isset($installedExtensions[$entrypoint]) &&
version_compare($installedExtensions[$entrypoint], (string) $ext['version']) < 0
) {
Minz_Log::notice('Updating extension: ' . $ext['name'] . ' from v' . $installedExtensions[$entrypoint] . ' to v' . $ext['version']);

if ($this->downloadExtension($ext)) {
$updatedCount++;
} else {
$failedCount++;
$failedNames[] = $ext['name'];
}
}
}

if ($failedCount === 0 && $updatedCount > 0) {
Minz_Request::good(_t('feedback.extensions.update_all.ok', $updatedCount), $url_redirect);
} elseif ($updatedCount === 0 && $failedCount === 0) {
Minz_Request::good(_t('feedback.extensions.update_all.none'), $url_redirect);
} else {
$message = _t('feedback.extensions.update_all.partial', $updatedCount, $failedCount);
if (!empty($failedNames)) {
$message .= ' ' . _t('feedback.extensions.update_all.failed_list', implode(', ', $failedNames));
}
Minz_Request::bad($message, $url_redirect);
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the request timeout if there are a lot of extensions and the server is slow?
Maybe update progress should be shown via JS and each extension updated in individual requests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A timeout might happen. The updateallAction was more a quick addition than a finely planned out solution. In my testing, I never saw any issues, and most plugins are never / rarely updated and relatively small:

Extension Name Last Commit Repo Size
Af_Readability 2025-05-16 20:05:37 5.50 MB
Always togglable menu 2024-04-08 12:06:54 77.24 KB
ArticleSummary 2025-01-05 14:02:17 202.21 KB
Auto Refresh 2023-02-15 14:06:42 115.53 KB
AutoTTL 2024-10-25 11:44:24 163.42 KB
Black List 2024-11-14 06:31:33 502.17 KB
Clickable Links 2021-12-24 21:01:03 236.98 KB
Colorful List 2025-07-02 11:26:27 2.64 MB
Comics in feed 2024-11-04 23:39:21 102.35 KB
Copy2Clipboard 2024-11-14 06:31:33 502.17 KB
Date Format 2022-02-20 10:56:55 790.15 KB
Feed Priority Shortcut 2024-01-22 18:35:28 68.19 KB
FeedTitleBuilder 2024-11-14 06:31:33 500.08 KB
FilterTitle 2024-11-14 06:31:33 502.17 KB
Nav Menu 2017-03-27 11:02:23 8.30 MB
Fresh Vibes View 2025-07-02 07:35:16 1.06 MB
FreshRss FlareSolverr 2025-01-22 15:45:12 179.59 KB
GReader Redate 2023-12-24 22:20:18 47.79 KB
Image Cache 2025-02-18 12:16:02 454.47 KB
Image Proxy 2025-07-02 11:26:27 2.64 MB
Invidious Video Feed 2024-10-09 18:47:53 104.92 KB
Invidious Video Feed 2023-10-19 13:33:40 121.46 KB
Kagi Summarizer 2025-07-02 07:11:12 56.53 KB
Keep Folder State 2017-03-27 11:02:23 8.30 MB
LaTeX support 2024-03-30 13:56:21 3.07 MB
Mark Previous as Read 2025-03-27 22:52:28 88.97 KB
Mobile Scroll Menu 2017-03-27 11:02:23 8.30 MB
News Assistant 2025-05-18 10:15:19 161.10 KB
Pocket Button 2025-06-09 15:26:29 141.16 KB
Quick Collapse 2025-07-02 11:26:27 2.64 MB
RSS-Bridge 2023-03-21 11:32:44 128.34 KB
RateLimitFeeds 2025-06-15 09:25:14 53.86 KB
Readable 2025-01-16 12:40:35 120.95 KB
Readeck Button 2025-04-16 14:41:23 208.90 KB
ReadingTime 2025-07-02 11:26:27 2.64 MB
Reddit Image 2024-01-10 22:51:03 526.82 KB
Remove emojis 2024-11-14 06:31:33 502.17 KB
SendToMyJD2 2024-11-14 06:31:33 502.17 KB
Share By Email 2025-07-02 11:26:27 2.64 MB
ShowFeedID 2025-07-02 11:26:27 2.64 MB
Smart Mobile Menu 2017-03-27 11:02:23 8.30 MB
StarToPocket 2025-06-03 07:40:38 68.50 KB
Sticky Feeds 2025-07-02 11:26:27 2.64 MB
Theme Mode Synchronizer 2024-01-22 18:35:28 68.19 KB
ThreePanesView 2024-12-02 10:15:31 395.87 KB
Title-Wrap 2025-07-02 11:26:27 2.64 MB
Touch Control 2017-03-27 11:02:23 8.30 MB
TranslateTitlesCN 2024-11-21 18:30:45 1.56 MB
TwitchChannel2RssFeed 2024-04-17 01:47:10 38.66 KB
Wallabag Button 2025-01-23 21:50:55 222.92 KB
White List 2024-01-10 22:25:34 82.80 KB
Word highlighter 2025-07-02 11:26:27 2.64 MB
YouTube Video Feed 2025-07-02 11:26:27 2.64 MB
YouTubeChannel2RssFeed 2024-11-14 06:31:33 500.08 KB

I've considered it "good enough" for a first implementation, but this and the single update function could certainly be improved with regard to timeouts or batch processing with GUI updates. The code could be on a self-hosted GitLab / Gitea instance located on the moon, running on grandma's first computer for all we know.

Copy link
Member

@Inverle Inverle Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit date of extensions like YouTube Video Feed, Title-Wrap, ReadingTime etc. isn't correct because they are all in the same repository. (neither is the size)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit date of extensions like YouTube Video Feed, Title-Wrap, ReadingTime etc. isn't correct because they are all in the same repository. (neither is the size)

I'm aware of that. My point is that updating 5 extensions per 6 months with each being only a few KB / MB shouldn't be an issue (for now).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully not


$headers = [];
// GitHub API requires specific headers for API access.
if (str_contains($url, 'api.github.com')) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (str_contains($url, 'api.github.com')) {
if ($parsed['host'] === 'api.github.com') {

Comment on lines +762 to +767
private function downloadJson(string $url): ?array {
$ch = curl_init($url);
if ($ch === false) {
Minz_Log::warning('Failed to initialize cURL for: ' . $url);
return null;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The URL should be validated with checkUrl() from lib_rss.php

<?php } elseif (strval($this->extensions_installed[$ext['entrypoint']]) !== strval($ext['version'])) { ?>
<span class="alert alert-warn">
<?= _t('admin.extensions.update') ?>
<?= _t('admin.extensions.manage.update_available_detailed', $this->extensions_installed[$ext['entrypoint']], $ext['version']) ?>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<?= _t('admin.extensions.manage.update_available_detailed', $this->extensions_installed[$ext['entrypoint']], $ext['version']) ?>
<?= _t('admin.extensions.manage.update_available_detailed', htmlspecialchars($this->extensions_installed[$ext['entrypoint']], ENT_NOQUOTES, 'UTF-8'), htmlspecialchars($ext['version'], ENT_NOQUOTES, 'UTF-8')) ?>

<?php if (version_compare(strval($this->extensions_installed[$ext['entrypoint']]), strval($ext['version'])) >= 0) { ?>
<span class="alert alert-success">
<?= _t('admin.extensions.latest') ?>
<?= _t('admin.extensions.version_installed', $this->extensions_installed[$ext['entrypoint']]) ?>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<?= _t('admin.extensions.version_installed', $this->extensions_installed[$ext['entrypoint']]) ?>
<?= _t('admin.extensions.version_installed', htmlspecialchars($this->extensions_installed[$ext['entrypoint']], ENT_NOQUOTES, 'UTF-8')) ?>

} else {
$message = _t('feedback.extensions.update_all.partial', $updatedCount, $failedCount);
if (!empty($failedNames)) {
$message .= ' ' . _t('feedback.extensions.update_all.failed_list', implode(', ', $failedNames));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$message .= ' ' . _t('feedback.extensions.update_all.failed_list', implode(', ', $failedNames));
$message .= ' ' . _t('feedback.extensions.update_all.failed_list', htmlspecialchars(implode(', ', $failedNames), ENT_NOQUOTES, 'UTF-8'));

@Inverle
Copy link
Member

Inverle commented Jul 2, 2025

Since downloading new code onto the server is a very sensitive action, the administrator should be forced to reauthenticate when installing / updating an extension.

It should work similarly to the sudo mode on GitHub, as in that after reauthenticating once, you won't be asked to again for some time[1]

Additionally, an option to disable installing/updating extensions via Web UI in the data/config.php would help also for users with security concerns.

[1]: a 10 minute window of time should be enough

tryallthethings and others added 4 commits July 2, 2025 19:15
Co-authored-by: Inverle <inverle@proton.me>
Co-authored-by: Inverle <inverle@proton.me>
Co-authored-by: Inverle <inverle@proton.me>
Co-authored-by: Inverle <inverle@proton.me>
@tryallthethings
Copy link
Contributor Author

It should work similarly to the sudo mode on GitHub, as in that after reauthenticating once, you won't be asked to again for some time[1]

That's a great idea. But I suppose that this doesn't exist in FreshRSS yet? So we would need to add a timestamp to the user config with the last login / auth time. When the user wants to install an update, we check if the last_login_time + sudo-timeout > current time and redirect to the login page to do the auth and from there back to the extension page?

Additionally, an option to disable installing/updating extensions via Web UI in the data/config.php would help also for users with security concerns.

Also a good idea. I'd also add an option to define the timeout for requiring to re-auth for admin tasks.

And how do we continue from here? Should I accept / commit the proposed changes and test it again? Or is it on your to-do for the next release?

@Inverle
Copy link
Member

Inverle commented Jul 19, 2025

So we would need to add a timestamp to the user config with the last login / auth time. When the user wants to install an update, we check if the last_login_time + sudo-timeout > current time and redirect to the login page to do the auth and from there back to the extension page?

Maybe something like that but tied to the session instead.

Also a good idea. I'd also add an option to define the timeout for requiring to re-auth for admin tasks.

Yes, that should be an option.

And how do we continue from here? Should I accept / commit the proposed changes and test it again? Or is it on your to-do for the next release?

I may make a PR for the sudo mode soon.
As for the suggested changes, you can commit them if you don't see any issues.

note: suggested changes should be batched appropriately

@Inverle
Copy link
Member

Inverle commented Jul 25, 2025

@tryallthethings

I may make a PR for the sudo mode soon.

#7753

@Alkarex Alkarex modified the milestones: 1.27.0, 1.28.0 Aug 18, 2025
Alkarex pushed a commit that referenced this pull request Sep 14, 2025
It was supposed to be done in #7712 (comment), but that PR hasn't been yet finished for some time now.
@Alkarex Alkarex removed this from the 1.28.0 milestone Dec 16, 2025
@Alkarex Alkarex added this to the 1.29.0 milestone Dec 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants