Extension installation and updates via GUI#7712
Extension installation and updates via GUI#7712tryallthethings wants to merge 11 commits intoFreshRSS:edgefrom
Conversation
…ions and bulk updates + Added helper functions to detect repository type, find version matching https://raw.githubusercontent.com/FreshRSS/Extensions/master/extensions.json, download release, find release notes, sanitize extension files + Added translations for update actions * change in base CSS to display icon and text without wrapping (Plus icon and install will wrap by default since the column is too small)
…s/FreshRSS into extension-updater
* Some additional error handling + Added check and notification if GitHub rate limit is hit
| if ($info === null) { | ||
| $info = [ | ||
| 'name' => $entry, | ||
| 'entrypoint' => $entry, | ||
| 'version' => Minz_Request::paramString('version'), | ||
| 'url' => Minz_Request::paramString('url'), | ||
| 'method' => '', | ||
| 'directory' => Minz_Request::paramString('directory'), | ||
| ]; | ||
| } |
There was a problem hiding this comment.
| if ($info === null) { | |
| $info = [ | |
| 'name' => $entry, | |
| 'entrypoint' => $entry, | |
| 'version' => Minz_Request::paramString('version'), | |
| 'url' => Minz_Request::paramString('url'), | |
| 'method' => '', | |
| 'directory' => Minz_Request::paramString('directory'), | |
| ]; | |
| } | |
| if ($info === null) { | |
| Minz_Error::error(404); | |
| return; | |
| } |
There was a problem hiding this comment.
Extensions that are not on the list shouldn't be allowed to be installed (since such option isn't exposed via UI)
FreshRSS/docs/en/admins/15_extensions.md
Lines 24 to 26 in 1bf9bf3
There was a problem hiding this comment.
The list isn't that safe really. Updates aren't vetted, only the initial addition:
https://github.com/FreshRSS/Extensions/blob/2ca8d82b0868014d6289cb25d9758913f68a6895/generate.php
https://github.com/FreshRSS/Extensions/blob/2ca8d82b0868014d6289cb25d9758913f68a6895/.github/workflows/generate.yml
But if there's no UI option to input a URL then of course code to handle it ranges somewhere between not making sense and being a security risk.
There was a problem hiding this comment.
The list isn't that safe really. Updates aren't vetted, only the initial addition:
I'm aware, maybe something can be done about this?
(if it's the administrator installing the extension and not an XSS payload, then maybe an extra warning before installing that the extension could be potentially unsafe[1] would be good enough?)
[1]: only when the extension is outside of https://github.com/FreshRSS/Extensions
But if there's no UI option to input a URL then of course code to handle it ranges somewhere between not making sense and being a security risk.
It requires slightly more effort to get RCE if the malicious extension has to be on the list.
Or the attacker can just rely on some vulnerable extension that's already in the list if any.
There was a problem hiding this comment.
I'm aware, maybe something can be done about this?
Unfortunately I don't think so. That would result in too much automated PR noise that nobody wants to deal with.[1] The only other truly functional model is how it was before, namely that an actual human being who cares makes a PR. But in that case you miss out on knowing about most updates. So it's pretty much a choice between how it was and how it is. There's a similar discussion over here btw, in case anyone comes up with something workable.
Perhaps Copilot could provide AI slop reviews of the changes, but the added value of such a thing is likely minimal if not counterproductive by giving a false sense of security.
[1] Though at the moment it seems okay, I suppose.
| <input type="hidden" name="url" value="<?= htmlspecialchars($ext['url']) ?>" /> | ||
| <input type="hidden" name="version" value="<?= htmlspecialchars($ext['version']) ?>" /> | ||
| <input type="hidden" name="directory" value="<?= htmlspecialchars($ext['directory']) ?>" /> |
There was a problem hiding this comment.
| <input type="hidden" name="url" value="<?= htmlspecialchars($ext['url']) ?>" /> | |
| <input type="hidden" name="version" value="<?= htmlspecialchars($ext['version']) ?>" /> | |
| <input type="hidden" name="directory" value="<?= htmlspecialchars($ext['directory']) ?>" /> |
We probably don't need this if we only allow installing extensions from the list.
| if (!FreshRSS_Auth::hasAccess('admin')) { | ||
| Minz_Error::error(403); | ||
| } |
There was a problem hiding this comment.
| if (!FreshRSS_Auth::hasAccess('admin')) { | |
| Minz_Error::error(403); | |
| } | |
| if (!FreshRSS_Auth::hasAccess('admin')) { | |
| Minz_Error::error(403); | |
| return; | |
| } |
There was a problem hiding this comment.
Has the same effect, but just for clarity
| /** | ||
| * Removes potentially dangerous or unnecessary files from an extension directory. | ||
| * | ||
| * This function recursively scans for files and directories matching a list of | ||
| * patterns (e.g., VCS directories, temp files, config files) and deletes them. | ||
| * | ||
| * @param string $path Extension directory path. | ||
| */ | ||
| private function sanitizeExtension(string $path): void { | ||
| $patterns = [ | ||
| // VCS and CI | ||
| '.git', | ||
| '.gitignore', | ||
| '.gitattributes', | ||
| '.gitmodules', | ||
| '.gitlab-ci.yml', | ||
| '.travis.yml', | ||
| 'appveyor.yml', | ||
| '.github', | ||
| '.circleci', | ||
|
|
||
| // Development artifacts | ||
| 'tests', | ||
| 'test', | ||
| 'spec', | ||
| 'node_modules', | ||
| '.idea', | ||
| '.vscode', | ||
| '*.sublime*', | ||
| 'composer.phar', | ||
| 'yarn.lock', | ||
| 'package-lock.json', | ||
| '*.log', | ||
|
|
||
| // Config and sensitive files | ||
| '.env*', | ||
| '*.key', | ||
| '*.pem', | ||
| '*.p12', | ||
| '*.pfx', | ||
| '*.crt', | ||
| 'id_rsa*', | ||
| 'id_dsa*', | ||
| '*.secret', | ||
| '*.credentials', | ||
|
|
||
| // OS and temporary files | ||
| '*.swp', | ||
| '*.swo', | ||
| '*~', | ||
| '*.bak', | ||
| '*.tmp', | ||
| '.DS_Store', | ||
| 'Thumbs.db', | ||
| ]; | ||
|
|
||
| if (!is_dir($path)) { | ||
| return; | ||
| } | ||
|
|
||
| $iterator = new RecursiveIteratorIterator( | ||
| new RecursiveDirectoryIterator($path, RecursiveDirectoryIterator::SKIP_DOTS), | ||
| RecursiveIteratorIterator::CHILD_FIRST | ||
| ); | ||
|
|
||
| /** @var SplFileInfo $file */ | ||
| foreach ($iterator as $file) { | ||
| $filename = $file->getFilename(); | ||
| foreach ($patterns as $pattern) { | ||
| if (fnmatch($pattern, $filename)) { | ||
| Minz_Log::debug('Sanitizing: removing `' . $file->getPathname() . '` matching pattern `' . $pattern . '`'); | ||
| recursive_unlink($file->getPathname()); | ||
| // Once unlinked, continue to the next file in the iterator. | ||
| continue 2; | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
I'm not sure if this is necessary, it probably won't break any extensions, but I don't see the point.
It could remove files needed by the extension in some cases.
I think it's a bad assumption to make that required source code isn't also contained within test, tests or perhaps spec. Those directory names are somewhat generic.
There was a problem hiding this comment.
As someone who cloned gigantic .git (several GB) folders for a simple script with 10 lines of code as well as received more API keys / passwords than I can count in my career, it felt like the right thing to do. The list is certainly not perfect (and I already broke an extension as I had the vendor folder on the list 😄) but it's a starting point. I would prefer a curated list of updates with vetted releases, but that requires some manpower. Very few of the plugins use the GitHub / GitLab releases at all, and I have to download the full source of the plugin instead of a "cleaned" release .ZIP.
There was a problem hiding this comment.
As someone who cloned gigantic .git (several GB) folders for a simple script with 10 lines of code
Fair enough
The list is certainly not perfect (and I already broke an extension as I had the vendor folder on the list 😄)
Which is why it doesn't make sense to include generic names like the previously mentioned test, tests, spec or vendor
edit: was it this one? https://github.com/kapdap/freshrss-extensions/tree/master/xExtension-ClickableLinks
And getting rid of entire file extensions also seems like a little too much
But I'm okay with this part:
// VCS and CI
'.git',
'.gitignore',
'.gitattributes',
'.gitmodules',
'.gitlab-ci.yml',
'.travis.yml',
'appveyor.yml',
'.github',
'.circleci',and
'node_modules',
'.idea',
'.vscode',
'composer.phar',
'yarn.lock',
'package-lock.json', '.DS_Store',
'Thumbs.db',We should keep the list as small as possible.
There was a problem hiding this comment.
You can do a shallow clone in Git btw.
| /** | ||
| * Updates all installed extensions that have a newer version available. | ||
| * | ||
| * This action is restricted to administrators and must be called via a POST request. | ||
| */ | ||
| public function updateallAction(): void { | ||
| if (!FreshRSS_Auth::hasAccess('admin')) { | ||
| Minz_Error::error(403); | ||
| return; | ||
| } | ||
|
|
||
| $url_redirect = ['c' => 'extension', 'a' => 'index']; | ||
|
|
||
| if (!Minz_Request::isPost()) { | ||
| Minz_Request::forward($url_redirect, true); | ||
| return; | ||
| } | ||
|
|
||
| Minz_Log::notice('Starting batch extension update process.'); | ||
|
|
||
| // Build the installed extensions list like in indexAction | ||
| $installedExtensions = []; | ||
| foreach (Minz_ExtensionManager::listExtensions() as $ext) { | ||
| $installedExtensions[$ext->getEntrypoint()] = $ext->getVersion(); | ||
| } | ||
|
|
||
| $available = $this->getAvailableExtensionList(); | ||
| $updatedCount = 0; | ||
| $failedCount = 0; | ||
| $failedNames = []; | ||
|
|
||
| foreach ($available as $ext) { | ||
| $entrypoint = $ext['entrypoint']; | ||
| if ( | ||
| isset($installedExtensions[$entrypoint]) && | ||
| version_compare($installedExtensions[$entrypoint], (string) $ext['version']) < 0 | ||
| ) { | ||
| Minz_Log::notice('Updating extension: ' . $ext['name'] . ' from v' . $installedExtensions[$entrypoint] . ' to v' . $ext['version']); | ||
|
|
||
| if ($this->downloadExtension($ext)) { | ||
| $updatedCount++; | ||
| } else { | ||
| $failedCount++; | ||
| $failedNames[] = $ext['name']; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if ($failedCount === 0 && $updatedCount > 0) { | ||
| Minz_Request::good(_t('feedback.extensions.update_all.ok', $updatedCount), $url_redirect); | ||
| } elseif ($updatedCount === 0 && $failedCount === 0) { | ||
| Minz_Request::good(_t('feedback.extensions.update_all.none'), $url_redirect); | ||
| } else { | ||
| $message = _t('feedback.extensions.update_all.partial', $updatedCount, $failedCount); | ||
| if (!empty($failedNames)) { | ||
| $message .= ' ' . _t('feedback.extensions.update_all.failed_list', implode(', ', $failedNames)); | ||
| } | ||
| Minz_Request::bad($message, $url_redirect); | ||
| } | ||
| } |
There was a problem hiding this comment.
Could the request timeout if there are a lot of extensions and the server is slow?
Maybe update progress should be shown via JS and each extension updated in individual requests
There was a problem hiding this comment.
A timeout might happen. The updateallAction was more a quick addition than a finely planned out solution. In my testing, I never saw any issues, and most plugins are never / rarely updated and relatively small:
| Extension Name | Last Commit | Repo Size |
|---|---|---|
| Af_Readability | 2025-05-16 20:05:37 | 5.50 MB |
| Always togglable menu | 2024-04-08 12:06:54 | 77.24 KB |
| ArticleSummary | 2025-01-05 14:02:17 | 202.21 KB |
| Auto Refresh | 2023-02-15 14:06:42 | 115.53 KB |
| AutoTTL | 2024-10-25 11:44:24 | 163.42 KB |
| Black List | 2024-11-14 06:31:33 | 502.17 KB |
| Clickable Links | 2021-12-24 21:01:03 | 236.98 KB |
| Colorful List | 2025-07-02 11:26:27 | 2.64 MB |
| Comics in feed | 2024-11-04 23:39:21 | 102.35 KB |
| Copy2Clipboard | 2024-11-14 06:31:33 | 502.17 KB |
| Date Format | 2022-02-20 10:56:55 | 790.15 KB |
| Feed Priority Shortcut | 2024-01-22 18:35:28 | 68.19 KB |
| FeedTitleBuilder | 2024-11-14 06:31:33 | 500.08 KB |
| FilterTitle | 2024-11-14 06:31:33 | 502.17 KB |
| Nav Menu | 2017-03-27 11:02:23 | 8.30 MB |
| Fresh Vibes View | 2025-07-02 07:35:16 | 1.06 MB |
| FreshRss FlareSolverr | 2025-01-22 15:45:12 | 179.59 KB |
| GReader Redate | 2023-12-24 22:20:18 | 47.79 KB |
| Image Cache | 2025-02-18 12:16:02 | 454.47 KB |
| Image Proxy | 2025-07-02 11:26:27 | 2.64 MB |
| Invidious Video Feed | 2024-10-09 18:47:53 | 104.92 KB |
| Invidious Video Feed | 2023-10-19 13:33:40 | 121.46 KB |
| Kagi Summarizer | 2025-07-02 07:11:12 | 56.53 KB |
| Keep Folder State | 2017-03-27 11:02:23 | 8.30 MB |
| LaTeX support | 2024-03-30 13:56:21 | 3.07 MB |
| Mark Previous as Read | 2025-03-27 22:52:28 | 88.97 KB |
| Mobile Scroll Menu | 2017-03-27 11:02:23 | 8.30 MB |
| News Assistant | 2025-05-18 10:15:19 | 161.10 KB |
| Pocket Button | 2025-06-09 15:26:29 | 141.16 KB |
| Quick Collapse | 2025-07-02 11:26:27 | 2.64 MB |
| RSS-Bridge | 2023-03-21 11:32:44 | 128.34 KB |
| RateLimitFeeds | 2025-06-15 09:25:14 | 53.86 KB |
| Readable | 2025-01-16 12:40:35 | 120.95 KB |
| Readeck Button | 2025-04-16 14:41:23 | 208.90 KB |
| ReadingTime | 2025-07-02 11:26:27 | 2.64 MB |
| Reddit Image | 2024-01-10 22:51:03 | 526.82 KB |
| Remove emojis | 2024-11-14 06:31:33 | 502.17 KB |
| SendToMyJD2 | 2024-11-14 06:31:33 | 502.17 KB |
| Share By Email | 2025-07-02 11:26:27 | 2.64 MB |
| ShowFeedID | 2025-07-02 11:26:27 | 2.64 MB |
| Smart Mobile Menu | 2017-03-27 11:02:23 | 8.30 MB |
| StarToPocket | 2025-06-03 07:40:38 | 68.50 KB |
| Sticky Feeds | 2025-07-02 11:26:27 | 2.64 MB |
| Theme Mode Synchronizer | 2024-01-22 18:35:28 | 68.19 KB |
| ThreePanesView | 2024-12-02 10:15:31 | 395.87 KB |
| Title-Wrap | 2025-07-02 11:26:27 | 2.64 MB |
| Touch Control | 2017-03-27 11:02:23 | 8.30 MB |
| TranslateTitlesCN | 2024-11-21 18:30:45 | 1.56 MB |
| TwitchChannel2RssFeed | 2024-04-17 01:47:10 | 38.66 KB |
| Wallabag Button | 2025-01-23 21:50:55 | 222.92 KB |
| White List | 2024-01-10 22:25:34 | 82.80 KB |
| Word highlighter | 2025-07-02 11:26:27 | 2.64 MB |
| YouTube Video Feed | 2025-07-02 11:26:27 | 2.64 MB |
| YouTubeChannel2RssFeed | 2024-11-14 06:31:33 | 500.08 KB |
I've considered it "good enough" for a first implementation, but this and the single update function could certainly be improved with regard to timeouts or batch processing with GUI updates. The code could be on a self-hosted GitLab / Gitea instance located on the moon, running on grandma's first computer for all we know.
There was a problem hiding this comment.
The commit date of extensions like YouTube Video Feed, Title-Wrap, ReadingTime etc. isn't correct because they are all in the same repository. (neither is the size)
There was a problem hiding this comment.
The commit date of extensions like YouTube Video Feed, Title-Wrap, ReadingTime etc. isn't correct because they are all in the same repository. (neither is the size)
I'm aware of that. My point is that updating 5 extensions per 6 months with each being only a few KB / MB shouldn't be an issue (for now).
|
|
||
| $headers = []; | ||
| // GitHub API requires specific headers for API access. | ||
| if (str_contains($url, 'api.github.com')) { |
There was a problem hiding this comment.
| if (str_contains($url, 'api.github.com')) { | |
| if ($parsed['host'] === 'api.github.com') { |
| private function downloadJson(string $url): ?array { | ||
| $ch = curl_init($url); | ||
| if ($ch === false) { | ||
| Minz_Log::warning('Failed to initialize cURL for: ' . $url); | ||
| return null; | ||
| } |
There was a problem hiding this comment.
The URL should be validated with checkUrl() from lib_rss.php
| <?php } elseif (strval($this->extensions_installed[$ext['entrypoint']]) !== strval($ext['version'])) { ?> | ||
| <span class="alert alert-warn"> | ||
| <?= _t('admin.extensions.update') ?> | ||
| <?= _t('admin.extensions.manage.update_available_detailed', $this->extensions_installed[$ext['entrypoint']], $ext['version']) ?> |
There was a problem hiding this comment.
| <?= _t('admin.extensions.manage.update_available_detailed', $this->extensions_installed[$ext['entrypoint']], $ext['version']) ?> | |
| <?= _t('admin.extensions.manage.update_available_detailed', htmlspecialchars($this->extensions_installed[$ext['entrypoint']], ENT_NOQUOTES, 'UTF-8'), htmlspecialchars($ext['version'], ENT_NOQUOTES, 'UTF-8')) ?> |
| <?php if (version_compare(strval($this->extensions_installed[$ext['entrypoint']]), strval($ext['version'])) >= 0) { ?> | ||
| <span class="alert alert-success"> | ||
| <?= _t('admin.extensions.latest') ?> | ||
| <?= _t('admin.extensions.version_installed', $this->extensions_installed[$ext['entrypoint']]) ?> |
There was a problem hiding this comment.
| <?= _t('admin.extensions.version_installed', $this->extensions_installed[$ext['entrypoint']]) ?> | |
| <?= _t('admin.extensions.version_installed', htmlspecialchars($this->extensions_installed[$ext['entrypoint']], ENT_NOQUOTES, 'UTF-8')) ?> |
| } else { | ||
| $message = _t('feedback.extensions.update_all.partial', $updatedCount, $failedCount); | ||
| if (!empty($failedNames)) { | ||
| $message .= ' ' . _t('feedback.extensions.update_all.failed_list', implode(', ', $failedNames)); |
There was a problem hiding this comment.
| $message .= ' ' . _t('feedback.extensions.update_all.failed_list', implode(', ', $failedNames)); | |
| $message .= ' ' . _t('feedback.extensions.update_all.failed_list', htmlspecialchars(implode(', ', $failedNames), ENT_NOQUOTES, 'UTF-8')); |
|
Since downloading new code onto the server is a very sensitive action, the administrator should be forced to reauthenticate when installing / updating an extension. It should work similarly to the sudo mode on GitHub, as in that after reauthenticating once, you won't be asked to again for some time[1] Additionally, an option to disable installing/updating extensions via Web UI in the [1]: a 10 minute window of time should be enough |
Co-authored-by: Inverle <inverle@proton.me>
Co-authored-by: Inverle <inverle@proton.me>
Co-authored-by: Inverle <inverle@proton.me>
Co-authored-by: Inverle <inverle@proton.me>
That's a great idea. But I suppose that this doesn't exist in FreshRSS yet? So we would need to add a timestamp to the user config with the last login / auth time. When the user wants to install an update, we check if the last_login_time + sudo-timeout > current time and redirect to the login page to do the auth and from there back to the extension page?
Also a good idea. I'd also add an option to define the timeout for requiring to re-auth for admin tasks. And how do we continue from here? Should I accept / commit the proposed changes and test it again? Or is it on your to-do for the next release? |
Maybe something like that but tied to the session instead.
Yes, that should be an option.
I may make a PR for the sudo mode soon. note: suggested changes should be batched appropriately |
|
It was supposed to be done in #7712 (comment), but that PR hasn't been yet finished for some time now.
Feel free to make changes. The only issue I've found is that hitting the GitHub rate limit is an option when testing 😄
Closes
#2900
FreshRSS/Extensions/issues/323
Changes proposed in this pull request:
How to test the feature manually:
Pull request checklist:
Additional information can be found in the documentation.