[4.0] Joomla Update Silent fails#36492
Conversation
I am not 100% certain that this is the best approach to this. @nikosdion @wilsonge please advise ## The Problem As reported in # if during an update a sql query fails then the updater quits the sql part of the update process and moves on. The joomla_update log gives no indication that there has been a problem etc. ## Proposed Partial Solution Adds a Start SQL and End SQL to the logs If there is an error in the sql then that is written to the logs as well AND it now says End SQL incomplete ### Testing For the testing we will be using the "Update your site by manually uploading the update package." and to make things quick I have prepared an empty update that you should download and use. ### Before Test 1 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### Before Test 2 Update your database to change the value in the #__schemas table so that the version_id is 4.0.0-2018-5-15 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### Before Test 3 Update your database to change the value in the #__schemas table so that the version_id is 4.0.3 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### After - Test 1 Update your database to change the value in the #__schemas table so that the version_id is 4.0.4 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### After - Test 2 Update your database to change the value in the #__schemas table so that the version_id is 4.0.0-2018-5-15 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### Before Test 3 Update your database to change the value in the #__schemas table so that the version_id is 4.0.3 Update your install and then check the `administrator\logs\joomla_update.php` It should look like
|
@brianteeman You haven't asked me for review but I noticed the PR and had a quick look. From code reading I think it should work, and I think it's an improvement, so ok for me. |
|
@richard67 Everyone is welcome to review. I just asked Nik and George specifically as they have the most intimate knowledge of the installer |
|
@brianteeman I'd count myself too to those who have such knowledge, at least to some large part. |
|
It's true that @richard67 has accrued a wealth of knowledge about the internal workings of the installer by fixing everyone else's broken crap (including mine) and working on the UTF8MB4 integration. @brianteeman you can tag him for installer and database issues, he's one of the few people who have this kind of intimate knowledge. |
|
My apologies no slight or offence intended |
|
OK, back to the PR. I really like what it's doing and I fully agree with the intention and the methodology. I would add only one more thing: log the Side note: The fact that the
We fix that and we try the update again. The update fails because the first file (1.0.1-202111011234) will be executed again. However, its changes are already applied and the ALTER TABLE commands it contains fail. There is no way to address this issue unless a. you uninstall the extension (and lose all data in it!) or b. edit the My idea is that as a user you should NOT have to do that. It's not your responsibility to keep track of minutiae like that. Joomla itself should know it has already executed 1.0.1-202111011234 and refrain from executing it again. The simple way to do that is add an update to the If @richard67 and @wilsonge agree I could help @brianteeman do that. It would make it SO MUCH EASIER for us 3PDs to provide support on failed updates without having us to ask for access to the client's database server or provide them with doctored installation ZIP files. |
I will take a look at adding that Files and foldersJust seen that the code already exists to log the actual files and folders removed. @PhilETaylor added it #31943 so if we want to add that its simple to do. Still ignores the bigger issue for me is that if the sql part fails with an error then we go directly to the success message and skip the files/folders altogether |
Log the schema version at the beginning it ok, but lok all the skipped update SQL because they are older than this schema version is redundant information, I think.
...
This should indeed be fixed. |
|
updated to show schema version at the start |
I think the general approach of using the log category is fine. Having said that it might need a bit more nuance - I think with this change removing any sort of message shown to a user when updating an extension (as this code is shared there) https://github.com/joomla/joomla-cms/pull/36492/files#diff-b99bfdb7da0a1b88f83ba167cfd61b61b3ee3f3875ba5e252258cb761b9bbfb1L1209 Obviously the end intention is correct to log this stuff when we're updating Joomla. Just need to make sure we don't make the extension update pathway worse in the process.
Obviously this sounds very logical and useful. I'm happy.
https://github.com/joomla/joomla-cms/blob/4.0-dev/administrator/components/com_joomlaupdate/src/Model/UpdateModel.php#L794-L802 So what should happen in this scenario is we run a full rollback. However because this is core we don't really have a rollback path. I guess I listen to @nikosdion as the expert here but I guess in the case here of upgrading core maybe attempting a rollback is probably not such a great idea as it is for extension as we won't rollback the files (and even if we did we'd have no way to remove any new files that had been added in the new release). |
Which is the main reason I asked for input as I didnt even know that AND now that I do it doesn't really help as I have almost zero experience with developing extensions |
|
@richard67 You said
No, it's not. When dealing with unstable versions (dev, alpha, beta, ...) PHP has a very weird and specific way of handling version_compare(). It's easy to mess it up. I did it, it took me 10' to realise that my problem was that I didn't think through how version_compare() actually works. If I had that in a log it would have taken me 1' instead 😜
IMHO we should improve the message logged to the user (instead of removing it), saying which file and SQL statement failed. Otherwise it's as good as not saying anything. This has frustrated me to no end trying to debug my own extension updates now that I am back into using the core database schema updates.
I'll do a PR to Brian's PR then! Awesome! This thing has bugged me for years.
Oof... I'll try to be as concise as possible but you'll have to bear with me. The TL;DR is that indeed it's not a good idea attempting a full rollback. Restoring from a backup will always yield better results which is why we suggest that. The only thing we can improve is communicating WHY the upgrade failed so that next time the user doesn't run into the exact same problem. And now, to the technical nitty–gritty. Updating Joomla is unlike every other extension. First of all, while Joomla registers itself as a files extension (so that SQL updates can execute) it is not installed as a files extension update. While that was the way it worked back in Joomla 1.6 and 1.7, it quickly became apparent that many shared hosting environment were so darned slow that unpacking the files would time out. The solution to that was Joomla! Update which runs the update in three stages:
Performing a full rollback would require maintaining a copy of the files which were replaced and a log of the new files created. Filesystem rollback would consist of copying the backups of the copied files and removing the new files. (Note that for extensions it's just a matter of copying over entire folders. There are fewer files and it doesn't time out. For Joomla itself it's not possible, it would be equivalent to taking a backup of the site on the same server which is likely to time out or have us run out of disk space). Likewise, performing a full rollback would require rolling back database changes. This is where things get VERY hairy. At the very least we'd need to have anti–update SQL files for each update SQL file we have. This means going back and doing that for every little update file which is a mammoth task, not guaranteed to succeed. We'd also need to know at which point to stop the rollback i.e. not roll back to a version before the one we started upgrading from. That's possible but convoluted. Even if we have that, there are a few questions left to answer. What happens when we drop an old table? We obviously cannot restore it. Instead of dropping tables we'd have to rename them to a temporary name and at the end of the post–update code drop them (if the update was successful) or rename them back (if the update failed). This is complicated and means we can no longer use the same update code as extensions OR we have to introduce a hard b/c break for extension updates. Damned if you do, damned if you don't kind of situation. Even worse, this kind of full rollback may actually take so long that the process times out and we have an inconsistent state of the site. I would actually say that on most servers this is exactly what will happen. In other words, there are no guarantees that we can ever perform a full rollback to a consistent state or at the very least detect when things are broken beyond our ability to fix them. Incidentally, the latter state is exactly what will happen with a failed update. The best advice you can give to users is to take a backup BEFORE updating so they can restore their site if the update fails. We already do that. Moreover, we have to tell them why the update failed so they can fix that before they retry the update. This is something that didn't use to happen but was largely fixed in 4.0.4 with the new Joomla Update backend code I contributed. It will tell you why it broke and how to fix it. The only thing missing is communicating correctly the reason why the database part of the update failed. The logging added here should help tons. Beyond that, I don't know what else we could do. But, no, a full rollback is not practical unless we decide that the only way to do a Joomla Update is via the command line which makes Joomla useless for the majority of its target audience. And even then we can't really be 100% sure that the rollback will succeed. Continuing on the same train of thought and since it's something we had started discussing back in 2015 at the J4WG, no, having a schema updater a la Doctrine won't help either. I've used that with Python (managing the updates on a Weblate installation) and I found that they are WORSE in the messy real world than Joomla's SQL update files. They can't do rollback any more than Joomla's SQL update files currently do (that is to say: not at all!) and they are even more sensitive to starting conditions. If the schema you start with does not match exactly the previous schema version's definition you're crap outta luck. You need to restore from a backup, make manual DB changes and hope it works this time. This means deciphering the metalanguage used to describe the schema and figuring out how it would be translated to the actual database schema. This works great if you have a fleet of servers with tightly controlled updates, it does not work in the messy real world with end users performing their own updates. That's why I was so against a DBAL for handling schema updates. |
this is my motivation for doing this as it really doesnt tell you what broke and it lies to you and says it is successful |
@brianteeman Well, as people very likely did lie before when they checked the "I have made a backup ..." check box, I think that's ok 😄 (sorry for silly joke) |
|
|
|
I think we should do the update of the schema version in database after each SQL script in a separate PR and let this one here handle the log messages and other messages, i.e. not try to make this one here solve everything because that could be too much in one PR. |
|
For testing it is just sufficient to add some SQL which results in an error to one of the last update SQL (not the early ones because if those break then almost all is broken), e.g. a drop table statement for a table which does not exist. That should be enough to cause a failure at the desired point in an update SQL script. |
|
E.g. if we put the statement which causes an error into script "4.0.3-2021-09-04.sql", then script "4.0.3-2021-09-05.sql" will not run. The last successfully run script would then be "4.0.0-2021-08-17.sql" when updating anything older that 4.0.3, with nothing happening when updating a 4.0.0 or 4.0.1 or 4.0.2 so for testing one should update from 3.10 or a 4.0.0-beta-x, and "4.0.0-2021-08-17" should be the schema version in DB after that. |
OK. I can do that tomorrow morning. I need to finish up some stuff for my own software today — and I am taking it slow and easy as I am technically on vacation :) |
|
@richard67 my test instructions already simulate the errors |
|
@nikosdion The tricky thing could be - if I remember right - that at the end of the update when it thinks it was successful, there is already an update of the schema version in the DB, and if I remember right it just sets the schema version according to the latest present file, regardless if that has been run with success or not. This could then make you trouble and at the end overwrite the result which your new functionality will have written. I thought I let you know in case if you run into this when testing your changes. |
@brianteeman That's right. I just thought I could simplify. |
|
@richard67 The trick is that we'd be removing this code since we update the schema version with the latest actually installed SQL update file. From a 3PD's perspective, this means that isolating each atomic change into its file guarantees that failures are recoverable. It is indeed one of the things I already have in mind. I even have a test fake component to test what happens with borked updates. I had built that to know for certain how Joomla behaves so I know how to provide support. I just need to shape it up so it can be used for testing the PR. |
@nikosdion I understood that. In fact I wanted to make that since some 2 years. I just thought I should mention that we need to remove a piece of code somewhere. |
|
@brianteeman I assume right that the last section header in the testing instructions should be "After Test 3" and not "Before Test 3"? |
|
yes - typo - now corrected |
|
@brianteeman I've found 2 glitches (one already in 4.x-dev and another one in this PR) and made a PR for you with a fix. Please check brianteeman#155 . Thanks in advance. @wilsonge The 2nd glitch is the one which you and me had expected, that the warning alert shown in backend will get lost with this PR here as it is now. |
* Fix exception type * Add back error alert
@brianteeman Cool, the new logs. Really a good improvement, together with the fix of the wrong exception type. |
|
I have tested this item ✅ successfully on dcf24bd This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/36492. |
|
@wilsonge I think this is a release blocker and has to go into 4.0.6 since it fixes also the wrong exception type, see screenshots in brianteeman#155 . |
|
Thanks! |
|
thanks |
* [4.0] Backup message (#36487) Many users who use their hosts backup facility and this is typically __only__ for files This update reminds them to backup the database as well * Remove reference & in usort callback function (#36503) * Update paragonie sodium_compat to 1.17.0 (#36491) Co-authored-by: Benjamin Trenkle <bembelimen@users.noreply.github.com> * [3.10] Finder plugins modify component params registry / Backport #35855 (#36141) * Backport #35855 to 3.10 * one space to much * [4.0] xml code style (#36532) Fixes a few spaces to tabs issues * [4.0] media manager breadcrumbs (#36154) * [4.0] media manager breadcrumbs The RTL specific css was wrong as can be seen in the before screenshot This PR removes the RTL specific css (should have been scss) and updates the scss to use logical properties instead. This PR also removes an unused class `breadcrumb-item`. To test RTL you will need to install arabic or persian. and dont forget to npm ci There is no visible change in LTR and the RTL changes are as shown below ## Before LTR ## Before RTL ## After LTR ## After RTL * revert * [4.0] RTL: Correcting Admin login 2factor and feedback hints (#36024) This is a replacement to the merged PR #32104 It does exactly the same thing but by using css logical properties we avoid the need to maintain both an LTR and an RTL version[4.0] There is no visual change * [4.0] Category list (#36536) Remove extra div that is not needed * [4.0] Make routing work in console applications (#36332) * Update list-fancy-select.php (#36543) replace tab with space code review * [4.0] archive icon (#36263) All the state icons are grey so that the published icon, which is green, stands out visually. The archived icon however was black. This PR changes that icon to grey. ### Before ### After * [4.0] Banner Client tips (#36262) * [4.0] Banner Client tips Adds a tooltip to the banner client numbers so that they appear the same as category numbers. Also corrects the class on the trashed number so that it has the correct colour To test Create one banner client Create a banner for the client in each of the published states * Update administrator/components/com_banners/tmpl/clients/default.php oops Co-authored-by: Quy <quy@fluxbb.org> Co-authored-by: Quy <quy@fluxbb.org> * Fix article creation from frontend (#36542) * [4.0] Handle when toggle icon is not available (#36329) * Handle when toggle icon is not available * cs * Take workflow extension into account when setting a default (#36527) * Fix uploadlimits checks (#36023) * Make edit_lock layout reusable for other extensions (#35112) * Allow to set a value for SubformField (#35921) * [4.0] Fix example in cleanImageURL docblock (#36554) * Fix Codemirror behind tabs (#36140) * Fix Codemirror behind tabs * jscs * Remove dependency on Factory in AdminController * [4.0] Ensure path contains api in the router (#36315) * [4.0] Check model is set before calling its methods (#36378) * Check model is set before calling its methods * Update OverridesController.php * Category edit form should use return url (#36560) Currently the article edit form respects return URLs; the category edit form does not. Adding it here makes for a more consistent editing experience. * [4.0] Sample Data plugin images (#36237) Updates the plugin to use the new image data used in lazy loading etc. This avoids confusion for users who see different source code for their created articles when compared to the same data plugins To test. Just make sure the sample data still installs and works as expected * [4.0] If the Mail Template was not found in the db, return false (#36489) * [4.0] Code Style (#36564) * Use an absolute path in image list field (#36063) * Correct Cache setWorkarounds to take account of multiple script or style declarations being used that mean that we have an array of scripts or stylesheets to be cached and not just a string (#36068) Cache setWorkarounds returns an array not a string * [3] privacy request by logged in user only (#35470) * privacy request logged in only * privacy request logged in only * privacy request logged in only * privacy request logged in only * privacy request logged in only * Update components/com_privacy/controller.php Co-authored-by: Quy <quy@fluxbb.org> Co-authored-by: Quy <quy@fluxbb.org> * Feature - Add always count option to 404 redirects (#34640) * Feature - Add always count option to 404 redirects * Add missiing space for code sniffer * Return line for if statement * Update plugins/system/redirect/redirect.php Co-authored-by: Quy <quy@fluxbb.org> * Alpha Ordering - Change Constant too. * update constant * Update en-GB.plg_system_redirect.ini * remove trailing whtiespace * Remove Added Language Strings * Update plugins/system/redirect/redirect.php Co-authored-by: Phil E. Taylor <phil@phil-taylor.com> * Update redirect.xml * Update redirect.php * Update plugins/system/redirect/redirect.php Co-authored-by: Quy <quy@fluxbb.org> * Update plugins/system/redirect/redirect.php Co-authored-by: Benjamin Trenkle <bembelimen@users.noreply.github.com> * Update plugins/system/redirect/redirect.php Co-authored-by: Quy <quy@fluxbb.org> Co-authored-by: Phil E. Taylor <phil@phil-taylor.com> Co-authored-by: Tobias Zulauf <zero-24@users.noreply.github.com> Co-authored-by: Benjamin Trenkle <bembelimen@users.noreply.github.com> * php8 deprecation str_replace Passing null to param (#36400) * add php 8.1 to the version array (#36577) * Add function parameters for callbacks. Fixes #36010 * PHP 8.1 deprecation fixes (#36306) * Backport some deprecation fixes from the J4 branch * Ensure htmlspecialchars is passed a string * Ignore PHPCS errors Co-authored-by: Tobias Zulauf <zero-24@users.noreply.github.com> * Update simplepie to 1.3.3 (#36358) * Update simplepie to 1.3.2 & phar-stream-wrapper to 2.2.2 * update to simplepie 1.3.3 to fix the version * [3.10] Dont show the pre upgrade checker when the upgrade is not supported (#36312) * [3.10] Fix Progressive caching to cache modules per custom menu assignment (#36324) * Fix progressive caching * Update libraries/src/Document/HtmlDocument.php Co-authored-by: Phil E. Taylor <phil@phil-taylor.com> * cast int * Update libraries/src/Document/HtmlDocument.php * Update libraries/src/Document/HtmlDocument.php * Update libraries/src/Document/HtmlDocument.php * Update libraries/src/Document/HtmlDocument.php * Update libraries/src/Document/HtmlDocument.php * Update libraries/src/Document/HtmlDocument.php Co-authored-by: Phil E. Taylor <phil@phil-taylor.com> Co-authored-by: Tobias Zulauf <zero-24@users.noreply.github.com> * Bump registry to 1.6.4 (#36611) * [4.0] Load the subform data correctly (#36598) * Load the subform data correctly * correct order * don't remove <joomla-hidden-mail> (#36602) * [4.0] Function to unregister a provider in media manager (#36594) * [4.0] Latests/Popular Admin modules permissions (#36559) * Allow our phpcode sniffer composer plugin to run * PHP 8.1 fixes ported from #36355 * Fix codestyle. Thanks @brianteeman * Improve security and type checking of the installer task * Ensure the rule modifier is a string by default * Ensure cpanel dashboard name is type safe (PHP 8.1) * Fix mistake in param ordering * Fix PHP 8.1 notice when creating an article * Fix URL Field giving deprecated notices in PHP 8.1 * [4.0] Fix template preview (#36268) * Fix template preview * Fix template preview * Fix Progressive caching to cache modules per custom menu assignment (#36193) * Cross Origin for Google Fonts (#36592) * Update Tobscure library to be PHP 8.1 compat * Tag 4.0.6 RC1 * Reset to dev * Translation Update (#36356) * [4.0] Fix images (#35780) * [4.0] Joomla Update Silent fails (#36492) * [4.0] Joomla Update Silent fails I am not 100% certain that this is the best approach to this. @nikosdion @wilsonge please advise ## The Problem As reported in # if during an update a sql query fails then the updater quits the sql part of the update process and moves on. The joomla_update log gives no indication that there has been a problem etc. ## Proposed Partial Solution Adds a Start SQL and End SQL to the logs If there is an error in the sql then that is written to the logs as well AND it now says End SQL incomplete ### Testing For the testing we will be using the "Update your site by manually uploading the update package." and to make things quick I have prepared an empty update that you should download and use. ### Before Test 1 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### Before Test 2 Update your database to change the value in the #__schemas table so that the version_id is 4.0.0-2018-5-15 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### Before Test 3 Update your database to change the value in the #__schemas table so that the version_id is 4.0.3 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### After - Test 1 Update your database to change the value in the #__schemas table so that the version_id is 4.0.4 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### After - Test 2 Update your database to change the value in the #__schemas table so that the version_id is 4.0.0-2018-5-15 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### Before Test 3 Update your database to change the value in the #__schemas table so that the version_id is 4.0.3 Update your install and then check the `administrator\logs\joomla_update.php` It should look like * Fix null being passed to DateTime::__construct causing PHP 8.1 deprecation error * Missing param default to fix uploads in PHP 8.1 * Tag 4.0.6 RC2 * Reset to dev * Missing param default to fix renaming media in PHP 8.1 * Stub generator now handles deprecated namespaced classes (#35144) * Fix error in article model (#36617) * Fix code comment * Temporary remove phpnext tests (#36624) * Sign drone.yml * Translation Update (#36645) * Update extract.php (#36635) * Update composer.lock Co-authored-by: Brian Teeman <brian@teeman.net> Co-authored-by: Harald Leithner <leithner@itronic.at> Co-authored-by: Tobias Zulauf <zero-24@users.noreply.github.com> Co-authored-by: Allon Moritz <allon.moritz@digital-peak.com> Co-authored-by: Quy <quy@fluxbb.org> Co-authored-by: Nicola Galgano <optimus4joomla@gmail.com> Co-authored-by: Roland Dalmulder <contact@rolandd.com> Co-authored-by: Tuan Pham Ngoc <github@joomdonation.com> Co-authored-by: Fedir Zinchuk <getthesite@gmail.com> Co-authored-by: wilsonge <georgejameswilson@googlemail.com> Co-authored-by: Phil E. Taylor <phil@phil-taylor.com> Co-authored-by: Crystal Dionysopoulos <9141288+crystalenka@users.noreply.github.com> Co-authored-by: heelc29 <66922325+heelc29@users.noreply.github.com> Co-authored-by: Geraint Edwards <github@copyn.plus.com> Co-authored-by: Tony Partridge <tony@xws.im> Co-authored-by: frostmakk <sveinung.larsen@lyse.net> Co-authored-by: Dimitris Grammatikogiannis <d.grammatiko@gmail.com> Co-authored-by: joomla-translation-bot <87496051+joomla-translation-bot@users.noreply.github.com> Co-authored-by: Richard Fath <richard67@users.noreply.github.com> Co-authored-by: Sveinung Larsen <frostmakk@users.noreply.github.com>
* [4.0] Backup message (#36487) Many users who use their hosts backup facility and this is typically __only__ for files This update reminds them to backup the database as well * Remove reference & in usort callback function (#36503) * Update paragonie sodium_compat to 1.17.0 (#36491) Co-authored-by: Benjamin Trenkle <bembelimen@users.noreply.github.com> * [3.10] Finder plugins modify component params registry / Backport #35855 (#36141) * Backport #35855 to 3.10 * one space to much * [4.0] xml code style (#36532) Fixes a few spaces to tabs issues * [4.0] media manager breadcrumbs (#36154) * [4.0] media manager breadcrumbs The RTL specific css was wrong as can be seen in the before screenshot This PR removes the RTL specific css (should have been scss) and updates the scss to use logical properties instead. This PR also removes an unused class `breadcrumb-item`. To test RTL you will need to install arabic or persian. and dont forget to npm ci There is no visible change in LTR and the RTL changes are as shown below ## Before LTR ## Before RTL ## After LTR ## After RTL * revert * [4.0] RTL: Correcting Admin login 2factor and feedback hints (#36024) This is a replacement to the merged PR #32104 It does exactly the same thing but by using css logical properties we avoid the need to maintain both an LTR and an RTL version[4.0] There is no visual change * [4.0] Category list (#36536) Remove extra div that is not needed * [4.0] Make routing work in console applications (#36332) * Update list-fancy-select.php (#36543) replace tab with space code review * [4.0] archive icon (#36263) All the state icons are grey so that the published icon, which is green, stands out visually. The archived icon however was black. This PR changes that icon to grey. ### Before ### After * [4.0] Banner Client tips (#36262) * [4.0] Banner Client tips Adds a tooltip to the banner client numbers so that they appear the same as category numbers. Also corrects the class on the trashed number so that it has the correct colour To test Create one banner client Create a banner for the client in each of the published states * Update administrator/components/com_banners/tmpl/clients/default.php oops Co-authored-by: Quy <quy@fluxbb.org> Co-authored-by: Quy <quy@fluxbb.org> * Fix article creation from frontend (#36542) * [4.0] Handle when toggle icon is not available (#36329) * Handle when toggle icon is not available * cs * Take workflow extension into account when setting a default (#36527) * Fix uploadlimits checks (#36023) * Make edit_lock layout reusable for other extensions (#35112) * Allow to set a value for SubformField (#35921) * [4.0] Fix example in cleanImageURL docblock (#36554) * Fix Codemirror behind tabs (#36140) * Fix Codemirror behind tabs * jscs * Remove dependency on Factory in AdminController * [4.0] Ensure path contains api in the router (#36315) * [4.0] Check model is set before calling its methods (#36378) * Check model is set before calling its methods * Update OverridesController.php * Category edit form should use return url (#36560) Currently the article edit form respects return URLs; the category edit form does not. Adding it here makes for a more consistent editing experience. * [4.0] Sample Data plugin images (#36237) Updates the plugin to use the new image data used in lazy loading etc. This avoids confusion for users who see different source code for their created articles when compared to the same data plugins To test. Just make sure the sample data still installs and works as expected * [4.0] If the Mail Template was not found in the db, return false (#36489) * [4.0] Code Style (#36564) * Use an absolute path in image list field (#36063) * Correct Cache setWorkarounds to take account of multiple script or style declarations being used that mean that we have an array of scripts or stylesheets to be cached and not just a string (#36068) Cache setWorkarounds returns an array not a string * [3] privacy request by logged in user only (#35470) * privacy request logged in only * privacy request logged in only * privacy request logged in only * privacy request logged in only * privacy request logged in only * Update components/com_privacy/controller.php Co-authored-by: Quy <quy@fluxbb.org> Co-authored-by: Quy <quy@fluxbb.org> * Feature - Add always count option to 404 redirects (#34640) * Feature - Add always count option to 404 redirects * Add missiing space for code sniffer * Return line for if statement * Update plugins/system/redirect/redirect.php Co-authored-by: Quy <quy@fluxbb.org> * Alpha Ordering - Change Constant too. * update constant * Update en-GB.plg_system_redirect.ini * remove trailing whtiespace * Remove Added Language Strings * Update plugins/system/redirect/redirect.php Co-authored-by: Phil E. Taylor <phil@phil-taylor.com> * Update redirect.xml * Update redirect.php * Update plugins/system/redirect/redirect.php Co-authored-by: Quy <quy@fluxbb.org> * Update plugins/system/redirect/redirect.php Co-authored-by: Benjamin Trenkle <bembelimen@users.noreply.github.com> * Update plugins/system/redirect/redirect.php Co-authored-by: Quy <quy@fluxbb.org> Co-authored-by: Phil E. Taylor <phil@phil-taylor.com> Co-authored-by: Tobias Zulauf <zero-24@users.noreply.github.com> Co-authored-by: Benjamin Trenkle <bembelimen@users.noreply.github.com> * php8 deprecation str_replace Passing null to param (#36400) * add php 8.1 to the version array (#36577) * Add function parameters for callbacks. Fixes #36010 * PHP 8.1 deprecation fixes (#36306) * Backport some deprecation fixes from the J4 branch * Ensure htmlspecialchars is passed a string * Ignore PHPCS errors Co-authored-by: Tobias Zulauf <zero-24@users.noreply.github.com> * Update simplepie to 1.3.3 (#36358) * Update simplepie to 1.3.2 & phar-stream-wrapper to 2.2.2 * update to simplepie 1.3.3 to fix the version * [3.10] Dont show the pre upgrade checker when the upgrade is not supported (#36312) * [3.10] Fix Progressive caching to cache modules per custom menu assignment (#36324) * Fix progressive caching * Update libraries/src/Document/HtmlDocument.php Co-authored-by: Phil E. Taylor <phil@phil-taylor.com> * cast int * Update libraries/src/Document/HtmlDocument.php * Update libraries/src/Document/HtmlDocument.php * Update libraries/src/Document/HtmlDocument.php * Update libraries/src/Document/HtmlDocument.php * Update libraries/src/Document/HtmlDocument.php * Update libraries/src/Document/HtmlDocument.php Co-authored-by: Phil E. Taylor <phil@phil-taylor.com> Co-authored-by: Tobias Zulauf <zero-24@users.noreply.github.com> * Bump registry to 1.6.4 (#36611) * [4.0] Load the subform data correctly (#36598) * Load the subform data correctly * correct order * don't remove <joomla-hidden-mail> (#36602) * [4.0] Function to unregister a provider in media manager (#36594) * Joomla! 3.10.5 Release Candidate 1 * [4.0] Latests/Popular Admin modules permissions (#36559) * Allow our phpcode sniffer composer plugin to run * PHP 8.1 fixes ported from #36355 * Fix codestyle. Thanks @brianteeman * Improve security and type checking of the installer task * Ensure the rule modifier is a string by default * Ensure cpanel dashboard name is type safe (PHP 8.1) * Fix mistake in param ordering * Fix PHP 8.1 notice when creating an article * Fix URL Field giving deprecated notices in PHP 8.1 * [4.0] Fix template preview (#36268) * Fix template preview * Fix template preview * Fix Progressive caching to cache modules per custom menu assignment (#36193) * Cross Origin for Google Fonts (#36592) * Update Tobscure library to be PHP 8.1 compat * Tag 4.0.6 RC1 * Reset to dev * reset to dev * Translation Update (#36356) * [4.0] Fix images (#35780) * [4.0] Joomla Update Silent fails (#36492) * [4.0] Joomla Update Silent fails I am not 100% certain that this is the best approach to this. @nikosdion @wilsonge please advise ## The Problem As reported in # if during an update a sql query fails then the updater quits the sql part of the update process and moves on. The joomla_update log gives no indication that there has been a problem etc. ## Proposed Partial Solution Adds a Start SQL and End SQL to the logs If there is an error in the sql then that is written to the logs as well AND it now says End SQL incomplete ### Testing For the testing we will be using the "Update your site by manually uploading the update package." and to make things quick I have prepared an empty update that you should download and use. ### Before Test 1 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### Before Test 2 Update your database to change the value in the #__schemas table so that the version_id is 4.0.0-2018-5-15 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### Before Test 3 Update your database to change the value in the #__schemas table so that the version_id is 4.0.3 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### After - Test 1 Update your database to change the value in the #__schemas table so that the version_id is 4.0.4 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### After - Test 2 Update your database to change the value in the #__schemas table so that the version_id is 4.0.0-2018-5-15 Update your install and then check the `administrator\logs\joomla_update.php` It should look like ### Before Test 3 Update your database to change the value in the #__schemas table so that the version_id is 4.0.3 Update your install and then check the `administrator\logs\joomla_update.php` It should look like * Fix null being passed to DateTime::__construct causing PHP 8.1 deprecation error * Missing param default to fix uploads in PHP 8.1 * Tag 4.0.6 RC2 * Reset to dev * Missing param default to fix renaming media in PHP 8.1 * Stub generator now handles deprecated namespaced classes (#35144) * Fix error in article model (#36617) * Fix code comment * Temporary remove phpnext tests (#36624) * Sign drone.yml * Use the correct namespace for the taggable trait (#36640) * Translation Update (#36645) * Update extract.php (#36635) * Updated MediaHelper::canUpload function * Tag RC3 * Reset to dev Co-authored-by: Brian Teeman <brian@teeman.net> Co-authored-by: Harald Leithner <leithner@itronic.at> Co-authored-by: Tobias Zulauf <zero-24@users.noreply.github.com> Co-authored-by: Allon Moritz <allon.moritz@digital-peak.com> Co-authored-by: Quy <quy@fluxbb.org> Co-authored-by: Nicola Galgano <optimus4joomla@gmail.com> Co-authored-by: Roland Dalmulder <contact@rolandd.com> Co-authored-by: Tuan Pham Ngoc <github@joomdonation.com> Co-authored-by: Fedir Zinchuk <getthesite@gmail.com> Co-authored-by: wilsonge <georgejameswilson@googlemail.com> Co-authored-by: Phil E. Taylor <phil@phil-taylor.com> Co-authored-by: Crystal Dionysopoulos <9141288+crystalenka@users.noreply.github.com> Co-authored-by: heelc29 <66922325+heelc29@users.noreply.github.com> Co-authored-by: Geraint Edwards <github@copyn.plus.com> Co-authored-by: Tony Partridge <tony@xws.im> Co-authored-by: frostmakk <sveinung.larsen@lyse.net> Co-authored-by: Dimitris Grammatikogiannis <d.grammatiko@gmail.com> Co-authored-by: joomla-translation-bot <87496051+joomla-translation-bot@users.noreply.github.com> Co-authored-by: Richard Fath <richard67@users.noreply.github.com> Co-authored-by: Sveinung Larsen <frostmakk@users.noreply.github.com> Co-authored-by: Akshit Rattan <akshitrattan@Akshits-MacBook-Air.local>
I am not 100% certain that this is the best approach to this. @nikosdion @wilsonge please advise
The Problem
As reported in #36485 if during an update a sql query fails then the updater quits the sql part of the update process and skips the com_admin\scripts.php files/folders changes and moves on. The joomla_update log gives no indication that there has been a problem etc.
Proposed Partial Solution
Adds a Start SQL and End SQL to the logs
updated Log "The current database version (schema) is xxxxxx"
If there is an error in the sql then that is written to the logs as well AND it now says End SQL incomplete
Testing
For the testing we will be using the "Update your site by manually uploading the update package." and to make things quick I have prepared an empty update that you should download and use.
Before Test 1
Update your install and then check the
administrator\logs\joomla_update.phpIt should look like
Before Test 2
Update your database to change the value in the #__schemas table so that the version_id is 4.0.0-2018-5-15
Update your install and then check the
administrator\logs\joomla_update.phpIt should look like
Note that the sql queries did not complete AND the deletion of files/folders from com_admin\script.php never took place - but you only know that because I told you
Before Test 3
Update your database to change the value in the #__schemas table so that the version_id is 4.0.3
Update your install and then check the
administrator\logs\joomla_update.phpIt should look like
Note that the sql queries did complete (only because I told you) AND the deletion of files/folders from com_admin\script.php took place
After - Test 1
Update your database to change the value in the #__schemas table so that the version_id is 4.0.4
Update your install and then check the
administrator\logs\joomla_update.phpIt should look like
Note the new start and end sql messages
After - Test 2
Update your database to change the value in the #__schemas table so that the version_id is 4.0.0-2018-5-15
Update your install and then check the
administrator\logs\joomla_update.phpIt should look like
Note that you can now see the start of the sql messages, the query that failed together with the error AND End of SQL updates - INCOMPLETE
It still skipped the remove files/folders (without indication) and says that the update is complete when it is not.
After Test 3
Update your database to change the value in the #__schemas table so that the version_id is 4.0.3
Update your install and then check the
administrator\logs\joomla_update.phpIt should look like
Note we can now see that the sql queries did complete AND the deletion of files/folders from com_admin\script.php took place
This PR can either be considered as a small step in resolving #36485 and can be merged OR pull requests will be welcome to resolve the missing parts