[4.0] Improve the generator tool for lists of files and folders to be removed or renamed on update by richard67 · Pull Request #20 · richard67/joomla-cms

richard67 · 2021-11-12T13:57:08Z

Pull Request for Issue # .

Replaces #19 .

Summary of Changes

This pull request (PR) is just to document the changed deleted files and folders generator script which I am currently using to generate the lists for updating script.php.

The change here is not to create the result files but to append the results to the files if they aren't already included in that file.

Testing Instructions

Currently I use following Bash script to generate the lists:

#!/usr/bin/env bash

# Remove old results
rm -f ./deleted_files.txt
rm -f ./deleted_folders.txt
rm -f ./renamed_files.txt

# From 3.10-dev to 4.0-dev
echo "// From 3.10 to 4.0" > ./deleted_files.txt
echo "// From 3.10 to 4.0" > ./deleted_folders.txt
echo "// From 3.10 to 4.0" > ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_3.10.4-dev-Development-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.5-rc2-dev-Development-Full_Package

# From 4.0.0 Beta 1 to 4.0.0 Beta 2 - ignore results from previous comparisons
echo "// 4.0 from Beta 1 to Beta 2" >> ./deleted_files.txt
echo "// 4.0 from Beta 1 to Beta 2" >> ./deleted_folders.txt
echo "// 4.0 from Beta 1 to Beta 2" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta1-Beta-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta2-Beta-Full_Package

# From 4.0.0 Beta 2 to 4.0.0 Beta 3 - ignore results from previous comparisons
echo "// 4.0 from Beta 2 to Beta 3" >> ./deleted_files.txt
echo "// 4.0 from Beta 2 to Beta 3" >> ./deleted_folders.txt
echo "// 4.0 from Beta 2 to Beta 3" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta2-Beta-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta3-Beta-Full_Package

# From 4.0.0 Beta 3 to 4.0.0 Beta 4 - ignore results from previous comparisons
echo "// 4.0 from Beta 3 to Beta 4" >> ./deleted_files.txt
echo "// 4.0 from Beta 3 to Beta 4" >> ./deleted_folders.txt
echo "// 4.0 from Beta 3 to Beta 4" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta3-Beta-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta4-Beta-Full_Package

# From 4.0.0 Beta 4 to 4.0.0 Beta 5 - ignore results from previous comparisons
echo "// 4.0 from Beta 4 to Beta 5" >> ./deleted_files.txt
echo "// 4.0 from Beta 4 to Beta 5" >> ./deleted_folders.txt
echo "// 4.0 from Beta 4 to Beta 5" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta4-Beta-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta5-Beta-Full_Package

# From 4.0.0 Beta 5 to 4.0.0 Beta 6 - ignore results from previous comparisons
echo "// 4.0 from Beta 5 to Beta 6" >> ./deleted_files.txt
echo "// 4.0 from Beta 5 to Beta 6" >> ./deleted_folders.txt
echo "// 4.0 from Beta 5 to Beta 6" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta5-Beta-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta6-Beta-Full_Package

# From 4.0.0 Beta 6 to 4.0.0 Beta 7 - ignore results from previous comparisons
echo "// 4.0 from Beta 6 to Beta 7" >> ./deleted_files.txt
echo "// 4.0 from Beta 6 to Beta 7" >> ./deleted_folders.txt
echo "// 4.0 from Beta 6 to Beta 7" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta6-Beta-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta7-Beta-Full_Package

# From 4.0.0 Beta 7 to 4.0.0 RC 1 - ignore results from previous comparisons
echo "// 4.0 from Beta 7 to RC 1" >> ./deleted_files.txt
echo "// 4.0 from Beta 7 to RC 1" >> ./deleted_folders.txt
echo "// 4.0 from Beta 7 to RC 1" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-beta7-Beta-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc1-Release_Candidate-Full_Package

# From 4.0.0 RC 1 to RC 2 - ignore results from previous comparisons
echo "// 4.0 from RC 1 to RC 2" >> ./deleted_files.txt
echo "// 4.0 from RC 1 to RC 2" >> ./deleted_folders.txt
echo "// 4.0 from RC 1 to RC 2" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc1-Release_Candidate-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc2-Release_Candidate-Full_Package

# From 4.0.0 RC 2 to RC 3 - ignore results from previous comparisons
echo "// 4.0 from RC 2 to RC 3" >> ./deleted_files.txt
echo "// 4.0 from RC 2 to RC 3" >> ./deleted_folders.txt
echo "// 4.0 from RC 2 to RC 3" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc2-Release_Candidate-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc3-Release_Candidate-Full_Package

# From 4.0.0 RC 3 to RC 4 - ignore results from previous comparisons
echo "// 4.0 from RC 3 to RC 4" >> ./deleted_files.txt
echo "// 4.0 from RC 3 to RC 4" >> ./deleted_folders.txt
echo "// 4.0 from RC 3 to RC 4" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc3-Release_Candidate-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc4-Release_Candidate-Full_Package

# From 4.0.0 RC 4 to RC 5 - ignore results from previous comparisons
echo "// 4.0 from RC 4 to RC 5" >> ./deleted_files.txt
echo "// 4.0 from RC 4 to RC 5" >> ./deleted_folders.txt
echo "// 4.0 from RC 4 to RC 5" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc4-Release_Candidate-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc5-Release_Candidate-Full_Package

# From 4.0.0 RC 5 to RC 6 - ignore results from previous comparisons
echo "// 4.0 from RC 5 to RC 6" >> ./deleted_files.txt
echo "// 4.0 from RC 5 to RC 6" >> ./deleted_folders.txt
echo "// 4.0 from RC 5 to RC 6" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc5-Release_Candidate-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc6-Release_Candidate-Full_Package

# From 4.0.0 RC 6 to 4.0.0 (stable) - ignore results from previous comparisons
echo "// 4.0 from RC 6 to 4.0.0 (stable)" >> ./deleted_files.txt
echo "// 4.0 from RC 6 to 4.0.0 (stable)" >> ./deleted_folders.txt
echo "// 4.0 from RC 6 to 4.0.0 (stable)" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-rc6-Release_Candidate-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-Stable-Full_Package

# From 4.0.0 to 4.0.1 - ignore results from previous comparisons
echo "// From 4.0.0 to 4.0.1" >> ./deleted_files.txt
echo "// From 4.0.0 to 4.0.1" >> ./deleted_folders.txt
echo "// From 4.0.0 to 4.0.1" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.0-Stable-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.1-Stable-Full_Package

# From 4.0.1 to 4.0.2 - ignore results from previous comparisons
echo "// From 4.0.1 to 4.0.2" >> ./deleted_files.txt
echo "// From 4.0.1 to 4.0.2" >> ./deleted_folders.txt
echo "// From 4.0.1 to 4.0.2" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.1-Stable-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.2-Stable-Full_Package

# From 4.0.2 to 4.0.3 - ignore results from previous comparisons
echo "// From 4.0.2 to 4.0.3" >> ./deleted_files.txt
echo "// From 4.0.2 to 4.0.3" >> ./deleted_folders.txt
echo "// From 4.0.2 to 4.0.3" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.2-Stable-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.3-Stable-Full_Package

# From 4.0.3 to 4.0.4 - ignore results from previous comparisons
echo "// From 4.0.3 to 4.0.4" >> ./deleted_files.txt
echo "// From 4.0.3 to 4.0.4" >> ./deleted_folders.txt
echo "// From 4.0.3 to 4.0.4" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.3-Stable-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.4-Stable-Full_Package

# From 4.0.4 to 4.0-dev - ignore results from previous comparisons
echo "// From 4.0.4 to 4.0.5" >> ./deleted_files.txt
echo "// From 4.0.4 to 4.0.5" >> ./deleted_folders.txt
echo "// From 4.0.4 to 4.0.5" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.4-Stable-Full_Package \
    --to=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.5-rc2-dev-Development-Full_Package

When keeping the intermediate results from comparing older 4.0 versions since Beta 1, it only needs to do the first comparison from current 3.10-dev to current 4.0-dev and then the last 2 comparisons. The 2nd last because something might have been forgotten in script.php when the last release was made, and the last one for the last release and the current branch.

At the end I can take the complete content of each file, intend it by 3 tabs and copy it into script PHP and can see the changes in Git.

Then it needs only 3 manual steps:

Add back the intermediate 4.0.0 update SQL scripts which I had removed between 2 beta versions when combining the update SQL script, so the comparisons don't know about them.
Move deletion of folder "/libraries/vendor/paragonie/random_compat" from section "// From 3.10 to 4.0" to section "// 4.0 from Beta 7 to RC 1". The reason is that we have different child folders of that being deleted when updating from 3.10 or when updating from 4.0 Beta 7, so we get the folder 2 times, and the duplicate check added in this PR here will add it do the 1st and not to the 2nd section, but in this case it should be in the 2nd.
Remove section comments for empty sections

This can be automated later with further improvements, but the question now is if what I am doing here is the right way to go.

For 4.1-dev I run the following shell script after having run the previously mentioned one for 4.0-dev:

#!/usr/bin/env bash

# Copy results from 4.0-dev
cp -f /home/richard/lamp/public_html/joomla-cms-4.0-dev/build/deleted_files.txt ./
cp -f /home/richard/lamp/public_html/joomla-cms-4.0-dev/build/deleted_folders.txt ./
cp -f /home/richard/lamp/public_html/joomla-cms-4.0-dev/build/renamed_files.txt ./

# From 4.0-dev to 4.1-dev - ignore results from previous comparisons
echo "// From 4.0.5 to 4.1.0" >> ./deleted_files.txt
echo "// From 4.0.5 to 4.1.0" >> ./deleted_folders.txt
echo "// From 4.0.5 to 4.1.0" >> ./renamed_files.txt
php ./deleted_file_check.php \
    --from=/home/richard/lamp/public_html/4.0-compare-releases/Joomla_4.0.5-rc2-dev-Development-Full_Package \
    --to=/home/richard/lamp/public_html/4.1-compare-releases/Joomla_4.1.0-dev-Development-Full_Package

Actual result BEFORE applying this Pull Request

I have to identify and remove duplicates myself from the right section of the deleted files and folder lists in script.php.

Expected result AFTER applying this Pull Request

Files and folders which have been identified to be removed when updating from the previous major (or minor) release on another branch, e.g. in case of J4 from 3.10, are not to be removed again when updating from a previous version of the current release to be checked.

To be Done

When some file or folder is removed between 2 versions and later added back with another, later version, the tool is not able to handle that yet.

Documentation Changes Required

There is no documentation yet on how to use deleted_file_check.php.

richard67 · 2021-12-04T20:24:07Z

Closing in favour of #21 .

Moved xml outside of src folder and load via trait

…-notification-email Prevent update notification emails when auto updater is active

richard67 mentioned this pull request Nov 12, 2021

[4.0] Improve the generator tool for lists of files and folders to be removed or renamed on update #19

Closed

richard67 force-pushed the 4.0-dev-deleted-files-and-folders-improvements-2 branch from 0a91ecf to fa85e51 Compare November 14, 2021 10:11

richard67 mentioned this pull request Nov 30, 2021

[4.0] Remove _com_joomlaupdate.scss from deleted files in script.php joomla/joomla-cms#36161

Merged

richard67 added 3 commits December 4, 2021 16:30

Let deleted files check append and not replace results

41b9f18

Append only if some results

7293208

Report both versions at the end

a0c6391

richard67 force-pushed the 4.0-dev-deleted-files-and-folders-improvements-2 branch from fa85e51 to a0c6391 Compare December 4, 2021 15:30

richard67 mentioned this pull request Dec 4, 2021

[4.0] Improve the generator tool for lists of files and folders to be removed or renamed on update #21

Closed

richard67 closed this Dec 4, 2021

richard67 pushed a commit that referenced this pull request Oct 21, 2022

The depreciation was only valid for 5.10.x (#20)

b49e2b7

richard67 pushed a commit that referenced this pull request Jul 14, 2023

Merge pull request #20 from joomla-projects/bembelimen/improvements

736a906

Moved xml outside of src folder and load via trait

richard67 pushed a commit that referenced this pull request Mar 16, 2025

Merge pull request #20 from joomla-projects/automated-updates/disable…

8cc4343

…-notification-email Prevent update notification emails when auto updater is active

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[4.0] Improve the generator tool for lists of files and folders to be removed or renamed on update#20

[4.0] Improve the generator tool for lists of files and folders to be removed or renamed on update#20
richard67 wants to merge 3 commits into4.0-devfrom
4.0-dev-deleted-files-and-folders-improvements-2

richard67 commented Nov 12, 2021 •

edited

Loading

Uh oh!

richard67 commented Dec 4, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

richard67 commented Nov 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of Changes

Testing Instructions

Actual result BEFORE applying this Pull Request

Expected result AFTER applying this Pull Request

To be Done

Documentation Changes Required

Uh oh!

richard67 commented Dec 4, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

richard67 commented Nov 12, 2021 •

edited

Loading