Skip to content

New Windows installer#7315

Open
bitraid wants to merge 28 commits intoqutebrowser:mainfrom
bitraid:nsis-3
Open

New Windows installer#7315
bitraid wants to merge 28 commits intoqutebrowser:mainfrom
bitraid:nsis-3

Conversation

@bitraid
Copy link
Copy Markdown
Contributor

@bitraid bitraid commented Jul 15, 2022

This took a really long time, but it’s finally here: The new NSIS script - written completely from scratch, and has also undergone several major refactors. It has many new features and improvements, and the code is much more mature. There are still a few small things that need to be decided - I’ll sum them up at the end.

Major features

  • Allow per-machine and per-user installations to coexist: This is done primary because the opposite cannot really be enforced on multi-user systems, but also because it might be something that could be useful in some cases. Now, their shortcuts have different names (per-user includes (current user)), and they are registered as distinct application, so both can be selected as default app.
  • Better detection of previous installers: Maybe not as useful anymore, but on upgrades, the version number and install path of installations from previous installers, are detected and seemingly integrated.
  • Improved workaround of Windows force-elevation of uninstallers: There is the /user command line option, which restarts the uninstaller unelevated, and is used when called from the system settings. This takes care of the bug on older versions of Windows that elevate per-user uninstalls, and also makes sure that on per-machine uninstalls, the Clear Cache and Clear Config options are for the user that run the uninstall (in case a different account was used for elevation - something that happens before the uninstaller is executed). This option is also available for the installer, and can be used for example, when it is started from an elevated prompt, but needs to install for the current (standard) user.
  • Tweaked single instance: Now, the window of an already running instance of the installer/uninstaller is brought to front when the user runs the program a second time (instead of displaying the Retry/Cancel dialog box).
  • Detection of qutebrowser running: The detection happens only if qutebrowser is running form within the install path, or in case of an upgrade to a different directory, also the uninstall path. Running from anywhere else shouldn't bother the install (and using only the process name for detection is weak).
  • Command line options for fully configurable unattended setups: Components are set with /{DesktopIcon, StartMenuIcon, RegisterBrowser, ClearCache, ClearConfig}[=off]. There is also /InstallDir as a /D alternative that doesn’t have to be the last parameter and accepts quotes as usual, and /Silent as a more descriptive /S alternative which is also not case sensitive.
  • Enhanced Components page: On upgrades, unless set from the command line, the shortcut and registry selections are based on the existing setup. The uninstaller displays which components are installed and are going to be removed. The Clear Cache and Clear Config components are only available if the directories exist, and are also available for the installer. A confirmation is requested if Clear Config is selected.
  • Enhanced Install Directory page: The installation path typed by the user is processed on the fly: rejects illegal characters, navigates \.\ and \..\, converts / to \ while allowing only one, converts short names to long names, and also disallows the selection of some paths (the root of the drive, directly into Program Files, anywhere inside Windows). On fresh installs (or upgrade to new directory), if the selected directory exists and contains files, a message box informs the user and asks for confirmation (unless it only contains the default cache directory, which is the case with the default path for per-user installations).
  • Clean up before extraction / after failure: If the installation directory contains files, any existing files that belong to the installation are removed (failure to remove a file will abort the install before the extraction begins), and if an existing subdirectory that belongs to the installation still contains files, the user is asked if the extra files should be removed or not (same with the uninstaller). If an installation fails to complete, the installer removes any written files/folders/registry. On upgrades, if the uninstaller is missing or fails to execute, the uninstall is performed by the installer.
  • Ability for the user to interrupt and abort/resume an ongoing installation: The cancel button is enabled and available for the user to abort an active install, in which case the incomplete installation is removed.
  • Verification of extracted files using SHA-256: After the extraction of each file, its hash is calculated and is compared with its original value stored at compile time. On a mismatch the user is informed and is given the option to retry the extraction or abort (and remove the incomplete install).
  • Validation of registry operations: If the installer fails to make a registry change, the user is informed and is given the option to retry, abort, or continue without adding registry settings (in which case, attempts to remove any written key/value). If the uninstaller fails to remove a key/value, the user can retry, abort or continue ignoring the specific item.
  • Log file support: Install/uninstall operations can be logged to a file by using the command line /Log option. It can accept as parameter the path of the log file, or an existing directory where the log will be saved with the default name (installer name + .log). If no parameter is provided, the log is created with the default name at the same directory as the .exe. On upgrades, both the installer and uninstaller write at the same log file.
  • (Optional) MSI wrapper for deployment using MSCCM or Active Directory: This is similar to what Crome and Firefox provide with their "Enterprise" installers. It requires the WiX3 toolset, which can be downloaded by the script at compile time.
  • (Early) Dark theme support: The setup window uses dark colors when dark mode for apps is set. There is no proper dark mode support for win32 apps from Windows yet (most existing methods are undocumented - more info: Discussion: Dark mode for applications microsoft/WindowsAppSDK#41). Currently, the detection is done through registry, and the colors are hardcoded. Some things, like message boxes and the "Browse Folder" dialog, don't have dark colors.

Building the installer

The script uses PowerShell to generate a .nsh file containing macros that can pass every file, its calculated hash, and every folder of the release, to any other macro or instruction. It also downloads the plugins (NsisMultiUser, UAC, StdUtils), and (if required) downloads and extracts the WiX binaries. The installer can also be built under Linux (with the POSIX build of makensis) if there is PowerShell Core installed (for the MSI package it also requires wine, but I haven’t test that).

Passing the script form its default location without setting any defines, builds both x86 and x64 for the version that gets from __init__.py. The most important configuration defines are the following:

  • ARCH: Set the targeted CPU architecture. Can be either 'x86' or 'x64'. If left undefined, both architectures are built.
  • VERSION: Set the release version number.
  • MSI: Enable the creation of MSI wrappers. If defined empty, the WiX binaries are expected in $path. If the value is an http(s) link, it treats it as a link to the binaries .zip file - if only the protocol is defined, it uses a predefined link for the .zip download. Any other value is treated as the local path to the WiX binaries.
  • SIGN: If defined it signs the installer and uninstaller executables, and (if created) the .msi package. It accepts the command string as its value, with %1 being replaced by the target file. If defined empty, it uses the predefined string signtool.exe sign /a /tr http://timestamp.digicert.com /td sha256 /fd sha256 "%1". Might come useful for Code signing on Windows #7249.
  • DEBUG: If defined, a debug build of the setup will be created. Running the installer/uninstaller creates a debug log file in the same directory that the installer was created, or in a directory that is set as a value to the define.

Running the installer

Command line options for install:

qutebrowser-setup.exe [</AllUsers | /CurrentUser> [/Silent]] [/User]
                      [/RegisterBrowser[=off]] [/DesktopIcon[=off]] [/StartMenuIcon[=off]]
                      [/ClearCache] [/ClearConfig]
                      [/InstallDir=<{install_path}>] [/Log[={log_path}]]

Command line options for uninstall using the installer:

qutebrowser-setup.exe </Uninstall> </AllUsers | /CurrentUser> [/Silent] [/User]
                      [/ClearCache] [/ClearConfig] [/Log[={log_path}]]

Command line options for uninstaller:

uninstall.exe [</AllUsers | /CurrentUser> [/Silent]] [/User]
              [/ClearCache] [/ClearConfig] [/Log[={log_path}]]

Using the MSI:
The .msi is intended for system administrators and use with deployment tools like Active Directory or Microsoft System Center Configuration Manager. They run unattended and the installation is performed for all users. An MSI database editor like Orca can be used to create transforms (or make direct changes). These public properties are provided for configuration:

  • INSTALL_PATH: Set the installation directory. Must be an absolute path.
  • DESKTOP_ICON: Create Desktop shortcut icon. Can be on or off, default in on.
  • START_MENU_ICON: Create Start Menu shortcut icon. Can be on or off, default in on.
  • REGISTER_BROWSER: Register the browser with Windows. Can be on or off, default in on.
  • LOG: The path of the log file or the log directory, default is \\.\nul.

They can also be set from command line, for example:

msiexec /package qutebrowser-setup.msi REGISTER_BROWSER=off DESKTOP_ICON=off INSTALL_PATH="C:\qb"

Reviewing the code

The code is structured in the following way:

  • qutebrowser.nsi: The main script. It configures the plugins, has the installer and uninstaller Sections, and sources the rest of the files.
  • config.nsh: Configures the setup build and provides a macro with the compile time commands.
  • commands.nsh: Contains commands with operations that are repeated at different parts of the code, or are shared between the installer and uninstaller. The code of the functions is defined by macros, and imported for both installer and uninstaller for shared commands.
  • callbacks.nsh: Contains the callback functions. Some functions are shared by the installer and uninstaller, that's why it is sourced twice, with a different function prefix each time.
  • language-english.nsh: The English language strings. Where possible, it reuses strings from NSIS MUI. To add a new language, copy this file, change the language name and translate the strings. Then include the new file from the main script. Keep in mind though, that although MUI supports many languages, there are also a few strings from MultiUser plugin, which are only translated to Bulgarian and German (for other languages, they are displayed in English).

Things that might need changes

  • The script is designed to download the 3rd party files to a temporary directory, build both architectures if needed, and remove the directory. But now build_release.py builds each installer separately, so it downloads the files twice. This could be overcomed without actual code changes, by setting KEEP_SCRATCHDIR for the first build and NO_DOWNLOADS for the second, while setting for both a temporary path in SCRATCHDIR. Or maybe it's not so important since the 32-bit build is going to be dropped?
  • If the required versions of Win10/11 are going to be from a specific build number and above, a new detection method must be written.
  • Maybe the message when the installer is started under 32-bit should be different, if it's not going to be a 32-bit version.
  • The dark theme colors are set to match the Windows 11 dark theme, which is not completely black. Setting DARK_BGCOLOR_0=0, DARK_BGCOLOR_1=0 makes it look more like what Windows 10 uses.
  • For the Welcome/Finish pages, I used a more modern looking image which is included with latest NSIS, and applied a small modification (color and logo). I think it looks ok, but maybe something different is preferred.

@The-Compiler
Copy link
Copy Markdown
Member

Cool stuff! Haven't been able to look at things in detail yet, but two remarks re platform support:

If the required versions of Win10/11 are going to be from a specific build number and above, a new detection method must be written.

Qt 6.3+ supports Windows 10 and 11 on 21H2 or later. I don't quite understand how that's related to the detection method, though? Perhaps we should prevent installation altogether on unsupported Windows installations?

Or maybe it's not so important since the 32-bit build is going to be dropped?

Yup. Maybe let's remove it (and display an error) with the new installer altogether? The next (non-bugfix) release is going to be 3.0.0 with Qt 6, which indeed does not support 32-bit Windows anymore.

@bitraid
Copy link
Copy Markdown
Contributor Author

bitraid commented Jul 15, 2022

Qt 6.3+ supports Windows 10 and 11 on 21H2 or later. I don't quite understand how that's related to the detection method, though? Perhaps we should prevent installation altogether on unsupported Windows installations?

Yes - I meant the OS detection method, because now we rely on the logiclib extensions of WinVer.nsh, where there is no support for build number check. But that should be no problem, as the last version of NSIS introduced the GetWinVer command, and there is also ${StdUtils.GetRealOSBuildNo} from StdUtils plugin.

@bitraid
Copy link
Copy Markdown
Contributor Author

bitraid commented Apr 11, 2023

The installer now builds only for 64-bit. The OS check only passes for x86_64 Windows 10 21H2 and later, and any version of Windows 11 (Windows 11 arm64 has emulation for x86 and x86_x64 while Windows 10 has only for x86). A couple more APIs are now used for dark mode - they are undocumented but they exist on all current supported Windows builds.

Also, the /user switch could now be removed or at least not be used when the uninstaller is called from App settings (the elevation bug of per-user uninstalls is fixed for the supported Windows builds. But for all-users uninstalls that start from App settings by a non-admin user, would cause the Clear Cache/Settings to work for the account used for elevation. On the plus side, removing /user would eliminate the double elevation request).

@toofar toofar added this to the v4.0.0 milestone Jul 23, 2023
@toofar
Copy link
Copy Markdown
Member

toofar commented Jul 23, 2023

Hi @bitraid . I'm planning on looking at this after we get the 3.0 release out. Mostly because you are maintaining the prior version already so if you say it needs a new version then I guess it does! I added it to the 4.0 milestone but I guess that would make it 3.1 really.

What do you think about adding a readme file with some helpful text for other contributors in it? Your PR description has a lot of stuff in it already that I think would be useful in a readme.

I do have a question that might seem impolite given your investment so far in this. The idea of you not being able to contribute your time and us having to maintain this is a bit scary. Is it normal for a project to have 2k+ lines of installer scripts for just one platform? Questions in my mind are: is there some other installer framework which has these features builtin? Is there some of this stuff (like clearing caches?) we could move into python?

When I do have a look at this I'll probably go through and ask a bunch of questions (the answers of which might end up in the readme) about it and test some core stuff. But honestly we might leave a bit of testing to our end users. Maybe some question like:

  • Why is there WiX stuff in here for the MSI build, isn't that a competing installer framework?
  • Where do the old product codes come from?
  • ...

@bitraid
Copy link
Copy Markdown
Contributor Author

bitraid commented Jul 28, 2023

Hi @bitraid . I'm planning on looking at this after we get the 3.0 release out. Mostly because you are maintaining the prior version already so if you say it needs a new version then I guess it does! I added it to the 4.0 milestone but I guess that would make it 3.1 really.

Hi @toofar, sorry for the delayed reply. I wouldn't necessarily say that the new version is needed - I think it would add to the quality of the project, but at the same time it would probably be enough to just update the OS requirements of the current installer instead (something that should probably be done anyway for 3.0?).

What do you think about adding a readme file with some helpful text for other contributors in it? Your PR description has a lot of stuff in it already that I think would be useful in a readme.

Sure, we could do a readme.

I do have a question that might seem impolite given your investment so far in this. The idea of you not being able to contribute your time and us having to maintain this is a bit scary. Is it normal for a project to have 2k+ lines of installer scripts for just one platform? Questions in my mind are: is there some other installer framework which has these features builtin? Is there some of this stuff (like clearing caches?) we could move into python?

I remember reading somewhere that "NSIS is not an installer engine, but rather an engine for creating installer engines". It allows for great flexibility that is not usually found elsewhere, but the scripts can become huge (have a look for example at the 7k+ lines monstrosity of firefox installer).I understand that the size increase seems scary, but on the plus side, I think the code is a little easier to read (e.g. named registers) and test (debug build). There are parts of the code though, that are too big for what they do, for example the dark mode support. I think it's a cool (and unique) feature, but it could certainly be removed (along with probably some other stuff, too) to trim down the size. Hopefully, in the near future qutebrowser could be packaged as .msix (I'm interested in experimenting with the new Win32 app isolation security feature).

When I do have a look at this I'll probably go through and ask a bunch of questions (the answers of which might end up in the readme) about it and test some core stuff. But honestly we might leave a bit of testing to our end users.[...]

Sure, ask anytime. Meanwhile, I'll try to add some more comments, and maybe open a PR with an update for the current version?

[...] Maybe some question like:

  • Why is there WiX stuff in here for the MSI build, isn't that a competing installer framework?

NSIS doesn't create .msi files, so here WiX is used almost as a compile-time plugin to create an .msi wrapped .exe. This could also be removed/moved outside of the script.

  • Where do the old product codes come from?

They have been extracted from the old msi packages. Maybe those (version checks) could also be removed, since they're too old now.

@toofar
Copy link
Copy Markdown
Member

toofar commented Jul 28, 2023

maybe open a PR with an update for the current version?

Maybe? #7804 mentions something about NSIS changes, so they would be welcome. But if we aren't building 32bit installers that's probably enough in a pinch.

@bitraid
Copy link
Copy Markdown
Contributor Author

bitraid commented Jul 30, 2023

maybe open a PR with an update for the current version?

Maybe? #7804 mentions something about NSIS changes, so they would be welcome. But if we aren't building 32bit installers that's probably enough in a pinch.

From what I see, Qt-6.5 requires Windows 10 1809 or later. I would suggest this change to the current installer:

diff --git a/misc/nsis/install.nsh b/misc/nsis/install.nsh
index e7d8b4956..8bd50e808 100755
--- a/misc/nsis/install.nsh
+++ b/misc/nsis/install.nsh
@@ -430,8 +430,25 @@ SectionEnd
 ; Callbacks
 Function .onInit
   StrCpy $KeepReg 1
-  !insertmacro CheckPlatform ${PLATFORM}
-  !insertmacro CheckMinWinVer ${MIN_WIN_VER}
+
+; OS version check
+  ${If} ${RunningX64}
+    GetWinVer $R0 Major
+    IntCmpU $R0 10 0 _os_check_fail _os_check_pass
+    GetWinVer $R1 Build
+    ${If} $R1 >= 22000 ; Windows 11 21H2
+      Goto _os_check_pass
+    ${ElseIf} $R1 >= 17763 ; Windows 10 1809
+    ${AndIf} ${IsNativeAMD64} ; Windows 10 has no x86_64 emulation on arm64
+      Goto _os_check_pass
+    ${EndIf}
+  ${EndIf}
+  _os_check_fail:
+  MessageBox MB_OK|MB_ICONSTOP "This version of ${PRODUCT_NAME} requires a 64-bit$\r$\n\
+    version of Windows 10 1809 or later."
+  Abort
+  _os_check_pass:
+
   ${ifnot} ${UAC_IsInnerInstance}
     !insertmacro CheckSingleInstance "Setup" "Global" "${SETUP_MUTEX}"
     !insertmacro CheckSingleInstance "Application" "Local" "${APP_MUTEX}"

@toofar
Copy link
Copy Markdown
Member

toofar commented Aug 2, 2023

Hmm, thanks for the thought, but the changelog says

It should be possible to build a custom .exe with those versions [Windows 8.1], but this is unsupported and not recommended.

So I guess we shouldn't be blocking that for now. The only other change I can think of is to pull out all the support for multiple architectures, and I'm not sure it's worth it given it's already queued up in this PR. I'll ask for clarification.

@bitraid
Copy link
Copy Markdown
Contributor Author

bitraid commented Aug 2, 2023

Hmm, thanks for the thought, but the changelog says

It should be possible to build a custom .exe with those versions [Windows 8.1], but this is unsupported and not recommended.

So I guess we shouldn't be blocking that for now. The only other change I can think of is to pull out all the support for multiple architectures, and I'm not sure it's worth it given it's already queued up in this PR. I'll ask for clarification.

If someone builds a custom .exe using Qt 5.15, and needs an installer for it (that runs on old Windows), I guess she/he can make that change too. I think the important thing here is for the user who downloads and runs the setup, to be informed whether the OS supports the program or not.

@toofar
Copy link
Copy Markdown
Member

toofar commented Aug 5, 2023

I think the important thing here is for the user who downloads and runs the setup, to be informed whether the OS supports the program or not.

That makes sense. And we can probably put it behind a Qt version conditional since we'll be building different installers for each one anyway.

Also, not that this is authoritative at all, but I just noticed my old ie-vm windows test box is 1803/4 depending on where you look and it seems to work fine there.
image
Weird.

@bitraid
Copy link
Copy Markdown
Contributor Author

bitraid commented Aug 11, 2023

I tested qutebrowser/Qt-6.5.2 under different Windows 10 versions: It seems that it works for as early as 1607 (build 14393). Only 1507 and 1511 fail with error:
qbqt6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Small old PRs

Development

Successfully merging this pull request may close these issues.

3 participants