Switch to create-dmg #2207

ann0see · 2022-01-02T20:51:01Z

Short description of changes

Just a draft on how we could switch to create-dmg. Probably it's worth changing more in the macOS build process.
Context: Fixes an issue?

Fixes: #1797
Does this change need documentation? What needs to be documented and how?
No

Status of this Pull Request

Working semi-tested version
What is missing until this pull request can be merged?
Someone with a Mac should test if it does what it should. Especially the placing of files in the .dmg
And it should be squashed and merged

Checklist

I've verified that this Pull Request follows the general code principles
I tested my code and it does what I want
My code follows the style guide
I waited some time after this Pull Request was opened and all GitHub checks completed without errors.
I've filled all the content above

.github/workflows/autobuild.yml

ann0see · 2022-01-02T20:55:01Z

This should NOT be merged as is.

ann0see

The .DS_Store file doesn't seem to be created.

ann0see · 2022-01-07T19:47:39Z

@emlynmac could you please test if this build works on a modern Mac? I couldn’t really test it on my macOS VM…

emlynmac · 2022-01-07T21:20:36Z

@emlynmac could you please test if this build works on a modern Mac? I couldn’t really test it on my macOS VM…

Sure - where can I find the build?

ann0see · 2022-01-07T21:40:56Z

The .dmg is in the actions tab as artifact; however I‘m more interested in a local creation of the .dmg file. On GH actions it seems as if it doesn’t correct create a .DS_Store file and therefore doesn’t add a background image

emlynmac · 2022-01-07T21:47:11Z

The .dmg is in the actions tab as artifact; however I‘m more interested in a local creation of the .dmg file. On GH actions it seems as if it doesn’t correct create a .DS_Store file and therefore doesn’t add a background image

I'd not be concerned about the .DS_Store. It's just a part of the filesystem.
The image should be a different file somewhere.
The DMG opened fine on my 12.0.1 macPro.

I wonder if create-dmg is more picky about absolute file paths vs relative paths and whether that is why the resource is not being pulled in? Or maybe even the PNG not being the right format or something?

ann0see · 2022-01-08T06:40:47Z

I think the background-image file is included in the dmg (see hidden folders).

.github/workflows/autobuild.yml

mac/deploy_mac.sh

ann0see · 2022-01-17T20:36:18Z

I seem to get what the problem is now... We need to copy the .DS_Store we already had so far to the new installer since CI doesn't like GUI stuff (and we don't get permissions to execute a apple script create-dmg uses to set the background images)

ann0see · 2022-01-17T21:15:27Z

... or we use a different tool.

ann0see · 2022-01-17T21:23:00Z

Nope. Copying the .DS_Store also doesn't work...

mac/deploy_mac.sh

ann0see · 2022-01-18T07:29:49Z

@emlynmac @pljones ready to review.

mac/deploy_mac.sh

ann0see · 2022-01-18T07:35:07Z

It even seems to fix the scroll bars being shown on Big Sur upwards

ann0see · 2022-01-19T12:52:45Z

Sorry to say that, but I am no longer that confident that the scroll bar issue is fixed. I just realised that the Mac I tested the dmg on was running some OS before big sur.

hoffie · 2022-01-26T11:07:46Z

I've had a quick look at the .dmg build artifact and noticed that it seems to be a different format. I can't yet tell what's better or if it makes a difference at all, but I wanted to mention it.

7z x ../jamulus_3.8.1dev-6172bc8_mac.dmg (this PR)

$ 7z x ../jamulus_3.8.1dev-6172bc8_mac.dmg

7-Zip [64] 17.04 : Copyright (c) 1999-2021 Igor Pavlov : 2017-08-28
p7zip Version 17.04 (locale=de_DE.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)

Scanning the drive for archives:
1 file, 24495216 bytes (24 MiB)

Extracting archive: ../jamulus_3.8.1dev-6172bc8_mac.dmg
--
Path = ../jamulus_3.8.1dev-6172bc8_mac.dmg
Type = Dmg
Physical Size = 24495216
Method = Zero2 ZLIB CRC
Blocks = 68
----
Path = 4.hfs
Size = 85983232
Packed Size = 24452455
Comment = disk image (Apple_HFS : 4)
Method = Zero2 ZLIB CRC
--
Path = 4.hfs
Type = HFS
Physical Size = 85983232
Method = HFS+
Cluster Size = 4096
Free Space = 27480064
Created = 2022-01-18 09:57:57
Modified = 2022-01-18 10:58:31

Everything is Ok                                               

Folders: 108
Files: 167
Size:       57592322
Compressed: 24495216

7z x ../jamulus_3.8.1dev-b324ee8_mac.dmg (master)

$ 7z x ../jamulus_3.8.1dev-b324ee8_mac.dmg # master

7-Zip [64] 17.04 : Copyright (c) 1999-2021 Igor Pavlov : 2017-08-28
p7zip Version 17.04 (locale=de_DE.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs x64)

Scanning the drive for archives:
1 file, 23274493 bytes (23 MiB)

Extracting archive: ../jamulus_3.8.1dev-b324ee8_mac.dmg
WARNING:
../jamulus_3.8.1dev-b324ee8_mac.dmg
Can not open the file as [Dmg] archive
The file is open as [bzip2] archive

--
Path = ../jamulus_3.8.1dev-b324ee8_mac.dmg
Open WARNING: Can not open the file as [Dmg] archive
Type = bzip2

ERROR: There are some data after the end of the payload data : jamulus_3.8.1dev-b324ee8_mac

Sub items Errors: 1

Archives with Errors: 1

Sub items Errors: 1

ann0see · 2022-01-27T19:08:07Z

Should be tested once again and then squash merged. There are no SDK changes anymore

mac/deploy_mac.sh

hoffie · 2022-01-27T20:41:48Z

I've had a quick look at the .dmg build artifact and noticed that it seems to be a different format. I can't yet tell what's better or if it makes a difference at all, but I wanted to mention it.

I think someone with macOS knowledge should judge what this means. To me it just means that this PR does not only change the process, but it also changes the result, either as a result of the different tooling or as a result of a different build environment (hdituil?). This change might be positive for us and for users or it might unexpectedly break for older macOS releases. I can't tell, but someone should. :)

hoffie

Another nit. Happy to approve once there's confirmation regarding the file format.
This is probably the explanation for the format within the old build, but we still don't know why this format was chosen:
https://github.com/jamulussoftware/jamulus/pull/2207/files#diff-8f016f4f41c7ad708e4ff8d2546ce6fc8a4e3146159b2a621688c472f912ff7cL38

According to git history, this was introduced along the initial mac build logic by @doloopuntil in 8411deb.

mac/deploy_mac.sh

ann0see · 2022-01-27T21:32:45Z

I think the change in format can be explained by the use of another tool.

The dmg opens fine on Catalina, which is the oldest version I have access to. @JohannesBrx has an older Mac and he should be able to test the .dmg

This enables building the macOS installer on macOS BigSur or later

hoffie · 2022-02-07T09:12:45Z

Looks like create-dmg is hardcoded to use HFS:
https://github.com/create-dmg/create-dmg/blob/master/create-dmg#L271-L275

It also looks like HFS is the more legacy-friendly option:
electron-userland/electron-builder#5850 (comment)
#2148 (reply in thread)

APFS seems to require 10.2+ (which might be fine, at least for the regular builds?).

I don't think this is a blocker for using create-dmg though.

Edit: The create-dmg-based build seems to work on M1, so no need to change anything: #2148 (reply in thread)

JohannesBrx · 2022-02-17T22:17:33Z

I think the change in format can be explained by the use of another tool.

The dmg opens fine on Catalina, which is the oldest version I have access to. @JohannesBrx has an older Mac and he should be able to test the .dmg

@ann0see: Sorry for replying so late. Some time ago I upgraded to MacOS High Sierra because the list of unsupported software was getting longer... ;-)

hoffie

@JohannesBrx Thanks for the feedback.

So... as this is a requirement for moving to more recent Mac build environments and because there are no known downsides, I'd vote for merging this early for 3.9.0 so that it can be part of one of the betas.

@ann0see Thanks for working on this :)

ann0see · 2022-02-20T09:27:43Z

Should be merged after upcoming release (not squash merged)

ann0see · 2022-02-20T18:23:18Z

CHANGELOG: Autobuild: Build macOS .dmg files by create-dmg for building compatibility with further versions of macOS #2420

ann0see · 2022-06-24T10:39:48Z

@hoffie this PR also seems to be wrongly linked in the changelog

hoffie · 2022-06-27T22:00:31Z

@hoffie this PR also seems to be wrongly linked in the changelog

What exactly? Both Github release and git log seem to confirm that it belongs to 3.9.0, doesn't it?

ann0see · 2022-06-28T05:32:04Z

Strange. I thought we already had that in in some previous release.
So it's my memory playing wrong 😑

ann0see commented Jan 2, 2022

View reviewed changes

.github/workflows/autobuild.yml Outdated Show resolved Hide resolved

ann0see commented Jan 2, 2022

View reviewed changes

softins reviewed Jan 8, 2022

View reviewed changes

.github/workflows/autobuild.yml Show resolved Hide resolved

ann0see force-pushed the autobuild/updateOS branch 3 times, most recently from ed54a0e to d30f126 Compare January 8, 2022 21:52

pljones reviewed Jan 8, 2022

View reviewed changes

mac/deploy_mac.sh Outdated Show resolved Hide resolved

pljones reviewed Jan 9, 2022

View reviewed changes

mac/deploy_mac.sh Show resolved Hide resolved

ann0see force-pushed the autobuild/updateOS branch 2 times, most recently from 2c26ed5 to bf43017 Compare January 17, 2022 21:14

pljones reviewed Jan 17, 2022

View reviewed changes

mac/deploy_mac.sh Outdated Show resolved Hide resolved

ann0see force-pushed the autobuild/updateOS branch 3 times, most recently from cb0f738 to 9df6013 Compare January 18, 2022 07:28

ann0see marked this pull request as ready for review January 18, 2022 07:30

ann0see requested a review from pljones January 18, 2022 07:30

ann0see force-pushed the autobuild/updateOS branch from 9df6013 to f754eec Compare January 18, 2022 07:31

ann0see commented Jan 18, 2022

View reviewed changes

mac/deploy_mac.sh Outdated Show resolved Hide resolved

ann0see linked an issue Jan 18, 2022 that may be closed by this pull request

Mac build: Find dmgbuild successor #1797

Closed

ann0see mentioned this pull request Jan 25, 2022

Mac build: Find dmgbuild successor #1797

Closed

hoffie mentioned this pull request Jan 26, 2022

Build: Use actions/cache #2277

Closed

6 tasks

hoffie marked this pull request as draft January 26, 2022 10:45

hoffie marked this pull request as ready for review January 26, 2022 11:53

pljones approved these changes Jan 27, 2022

View reviewed changes

hoffie added this to the Release 3.9.0 milestone Jan 27, 2022

hoffie reviewed Jan 27, 2022

View reviewed changes

mac/deploy_mac.sh Outdated Show resolved Hide resolved

hoffie reviewed Jan 27, 2022

View reviewed changes

mac/deploy_mac.sh Outdated Show resolved Hide resolved

ann0see force-pushed the autobuild/updateOS branch 2 times, most recently from 6962b56 to fa5e0ed Compare February 2, 2022 20:02

Switch macOS build to create-dmg

bc950a2

This enables building the macOS installer on macOS BigSur or later

ann0see force-pushed the autobuild/updateOS branch from fa5e0ed to bc950a2 Compare February 2, 2022 20:03

hoffie mentioned this pull request Feb 7, 2022

Native Apple M1 releases (arm64) #2357

Closed

6 tasks

hoffie approved these changes Feb 17, 2022

View reviewed changes

ann0see merged commit 0c7dcb5 into jamulussoftware:master Feb 20, 2022

ann0see deleted the autobuild/updateOS branch February 20, 2022 18:23

hoffie mentioned this pull request Feb 20, 2022

Build: Cache Mac's brew install (create-dmg) #2412

Closed

2 tasks

hoffie mentioned this pull request Feb 28, 2022

Build: Cache brew/create-dmg install #2420

Merged

5 tasks

Switch to create-dmg #2207

Switch to create-dmg #2207

Uh oh!

Conversation

ann0see commented Jan 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

Uh oh!

ann0see commented Jan 2, 2022

Uh oh!

ann0see left a comment

Choose a reason for hiding this comment

Uh oh!

ann0see commented Jan 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emlynmac commented Jan 7, 2022

Uh oh!

ann0see commented Jan 7, 2022

Uh oh!

emlynmac commented Jan 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ann0see commented Jan 8, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ann0see commented Jan 17, 2022

Uh oh!

ann0see commented Jan 17, 2022

Uh oh!

ann0see commented Jan 17, 2022

Uh oh!

Uh oh!

ann0see commented Jan 18, 2022

Uh oh!

Uh oh!

ann0see commented Jan 18, 2022

Uh oh!

ann0see commented Jan 19, 2022

Uh oh!

hoffie commented Jan 26, 2022

Uh oh!

ann0see commented Jan 27, 2022

Uh oh!

Uh oh!

hoffie commented Jan 27, 2022

Uh oh!

hoffie left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ann0see commented Jan 27, 2022

Uh oh!

hoffie commented Feb 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JohannesBrx commented Feb 17, 2022

Uh oh!

hoffie left a comment

Choose a reason for hiding this comment

Uh oh!

ann0see commented Feb 20, 2022

Uh oh!

ann0see commented Feb 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ann0see commented Jun 24, 2022

Uh oh!

hoffie commented Jun 27, 2022

Uh oh!

ann0see commented Jun 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

ann0see commented Jan 2, 2022 •

edited

Loading

ann0see commented Jan 7, 2022 •

edited

Loading

emlynmac commented Jan 7, 2022 •

edited

Loading

hoffie commented Feb 7, 2022 •

edited

Loading

ann0see commented Feb 20, 2022 •

edited

Loading

ann0see commented Jun 28, 2022 •

edited

Loading