Skip to content

subversion: build javahl on Apple Silicon#76900

Closed
markphip wants to merge 1 commit intoHomebrew:masterfrom
markphip:javahl-m1
Closed

subversion: build javahl on Apple Silicon#76900
markphip wants to merge 1 commit intoHomebrew:masterfrom
markphip:javahl-m1

Conversation

@markphip
Copy link
Copy Markdown
Contributor

@markphip markphip commented May 9, 2021

This was initially disabled due to OpenJDK not
being available.

  • Have you followed the guidelines for contributing?
  • Have you checked that there aren't other open pull requests for the same formula update/change?
  • Have you built your formula locally with brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
  • Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
  • Does your build pass brew audit --strict <formula> (after doing brew install <formula>)?

I verified that the JavaHL library works by using it via Subclipse in Eclipse.
Note there is a sqlite related problem with JavaHL that already exists on Intel
and also exists on Apple Silicon. This either came about with Big Sur or Homebrew 3.0
I am investigating options and will possibly submit a separate PR for that.

@BrewTestBot BrewTestBot added java Java use is a significant feature of the PR or issue perl Perl use is a significant feature of the PR or issue ruby Ruby use is a significant feature of the PR or issue labels May 9, 2021
Copy link
Copy Markdown
Member

@carlocab carlocab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your PR, @markphip.

Please bump the revision (either add revision 1 or increment the revision right after license) so that users who have this installed get the updated version of the formula when they do brew upgrade.

@markphip
Copy link
Copy Markdown
Contributor Author

markphip commented May 9, 2021

ok, thanks @carlocab I was not sure if I was supposed to do that or not since this formula did not already have revision specified. I have added it to the formula.

@carlocab
Copy link
Copy Markdown
Member

carlocab commented May 9, 2021

That's fine. Forgot to mention: please squash your commits together. Thanks!

@markphip
Copy link
Copy Markdown
Contributor Author

markphip commented May 9, 2021

Thanks for the help @carlocab

@carlocab
Copy link
Copy Markdown
Member

carlocab commented May 9, 2021

Sorry, one last thing: let's put the --with-jdk and --enable-javahl flags back in the args array, reverting the relevant parts from this commit: 41f685f (squash your changes again please, thanks)

This was initially disabled due to OpenJDK not
being available.
@markphip
Copy link
Copy Markdown
Contributor Author

markphip commented May 9, 2021

@carlocab that makes sense, nice catch. Done

Copy link
Copy Markdown
Member

@carlocab carlocab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your first contribution to Homebrew/core, @markphip.

BrewTestBot will merge this soon after the CI run is complete.

@BrewTestBot
Copy link
Copy Markdown
Contributor

🤖 A scheduled task has triggered a merge.

@moonfruit
Copy link
Copy Markdown
Contributor

Why not build subversion Perl module for Apple Silicon?

@carlocab
Copy link
Copy Markdown
Member

Does it work? It was deactivated in bdd7b9f because it didn't at the time. But if you know it works, you are welcome to open a PR to enable it.

@markphip
Copy link
Copy Markdown
Contributor Author

@moonfruit I am not a Perl user so would not know how to verify it works.

I am toying with opening a new PR to change sqlite to use the system version from MacOS. I am not sure how others would feel about it. Since Big Sur on Intel and Apple Silicon, JavaHL no longer works in Eclipse (at least the working copy code that needs SQLite does not work). This is because for some reason the system library loads and there is a version mismatch. I can work around the problem by starting Eclipse from a Terminal after running: export DYLD_LIBRARY_PATH=/opt/homebrew/opt/sqlite/lib. This does not feel like a very good solution as most Eclipse users would not be able to do this.

Building Subversion to use the system sqlite fixes the problem. I do not have easy access to Mojave and Catalina to know if it is a good solution for those versions.

Thoughts? Should I just send a PR to have this discussion there?

From what I can tell the JavaHL library is already building correctly. I am not sure what changed in Big Sur that makes this no longer work. That said, Big Sur and Homebrew 3.0 all happened around the same time so it is hard to know for certain where the root problem lives.

@markphip
Copy link
Copy Markdown
Contributor Author

This is what removing sqlite dependency would look like: markphip@cf8b319

@markphip
Copy link
Copy Markdown
Contributor Author

@moonfruit I tried to remove the Intel conditions on the Perl bindings. It seems to build and install OK but when you run the tests it fails.

brew test subversion
==> Testing subversion
==> /opt/homebrew/Cellar/subversion/1.14.1_2/bin/svnadmin create test
==> /opt/homebrew/Cellar/subversion/1.14.1_2/bin/svnadmin verify test
==> /usr/bin/perl -e use SVN::Client; new SVN::Client()
Last 15 lines from /Users/markphip/Library/Logs/Homebrew/subversion/test.03.perl:
2021-05-10 10:26:58 -0400

/usr/bin/perl
-e
use SVN::Client; new SVN::Client()

Can't locate SVN/Client.pm in @INC (you may need to install the SVN::Client module) (@INC contains: /opt/homebrew/Cellar/subversion/1.14.1_2/lib/perl5/site_perl/5.30.2/darwin-thread-multi-2level /Library/Perl/5.30/darwin-thread-multi-2level /Library/Perl/5.30 /Network/Library/Perl/5.30/darwin-thread-multi-2level /Network/Library/Perl/5.30 /Library/Perl/Updates/5.30.2 /System/Library/Perl/5.30/darwin-thread-multi-2level /System/Library/Perl/5.30 /System/Library/Perl/Extras/5.30/darwin-thread-multi-2level /System/Library/Perl/Extras/5.30) at -e line 1.
BEGIN failed--compilation aborted at -e line 1.
Error: subversion: failed
An exception occurred within a child process:
  BuildError: Failed executing: /usr/bin/perl -e use\ SVN::Client;\ new\ SVN::Client()

@carlocab
Copy link
Copy Markdown
Member

subversion has used brewed sqlite forever, and it's hard to say what the consequences of changing it will be. I'd be reluctant to make that change given that no one's really asking for it (since you have your own workaround).

Still, you may wish to open a PR so that it can be reviewed and discussed appropriately.

@markphip
Copy link
Copy Markdown
Contributor Author

subversion has used brewed sqlite forever, and it's hard to say what the consequences of changing it will be. I'd be reluctant to make that change given that no one's really asking for it (since you have your own workaround).

Still, you may wish to open a PR so that it can be reviewed and discussed appropriately.

I agree and I am also reluctant. I am a member of the Subversion PMC and the maintainer for Subclipse. I have already pinged some others from Subversion to see if they have any other ideas on better workarounds I could pursue or try to understand what has changed to cause the problem.

I would feel slightly more comfortable if this was limited to Big Sur where I can at least see that Apple is providing a fairly current version of sqlite in the OS. I have no idea what version is provided in Mojave and Catalina.

I will submit the PR. I just did not know if that was an acceptable way to start this conversation.

@Bo98
Copy link
Copy Markdown
Member

Bo98 commented May 10, 2021

it's hard to say what the consequences of changing it will be.

There was some consistency issues where some dependents used system libs and some did not. That's no longer the case so I'm happy to just use system SQLite if it works.

@markphip
Copy link
Copy Markdown
Contributor Author

Thanks @Bo98 .. I opened a new PR #76970 so let's move the conversation over there.

@carlocab
Copy link
Copy Markdown
Member

I will submit the PR. I just did not know if that was an acceptable way to start this conversation.

Code changes are usually small and relatively isolated in Homebrew/core, so PRs are always our preferred means of discussion.

I suspect also that the issue you mention in #76900 (comment) could be a bug in Eclipse, as it sounds like it is assuming the default library search path, but shouldn't be.

@markphip markphip mentioned this pull request May 10, 2021
5 tasks
@moonfruit
Copy link
Copy Markdown
Contributor

moonfruit commented May 11, 2021

@markphip Sorry, I have no M1 chip MacBook, so I cannot know what's happened.

But according to the output you provide, I think you should find what's in /opt/homebrew/Cellar/subversion/1.14.1_2/lib/perl5/site_perl/5.30.2/darwin-thread-multi-2level/. It seems there is no subversion perl module in that directory.

Maybe this path is not correct on Apple Silicon macOS. Can you paste what you see in /opt/homebrew/Cellar/subversion/1.14.1_2/lib/perl5/, /opt/homebrew/Cellar/subversion/1.14.1_2/lib/perl5/site_perl/ and /opt/homebrew/Cellar/subversion/1.14.1_2/lib/perl5/site_perl/5.30.2/

@markphip
Copy link
Copy Markdown
Contributor Author

@moonfruit it looks like the issue is that the folder name that gets installed in that folder is 5.32.1 not 5.30.2 ?

@moonfruit
Copy link
Copy Markdown
Contributor

OK, I know, this is because you installed homebrew's perl which is 5.32.1.
Now, subversion build perl module against to perl which in the PATH.

This patch is for only building perl module to system perl.
I think should be applied to #76964 or I will make a pull request after that is accepted.

diff --git a/Formula/subversion.rb b/Formula/subversion.rb
index 1eac4ac5b6..7cfbcb3462 100644
--- a/Formula/subversion.rb
+++ b/Formula/subversion.rb
@@ -112,6 +112,7 @@ class Subversion < Formula
       --without-jikes
       PYTHON=#{Formula["python@3.9"].opt_bin}/python3
       RUBY=/usr/bin/ruby
+      PERL=/usr/bin/perl
     ]

     inreplace "Makefile.in",
@@ -137,34 +138,25 @@ class Subversion < Formula
     system "make", "javahl"
     system "make", "install-javahl"

-    if Hardware::CPU.intel?
-      perl_archlib = Utils.safe_popen_read("perl", "-MConfig", "-e", "print $Config{archlib}")
-      perl_core = Pathname.new(perl_archlib)/"CORE"
-      perl_extern_h = perl_core/"EXTERN.h"
+    perl_version = Utils.safe_popen_read("/usr/bin/perl", "--version")[/v(\d+\.\d+)(?:\.\d+)?/, 1]
+    perl_core = MacOS.sdk_path/"System/Library/Perl"/perl_version/"darwin-thread-multi-2level/CORE"
+    perl_extern_h = perl_core/"EXTERN.h"

-      unless perl_extern_h.exist?
-        # No EXTERN.h, maybe it's system perl
-        perl_version = Utils.safe_popen_read("perl", "--version")[/v(\d+\.\d+)(?:\.\d+)?/, 1]
-        perl_core = MacOS.sdk_path/"System/Library/Perl"/perl_version/"darwin-thread-multi-2level/CORE"
-        perl_extern_h = perl_core/"EXTERN.h"
-      end
-
-      onoe "'#{perl_extern_h}' does not exist" unless perl_extern_h.exist?
+    onoe "'#{perl_extern_h}' does not exist" unless perl_extern_h.exist?

-      inreplace "Makefile" do |s|
-        s.change_make_var! "SWIG_PL_INCLUDES",
-          "$(SWIG_INCLUDES) -arch x86_64 -g -pipe -fno-common " \
-          "-DPERL_DARWIN -fno-strict-aliasing -I#{HOMEBREW_PREFIX}/include -I#{perl_core}"
-      end
-      system "make", "swig-pl"
-      system "make", "install-swig-pl"
-
-      # This is only created when building against system Perl, but it isn't
-      # purged by Homebrew's post-install cleaner because that doesn't check
-      # "Library" directories. It is however pointless to keep around as it
-      # only contains the perllocal.pod installation file.
-      rm_rf prefix/"Library/Perl"
+    inreplace "Makefile" do |s|
+      s.change_make_var! "SWIG_PL_INCLUDES",
+        "$(SWIG_INCLUDES) -arch x86_64 -g -pipe -fno-common " \
+        "-DPERL_DARWIN -fno-strict-aliasing -I#{HOMEBREW_PREFIX}/include -I#{perl_core}"
     end
+    system "make", "swig-pl"
+    system "make", "install-swig-pl"
+
+    # This is only created when building against system Perl, but it isn't
+    # purged by Homebrew's post-install cleaner because that doesn't check
+    # "Library" directories. It is however pointless to keep around as it
+    # only contains the perllocal.pod installation file.
+    rm_rf prefix/"Library/Perl"
   end

   def caveats
@@ -185,11 +177,9 @@ class Subversion < Formula
     system "#{bin}/svnadmin", "create", "test"
     system "#{bin}/svnadmin", "verify", "test"

-    if Hardware::CPU.intel?
-      perl_version = Utils.safe_popen_read("/usr/bin/perl", "--version")[/v(\d+\.\d+(?:\.\d+)?)/, 1]
-      ENV["PERL5LIB"] = "#{lib}/perl5/site_perl/#{perl_version}/darwin-thread-multi-2level"
-      system "/usr/bin/perl", "-e", "use SVN::Client; new SVN::Client()"
-    end
+    perl_version = Utils.safe_popen_read("/usr/bin/perl", "--version")[/v(\d+\.\d+(?:\.\d+)?)/, 1]
+    ENV["PERL5LIB"] = "#{lib}/perl5/site_perl/#{perl_version}/darwin-thread-multi-2level"
+    system "/usr/bin/perl", "-e", "use SVN::Client; new SVN::Client()"
   end
 end

@markphip
Copy link
Copy Markdown
Contributor Author

@moonfruit your patch still fails during the test command:

==> /usr/bin/perl -e use SVN::Client; new SVN::Client()
Last 15 lines from /Users/markphip/Library/Logs/Homebrew/subversion/test.03.perl:
2021-05-11 09:24:17 -0400

/usr/bin/perl
-e
use SVN::Client; new SVN::Client()

Can't load '/opt/homebrew/Cellar/subversion/1.14.1_1/lib/perl5/site_perl/5.30.2/darwin-thread-multi-2level/auto/SVN/_Core/_Core.bundle' for module SVN::_Core: dlopen(/opt/homebrew/Cellar/subversion/1.14.1_1/lib/perl5/site_perl/5.30.2/darwin-thread-multi-2level/auto/SVN/_Core/_Core.bundle, 0x0001): symbol '_svn_swig_pl_cancel_func' not found, expected in flat namespace by '/opt/homebrew/Cellar/subversion/1.14.1_1/lib/perl5/site_perl/5.30.2/darwin-thread-multi-2level/auto/SVN/_Core/_Core.bundle' at /System/Library/Perl/5.30/darwin-thread-multi-2level/DynaLoader.pm line 197.
 at /opt/homebrew/Cellar/subversion/1.14.1_1/lib/perl5/site_perl/5.30.2/darwin-thread-multi-2level/SVN/Base.pm line 59.
BEGIN failed--compilation aborted at /opt/homebrew/Cellar/subversion/1.14.1_1/lib/perl5/site_perl/5.30.2/darwin-thread-multi-2level/SVN/Core.pm line 5.
Compilation failed in require at /opt/homebrew/Cellar/subversion/1.14.1_1/lib/perl5/site_perl/5.30.2/darwin-thread-multi-2level/SVN/Client.pm line 4.
BEGIN failed--compilation aborted at /opt/homebrew/Cellar/subversion/1.14.1_1/lib/perl5/site_perl/5.30.2/darwin-thread-multi-2level/SVN/Client.pm line 4.
Compilation failed in require at -e line 1.
BEGIN failed--compilation aborted at -e line 1.
Error: subversion: failed

FWIW, this is the diff of your change and the one I was previously testing:

diff --git a/Formula/subversion.rb b/Formula/subversion.rb
index 07d585569a..7cfbcb3462 100644
--- a/Formula/subversion.rb
+++ b/Formula/subversion.rb
@@ -5,7 +5,7 @@ class Subversion < Formula
   mirror "https://archive.apache.org/dist/subversion/subversion-1.14.1.tar.bz2"
   sha256 "2c5da93c255d2e5569fa91d92457fdb65396b0666fad4fd59b22e154d986e1a9"
   license "Apache-2.0"
-  revision 2
+  revision 1
 
   bottle do
     sha256 arm64_big_sur: "cc10a37f931098a527772624566986313996921ef4db503574894e7c879a148e"
@@ -112,6 +112,7 @@ class Subversion < Formula
       --without-jikes
       PYTHON=#{Formula["python@3.9"].opt_bin}/python3
       RUBY=/usr/bin/ruby
+      PERL=/usr/bin/perl
     ]
 
     inreplace "Makefile.in",
@@ -137,17 +138,10 @@ class Subversion < Formula
     system "make", "javahl"
     system "make", "install-javahl"
 
-    perl_archlib = Utils.safe_popen_read("perl", "-MConfig", "-e", "print $Config{archlib}")
-    perl_core = Pathname.new(perl_archlib)/"CORE"
+    perl_version = Utils.safe_popen_read("/usr/bin/perl", "--version")[/v(\d+\.\d+)(?:\.\d+)?/, 1]
+    perl_core = MacOS.sdk_path/"System/Library/Perl"/perl_version/"darwin-thread-multi-2level/CORE"
     perl_extern_h = perl_core/"EXTERN.h"
 
-    unless perl_extern_h.exist?
-      # No EXTERN.h, maybe it's system perl
-      perl_version = Utils.safe_popen_read("perl", "--version")[/v(\d+\.\d+)(?:\.\d+)?/, 1]
-      perl_core = MacOS.sdk_path/"System/Library/Perl"/perl_version/"darwin-thread-multi-2level/CORE"
-      perl_extern_h = perl_core/"EXTERN.h"
-    end
-
     onoe "'#{perl_extern_h}' does not exist" unless perl_extern_h.exist?
 
     inreplace "Makefile" do |s|

@moonfruit
Copy link
Copy Markdown
Contributor

This patch mostly applies to #76964 and will be solved there. If there are some problems, you can post there.

@gromgit gromgit mentioned this pull request Jun 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

java Java use is a significant feature of the PR or issue perl Perl use is a significant feature of the PR or issue ruby Ruby use is a significant feature of the PR or issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants