Skip to content

Conversation

@MeFisto94
Copy link
Member

So I looked into building android natives and made them work on travis despite the jdk-11 changes which broke them.

I added another build to the matrix, which is android. That way the Android NDK is just easily available.

The header generation has been moved into jme3-android, which generates headers during the compilation which happens anyway (though now headers are generated if noone wants to build natives).

OpenJDK 10 is critical here, because the JDK Version 11 does not support Java EE Modules (XML) anymore, which are required by Android Tools.

What is left is the integration, there are two options:

  1. Do it like Bullet-Native: Commit the binaries to the repository. These are already here, because when someone wants to build jme without having the Android NDK the existing binaries are just copied. The downside is that a release tag must not be an android-commit because the binaries are only updated the commit after.
  2. Do it like one would do: Ensure jme3-android.jar is published to maven by this task and not by all the other envs (two linuxes, one osx).

Personally I am in for a hybrid approach: 1 + 2, usually 2 is relevant for most users but we need number 1 to allow users to build the engine (including the jme3-android) module without any issues.

Copy link
Member

@stephengold stephengold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for your contribution. Some of the changes seem unnecessary:

  1. In .travis.yml, the added commented code (from googles NDK example) is more confusing than helpful.
  2. In .travis.yml, changing the sudo setting is a good idea, but unless it's necessary for building Android natives, it should be a separate PR.
  3. the changes to jme3-android-native/build.gradle look unnecessary to me.

Also, could you do a similar PR for building the jme3-bullet-native-android library?

@MeFisto94
Copy link
Member Author

  1. That is true, I will check if it is relevant for bullet though first, as it installs cmake and others.

  2. I thought that a separate PR is unnecessary for 3 characters changed. Especially since the testing is the same: if something breaks with sudo: true, it would also fail with sudo: false. But if desired, I will do so.

  3. Correct. To explain a bit on that: I was experimenting and had the issue that TASKS are global and thus can't have the same name in different files, but Strings/Variables can. So I first thought the order of declaration matters, thus I changed that order. I will revert that.

  4. Bullet Native: I wonder if I can/should integrate it into the already existing build system, as there is no point to build 2 libraries separate from the third. Or is this relevant because of artifacts? (Because when being part of jme3-android[-natives], bullet native can't be excluded anymore, however I think someone said that jbullet doesn't work on android anyway or at least not fast enough?)

@stephengold
Copy link
Member

Re 2: Please make a separate PR so the change can be a separate commit to the repo. Among other things, separate commits for separate issues simplify cherrypicking.

Re 4: It's good to keep jme3-bullet-native-android separate from jme3-bullet-native. If someone is building an Android-only app or a desktop-only app, they don't want to include both sets of native libs.

@MeFisto94
Copy link
Member Author

3: Couldn't solve that entirely because I had a whiteline at the end and now git refuses to commit because the file only has a whitespace change.
2. has been changed.

  1. will be solved when working on 4.

@MeFisto94
Copy link
Member Author

so 4 will be a separate pull request later, because it affects jme3-bullet(-native), as it looks like bullet-native-android doesn't contain any native code at all but seems to copy it from jme3-bullet.

@stephengold stephengold merged commit 90d3b69 into jMonkeyEngine:master Sep 14, 2019
@MeFisto94
Copy link
Member Author

Thanks for the merge, just one thing: The deployment still has to be thought of. Currently it would try to deploy but I guess it depends on which OS has compiled first. It will work fine until the first tagged commit.

That means excluding the tasks from the other build matrices. Then deployment works. The second step would be to modify the upload script from bullet to also upload the android natives though, because otherwise while the artifacts are updated, everyone building the engine manually would be left of with the ancient files in this repository (prebuilt libraries)

@stephengold
Copy link
Member

stephengold commented Sep 15, 2019

I'm not sure I understand. The way I see it, deployment should happen in 2 stages, as it has for jme3-bullet-native in recent years:

  1. For each commit that changes a native library, a CI process builds the new .so and commits it to a libs directory in the repository, commenting it with [ci skip].
  2. For each tagged commit, a CI process builds new .jar, .pom, and .asc files and deploys them to JFrog.

Unless commits come close together, or a tagged commit changes a native library, there should be no race between these processes. The main downside I see to this approach is that the repo grows rapidly.

@MeFisto94
Copy link
Member Author

Exactly. Even when a tagged commit changes a native library there isn't much of a problem because the .jar will contain the new libraries. Repo Size Growth can't be avoided without changing this completely, that's true as well.

What has to be done:

  1. We have a script for that, it currently only checks for bullet-native, the two android builds have to be added there
  2. For this, the regular build could exclude the native modules from being deployed, because then they get deployed from the android build. I mean: the main build excludes jme3-android-{native, bullet-native} and they get deployed by the build I added with this PR. Otherwise there would have to be yet another roundtrip with [ci skip].

@stephengold
Copy link
Member

Are you working on this?

@MeFisto94
Copy link
Member Author

I wasn't so far but yeah I should do these changes

MeFisto94 added a commit that referenced this pull request Sep 17, 2019
…o they don't conflict with the android build matrix (#1171)
MeFisto94 added a commit that referenced this pull request Sep 17, 2019
@MeFisto94
Copy link
Member Author

MeFisto94 commented Sep 17, 2019

I am done. The problem is: we can only see if it works on the next tagged release and/or as soon as some natives change. Unless 21d0854 already commits something because things have changed.
The only thing which could be changed is that all 3 natives are updated in one commit, but that way it's more clean I think

Edit: Alternatively we could update openAL for android, but we dont have people testing if this brings in more bugs than before, so...

@stephengold
Copy link
Member

Luckily I have some native change planned, along with a tagged release!

@stephengold
Copy link
Member

I just committed 30d1eca which should trigger recompilation of all the Bullet natives.

@stephengold
Copy link
Member

Hmm. The OSX/Windows native libraries got updated, but so far I haven't seen any evidence that the Android/Linux ones did. Please check.

@stephengold
Copy link
Member

The Android build failed to commit the native libs because they weren't staged.

@MeFisto94
Copy link
Member Author

I see two oddities there. https://travis-ci.org/jMonkeyEngine/jmonkeyengine/jobs/586240198#L2291 Android Natives seem to have changed as well, why didn't they do that before? Or did I just not look closely enough?

The second one, you are right, is not staging them, however there is https://github.com/jMonkeyEngine/jmonkeyengine/blob/master/private/upload_native.sh#L31
Why doesn't that work? Do I need a star * so that it picks up the subfolders?

Also given that android-natives changed as well, I was expecting to see that output twice.

Since you are more experienced with the git command line: Do you spot some issues there?

@stephengold
Copy link
Member

stephengold commented Sep 18, 2019

Actually, since I use GUI tools for most of my gitwork, I'm unfortunately very ignorant of the git command line.

Yesterday I looked carefully at upload_native.sh, experimented with the git add command on my desktop, and didn't find the issue. I will keep looking, since I don't want to cut the alpha3 release with incorrect native libraries.

At this point, my inclination would be to add a bunch of echo statements to the script.

If we're both stuck, perhaps we should get others involved in the debugging.

@stephengold
Copy link
Member

I'm about to add some debugging commands to upload_native.sh and trigger another build.

@stephengold
Copy link
Member

I just noticed this failure in the Linux build: https://travis-ci.org/jMonkeyEngine/jmonkeyengine/jobs/586791631#L386

As for the Android build, it looks like the upload_native.sh script terminated silently somewhere before Push changes in jme3-bullet-native. Non-zero error status from the git commit command?

Also, I may have figured out why the Android build is trying to commit jme3-bullet-native libraries: a race condition. TRAVIS_COMMIT refers to the commit that triggered the CI build, but during the after_success stage, this might not be the most recent commit. The git diff-tree --name-only "$TRAVIS_COMMIT" -- jme3-bullet-native is probably detecting changes that were committed by another CI process such as the Windows process at Appveyor. I'll research the Git command line to see if I can solve the race.

@stephengold
Copy link
Member

I've confirmed that git commit sets exit code 1 when there is nothing to commit. That, combined with the set -euo pipefail at the top of upload_native.sh, explains why the Android build aborted halfway through that script.

@stephengold
Copy link
Member

Using git diff got us closer to success, but now the pull-with-rebase fails:
https://travis-ci.org/jMonkeyEngine/jmonkeyengine/jobs/586809734#L2341

@stephengold
Copy link
Member

Android side is working. Looking at the Linux bulletjme error.

@stephengold
Copy link
Member

Resembles #915

@stephengold
Copy link
Member

I foolishly tested a .travis.yml change in the main repo. Should've done that in my own fork. Now the build is broken, but hopefully that'll be temporary.

@stephengold stephengold added this to the v3.3.0 milestone Mar 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants