Skip to content

Add support for cross-compiling for ppc64le platform#9284

Merged
ejona86 merged 13 commits intogrpc:masterfrom
sumitd2:ppc64le-support
Jun 23, 2022
Merged

Add support for cross-compiling for ppc64le platform#9284
ejona86 merged 13 commits intogrpc:masterfrom
sumitd2:ppc64le-support

Conversation

@sumitd2
Copy link
Contributor

@sumitd2 sumitd2 commented Jun 17, 2022

..on the lines of #6441

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Jun 17, 2022 •
edited
Loading

CLA Signed

The committers listed above are authorized under a signed CLA.

@sumitd2 sumitd2 changed the title Add support for cross-compiling for aarch64 platform Add support for cross-compiling for ppc64le platform Jun 17, 2022
@sumitd2 sumitd2 mentioned this pull request Jun 17, 2022
Closed
ejona86
ejona86 reviewed Jun 21, 2022
View reviewed changes
@ejona86 ejona86 added the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 21, 2022
@grpc-kokoro grpc-kokoro removed the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 21, 2022
@ejona86
Copy link
Member

ejona86 commented Jun 21, 2022

Looks like we need to check what distro and stuff is being used in kokoro. :-/

+ sudo apt-get install -y g++-10-powerpc64-linux-gnu
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package g++-10-powerpc64-linux-gnu
E: Couldn't find any package by regex 'g++-10-powerpc64-linux-gnu'

@ejona86 ejona86 added the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 21, 2022
@grpc-kokoro grpc-kokoro removed kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run labels Jun 21, 2022
@ejona86 ejona86 added the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 21, 2022
@grpc-kokoro grpc-kokoro removed the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 21, 2022
@sumitd2
Copy link
Contributor Author

sumitd2 commented Jun 22, 2022

Not sure how to fix this for cross compilation:

> Task :grpc-compiler:linkJava_pluginExecutable FAILED
/usr/lib/gcc-cross/powerpc64-linux-gnu/5/../../../../powerpc64-linux-gnu/bin/ld: skipping incompatible /tmp/protobuf/lib/libprotoc.a when searching for -lprotoc
/usr/lib/gcc-cross/powerpc64-linux-gnu/5/../../../../powerpc64-linux-gnu/bin/ld: skipping incompatible //usr/local/lib/libprotoc.a when searching for -lprotoc
/usr/lib/gcc-cross/powerpc64-linux-gnu/5/../../../../powerpc64-linux-gnu/bin/ld: cannot find -lprotoc
/usr/lib/gcc-cross/powerpc64-linux-gnu/5/../../../../powerpc64-linux-gnu/bin/ld: skipping incompatible /tmp/protobuf/lib/libprotobuf.a when searching for -lprotobuf
/usr/lib/gcc-cross/powerpc64-linux-gnu/5/../../../../powerpc64-linux-gnu/bin/ld: skipping incompatible //usr/local/lib/libprotobuf.a when searching for -lprotobuf
/usr/lib/gcc-cross/powerpc64-linux-gnu/5/../../../../powerpc64-linux-gnu/bin/ld: cannot find -lprotobuf
collect2: error: ld returned 1 exit status

@ejona86 ejona86 added the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 22, 2022
@grpc-kokoro grpc-kokoro removed the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 22, 2022
@ejona86
Copy link
Member

ejona86 commented Jun 22, 2022

Those incompatible libraries are most likely x86 libraries. The incompatible libraries aren't much of a problem; they will be skipped. The real problem is cannot find -lprotoc, which means the ppc libraries were not available. If you look at the end of make_dependencies.sh, you'll see:

export LDFLAGS=-L/tmp/protobuf/lib
export CXXFLAGS=-I/tmp/protobuf/include
export LD_LIBRARY_PATH=/tmp/protobuf/lib

-L in LDFLAGS adds a path to be searched when linking (the step you are seeing an error). And LD_LIBRARY_PATH adds a path to be searched when running the binary.

@sumitd2
Copy link
Contributor Author

sumitd2 commented Jun 22, 2022

The host string looks wrong to me. Adding the fix...

+ ./configure --disable-shared --host=ppc64le-linux-gnu --prefix=/tmp/protobuf-cache/3.19.2/Linux-ppcle_64
checking host system type... powerpc64le-unknown-linux-gnu
checking target system type... powerpc64le-unknown-linux-gnu
checking for ppc64le-linux-gnu-gcc... no
checking for gcc... gcc
**checking whether we are cross compiling... no**
checking for ppc64le-linux-gnu-g++... no
checking for g++... g++

+ ./configure --disable-shared --host=aarch64-linux-gnu --prefix=/tmp/protobuf-cache/3.19.2/Linux-aarch_64
checking build system type... x86_64-pc-linux-gnu
checking host system type... aarch64-unknown-linux-gnu
checking target system type... aarch64-unknown-linux-gnu
checking for aarch64-linux-gnu-gcc... aarch64-linux-gnu-gcc
**checking whether we are cross compiling... yes**

@ejona86 ejona86 added the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 22, 2022
@grpc-kokoro grpc-kokoro removed the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 22, 2022
@sumitd2
Copy link
Contributor Author

sumitd2 commented Jun 22, 2022 •
edited
Loading

exe was built successfully. Are we using the wrong objdump?

> Task :grpc-compiler:checkArtifacts FAILED

Checking format of build/artifacts/java_plugin/protoc-gen-grpc-java.exe
Format=elf64-little
Assertion failed:  "elf64-little" == "elf64-powerpcle"
File "check-artifact.sh", line 65

FAILURE: Build failed with an exception.

@ejona86
Copy link
Member

ejona86 commented Jun 22, 2022

Oh, it is close!

Hmmm... That's an explicit check in the check-artifact.sh script. It does seem broken that arm and ppc would have the same string. Line 55 does call objdump without a prefix.

@sumitd2
Copy link
Contributor Author

sumitd2 commented Jun 22, 2022

@ejona86
Confirmed, its the wrong objdump. Appended is the output for power exe on x86.
Have put in a fix.

[root@acs-x86-node1-ghatwala-rhel sumit]# objdump -f protoc-gen-grpc-java
protoc-gen-grpc-java:     file format elf64-little
architecture: UNKNOWN!, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x0000000010008628

@ejona86 ejona86 added the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 22, 2022
@grpc-kokoro grpc-kokoro removed the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 22, 2022
@sumitd2
Copy link
Contributor Author

sumitd2 commented Jun 23, 2022

@ejona86 Looks like the Bazel build failed due to a JVM crash. All else LGTM.

ejona86
ejona86 approved these changes Jun 23, 2022
View reviewed changes
Copy link
Member

@ejona86 ejona86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change in check-artifacts for getting the format should probably be cleaned up in the future, but we can do that later.

@ejona86 ejona86 requested a review from sanjaypujare June 23, 2022 15:12
@ejona86 ejona86 added the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 23, 2022
@grpc-kokoro grpc-kokoro removed the kokoro:run Add this label to a PR to tell Kokoro the code is safe and tests can be run label Jun 23, 2022
sanjaypujare
sanjaypujare reviewed Jun 23, 2022
View reviewed changes
sanjaypujare
sanjaypujare approved these changes Jun 23, 2022
View reviewed changes
Copy link
Contributor

@sanjaypujare sanjaypujare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ejona86 ejona86 enabled auto-merge (squash) June 23, 2022 16:05
@ejona86 ejona86 disabled auto-merge June 23, 2022 16:06
@ejona86 ejona86 enabled auto-merge (squash) June 23, 2022 16:07
@ejona86 ejona86 merged commit 04f9bdc into grpc:master Jun 23, 2022
@sumitd2 sumitd2 deleted the ppc64le-support branch June 23, 2022 16:18
larry-safran pushed a commit to larry-safran/grpc-java that referenced this pull request Jun 28, 2022
@sumitd2 @larry-safran
compiler: Cross-compile for ppc64le platform (grpc#9284)
7868893 
Fixes grpc#9274
@haubenr haubenr mentioned this pull request Aug 18, 2022
Merged
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@ejona86 ejona86 ejona86 approved these changes

+1 more reviewer

@sanjaypujare sanjaypujare sanjaypujare approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sumitd2 @ejona86 @sanjaypujare @grpc-kokoro