Fix Android build by Frenzie · Pull Request #482 · koreader/koreader-base

Frenzie · 2017-04-11T13:39:50Z

houqp · 2017-04-12T09:14:15Z

@Frenzie I assume you have tested this on an android device? What NDK version did you test with?

houqp · 2017-04-24T06:41:34Z

I think this patch fixed the random build failure on gitlab! Although I still don't know why it would randomly build libraries without version suffixes. It also explains why it was working for me previously. I really wonder where is the entropy coming from :(

Frenzie · 2017-04-24T06:45:51Z

How does it explain that it was working for you?

…

On Apr 24, 2017 08:41, "Qingping Hou" ***@***.***> wrote: I think this patch fixed the random build failure on gitlab! Although I still don't know why it would randomly build libraries without version suffixes. It also explains why it was working for me previously. I really wonder where is the entropy coming from :( — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#482 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMYBQIv9ICHlZsSEFLhqzQ4ciZMHv1tks5rzEQfgaJpZM4M6HIX> .

houqp · 2017-04-24T06:50:03Z

Because I was using the exact same toolchain to build android without issue and I was like why the hell is it broken for me now. If it will randomly produce builds without version suffix, then it sort of makes sense.

Frenzie · 2017-04-24T06:56:11Z

Randomly produce versionless? Make sense? Lol!

…

On Apr 24, 2017 08:50, "Qingping Hou" ***@***.***> wrote: Because I was using the exact same toolchain to build android without issue and I was like why the hell is it broken for me now. If it will randomly produce builds without version suffix, then it sort of makes sense. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#482 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMYBbd3CHUSnxchfns4zwL5ESQS5rvoks5rzEYbgaJpZM4M6HIX> .

houqp · 2017-04-24T07:40:34Z

Makes sense in the sense that it breaks my local build ;)

chrox · 2017-06-02T04:58:09Z

This definitely will not work. On Android we use a custom dl function to recursively load dynamic libraries by parsing the ELF dynamic section and loading NEEDED shared library.

libs $ greadelf -d libmupdf.so

Dynamic section at offset 0x303a1c contains 35 entries:
  Tag        Type                         Name/Value
 0x00000003 (PLTGOT)                     0x304d98
 0x00000002 (PLTRELSZ)                   1208 (bytes)
 0x00000017 (JMPREL)                     0x24a48
 0x00000014 (PLTREL)                     REL
 0x00000011 (REL)                        0x13aa0
 0x00000012 (RELSZ)                      69544 (bytes)
 0x00000013 (RELENT)                     8 (bytes)
 0x6ffffffa (RELCOUNT)                   8690
 0x00000006 (SYMTAB)                     0x148
 0x0000000b (SYMENT)                     16 (bytes)
 0x00000005 (STRTAB)                     0x7968
 0x0000000a (STRSZ)                      33642 (bytes)
 0x00000004 (HASH)                       0xfcd4
 0x00000001 (NEEDED)                     Shared library: [libz.so.1]
 0x00000001 (NEEDED)                     Shared library: [libjpeg.so.8]
 0x00000001 (NEEDED)                     Shared library: [libfreetype.so.6]
 0x00000001 (NEEDED)                     Shared library: [libm.so]
 0x00000001 (NEEDED)                     Shared library: [liblog.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]
 0x00000001 (NEEDED)                     Shared library: [libdl.so]
 0x0000000e (SONAME)                     Library soname: [libmupdf.so]
 0x0000001a (FINI_ARRAY)                 0x30495c
 0x0000001c (FINI_ARRAYSZ)               8 (bytes)
 0x00000019 (INIT_ARRAY)                 0x304a18
 0x0000001b (INIT_ARRAYSZ)               4 (bytes)
 0x0000000f (RPATH)                      Library rpath: [$ORIGIN]
 0x00000010 (SYMBOLIC)                   0x0
 0x0000001e (FLAGS)                      SYMBOLIC BIND_NOW
 0x6ffffffb (FLAGS_1)                    Flags: NOW
 0x6ffffff0 (VERSYM)                     0x12b00
 0x6ffffffc (VERDEF)                     0x13a04
 0x6ffffffd (VERDEFNUM)                  1
 0x6ffffffe (VERNEED)                    0x13a20
 0x6fffffff (VERNEEDNUM)                 4
 0x00000000 (NULL)                       0x0

It won't find

0x00000001 (NEEDED)                     Shared library: [libjpeg.so.8]
 0x00000001 (NEEDED)                     Shared library: [libfreetype.so.6]

these two libs any more as the version suffixes are omitted.

libs $ ls
libcrengine.so      libfreetype.so      libkoreader-cre.so  libluacompat52.so   libpng16.so         libturbojpeg.so     libzyre.so
libcrypto.so.1.0.0  libgif.so           libkoreader-lfs.so  libluajit.so        libsqlite3.so       libwrap-mupdf.so
libczmq.so          libjpeg.so          liblept.so.4        liblualongnumber.so libssl.so.1.0.0     libz.so.1
libfmq.so           libk2pdfopt.so.2    liblodepng.so       libmupdf.so         libtesseract.so.3   libzmq.so

One solution might be making https://github.com/koreader/android-luajit-launcher/blob/master/assets/dl.lua#L51 handling so files without version suffixes.

houqp · 2017-06-02T05:37:19Z

Weird, why is it linked to libfreetype.so.6 when the actual dynamic library produced is libfreetype.so?

chrox · 2017-06-02T05:47:39Z

Generally the file without version suffix is just a soft link to the one with suffix and the soname certainly contains version suffix.

houqp · 2017-06-02T06:05:01Z

I know that's the case for standard gnu toolchain. I was under the impression that android NDK toolchain actually produces some of the dynamic libraries without the suffix. But I might completely wrong since the last time I touched android build was a long time ago ;)

chrox · 2017-06-03T01:03:14Z

It can be easily verified with this:

libs $ ls
libcrengine.so      libfreetype.so      libkoreader-cre.so  libluacompat52.so   libpng16.so         libturbojpeg.so     libzyre.so
libcrypto.so.1.0.0  libgif.so           libkoreader-lfs.so  libluajit.so        libsqlite3.so       libwrap-mupdf.so
libczmq.so          libjpeg.so          liblept.so.4        liblualongnumber.so libssl.so.1.0.0     libz.so.1
libfmq.so           libk2pdfopt.so.2    liblodepng.so       libmupdf.so         libtesseract.so.3   libzmq.so
libs $ greadelf -d libfreetype.so

Dynamic section at offset 0x61df0 contains 28 entries:
  Tag        Type                         Name/Value
 0x00000003 (PLTGOT)                     0x62f64
 0x00000002 (PLTRELSZ)                   288 (bytes)
 0x00000017 (JMPREL)                     0x4aa8
 0x00000014 (PLTREL)                     REL
 0x00000011 (REL)                        0x2838
 0x00000012 (RELSZ)                      8816 (bytes)
 0x00000013 (RELENT)                     8 (bytes)
 0x6ffffffa (RELCOUNT)                   1099
 0x00000006 (SYMTAB)                     0x148
 0x0000000b (SYMENT)                     16 (bytes)
 0x00000005 (STRTAB)                     0xfc8
 0x0000000a (STRSZ)                      4007 (bytes)
 0x00000004 (HASH)                       0x1f70
 0x00000001 (NEEDED)                     Shared library: [libc.so]
 0x0000000e (SONAME)                     Library soname: [libfreetype.so.6]
 0x0000001a (FINI_ARRAY)                 0x6280c
 0x0000001c (FINI_ARRAYSZ)               8 (bytes)
 0x00000019 (INIT_ARRAY)                 0x62dec
 0x0000001b (INIT_ARRAYSZ)               4 (bytes)
 0x00000010 (SYMBOLIC)                   0x0
 0x0000001e (FLAGS)                      SYMBOLIC BIND_NOW
 0x6ffffffb (FLAGS_1)                    Flags: NOW
 0x6ffffff0 (VERSYM)                     0x262c
 0x6ffffffc (VERDEF)                     0x27fc
 0x6ffffffd (VERDEFNUM)                  1
 0x6ffffffe (VERNEED)                    0x2818
 0x6fffffff (VERNEEDNUM)                 1
 0x00000000 (NULL)                       0x0

libfreetype.so in the latest nightly build android release has a soname libfreetype.so.6.

Frenzie · 2017-06-03T07:25:27Z

How old is libtool? See e.g. http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=8eeeb00daef8c4f720c9b79a0cdb89225d9909b6 and moritz-wundke/Boost-for-Android#44

houqp · 2017-06-03T09:14:02Z

I think there are couple options here:

Compile all the binaries without version suffix in soname for all platforms. For many libraries, there should be a configure flag to set it. For others, we can patch the binary. I think the version suffix is mostly useful when the libraries is installed and shared system wide. In our case, those are private libraries packaged with koreader, so the version doesn't provide that much value to us. That's one of the reason why android doesn't work well with soname versioning because libraries in each app is only supposed to be used by the app itself.
Like @chrox said, change the loader to search for both
Rename the library with version suffix on android to make our loader happy

Frenzie · 2017-06-03T09:20:30Z

Compile all the binaries without version suffix in soname for all platforms.

I'm not vehemently opposed or anything, but doing something like that on all platforms just for Android gets a definite "meh" from me.

change the loader to search for both

Sounds good!

Rename the library with version suffix on android to make our loader happy

That makes the linker unhappy (which is what this PR was about) so I'm not sure if that's really an option.

chrox · 2017-06-05T16:52:44Z

It turns out to be the result of different versions of libtool that makes the discrepancy. Ubuntu 16.04 comes with libtool 2.4.6 which will generate a soname_spec='$libname$release$shared_ext' while older libtool (on my machine, 2.4.2) will generate a soname_spec='$libname$release$shared_ext$major' with major version suffix. There seems to be no simple parameters to accommodate both versions of libtool. I would like to keep current fix and add a libtool version dependency in the README.

@houqp And we should upgrade the libtool in the building machine too. ref: https://www.gnu.org/software/libtool/

chrox · 2017-06-05T16:58:53Z

change the loader to search for both

Tried that, although it can load the freetype lib, it still failed on loading mupdf with system dlopen. It will fail when it cannot find the needed library.

Fix Android build

fda8e2b

Fixes koreader/koreader#2647

houqp merged commit 387f4b9 into koreader:master Apr 16, 2017

Frenzie deleted the fix_android_build branch April 24, 2017 15:22

chrox mentioned this pull request Jun 2, 2017

Not able to load dynamic library: libs/libmupdf.so koreader/koreader#2853

Closed

Frenzie mentioned this pull request Jun 6, 2017

[finished]Android fullscreen switcher koreader/koreader#2938

Merged

chrox mentioned this pull request Jun 6, 2017

[fix] patch libtool version < 2.4.6 to generate lib without version suffix for Android #493

Merged

Conversation

Frenzie commented Apr 11, 2017

Uh oh!

houqp commented Apr 12, 2017

Uh oh!

houqp commented Apr 24, 2017

Uh oh!

Frenzie commented Apr 24, 2017 via email

Uh oh!

houqp commented Apr 24, 2017

Uh oh!

Frenzie commented Apr 24, 2017 via email

Uh oh!

houqp commented Apr 24, 2017

Uh oh!

chrox commented Jun 2, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

houqp commented Jun 2, 2017

Uh oh!

chrox commented Jun 2, 2017

Uh oh!

houqp commented Jun 2, 2017

Uh oh!

chrox commented Jun 3, 2017

Uh oh!

Frenzie commented Jun 3, 2017

Uh oh!

houqp commented Jun 3, 2017

Uh oh!

Frenzie commented Jun 3, 2017

Uh oh!

chrox commented Jun 5, 2017

Uh oh!

chrox commented Jun 5, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chrox commented Jun 2, 2017 •

edited

Loading