Skip to content

add patch to extend libiconv with vs2019+win/arm64 support and revert submodules libxml2 and libxslt2#1

Merged
scoder merged 3 commits intolxml:masterfrom
niyas-sait:refactor_libiconv_revert_submodules
Nov 19, 2021
Merged

add patch to extend libiconv with vs2019+win/arm64 support and revert submodules libxml2 and libxslt2#1
scoder merged 3 commits intolxml:masterfrom
niyas-sait:refactor_libiconv_revert_submodules

Conversation

@niyas-sait
Copy link
Copy Markdown

This patch extends the libiconv submodule with VS project files for VS 2019 and with support for win/arm64 target.

The patch also reverts submodule updates for the following modules

  • libiconv ( Reason - Latest libiconv does not support MSVC 9.0 )
  • libxml2 and libxslt ( Reason - python lxml compilation fails due to missing symbols)

@scoder
Copy link
Copy Markdown
Member

scoder commented Nov 6, 2021

Thanks. Any recent libiconv version will probably do, but IIUC you reverted libxml2 to winlibs/libxml2@cece36e, right? That's too old. I'd like to stick with winlibs/libxml2@6a6690c. What were the missing symbols? Do you have a build log for that?

@niyas-sait
Copy link
Copy Markdown
Author

@scoder I looked into it a bit more and it seems like the problem is due to libxml2 header files by default trying to link against DLL by adding the import library prefixes which causes the many link error I saw previously.

etree.obj : error LNK2001: unresolved external symbol __imp_xmlC14NDocSave
etree.obj : error LNK2001: unresolved external symbol __imp_xmlSchematronValidateDoc
etree.obj : error LNK2001: unresolved external symbol __imp_xmlSchematronNewParserCtxt
etree.obj : error LNK2001: unresolved external symbol __imp_xmlSchematronNewDocParserCtxt

I think the fix would be to add a compile flag while building lxml extension modules to link against the static library. Something like following would work

--- a/setupinfo.py
+++ b/setupinfo.py
@@ -122,7 +122,7 @@ def ext_modules(static_include_dirs, static_library_dirs,
     _library_dirs = _prefer_reldirs(base_dir, library_dirs(static_library_dirs))
     _cflags = cflags(static_cflags)
     _ldflags = ['-isysroot', get_xcode_isysroot()] if sys.platform == 'darwin' else None
-    _define_macros = define_macros()
+    _define_macros = [('LIBXML_STATIC', 1)]
     _libraries = libraries()

     if _library_dirs:

I have reverted the submodule update for libxml2 and libxslt to use the latest as before. So this can be merged if you are okay.

And I can open a PR for lxml to add this extra change build against the static library

@niyas-sait
Copy link
Copy Markdown
Author

@scoder Just pinging in case you missed previous update

@scoder
Copy link
Copy Markdown
Member

scoder commented Nov 18, 2021

Not sure why you want a macro for this. Isn't it just something to add to the extra_objects of the Extension setup further down? That's what the static_binaries argument is there for.

@niyas-sait
Copy link
Copy Markdown
Author

Not sure why you want a macro for this. Isn't it just something to add to the extra_objects of the Extension setup further down? That's what the static_binaries argument is there for.

There may be a better way to add this flag. I am not sure. But for compiling against static library we will need to define LIBXML_STATIC during the c/c++ compilation.

If you are happy we could merge this patch and could look into options of defining the macro while building lxml ?

@scoder
Copy link
Copy Markdown
Member

scoder commented Nov 19, 2021

I pushed lxml/lxml@7837d13. Does that solve the issue?

@niyas-sait
Copy link
Copy Markdown
Author

I pushed lxml/lxml@7837d13. Does that solve the issue?

Yes I think that should work. Thanks

@scoder scoder merged commit 3fb0529 into lxml:master Nov 19, 2021
@scoder
Copy link
Copy Markdown
Member

scoder commented Nov 19, 2021

Thanks a lot. Let's see what appveyor makes of this.

@niyas-sait
Copy link
Copy Markdown
Author

Thanks, @scoder. Could you make a release with the appveyor artifacts please?

@niyas-sait
Copy link
Copy Markdown
Author

Thanks @scoder. I've tried with top of lxml and everything seems to be working fine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants