Skip to content

feat: Windows support using MinGW-w64#4201

Merged
maxbrunsfeld merged 1 commit intotree-sitter:masterfrom
MisterDA:cygwin-works-gcc-mingw-w64
Feb 13, 2025
Merged

feat: Windows support using MinGW-w64#4201
maxbrunsfeld merged 1 commit intotree-sitter:masterfrom
MisterDA:cygwin-works-gcc-mingw-w64

Conversation

@MisterDA
Copy link
Contributor

@MisterDA MisterDA commented Feb 11, 2025 •
edited
Loading

Building tree-sitter on Windows within a Cygwin or MSYS2 environment for MinGW-w64 targets should work. As there's no configure-like step, the tools have to be specified with:

make CC=x86_64-w64-mingw32-gcc \
     AR=x86_64-w64-mingw32-ar \
     STRIP=x86_64-w64-mingw32-strip

Exported symbols should be tagged.
Useful reference: How to build and use DLLs on Windows.

This PR doesn't include support to build libtree-sitter with MSVC or clang-cl, but generates a libtree-sitter.lib file for MSVC/clang-cl consumers.

Cf #3418 #3417
cc @ObserverOfTime

@ObserverOfTime
Copy link
Member

ObserverOfTime commented Feb 11, 2025

But it does not produce a .dll

@MisterDA
Copy link
Contributor Author

MisterDA commented Feb 11, 2025 •
edited
Loading

But it does not produce a .dll

It does, but just with the .so extension. Maybe we can just set SOEXT = dll inside the Makefile's OS-specific bits if dumpmachine contains cygwin or mingw32?

$ make
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/alloc.o lib/src/alloc.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/get_changed_ranges.o lib/src/get_changed_ranges.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/language.o lib/src/language.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/lexer.o lib/src/lexer.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/node.o lib/src/node.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/parser.o lib/src/parser.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/query.o lib/src/query.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/stack.o lib/src/stack.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/subtree.o lib/src/subtree.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/tree.o lib/src/tree.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/tree_cursor.o lib/src/tree_cursor.c
cc -O3 -Wall -Wextra -Wshadow -Wpedantic -Werror=incompatible-pointer-types -std=c11 -fPIC -fvisibility=hidden -D_POSIX_C_SOURCE=200112L -D_DEFAULT_SOURCE -Ilib/src -Ilib/src/wasm -Ilib/include   -c -o lib/src/wasm_store.o lib/src/wasm_store.c
sed -e 's|@PROJECT_VERSION@|0.25.1|' \
        -e 's|@CMAKE_INSTALL_LIBDIR@|lib|' \
        -e 's|@CMAKE_INSTALL_INCLUDEDIR@|include|' \
        -e 's|@PROJECT_DESCRIPTION@|An incremental parsing system for programming tools|' \
        -e 's|@PROJECT_HOMEPAGE_URL@|https://tree-sitter.github.io/tree-sitter/|' \
        -e 's|@CMAKE_INSTALL_PREFIX@|/usr/local|' lib/tree-sitter.pc.in > tree-sitter.pc
ar rcs libtree-sitter.a lib/src/alloc.o lib/src/get_changed_ranges.o lib/src/language.o lib/src/lexer.o lib/src/node.o lib/src/parser.o lib/src/query.o lib/src/stack.o lib/src/subtree.o lib/src/tree.o lib/src/tree_cursor.o lib/src/wasm_store.o
cc  -shared -Wl,-soname,libtree-sitter.so.0.25 lib/src/alloc.o lib/src/get_changed_ranges.o lib/src/language.o lib/src/lexer.o lib/src/node.o lib/src/parser.o lib/src/query.o lib/src/stack.o lib/src/subtree.o lib/src/tree.o lib/src/tree_cursor.o lib/src/wasm_store.o
-o libtree-sitter.so
$ file libtree-sitter.so
libtree-sitter.so: PE32+ executable (DLL) (console) x86-64, for MS Windows, 18 sections

@MisterDA MisterDA marked this pull request as draft February 11, 2025 17:16
@MisterDA MisterDA force-pushed the cygwin-works-gcc-mingw-w64 branch from f664d14 to 2448435 Compare February 12, 2025 14:20
@MisterDA MisterDA changed the title fix(make): re-enable Makefile on Windows (for Cygwin/MSYS2 envs) feat: Windows support using MinGW-w64 Feb 12, 2025
@MisterDA MisterDA force-pushed the cygwin-works-gcc-mingw-w64 branch from 2448435 to 7e26e56 Compare February 12, 2025 14:25
@MisterDA
Copy link
Contributor Author

MisterDA commented Feb 12, 2025 •
edited
Loading

I think the build using MinGW-w64 succeeds. I'm not interested in supporting MSVC/clang-cl or Cygwin+cygwin1.dll builds with the Makefile. The ugly Windows thing is that symbol visibility has to be explicit, with the dreaded dllimport/dllexport. As libtree-sitter is a pure C library (as opposed to say, C++) a DLL produced by MinGW-w64 can be reused by an executable built with MSVC. I've found this blogpost How to build and use DLLs on Windows to be particularly helpful.

@MisterDA MisterDA force-pushed the cygwin-works-gcc-mingw-w64 branch 7 times, most recently from 7b10bf9 to d6a3153 Compare February 12, 2025 16:02
@MisterDA MisterDA marked this pull request as ready for review February 12, 2025 16:05
@ObserverOfTime
Copy link
Member

ObserverOfTime commented Feb 12, 2025

This is a bit too much.

@MisterDA
feat: Windows support using MinGW-w64
87a376a 
Building tree-sitter on Windows within a Cygwin or MSYS2 environment
for MinGW-w64 targets should work. As there's no configure-like step,
the tools have to be specified with:

    make CC=x86_64-w64-mingw32-gcc \
         AR=x86_64-w64-mingw32-ar \
         STRIP=x86_64-w64-mingw32-strip

Useful reference:
[How to build and use DLLs on Windows](https://nullprogram.com/blog/2021/05/31/)

This commit doesn't tag public functions with dllexport or
dllimport. This results in exporting non-static functions visible
between translation units, and generating slightly less efficient code
for calling exported functions from the DLL.

This commit doesn't include support to build libtree-sitter with MSVC or
clang-cl, but generates a libtree-sitter.lib file for MSVC/clang-cl
consumers.
@MisterDA MisterDA force-pushed the cygwin-works-gcc-mingw-w64 branch from d6a3153 to 87a376a Compare February 12, 2025 17:31
@MisterDA
Copy link
Contributor Author

MisterDA commented Feb 12, 2025

This is a bit too much.

Ok, I've removed the dllexport/dllimport machinery. This will result in the DLL exporting non-static functions visible between translation units, and the compiler generating slightly less efficient code for calling exported functions from the DLL; but now the PR only adds support for building with MinGW-w64 to the build system and doesn't change the C code.

@MisterDA
Copy link
Contributor Author

MisterDA commented Feb 12, 2025

The CI timed out but should succeed. Is the PR more acceptable now?

@maxbrunsfeld
Copy link
Contributor

maxbrunsfeld commented Feb 13, 2025

Thanks @MisterDA, looks good. And thanks for the reviews @ObserverOfTime.

@maxbrunsfeld maxbrunsfeld merged commit f0571b1 into tree-sitter:master Feb 13, 2025
19 checks passed
@MisterDA MisterDA deleted the cygwin-works-gcc-mingw-w64 branch February 13, 2025 09:13
@MisterDA MisterDA mentioned this pull request Feb 13, 2025
Merged
shonfeder added a commit to shonfeder/ocaml-tree-sitter-core that referenced this pull request Feb 19, 2025
@shonfeder @MisterDA
Detect CC, AR, and STRIP to build tree-sitter on MinGW-w64
56b93fc 
This is best-effort and should help picking the right cc, ar, and
strip utilities, whether they are prefixed (as when using the
MinGW-w64 cross-compiler) or not.

For a refresh of the shell features used, see:
POSIX 2018 2.6.2 Parameter Expansion.
https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html#tag_19_06_02

This includes an update the Makefile patch to build with MinGW-w64.

GNU Make 4.4.1 build for x86_64-pc-cygwin is perfectly capable of
reading the Makefile. It needs to be taught how to use MinGW-w64 to
build a DLL. This is a backport of patches accepted upstream.

See tree-sitter/tree-sitter#4201 for the
corresponding upstream fix.

Co-authored-by: Antonin Décimo <antonin@tarides.com>
mjambon pushed a commit to semgrep/ocaml-tree-sitter-core that referenced this pull request Feb 21, 2025
@shonfeder @MisterDA
Avoid use of -rpath on Windows (#86)
9ba9692 
* Detect CC, AR, and STRIP to build tree-sitter on MinGW-w64

This is best-effort and should help picking the right cc, ar, and
strip utilities, whether they are prefixed (as when using the
MinGW-w64 cross-compiler) or not.

For a refresh of the shell features used, see:
POSIX 2018 2.6.2 Parameter Expansion.
https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html#tag_19_06_02

This includes an update the Makefile patch to build with MinGW-w64.

GNU Make 4.4.1 build for x86_64-pc-cygwin is perfectly capable of
reading the Makefile. It needs to be taught how to use MinGW-w64 to
build a DLL. This is a backport of patches accepted upstream.

See tree-sitter/tree-sitter#4201 for the
corresponding upstream fix.

Co-authored-by: Antonin Décimo <antonin@tarides.com>

* Don't pass `-rpath` on Windows

---------

Co-authored-by: Antonin Décimo <antonin@tarides.com>
@clason clason mentioned this pull request Jul 10, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@maxbrunsfeld maxbrunsfeld maxbrunsfeld approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@MisterDA @ObserverOfTime @maxbrunsfeld