Skip to content

Fix a bug with using spaces in add_defines values#3

Closed
daviwil wants to merge 2 commits intoxmake-io:masterfrom
daviwil:fix-define-spaces
Closed

Fix a bug with using spaces in add_defines values#3
daviwil wants to merge 2 commits intoxmake-io:masterfrom
daviwil:fix-define-spaces

Conversation

@daviwil
Copy link
Copy Markdown
Contributor

@daviwil daviwil commented Apr 6, 2023

I ran into an issue with xmake.sh while trying to add a more complex define in my xmake.sh script, here's an example:

#!/bin/sh

set_project "my-lib"
set_version "0.0.1" "%Y%m%d%H%M"

target "lib"
    set_kind "static"
    add_files "src/*.c"
    add_files "src/*.cpp"
    # This should add a define that looks like: -DIMGUI_IMPL_API="extern \"C\" "
    add_defines "IMGUI_IMPL_API=\"extern \\\"C\\\" \""
    add_includedirs "lib" "{public}"
    add_includedirs "src" "{public}"

The goal is to have the flag -DIMGUI_IMPL_API="extern \"C\" " added to all compiler invocations, but the current code in master for the configure script ends up splitting the add_defines value based on spaces so the cflags end up looking like this:

-DIMGUI_IMPL_API=\"extern -D\\"C\\" -D\"

After my changes, they now look like this:

-DIMGUI_IMPL_API=\"extern \\"C\\" \"

This is still not right, however! The quotation marks are escaped when they shouldn't be in this case. The C compiler can't cope with them when IMGUI_IMPL_API gets used in source files.

I noticed that in _get_abstract_flag_for_gcc_clang all quotation marks are explicitly escaped for defines using string_replace "${value}" '"' '\\\"'; value="${_ret}". Is there a way to disable this behavior in cases when the quotation marks should be written out literally?

@daviwil daviwil force-pushed the fix-define-spaces branch from e754f7d to 25b9563 Compare April 6, 2023 04:35
@waruqi
Copy link
Copy Markdown
Member

waruqi commented Apr 6, 2023

It will break on ci.

@daviwil
Copy link
Copy Markdown
Contributor Author

daviwil commented Apr 6, 2023

@waruqi It seems that the CI error is exposed by my change but not caused by it, I sent a separate PR to fix it: #4

Do you have any suggestion about what to do for the string escaping of quotation marks?

@waruqi waruqi closed this Apr 6, 2023
@waruqi waruqi reopened this Apr 6, 2023
@daviwil daviwil force-pushed the fix-define-spaces branch from 25b9563 to 7ec101e Compare April 6, 2023 06:51
@waruqi
Copy link
Copy Markdown
Member

waruqi commented Apr 6, 2023

Do you have any suggestion about what to do for the string escaping of quotation marks?

Check if it is \" and not automatically escape it?

@daviwil
Copy link
Copy Markdown
Contributor Author

daviwil commented Apr 6, 2023

Yep, I'll try that. I'm now dealing with another issue where one of the changes I made is breaking how other values are read out of the target map. Looking for a solution now.

@daviwil
Copy link
Copy Markdown
Contributor Author

daviwil commented Apr 6, 2023

@waruqi OK, I have solution that works now. Let me know if you think there are any possible issues with how I'm "escaping" the spaces in the define string. I'm using a sentinel value of ~!~ which hopefully should be uncommon. Happy to make further edits based on your suggestions.

@daviwil daviwil force-pushed the fix-define-spaces branch from 38a4992 to 912340f Compare April 6, 2023 08:20
@daviwil
Copy link
Copy Markdown
Contributor Author

daviwil commented Apr 6, 2023

That particular system configuration must not like the space in the -D flag somehow, I'll investigate a bit futher.

@waruqi
Copy link
Copy Markdown
Member

waruqi commented Apr 6, 2023

I tested it, it does not work for my xmake/tbox.

git clone https://github.com/tboox/tbox
./configure
make
make: *** Waiting for unfinished jobs....
src/tbox/utils/dump.c:39:5: error: use of undeclared identifier 'tbox'
    tb_trace_i("");
    ^
    add_defines "__tb_prefix__=\"tbox\""

@daviwil
Copy link
Copy Markdown
Contributor Author

daviwil commented Apr 6, 2023

Thanks! I'll adjust it and use your repo to verify.

@daviwil
Copy link
Copy Markdown
Contributor Author

daviwil commented Apr 6, 2023

OK, I'm not sure how to proceed here, let me see if I can explain the factors at play:

  • The tbox repo depends on automatic quote escaping for add_depends so that "tbox" gets inserted as a string literal in the C source to be used for tracing messages
  • My example needs to insert an unquoted string of C code (extern "C" ) into a header file via -D flag so that it can affect API signatures of the Dear Imgui library. If the outer quotes get auto-escaped, they will be inserted verbatim into the C code, causing a compiler error
  • There isn't a way, as far as I know, to un-escape the quotes when they reach the -D flag in the makefile, so at this point I would need to introduce some kind of hack to make my scenario work (or just keep my own forked xmake.sh that works for me; not ideal)

In my opinion, the default behavior (always escaping quotes) might be surprising to users who want to pass a string containing spaces to a -D flag without the escaped quotes included.

My recommendation would be to remove auto-escaping and for xmake.sh targets that need to pass a quoted string to do it explicitly:

add_defines "__tb_prefix__=\\\"tbox\\\""

It's certainly uglier and slightly inconvenient, but at least it enables the configure script to enable all possible usages of add_defines. On the other hand, the tbox repo uses the __tb_prefix__ a number of times, so I'm reluctant to change how everything works just to enable my own scenario, though I think I'm not the only person who will run into this problem.

Do you think there's a reasonable way to solve this problem? How does the Lua version of xmake deal with this?

If you're willing to remove auto-escaping of defines strings, I'd be happy to go update the xmake.sh files in the tbox repo to use the additional escaping needed to make it work again. Do you know if this pattern is used in other repositories that might need to be updated too?

@waruqi
Copy link
Copy Markdown
Member

waruqi commented Apr 6, 2023

How does the Lua version of xmake deal with this?

For xmake, it does not splice arguments to generate a string command, but passes all the argument items directly into execv as a table list

so, string escaping can be avoided.

        add_defines "TEST=\"test\""
        add_defines "IMPL_API=extern \"C\" "
os.execv("gcc", {"TEST=\"test\"", "IMPL_API=extern \"C\" "})

@waruqi
Copy link
Copy Markdown
Member

waruqi commented Apr 6, 2023

xmake.sh is a lightweight version of the xmake implementation, it is not used to handle complex things. Calling string_replace on each defines entry is very slow, as it calls the sed subprocess every time.

so we need only use add_cxflags instead of add_defines for this special case.

        add_defines "TEST=\"${name}\""
        add_cxflags "-DIMPL_API=\"extern \\\"C\\\" \""

It works for me. I have improved it.

f54115a

@daviwil
Copy link
Copy Markdown
Contributor Author

daviwil commented Apr 6, 2023

OK, I'm fine with using add_cxflags for that purpose, thanks for the suggestion! It looks like those group of functions don't support {public} but I think I should be fine without it in this case.

I just tried it out in my project and add_cxflags works just fine. Thanks again! I'll close this PR for now.

@daviwil daviwil closed this Apr 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants