Skip to content

add bin2obj support for glsl/hlsl2spv#7105

Merged
waruqi merged 4 commits intodevfrom
spv
Dec 8, 2025
Merged

add bin2obj support for glsl/hlsl2spv#7105
waruqi merged 4 commits intodevfrom
spv

Conversation

@waruqi
Copy link
Member

@waruqi waruqi commented Dec 7, 2025

  • Before adding new features and new modules, please go to issues to submit the relevant feature description first.
  • Write good commit messages and use the same coding conventions as the rest of the project.
  • Please commit code to dev branch and we will merge into master branch in feature
  • Ensure your edited codes with four spaces instead of TAB.

  • 增加新特性和新模块之前,请先到issues提交相关特性说明,经过讨论评估确认后,再进行相应的代码提交,避免做无用工作。
  • 编写友好可读的提交信息,并使用与工程代码相同的代码规范,代码请用4个空格字符代替tab缩进。
  • 请提交代码到dev分支,如果通过,我们会在特定时间合并到master分支上。
  • 为了规范化提交日志的格式,commit消息,不要用中文,请用英文描述。

@waruqi waruqi added this to the v3.0.6 milestone Dec 7, 2025
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @waruqi, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new bin2obj utility that enables the direct conversion of binary files into linkable object files. This functionality is integrated into the glsl2spv and hlsl2spv rules, offering a flexible way to embed SPIR-V shader binaries into executables without generating intermediate C header files. The changes include refactoring existing binary-to-object conversion logic and adding comprehensive test cases to ensure proper operation.

Highlights

  • New bin2obj Utility: A new Lua utility function, generate_objectfile, has been introduced in xmake/rules/utils/bin2obj/utils.lua. This function centralizes the logic for converting arbitrary binary files into platform-specific object files (COFF, ELF, or Mach-O) for direct linking.
  • Integration with glsl2spv Rule: The glsl2spv rule now supports a bin2obj option. When enabled, compiled GLSL shaders (SPIR-V binaries) are directly converted into linkable object files using the new utility, providing an alternative to embedding them as C header files.
  • Integration with hlsl2spv Rule: Similarly, the hlsl2spv rule has been updated to include a bin2obj option. This allows compiled HLSL shaders (SPIR-V binaries) to be converted into object files for direct linking, mirroring the functionality added for GLSL shaders.
  • Refactoring of utils.bin2obj Rule: The existing utils.bin2obj rule in xmake/rules/utils/bin2obj/xmake.lua has been refactored to leverage the new generate_objectfile utility, removing redundant code and improving maintainability.
  • New Test Projects: Two new test projects, glsl2spv_bin2obj and hlsl2spv_bin2obj, have been added. These projects demonstrate and verify the new bin2obj functionality by compiling GLSL and HLSL shaders into object files and then linking them into a C executable to print their sizes and initial bytes.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces bin2obj support for glsl2spv and hlsl2spv rules, which is a valuable addition for embedding shaders. The implementation is well-structured, including a new reusable bin2obj utility and corresponding test cases. My review focuses on a few areas for improvement:

  • A correctness issue where a null terminator is incorrectly added to SPIR-V binary data.
  • Opportunities to refactor the new C test code for better readability, correctness, and maintainability.
  • Minor code redundancy and style improvements in the Lua scripts.

Comment on lines +102 to +105
local objectfile = bin2obj_utils.generate_objectfile(target, batchcmds, spvfilepath, {
progress = opt.progress,
zeroend = true -- default enable zeroend for shader data
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Setting zeroend = true for SPIR-V binary data is incorrect. SPIR-V is a binary format with a specific structure, not a null-terminated string. Appending a null byte will alter the content and size, which can cause issues with SPIR-V parsers. The bin2c path correctly disables the null terminator. For consistency and correctness, zeroend should be false. Since false is the default, you can simply omit this option.

            local objectfile = bin2obj_utils.generate_objectfile(target, batchcmds, spvfilepath, {
                progress = opt.progress
            })

Comment on lines +1 to +35
#include <stdio.h>
#include <stdint.h>

extern const uint8_t _binary_test_vert_spv_start[];
extern const uint8_t _binary_test_vert_spv_end[];

extern const uint8_t _binary_test_frag_spv_start[];
extern const uint8_t _binary_test_frag_spv_end[];

int main(int argc, char** argv) {
const uint32_t vert_size = (uint32_t)(_binary_test_vert_spv_end - _binary_test_vert_spv_start);
const uint32_t frag_size = (uint32_t)(_binary_test_frag_spv_end - _binary_test_frag_spv_start);

printf("test.vert.spv: size: %u bytes\n", vert_size);
printf("test.frag.spv: size: %u bytes\n", frag_size);

// Print first few bytes
if (vert_size > 0) {
printf("test.vert.spv first 4 bytes: ");
for (uint32_t i = 0; i < 4 && i < vert_size; i++) {
printf("%02x ", _binary_test_vert_spv_start[i]);
}
printf("\n");
}

if (frag_size > 0) {
printf("test.frag.spv first 4 bytes: ");
for (uint32_t i = 0; i < 4 && i < frag_size; i++) {
printf("%02x ", _binary_test_frag_spv_start[i]);
}
printf("\n");
}

return 0;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This test program can be improved for correctness, readability and maintainability:

  1. The main function signature should be int main(void) as argc and argv are unused.
  2. For portability and correctness, size_t (from <stddef.h>) should be used for sizes, along with the %zu format specifier in printf.
  3. The code for printing vertex and fragment shader information is duplicated. Extracting it into a helper function would reduce redundancy.

Here is a suggested refactoring that addresses these points:

#include <stdio.h>
#include <stdint.h>
#include <stddef.h>

extern const uint8_t _binary_test_vert_spv_start[];
extern const uint8_t _binary_test_vert_spv_end[];

extern const uint8_t _binary_test_frag_spv_start[];
extern const uint8_t _binary_test_frag_spv_end[];

static void print_shader_info(const char* name, const uint8_t* start, const uint8_t* end) {
    const size_t size = (size_t)(end - start);
    printf("%s: size: %zu bytes\n", name, size);
    
    if (size > 0) {
        printf("%s first 4 bytes: ", name);
        const size_t print_count = size < 4 ? size : 4;
        for (size_t i = 0; i < print_count; i++) {
            printf("%02x ", start[i]);
        }
        printf("\n");
    }
}

int main(void) {
    print_shader_info("test.vert.spv", _binary_test_vert_spv_start, _binary_test_vert_spv_end);
    print_shader_info("test.frag.spv", _binary_test_frag_spv_start, _binary_test_frag_spv_end);
    
    return 0;
}

Comment on lines +1 to +35
#include <stdio.h>
#include <stdint.h>

extern const uint8_t _binary_test_vs_spv_start[];
extern const uint8_t _binary_test_vs_spv_end[];

extern const uint8_t _binary_test_ps_spv_start[];
extern const uint8_t _binary_test_ps_spv_end[];

int main(int argc, char** argv) {
const uint32_t vs_size = (uint32_t)(_binary_test_vs_spv_end - _binary_test_vs_spv_start);
const uint32_t ps_size = (uint32_t)(_binary_test_ps_spv_end - _binary_test_ps_spv_start);

printf("test.vs.spv: size: %u bytes\n", vs_size);
printf("test.ps.spv: size: %u bytes\n", ps_size);

// Print first few bytes
if (vs_size > 0) {
printf("test.vs.spv first 4 bytes: ");
for (uint32_t i = 0; i < 4 && i < vs_size; i++) {
printf("%02x ", _binary_test_vs_spv_start[i]);
}
printf("\n");
}

if (ps_size > 0) {
printf("test.ps.spv first 4 bytes: ");
for (uint32_t i = 0; i < 4 && i < ps_size; i++) {
printf("%02x ", _binary_test_ps_spv_start[i]);
}
printf("\n");
}

return 0;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This test program can be improved for correctness, readability and maintainability:

  1. The main function signature should be int main(void) as argc and argv are unused.
  2. For portability and correctness, size_t (from <stddef.h>) should be used for sizes, along with the %zu format specifier in printf.
  3. The code for printing vertex and pixel shader information is duplicated. Extracting it into a helper function would reduce redundancy.

Here is a suggested refactoring that addresses these points:

#include <stdio.h>
#include <stdint.h>
#include <stddef.h>

extern const uint8_t _binary_test_vs_spv_start[];
extern const uint8_t _binary_test_vs_spv_end[];

extern const uint8_t _binary_test_ps_spv_start[];
extern const uint8_t _binary_test_ps_spv_end[];

static void print_shader_info(const char* name, const uint8_t* start, const uint8_t* end) {
    const size_t size = (size_t)(end - start);
    printf("%s: size: %zu bytes\n", name, size);
    
    if (size > 0) {
        printf("%s first 4 bytes: ", name);
        const size_t print_count = size < 4 ? size : 4;
        for (size_t i = 0; i < print_count; i++) {
            printf("%02x ", start[i]);
        }
        printf("\n");
    }
}

int main(void) {
    print_shader_info("test.vs.spv", _binary_test_vs_spv_start, _binary_test_vs_spv_end);
    print_shader_info("test.ps.spv", _binary_test_ps_spv_start, _binary_test_ps_spv_end);
    
    return 0;
}

objectfile = target:objectfile(binaryfile)
-- adjust extension based on format (use .obj for COFF, .o for others)
local objext = (format == "coff") and ".obj" or ".o"
objectfile = objectfile:gsub("%.o$", objext):gsub("%.obj$", objext)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This logic for replacing the file extension is a bit complex. You can achieve the same result more cleanly by using path.replace_ext(), which will correctly handle replacing .o or any other extension.

        objectfile = path.replace_ext(objectfile, objext)

@waruqi waruqi merged commit c5ca6a6 into dev Dec 8, 2025
63 of 65 checks passed
@waruqi waruqi deleted the spv branch December 8, 2025 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant