Skip to content

[.NET 9] DllImport with Custom Marshaler Passes non-null address when null value for field is specified. #109033

@Sewer56

Description

@Sewer56

Description

In .NET 9 (Preview 2 and newer), passing null to a parameter which uses a CustomMarshaler results in a non-zero value passed to native code. This can lead to memory corruption or invalid memory reads, depending on what the native code does with the value.

Reproduction Steps

Attached reproduction: cimgui-stub.zip

To Reproduce

Run the csharp project (csharp.csproj) provided.

When ran with .NET 9 Preview 1 or older, the result is:

igDragInt called with:
  label address: 2941681900960
  v address: 750695999576
  format address: 0
  label value: label
  v value: 0
  v_speed: 0.1
  v_min: 0
  v_max: 100
  format: null
  flags: 0

When ran with .NET 9 Preview 2 or newer the result is:

igDragInt called with:
  label address: 1824962101440
  v address: 114923399832
  format address: 14829735431805717965
  label value: label
  v value: 0
  v_speed: 0.1
  v_min: 0
  v_max: 100
Fatal error. 0xC0000005
   at ImGui.DragInt(System.String, Int32 ByRef, Single, Int32, Int32, System.String, Int32)
   at Program.<Main>$(System.String[])

When null is passed to a string parameter with any CustomMarshaler specified, the generated DllImport stub passes a non-zero address. (Note: Marshaler is skipped, so implementation of marshaler used with CustomMarshaler is not relevant)


To Compile the Rust Library/ Native Code

[Note: I've included an x64 precompiled DLL in the csharp folder out of the box, this is only needed if you want to change the native DllImport target, e.g. reduce param count for debugging]

  1. Install rustup: https://static.rust-lang.org/rustup/dist/x86_64-pc-windows-msvc/rustup-init.exe
  2. cd into cimgui_export
  3. cargo build --release
  4. Result is in target/release/cimgui_export.dll
  5. Paste into csharp folder, replacing the existing file.

Expected behavior

format address should be 0 (null). Instead an unexpected address is used.

Actual behavior

format address is not 0 (null)

Regression?

.NET 9 (Preview 1) and older work fine.
Regression is introduced in Preview 2.

Known Workarounds

No response

Configuration

N/A

This reproduces in both x86 and x64 on Windows.

Other information

The reproduction is small enough that it can be included here for convenience:

var v = 0;
while (true)
{
    ImGui.DragInt("label", ref v, 0.1f, 0, 100, null!, 0);
    Thread.Sleep(1000);
}

static class ImGui
{
    [DllImport("cimgui_export", CallingConvention = CallingConvention.Cdecl, EntryPoint = "igDragInt")]
    [SuppressUnmanagedCodeSecurity]
    [return: MarshalAs(UnmanagedType.I1)]
    public static extern bool DragInt(
        [MarshalAs(UnmanagedType.CustomMarshaler, MarshalType = "csharp.UTF8Marshaller, csharp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null")]
        string label,
        ref int v,
        float v_speed,
        int v_min,
        int v_max,
        [MarshalAs(UnmanagedType.CustomMarshaler, MarshalType = "csharp.UTF8Marshaller, csharp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null")]
        string format,
        int flags
    );
}

Rust part:

use std::ffi::CStr;
use std::os::raw::{c_char, c_int, c_float, c_uchar};

#[no_mangle]
pub extern "C" fn igDragInt(
    label: *const c_char,
    v: *mut c_int,
    v_speed: c_float,
    v_min: c_int,
    v_max: c_int,
    format: *const c_char,
    flags: c_int,
) -> c_uchar {
    // Print addresses of all pointers
    println!("igDragInt called with:");
    println!("  label address: {:?}", label as usize);
    println!("  v address: {:?}", v as usize);
    println!("  format address: {:?}", format as usize);

    // Handle `label`
    if label.is_null() {
        println!("  label: null");
    } else {
        // Convert C string to Rust string
        let c_label = unsafe { CStr::from_ptr(label) };
        match c_label.to_str() {
            Ok(s) => println!("  label value: {}", s),
            Err(_) => println!("  label: Invalid UTF-8 string"),
        }
    }

    // Handle `v`
    if v.is_null() {
        println!("  v: null");
    } else {
        // Safely read the integer value
        let value = unsafe { *v };
        println!("  v value: {}", value);
    }

    // Print other parameters
    println!("  v_speed: {}", v_speed);
    println!("  v_min: {}", v_min);
    println!("  v_max: {}", v_max);

    // Handle `format`
    if format.is_null() {
        println!("  format: null");
    } else {
        // Convert C string to Rust string
        let c_format = unsafe { CStr::from_ptr(format) };
        match c_format.to_str() {
            Ok(s) => println!("  format value: {}", s),
            Err(_) => println!("  format: Invalid UTF-8 string"),
        }
    }

    println!("  flags: {}", flags);

    // For demonstration, return true (1) to indicate success
    1
}

I can across this issue when trying older code with .NET 9 that used dear imgui bindings made with CppSharp. So in the repro I tried to model that.

Metadata

Metadata

Assignees

Labels

area-Interop-coreclrin-prThere is an active PR which will close this issue when it is merged

Type

No type

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions