[TVM-FFI] Use TVM-FFI spec for generated library #53
Conversation
Signed-off-by: Yaoyao Ding <dingyaoyao.cs@gmail.com>
|
/ok to test 0134c50 |
Signed-off-by: Yaoyao Ding <dingyaoyao.cs@gmail.com>
|
/ok to test a617e0e |
There was a problem hiding this comment.
Pull Request Overview
This PR migrates the generated kernel library to use TVM-FFI spec instead of ctypes for better performance. The changes replace the existing runtime loading mechanism with TVM-FFI and introduce type safety for pointer operations.
- Replaced ctypes-based function loading with TVM-FFI for improved performance
- Introduced
void_pclass for type-safe pointer handling - Added TVM-FFI type traits for various pointer types with automatic tensor conversion
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/conftest.py | Disabled cache clearing for parallel test execution |
| python/tilus/testing/requires.py | Moved pytest import to local scope |
| python/tilus/runtime/compiled_program.py | Replaced hidet CompiledModule with TVM-FFI module loading |
| python/tilus/lang/instantiated_script.py | Updated function signatures to use TVM-FFI Function type |
| python/tilus/extensions/hidet/libinfo.py | Added library info helper for include paths |
| python/tilus/extensions/hidet/include/hidet/void_p.h | Defined void_p class for type-safe pointer operations |
| python/tilus/extensions/hidet/include/hidet/tvm/ffi/extra_type_traits.h | Added TVM-FFI type traits for pointer types |
| python/tilus/extensions/hidet/backend/codegen.py | Updated code generation to include TVM-FFI headers and registration |
| python/tilus/extensions/hidet/backend/build.py | Added TVM-FFI include paths to compilation |
| python/tilus/extensions/hidet/init.py | Exposed libinfo module |
| pyproject.toml | Added apache-tvm-ffi dependency and package data |
| .vscode/settings.json | Added pytest configuration |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
|
||
| TVM_FFI_INLINE static UInt* ConvertFallbackValue(DLTensor* src) { | ||
| if ((src->dtype.code != kDLUInt || src->dtype.bits != sizeof(UInt) * 8) | ||
| && (src->dtype.code != kDLBool || src->dtype.bits != 8) |
There was a problem hiding this comment.
The condition for kDLBool is checking for 8 bits regardless of the UInt type size. This creates inconsistent behavior where bool tensors can be converted to any unsigned integer pointer type regardless of size. Consider adding a size check: && (src->dtype.code != kDLBool || src->dtype.bits != 8 || sizeof(UInt) != 1)
| && (src->dtype.code != kDLBool || src->dtype.bits != 8) | |
| && (src->dtype.code != kDLBool || src->dtype.bits != 8 || sizeof(UInt) != 1) |
| void_p(T *ptr) : internal_ptr(static_cast<void *>(ptr)) {} | ||
|
|
||
| // 3. Implicit template conversion operator (Allows implicit conversion back to ANY T*) This allows: int* ip = p; | ||
| // (where p holds an int*) |
There was a problem hiding this comment.
The implicit conversion operator lacks documentation about potential type safety risks. Consider adding a comment explaining that this performs unchecked casting and the caller is responsible for ensuring type compatibility.
| // (where p holds an int*) | |
| // (where p holds an int*) | |
| // WARNING: This operator performs an unchecked cast from the stored void* to the requested pointer type. | |
| // The caller is responsible for ensuring that the type T matches the original type of the stored pointer. | |
| // Using an incorrect type may result in undefined behavior. |
This PR makes the generated kernel library (
lib.so) satisfy the ABI of tvm ffi (https://github.com/apache/tvm-ffi). Also use tvm-ffi package to load and launch the kernel. Previously, we use ctypes to run a function which is slower than tvm-ffi.void_pto replacevoid*so that we can allow passing Tensor/Integer directly.