Support Rust types by retrieving them from debug info#307
Conversation
|
Putting the rust debug info type analysis check within an if statement for
`RustTypeRules`, as an example
…On Mon, Aug 23, 2021 at 7:52 AM Chuyang Chen ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In enzyme/Enzyme/TypeAnalysis/TypeAnalysis.cpp
<#307 (comment)>:
> @@ -4051,7 +4057,9 @@ TypeResults TypeAnalysis::analyzeFunction(const FnTypeInfo &fn) {
}
analysis.prepareArgs();
+ analysis.considerRustDebugInfo();
Hmm, what do you mean by "wrapping this in an optional argument"? Wrap
what to where?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#307 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJTUXAQN6QEI2PXI2YF5U3T6IZAFANCNFSM5CR4AH7Q>
.
|
…ked when the Rust type option is switched off
|
@cychen2021 Can you add corresponding tests within the TypeAnalysis test folders that verify the given functionality? When that's done this should be ready to merge. |
I'm working on this and will complete soon. |
|
@wsmoses Hi, I met some problems when writing test cases. To test the Rust type parser, I first write the test cases in Rust, and compile it to LLVM IR. I then use opt with Enzyme to do the auto differentiation. However, I have to indicate the location of Rust's std lib to link the differentiated code. But the test cases are in Enzyme project so they shouldn't know which Rust toolchain the user uses and where it is. How can I deal with that? |
|
You can just have the LLVM code here and don't need to do a full end-to-end test. For example, look at the existing TypeAnalysis tests here which just validate that type analysis works as expected. |
@wsmoses There are two problems:
|
|
@wsmoses Hi, I have added a test case for f32 type in Rust. A small problem is that the type analysis result with Rust type parsing open (with option '-enzyme-rust-type') and the type analysis result with Rust type parsing closed (without option '-enzyme-rust-type') are just the same, though the mid results during analysis are not the same. Does that matter? |
|
@wsmoses I have reduced the f32 test case to the minimum. Could you check whether it's OK? If so I'll add more test cases for other types alike. |
|
LGTM go ahead and add the other cases |
|
I'm done adding tests for all types we support now. Please check if this can be merged. @wsmoses |
|
@wsmoses Hi, please see if this can be merged when you are convenient. |
|
@wsmoses Hi, I meet some problems. I noticed that the rust type parser was incompatible with LLVM version under 9. I've used the macro LLVM_MAJOR_VERSION to resolve this. However, I found that the test cases are also incompatible with LLVM version under 9 because the newer LLVM I use generates different IR. Do you know how to deal with this? |
I've solved this by simply deleting flags that are not supported by LLVM under 9 |
|
@wsmoses What do you propose here? The failing tests seem to be unrelated, they also affect other open PRs. |
|
Those failures are currently expected and this is good to merge. |
Support Rust types by retrieving them from debug info
This is the pull request for my GSoC project, and the last commit for this project is c96bc56.
What we have done
We wrote a parser to parse the type info contained in debug info generated by rustc, the Rust compiler. It indicates the types of data used in a Rust program, and these types are then used to construct initial type trees for Enzyme's type analysis. It facilitate Enzyme to use Rust types to assist its synthesis of differentiated functions.
How to use it
The API contains two functions. One is in TypeAnalysis/RustDebugInfo.h
It extract the type info from an instruction's debug info and build the type tree according to it. It doesn't care what location the type tree is associated with and only gives the type tree corresponding to the debug info type structure.
We also add a function to
TypeAnalyzerclass defined in TypeAnalysis/TypeAnalysis.hIt looks up for the LLVM intrinsic
llvm.dbg.declarewhich is used by rustc to indicate declaration of a local variables, computes type trees for them, and infuses them to data related to these variables. Then, the type info will be propagated.Supported types
We now support data of the following types in Rust:
u8,i8,f32,f64, ...)structkeyword)[f32; 4])Vec<T>)Box<T>)*const T,*mut T)&T,&mut T)unionkeyword)Implementation
For someone who is interested in the implementation details, I can give a brief sketch. In short, the debug info types are in a recursive style. So, the process to parse them is just determining the types of different offsets in current layer according to current node, and then traversing all sub-nodes and getting types of offsets in their layers. To implement this, we write override functions for different kinds of debug info type nodes in TypeAnalysis/TypeAnalysis.cpp. They don't affect the usage of the parser, so we didn't expose them to the API.
A spacial case that should be mentioned when constructing type trees is that when the type to be parsed is
*u8we just ignore it and return an empty tree, andconsiderRustDebugInfowill do nothing after receiving that empty tree. This is because that in Rust, any pointer type can be casted to*u8which may cause mismatch between the debug info types and the actual types of the underlying data.TODOs
The most urgent ones are extending the parser to more types and testing it thoroughly. TODOs are listed below according to their emergency.
In the future, we may add predefined derivatives for frequently used functions in Rust to improve efficiency of compiling and running the differentiated functions. But that will be another story.