[RFC] Add new CPP builder for inductor on pytorch Windows #124245
Description
🚀 The feature, motivation and pitch
Motivation
Current torch inductor only support non-Windows OS. This RFC is proposal to add a new CPP builder which also support Windows OS.
Firstly, we can list the gaps to enable inductor on Windows.
- Current CPP builder not support Windows cl compiler.
- Current ISA check use linux
proc file system, which not support Windows also.
Proposed Solution
CPP builder
Current CPP builder is hard code to only support Linux likes OS, Windows has different build options and command line rules. We need to design a abstract mechanism to adapt to both Windows and Linux likes OS(compiler).
Build options
A. Categories:
Compiler options genarally has same categories: cflags, ldflags, definations, include_dirs, libraries_dirs, libraries and _passthough_args.
B. Options prefix:
Linux likes OS use - as prefix, but Windows use / as prefix.
Example:
Setup compiler optimization level as O2, on Windows should be /O2, but on Linux likes OS should be -O2.
C. Common and especial options.
Common options:
| Options | Windows | Linux |
|---|---|---|
| Compile Optimization | /O2 | -O2 |
| Add defination XXX | /D XXX | -D XXX |
| Add cflag | /cflag | -cflag |
| Add include dir | /I include_dir_path | -Iinclude_dir_path |
The common options has same build option between Windows and Linux, which is only different with prefix character. We can store the common options and then format to command as prefix character + common option in different OS.
Especial options:
| Options | Windows | Linux |
|---|---|---|
| Enable AVX2 | /arch:AVX2 | -mavx2 |
| Enable AVX512 | /arch:AVX512 | -mavx512f -mavx512dq -mavx512vl -mavx512bw |
Especial options is different between Windows and Linux, we can pass them to command line via _passthough_args.
Command line rules
A. Parameters pass order
Windows: cl [ option... ] filename... [ /link linkoption... ]
Linux: no parameters order requirement.
Reference: MSVC Cmdline
B. Output Binary Path
Windows: /Fe BinaryName
Linux: -o BinaryPath
Reference: MSVC Output
C. MSVC Compiler options
Please reference to this doc.
Build options orgnazation & modularization
Modularization
We have categories the options by types in above part. And in pytorch scenario, we can also modularization these build options. Such as BuildOptionsBase, CppOptions, CppTorchOptions, and CppTorchCudaOptions.
These options will inherit build options level by level.
| modularization name | inherit to | Options categories |
|---|---|---|
| BuildOptionsBase | NA(base implemention) | NA |
| CppOptions | BuildOptionsBase | shared_lib, cc_optimze(O2), cpp_std, warning_flag |
| CppTorchOptions | CppOptions | cpp_wrapper_defination, custom_generated_macros, torch_includes_and_libs, openmp |
| CppTorchCudaOptions | CppTorchOptions | cuda_incs_and_libs |
Note: we can extend for more devices on demand, example to xpu and mps.
| modularization name | inherit to | Options categories |
|---|---|---|
| CppTorchXpuOptions | CppTorchOptions | xpu_incs_and_libs |
| CppTorchMpsOptions | CppTorchOptions | mps_incs_and_libs |
Orgnazation
We can use python class to organize the inherit relations.
Cross OS CppBuilder
It will format the compiler command line from the build options, for both Windows and Linux.
Class Diagram
ISA checker
- Export a pybind ISA checker from backend cpuinfo.
- Switch read /proc/cpuinfo method to new pybind functions.
Implement Plan
Step 1:
- Add cpp_builder code, the new cpp_builder support Windows OS.
- Add CPU ISA checker which is cross OS and exported from backend cpuinfo.
- Switch compiler ISA checker to new cpp_builder.
CppCodeCacheuse the new ISA checker.
Step 2:
- Switch
AotCodeCompilerandCppCodeCacheto new CPP builder. - Optimize code structure, reduce circle imports: [RFC] Add new CPP builder for inductor on pytorch Windows #124245 (comment)
- Remove old cpp builder code.
Step 3:
- Fix issue for
fb_code, which need Meta employee help on. - Make code support Windows.
- Add Windows inductor UTs.
Alternatives
No response
Additional context
No response
cc @peterjc123 @mszhanyi @skyline75489 @nbcsm @vladimir-aubrecht @iremyux @Blackhex @cristianPanaite @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @ezyang @msaroufim @bdhirsh @anijain2305 @chauhang @voznesenskym @penguinwu @EikanWang @Guobing-Chen @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @amjames @desertfire
### PR list
- [ ] https://github.com/pytorch/pytorch/pull/125849
- [ ] https://github.com/pytorch/pytorch/pull/128274
- [ ] https://github.com/pytorch/pytorch/pull/128686
- [ ] https://github.com/pytorch/pytorch/pull/128765
- [ ] https://github.com/pytorch/pytorch/pull/128770
- [ ] https://github.com/pytorch/pytorch/pull/129059
- [ ] https://github.com/pytorch/pytorch/pull/129441
- [ ] https://github.com/pytorch/pytorch/pull/129789
- [ ] https://github.com/pytorch/pytorch/pull/129816
- [ ] https://github.com/pytorch/pytorch/pull/129577
- [ ] https://github.com/pytorch/pytorch/pull/130105
- [ ] https://github.com/pytorch/pytorch/pull/130146
- [ ] https://github.com/pytorch/pytorch/pull/130132
- [ ] https://github.com/pytorch/pytorch/pull/130713
- [ ] https://github.com/pytorch/pytorch/pull/130127
- [ ] https://github.com/pytorch/pytorch/pull/132594
- [ ] https://github.com/pytorch/pytorch/pull/132766
- [ ] https://github.com/pytorch/pytorch/pull/131286
- [ ] https://github.com/pytorch/pytorch/pull/131504
- [ ] https://github.com/pytorch/pytorch/pull/131752
- [ ] https://github.com/pytorch/pytorch/pull/131767
- [ ] https://github.com/pytorch/pytorch/pull/131978
- [ ] https://github.com/pytorch/pytorch/pull/131980
- [ ] https://github.com/pytorch/pytorch/pull/132025
- [ ] https://github.com/pytorch/pytorch/pull/132117
- [ ] https://github.com/pytorch/pytorch/pull/132326
- [ ] https://github.com/pytorch/pytorch/pull/132387
- [ ] https://github.com/pytorch/pytorch/pull/132394
- [ ] https://github.com/pytorch/pytorch/pull/132463
- [ ] https://github.com/pytorch/pytorch/pull/132533
- [ ] https://github.com/pytorch/pytorch/pull/132491
- [ ] https://github.com/pytorch/pytorch/pull/132571
- [ ] https://github.com/pytorch/pytorch/pull/132580
- [ ] https://github.com/pytorch/pytorch/pull/132630
- [ ] https://github.com/pytorch/pytorch/pull/132615
- [ ] https://github.com/pytorch/pytorch/pull/132823
- [ ] https://github.com/pytorch/pytorch/pull/132841
- [ ] https://github.com/pytorch/pytorch/pull/132848
- [ ] https://github.com/pytorch/pytorch/pull/132973
- [ ] https://github.com/pytorch/pytorch/pull/133103
- [ ] https://github.com/pytorch/pytorch/pull/133161
- [ ] https://github.com/pytorch/pytorch/pull/129975
- [ ] https://github.com/pytorch/pytorch/pull/133184
- [ ] https://github.com/pytorch/pytorch/pull/133186
- [ ] https://github.com/pytorch/pytorch/pull/133188
- [ ] https://github.com/pytorch/pytorch/pull/133164
- [ ] https://github.com/pytorch/pytorch/pull/133718
- [ ] https://github.com/pytorch/pytorch/pull/133455
- [ ] https://github.com/pytorch/pytorch/pull/133892
- [ ] https://github.com/pytorch/pytorch/pull/134027
- [ ] https://github.com/pytorch/pytorch/pull/133226
- [ ] https://github.com/pytorch/pytorch/pull/132757
- [ ] https://github.com/pytorch/pytorch/pull/134221
- [ ] https://github.com/pytorch/pytorch/pull/134229
- [ ] https://github.com/pytorch/pytorch/pull/134348
- [ ] https://github.com/pytorch/pytorch/pull/134397
- [ ] https://github.com/pytorch/pytorch/pull/134365
- [ ] https://github.com/pytorch/pytorch/pull/134394
- [ ] https://github.com/pytorch/pytorch/pull/134806
- [ ] https://github.com/pytorch/pytorch/pull/134900
- [ ] https://github.com/pytorch/pytorch/pull/134909
- [ ] https://github.com/pytorch/pytorch/pull/134510
- [ ] https://github.com/pytorch/pytorch/pull/134033
- [ ] https://github.com/pytorch/pytorch/pull/134358
- [ ] https://github.com/pytorch/pytorch/pull/134400
- [ ] https://github.com/pytorch/pytorch/pull/134401
- [ ] https://github.com/pytorch/pytorch/pull/134402
- [ ] https://github.com/pytorch/pytorch/pull/134419
- [ ] https://github.com/pytorch/pytorch/pull/134420
- [ ] https://github.com/pytorch/pytorch/pull/134424
- [ ] https://github.com/pytorch/pytorch/pull/134425
- [ ] https://github.com/pytorch/pytorch/pull/134426
- [ ] https://github.com/pytorch/pytorch/pull/134427
- [ ] https://github.com/pytorch/pytorch/pull/134428
- [ ] https://github.com/pytorch/pytorch/pull/134429
- [ ] https://github.com/pytorch/pytorch/pull/134585
- [ ] https://github.com/pytorch/pytorch/pull/134586
- [ ] https://github.com/pytorch/pytorch/pull/134587
- [ ] https://github.com/pytorch/pytorch/pull/134588
- [ ] https://github.com/pytorch/pytorch/pull/135449
- [ ] https://github.com/pytorch/pytorch/pull/134444
- [ ] https://github.com/pytorch/pytorch/pull/134772
- [ ] https://github.com/pytorch/pytorch/pull/134973
- [ ] https://github.com/pytorch/pytorch/pull/134870
- [ ] https://github.com/pytorch/pytorch/pull/135209
- [ ] https://github.com/pytorch/pytorch/pull/135230
- [ ] https://github.com/pytorch/pytorch/pull/135307
- [ ] https://github.com/pytorch/pytorch/pull/134553
- [ ] https://github.com/pytorch/pytorch/pull/135187
- [ ] https://github.com/pytorch/pytorch/pull/134917
- [ ] https://github.com/pytorch/pytorch/pull/138331
- [ ] https://github.com/pytorch/pytorch/pull/138173
- [ ] https://github.com/pytorch/pytorch/pull/138937
### Issue list
- [ ] https://github.com/pytorch/pytorch/issues/132569
- [ ] https://github.com/pytorch/pytorch/issues/132561
- [ ] [inductor] fresh_inductor_cache not support delete loaded modules.
- [ ] CppBenchmarkRequest --> CDLL to DLLWrapper
- [ ] https://github.com/pytorch/pytorch/issues/135274
- [ ] https://github.com/pytorch/pytorch/issues/135927
- [ ] https://github.com/pytorch/pytorch/issues/135954
Metadata
Metadata
Assignees
Labels
Type
Projects
Status