Skip to content

[Refactor] Refactor building system and adding compiled products#261

Merged
yaoyaoding merged 2 commits intohidet-org:mainfrom
yaoyaoding:template
Jun 2, 2023
Merged

[Refactor] Refactor building system and adding compiled products#261
yaoyaoding merged 2 commits intohidet-org:mainfrom
yaoyaoding:template

Conversation

@yaoyaoding
Copy link
Copy Markdown
Member

Refactor building system and adding compiled products

Now, we can build Function, IRModule, Task and FlowGraph to CompiledFunction, CompiledModule, CompiledTask and CompiledModule. The compiled products can be saved on the disk and loaded for running.

The CompiledTask has a lib.so that contains all the scheduled kernels. So we will not have the problem of loading thousands of dynamic libraries when we support dynamic shape & kernel tuning (e.g., 100 operators * 200 schedules for each operator).

The CompiledGraph contains a C runtime that invokles the kernels and allocates memory. With this, we do not need CUDA graph to get rid of the framework overhead when inference. The CompiledGraph can be stored as a single file on the disk. Thus, we can compile one model on one machine and then tune & run the model on another machine.

Minors

  • Add target argument to build_task and build_ir_module to specify the target for code generation and underlying compiler (e.g., nvcc vs. gcc)
  • Move IRModule from hidet.ir.func to hidet.ir.module.
  • Changed the format to define tuning space (may change again in the future).

.
.

.

.

.

.

fix

.

.

.

.

.

.

.

.

.

.

.

.
@yaoyaoding yaoyaoding merged commit 3269ee5 into hidet-org:main Jun 2, 2023
@yaoyaoding yaoyaoding deleted the template branch June 2, 2023 15:29
vadiklyutiy added a commit that referenced this pull request Jul 22, 2024
vadiklyutiy added a commit that referenced this pull request Jul 23, 2024
vadiklyutiy added a commit that referenced this pull request Dec 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant