Skip to content

Migrate to bazel 8, start migrating to bazelmod#1445

Merged
asraa merged 2 commits into
google:mainfrom
j2kun:bzlmod
Feb 25, 2025
Merged

Migrate to bazel 8, start migrating to bazelmod#1445
asraa merged 2 commits into
google:mainfrom
j2kun:bzlmod

Conversation

@j2kun

@j2kun j2kun commented Feb 20, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Fixes #332

This PR sets up a hybrid Bazelmod + WORKSPACE configuration, where the "hard" dependencies remain in a workspace file while the supported dependencies are migrated to MODULE.bazel

This allows us to use the latest bazel version. I also went ahead and bumped versions of various dependencies, including rules_python and rules_hdl, which have material impacts on the project.

Summary of impact:

  • Bazel 8 uses _main instead of heir in many bazel runfiles paths, which broke yosys techmap file finding, lit configuration, and our heir-opt bazel rules.
  • Globs that have empty matches are now hard errors
  • Go dependencies now must come in a go.mod file
  • rules_python now has a different build target naming scheme (@heir_py_pip_deps_numba//:pkg -> @heir_py_pip_deps//numba)
  • rules_python now has stricter hermeticity conditions, breaking our use of LLVM's in-tree lit tool, so I moved lit to a pip dependency.
  • I updated our rules_python to use Pyhton3.11 by default (Cf. Which versions of Python should the frontend support? #1370)
  • Some deps we were building manually are on the bazel registry, which had minor changes to the import paths (e.g. eigen)
  • The new rules_go decides on bazel project names for you, so instead of @lattigo we have to do @com_github_tuneinsight_lattigo_v6
  • We now have a MODULE.bazel.lock which should improve hermeticity

The main dependencies that are still handled by WORKSPACE:

@j2kun j2kun requested a review from asraa February 20, 2025 23:51
@j2kun

j2kun commented Feb 21, 2025

Copy link
Copy Markdown
Collaborator Author

Bazel 8 uses _main instead of heir in many bazel runfiles paths, which broke yosys techmap file finding, lit configuration, and our heir-opt bazel rules.

I worked with @raghav198 on a bug related to this: apparently on MacOS platforms bazel starts inside _main, while on my linux system it was outside _main. Need to investigate this a bit closer.

@j2kun

j2kun commented Feb 21, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Still TODO:

@j2kun j2kun mentioned this pull request Feb 21, 2025
Closed
@j2kun

j2kun commented Feb 24, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

@asraa @AlexanderViand-Intel do you think we can get this PR in before the Python frontend? I think getting this in with the python version bump would help with the internal patch for Alex's python frontend PR. I will handle the rebase in the internal patch

asraa
asraa approved these changes Feb 24, 2025
View reviewed changes

@asraa asraa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I also remember getting a survey from the bazel folks at google to list out the blocking deps for bzlmod migration and at the time I didn't know exactly which were blocking. I'll send them over the remaining ones

Comment thread MODULE.bazel
Comment on lines +78 to +79
# See the project repo for more details and configuration options
# https://github.com/hedronvision/bazel-compile-commands-extractor

@asraa asraa Feb 24, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should use renovatebot to manage the sha update - should we file an issue? eventually we should have some dependency management automated, and bzlmod at least takes care of it for the

@j2kun j2kun Feb 24, 2025

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File an issue :)

@j2kun

j2kun commented Feb 24, 2025

Copy link
Copy Markdown
Collaborator Author

I think I can fix the remaining frontend issues, almost got it.

@j2kun j2kun added the pull_ready Indicates whether a PR is ready to pull. The copybara worker will import for internal testing label Feb 24, 2025
@AlexanderViand-Intel

AlexanderViand-Intel commented Feb 24, 2025 •
edited
Loading

Copy link
Copy Markdown

I think I can fix the remaining frontend issues, almost got it.

I just had a look and is_mlir_type_annotation(..) looks...interesting 😉 What made this necessary/broke the simple "is subclass" check?

copybara-service Bot pushed a commit that referenced this pull request Feb 24, 2025
@j2kun @copybara-github
Fix pip dep copybara rules for #1445
056a9d5 
PiperOrigin-RevId: 730620084
@copybara-service copybara-service Bot mentioned this pull request Feb 24, 2025
Closed
copybara-service Bot pushed a commit that referenced this pull request Feb 24, 2025
@j2kun @copybara-github
Fix pip dep copybara rules for #1445
05eb960 
PiperOrigin-RevId: 730620084
@j2kun

j2kun commented Feb 25, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

I think I can fix the remaining frontend issues, almost got it.

I just had a look and is_mlir_type_annotation(..) looks...interesting 😉 What made this necessary/broke the simple "is subclass" check?

OK I reverted this and now I can't reproduce my original problem... It's concerning but what was happening was the type being passed to the issubclass was actually the Secret type, which is not an instance of Secret but rather of _GenericAlias, and you needed to get the origin type. But now stepping through it's properly extracting the internal type as the code obviously is supposed to do. So I don't know what I was seeing anymore.

@j2kun

j2kun commented Feb 25, 2025

Copy link
Copy Markdown
Collaborator Author

OK the internal patch is now done and tested, just waiting on a Googler to click the button to merge it. Sorry for the delay all!

@asraa asraa merged commit 1a36236 into google:main Feb 25, 2025
@j2kun j2kun deleted the bzlmod branch August 19, 2025 02:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@asraa asraa asraa approved these changes

Assignees

No one assigned

Labels

pull_ready Indicates whether a PR is ready to pull. The copybara worker will import for internal testing

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Migrate to bazel 8

3 participants

@j2kun @AlexanderViand-Intel @asraa