Skip to content

[py] Modularize Bazel build with per-module targets#17012

Merged
titusfortner merged 12 commits intotrunkfrom
py_build
Jan 29, 2026
Merged

[py] Modularize Bazel build with per-module targets#17012
titusfortner merged 12 commits intotrunkfrom
py_build

Conversation

@titusfortner
Copy link
Member

@titusfortner titusfortner commented Jan 26, 2026

🔗 Related Issues

Follows up on #16993 (lazy imports)

Java's approach is to create a separate BUILD.bazel file for each directory.
Ruby defines everything from one file, and that's the one this copies.

💥 What does this PR do?

Modularizes the Python Bazel build by creating separate py_library targets for each module

Module targets added:

  • :exceptions - Base exception classes
  • :remote - Wire protocol and WebDriver commands
  • :bidi - BiDi protocol support
  • :common - Shared utilities, actions, selenium-manager, devtools
  • :support - Wait conditions, event listeners
  • :chromium - Base for Chrome/Edge
  • :chrome, :edge, :firefox, :safari, :ie, :webkitgtk, :wpewebkit - Browser drivers

🔧 Implementation Notes

  • The aggregate :selenium target is preserved for backwards compatibility
  • Test targets now depend on specific browser modules instead of the full :selenium aggregate
  • Each module has imports = ["."] for proper Python path resolution
  • DevTools generated code is included in :common via data attribute (same as before)
  • With lazy imports (merged in [py] Use lazy imports in webdriver __init__.py #16993), users can now depend on individual modules without ImportError

💡 Additional Considerations

  • Future optimization: tests could be further refined to depend on even more specific modules

🔄 Types of changes

  • New feature (non-breaking change which adds functionality)

@selenium-ci selenium-ci added C-py Python Bindings B-build Includes scripting, bazel and CI integrations labels Jan 26, 2026
@qodo-code-review
Copy link
Contributor

User description

🔗 Related Issues

Follows up on #16993 (lazy imports)

Java's approach is to create a separate BUILD.bazel file for each directory.
Ruby defines everything from one file, and that's the one this copies.

💥 What does this PR do?

Modularizes the Python Bazel build by creating separate py_library targets for each module

Module targets added:

  • :exceptions - Base exception classes
  • :remote - Wire protocol and WebDriver commands
  • :bidi - BiDi protocol support
  • :common - Shared utilities, actions, selenium-manager, devtools
  • :support - Wait conditions, event listeners
  • :chromium - Base for Chrome/Edge
  • :chrome, :edge, :firefox, :safari, :ie, :webkitgtk, :wpewebkit - Browser drivers

🔧 Implementation Notes

  • The aggregate :selenium target is preserved for backwards compatibility
  • Test targets now depend on specific browser modules instead of the full :selenium aggregate
  • Each module has imports = ["."] for proper Python path resolution
  • DevTools generated code is included in :common via data attribute (same as before)
  • With lazy imports (merged in [py] Use lazy imports in webdriver __init__.py #16993), users can now depend on individual modules without ImportError

💡 Additional Considerations

  • Next step is to get the tests to reference only the modules they need

🔄 Types of changes

  • New feature (non-breaking change which adds functionality)

PR Type

Enhancement


Description

  • Modularizes Python Bazel build into per-module targets

  • Creates separate py_library targets for exceptions, remote, bidi, common, support, chromium, and browser drivers

  • Preserves aggregate :selenium target for backwards compatibility

  • Updates test targets to depend on specific modules instead of full aggregate

  • Enables faster incremental builds and clearer dependency graph


Diagram Walkthrough

flowchart LR
  exceptions["exceptions<br/>Base exceptions"]
  remote["remote<br/>Wire protocol"]
  bidi["bidi<br/>BiDi protocol"]
  common["common<br/>Shared utilities"]
  support["support<br/>Wait/listeners"]
  chromium["chromium<br/>Chrome/Edge base"]
  chrome["chrome<br/>Chrome driver"]
  edge["edge<br/>Edge driver"]
  firefox["firefox<br/>Firefox driver"]
  safari["safari<br/>Safari driver"]
  ie["ie<br/>IE driver"]
  webkitgtk["webkitgtk<br/>WebKitGTK driver"]
  wpewebkit["wpewebkit<br/>WPEWebKit driver"]
  selenium["selenium<br/>Aggregate target"]
  
  exceptions --> remote
  exceptions --> bidi
  exceptions --> common
  exceptions --> support
  remote --> common
  bidi --> common
  remote --> chromium
  common --> chromium
  chromium --> chrome
  chromium --> edge
  remote --> firefox
  common --> firefox
  remote --> safari
  common --> safari
  remote --> ie
  common --> ie
  remote --> webkitgtk
  common --> webkitgtk
  remote --> wpewebkit
  common --> wpewebkit
  
  exceptions --> selenium
  remote --> selenium
  bidi --> selenium
  common --> selenium
  support --> selenium
  chromium --> selenium
  chrome --> selenium
  edge --> selenium
  firefox --> selenium
  safari --> selenium
  ie --> selenium
  webkitgtk --> selenium
  wpewebkit --> selenium
Loading

File Walkthrough

Relevant files
Enhancement
BUILD.bazel
Modularize Bazel targets by browser and functionality       

py/BUILD.bazel

  • Added 13 new modular py_library targets (exceptions, remote, bidi,
    common, support, chromium, chrome, edge, firefox, safari, ie,
    webkitgtk, wpewebkit)
  • Refactored aggregate :selenium target to depend on all module targets
    instead of directly including all sources
  • Updated browser test targets to depend on specific modules (e.g.,
    :chrome, :common, :support) instead of full :selenium aggregate
  • Added module-specific dependencies and data attributes (e.g.,
    firefox-driver-prefs for firefox, mutation-listener for bidi)
  • Organized code with clear section headers for module targets and
    aggregate target
+200/-23

@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Jan 26, 2026

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢
No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

  • Update
Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

@qodo-code-review
Copy link
Contributor

qodo-code-review bot commented Jan 26, 2026

PR Code Suggestions ✨

Latest suggestions up to 8ed92f0

CategorySuggestion                                                                                                                                    Impact
Possible issue
Make root package importable
Suggestion Impact:Instead of adding a separate `selenium_init` py_library and wiring it into module deps, the commit folded `selenium/__init__.py` and `selenium/webdriver/__init__.py` into the foundational `exceptions` target (comment updated accordingly) and removed those files from the `selenium` aggregate target by setting its `srcs = []`. This addresses the same underlying concern (ensuring package init files are included in a base library so imports work) but via a different structure than suggested.

code diff:

-# Exceptions (no deps - foundational)
+# Exceptions and package init (no deps - foundational)
 py_library(
     name = "exceptions",
-    srcs = glob(["selenium/common/**/*.py"]),
+    srcs = [
+        "selenium/__init__.py",
+        "selenium/webdriver/__init__.py",
+    ] + glob(["selenium/common/**/*.py"]),
     imports = ["."],
     visibility = ["//visibility:public"],
 )
@@ -397,10 +400,7 @@
 
 py_library(
     name = "selenium",
-    srcs = [
-        "selenium/__init__.py",
-        "selenium/webdriver/__init__.py",
-    ],
+    srcs = [],
     data = ["selenium/py.typed"],
     imports = ["."],
     visibility = ["//visibility:public"],

Create a dedicated py_library for root init.py files and add it as a
dependency to each new module target to ensure they are importable.

py/BUILD.bazel [316-422]

+py_library(
+    name = "selenium_init",
+    srcs = [
+        "selenium/__init__.py",
+        "selenium/webdriver/__init__.py",
+    ],
+    data = ["selenium/py.typed"],
+    imports = ["."],
+    visibility = ["//visibility:public"],
+)
+
 py_library(
     name = "chrome",
     srcs = glob(["selenium/webdriver/chrome/**/*.py"]),
     imports = ["."],
     visibility = ["//visibility:public"],
-    deps = [":chromium"],
+    deps = [
+        ":selenium_init",
+        ":chromium",
+    ],
 )
 
 py_library(
     name = "edge",
     srcs = glob(["selenium/webdriver/edge/**/*.py"]),
     imports = ["."],
     visibility = ["//visibility:public"],
-    deps = [":chromium"],
+    deps = [
+        ":selenium_init",
+        ":chromium",
+    ],
 )
 
 py_library(
     name = "firefox",
     srcs = glob(["selenium/webdriver/firefox/**/*.py"]),
     data = [":firefox-driver-prefs"],
     imports = ["."],
     visibility = ["//visibility:public"],
     deps = [
+        ":selenium_init",
         ":common",
         ":remote",
     ],
 )
 
 py_library(
     name = "selenium",
     srcs = [
         "selenium/__init__.py",
         "selenium/webdriver/__init__.py",
     ],
     data = ["selenium/py.typed"],
     imports = ["."],
     visibility = ["//visibility:public"],
     deps = [
         ":bidi",
         ":chrome",
         ":chromium",
         ":common",
         ":edge",
         ":exceptions",
         ":firefox",
         ":ie",
         ":remote",
         ":safari",
         ":support",
         ":webkitgtk",
         ":wpewebkit",
     ],
 )

[Suggestion processed]

Suggestion importance[1-10]: 9

__

Why: The suggestion correctly identifies a critical flaw in the PR's modularization effort where individual modules would not be importable, and provides a sound solution to fix it.

High
Add root package to tests

Add the proposed :selenium_init target to the test dependencies to prevent
import errors that would arise from removing the monolithic :selenium
dependency.

py/BUILD.bazel [678-681]

 deps = [
     ":init-tree",
+    ":selenium_init",
     ":webserver",
 ] + BROWSER_TESTS[browser]["deps"] + TEST_DEPS,
  • Apply / Chat
Suggestion importance[1-10]: 9

__

Why: This suggestion correctly identifies that tests will fail due to missing root __init__.py files after the dependency refactoring, and proposes a correct fix that is essential for the tests to pass.

High
Incremental [*]
Make test dependencies explicit

Add :remote and :exceptions as explicit dependencies to each browser's test
configuration to make the build more robust against future refactoring.

py/BUILD.bazel [612-656]

 "chrome": {
     "browser_srcs": ["test/selenium/webdriver/chrome/**/*.py"],
     "deps": [
         ":chrome",
         ":common",
         ":support",
+        ":remote",
+        ":exceptions",
     ],
     "bidi": True,
 },
 "edge": {
     "browser_srcs": ["test/selenium/webdriver/edge/**/*.py"],
     "deps": [
         ":edge",
         ":common",
         ":support",
+        ":remote",
+        ":exceptions",
     ],
 },
 "firefox": {
     "browser_srcs": [
         "test/selenium/webdriver/firefox/**/*.py",
     ],
     "deps": [
         ":firefox",
         ":common",
         ":support",
+        ":remote",
+        ":exceptions",
     ],
     "extra_excludes": ["test/selenium/webdriver/common/devtools_tests.py"],
     "bidi": True,
 },
 "ie": {
     "browser_srcs": ["test/selenium/webdriver/ie/**/*.py"],
     "deps": [
         ":ie",
         ":common",
         ":support",
+        ":remote",
+        ":exceptions",
     ],
 },
 "safari": {
     "browser_srcs": ["test/selenium/webdriver/safari/**/*.py"],
     "deps": [
         ":safari",
         ":common",
         ":support",
+        ":remote",
+        ":exceptions",
     ],
 },
  • Apply / Chat
Suggestion importance[1-10]: 6

__

Why: The suggestion correctly proposes making test dependencies explicit, which improves the build configuration's robustness and maintainability by not relying on transitive dependencies.

Low
  • Update

Previous suggestions

Suggestions up to commit dcf479f
CategorySuggestion                                                                                                                                    Impact
Possible issue
Exclude devtools and generate script

Update the exclude list for the common py_library's srcs to also filter out
generated devtools files (selenium/webdriver/common/devtools/v
/
) and the
generator script (selenium/webdriver/common/generate.py).
*

py/BUILD.bazel [268-273]

 srcs = glob(
     ["selenium/webdriver/common/**/*.py"],
     exclude = [
         "selenium/webdriver/common/bidi/**",
+        "selenium/webdriver/common/devtools/v*/**",
+        "selenium/webdriver/common/generate.py",
     ],
 ),
Suggestion importance[1-10]: 8

__

Why: This suggestion correctly identifies that the refactoring missed excluding generated devtools files and a generator script from the common library sources, which were excluded in the original build configuration. This prevents including unnecessary and potentially problematic files in the build target.

Medium
General
Use a more specific file path

Refine the srcs for the exceptions py_library to target only
selenium/common/exceptions.py instead of using a broad glob.

py/BUILD.bazel [223-229]

 # Exceptions (no deps - foundational)
 py_library(
     name = "exceptions",
-    srcs = glob(["selenium/common/**/*.py"]),
+    srcs = ["selenium/common/exceptions.py"],
     imports = ["."],
     visibility = ["//visibility:public"],
 )
Suggestion importance[1-10]: 7

__

Why: The suggestion correctly identifies that the glob for the exceptions library is too broad and will include unintended files. Specifying the exact file improves the build target's precision and correctness.

Medium
Remove redundant build dependency

Remove the redundant :chromium dependency from the selenium aggregate
py_library, as it is already a transitive dependency via :chrome and :edge.

py/BUILD.bazel [396-420]

 # Aggregate Target (backwards compatible - includes all modules)
 # ==============================================================================
 
 py_library(
     name = "selenium",
     srcs = [
         "selenium/__init__.py",
         "selenium/webdriver/__init__.py",
     ],
     data = ["selenium/py.typed"],
     imports = ["."],
     visibility = ["//visibility:public"],
     deps = [
         ":bidi",
         ":chrome",
-        ":chromium",
         ":common",
         ":edge",
         ":exceptions",
         ":firefox",
         ":ie",
         ":remote",
         ":safari",
         ":support",
         ":webkitgtk",
         ":wpewebkit",
     ],
 )
Suggestion importance[1-10]: 4

__

Why: The suggestion correctly points out a redundant dependency in the aggregate selenium target. Removing it improves code clarity and maintainability, which is a good practice.

Low
Learned
best practice
Centralize repeated BUILD boilerplate

Create a small local macro (or load one from a shared .bzl) that applies the
common imports/visibility defaults and use it for module targets to reduce
copy/paste and prevent drift.

py/BUILD.bazel [314-329]

-py_library(
+def selenium_module(name, srcs, deps = [], data = None):
+    py_library(
+        name = name,
+        srcs = srcs,
+        data = data,
+        imports = ["."],
+        visibility = ["//visibility:public"],
+        deps = deps,
+    )
+
+selenium_module(
     name = "chrome",
     srcs = glob(["selenium/webdriver/chrome/**/*.py"]),
-    imports = ["."],
-    visibility = ["//visibility:public"],
     deps = [":chromium"],
 )
 
-py_library(
+selenium_module(
     name = "edge",
     srcs = glob(["selenium/webdriver/edge/**/*.py"]),
-    imports = ["."],
-    visibility = ["//visibility:public"],
     deps = [":chromium"],
 )
Suggestion importance[1-10]: 6

__

Why:
Relevant best practice - Reduce duplication by centralizing repeated target boilerplate (e.g., shared imports/visibility/common attrs) into a macro/helper to keep module BUILD definitions consistent and easier to maintain.

Low

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR modularizes the Python Bazel build by creating separate py_library targets for each module, following Ruby's modular pattern from the same monorepo. The goal is to enable users to depend on individual modules (like :chrome or :firefox) without pulling in the entire Selenium library, building on the lazy import mechanism added in PR #16993.

Changes:

  • Creates 15 separate module targets: :exceptions, :remote, :bidi, :common, :support, :chromium, and individual browser drivers (:chrome, :edge, :firefox, :safari, :ie, :webkitgtk, :wpewebkit)
  • Preserves the aggregate :selenium target for backwards compatibility
  • Updates test targets to depend on specific modules instead of the full aggregate

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

titusfortner and others added 8 commits January 27, 2026 12:15
Add separate py_library targets for each module following Ruby's pattern:
- exceptions: Base exception classes
- remote: Wire protocol and WebDriver commands
- bidi: BiDi protocol support
- common: Shared utilities, actions, selenium-manager, devtools
- support: Wait conditions, event listeners
- chromium: Base for Chrome/Edge
- chrome, edge, firefox, safari, ie, webkitgtk, wpewebkit: Browser drivers

The aggregate :selenium target is preserved for backwards compatibility,
depending on all module targets. This enables:
- Faster incremental builds when editing a single module
- Clear dependency graph between modules
- Future ability for users to depend on subsets (with lazy imports)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update browser test targets to depend on specific modules instead
of the aggregate :selenium target:
- test-chrome → :chrome, :common, :support
- test-firefox → :firefox, :common, :support
- test-edge → :edge, :common, :support
- etc.

This enables faster incremental test builds when only one module
is modified.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add missing deps identified in PR review:
- :bidi needs :remote (imports websocket_connection)
- :support needs :common (imports by, alert)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move selenium/__init__.py and selenium/webdriver/__init__.py to the
:exceptions module so lazy imports (e.g., webdriver.ChromeOptions) work
when depending on individual modules like :chrome instead of :selenium.

This fixes integration test failures where tests use lazy import syntax.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes mypy error about incompatible types in assignment when
handler is assigned different RemoteConnection subclasses.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Move remote_firefox_profile_tests.py to firefox/ folder since it's
  Firefox-specific
- Change remote_hub_connection.py to use ArgOptions instead of Firefox
  Options since the test is about SSL cert verification, not Firefox

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Member

@cgoldberg cgoldberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you rename remote_hub_connection.py so it ends with _tests (remote_hub_connection_tests.py)? Otherwise, pytest won't find it when you run locally. It filters on ["test_*.py", "*_test.py", "*_tests.py"].

@cgoldberg
Copy link
Member

I think CI is failing because remote_profile_tests.py was moved out of the remote/ directory, so no Grid server exists when it runs without --remote.

@titusfortner
Copy link
Member Author

Generally speaking:

  • Browser specific features should be tested locally & remote
  • Remote only features shouldn't care what browser runs them

Based on that, a test about firefox profiles belongs in firefox directory, and if the test only applies when remote and not local, we need to generate targets based on that.

titusfortner and others added 2 commits January 28, 2026 09:51
Move remote_firefox_profile_tests.py back to remote/ folder where it
belongs - it needs the server fixture which only exists in remote mode.
Exclude it from chrome-remote tests since it's Firefox-specific.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@cgoldberg
Copy link
Member

cgoldberg commented Jan 28, 2026

Browser specific features should be tested locally & remote

The way the test is written in remote_profile_tests.py, it explicitly creates a Remote instance, but a similar test should also work with a local driver.

To support both, I think we can change the test to:


from conftest import Driver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile


def test_profile_is_used(request, server):
    ff_profile = FirefoxProfile()
    ff_profile.set_preference("browser.startup.page", "1")
    try:
        driver = Driver("firefox", request)
        driver._server = server
        driver.options.profile = ff_profile
        browser = driver.driver
        assert "browser/content/blanktab.html" in browser.current_url
    finally:
        driver.stop_driver()

... and it will work with remote or local Firefox.

If we make it worth both ways, the file should be renamed from remote_profile_tests.py to firefox_profile_tests.py

We could also refactor the Driver class in conftest to make this a little easier and not require the test to use the server fixture.

There are also some tests inside /firefox/firefox_sizing_tests.py that explicitly launch a local driver (even if called with --remote). We should update those to work with either remote or local also.

Use the Driver class from conftest to handle both local and remote
execution. The test now works with both --remote flag and without it.

Move test from remote/ to firefox/ folder since it's testing a
Firefox-specific feature (profiles), not a remote-only feature.

Rename from remote_firefox_profile_tests.py to firefox_profile_tests.py.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Member

@cgoldberg cgoldberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@titusfortner titusfortner merged commit a0ef99e into trunk Jan 29, 2026
40 checks passed
@titusfortner titusfortner deleted the py_build branch January 29, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

B-build Includes scripting, bazel and CI integrations C-py Python Bindings Review effort 2/5

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants