✨ feat(extras): add --extras flag for optional deps by gaborbernat · Pull Request #549 · tox-dev/pipdeptree

gaborbernat · 2026-03-03T06:08:00Z

pipdeptree has long ignored extra/optional dependencies defined via PEP 508 extra markers. When a user installs a package with extras (e.g. pip install jira[signedtoken]), the extra dependencies appear as unrelated top-level packages instead of showing up as children in the dependency tree. This has been a recurring pain point since issue #107 was opened, and previous attempts (#132, #138) were never merged.

The new --extras / -x flag enables transitive resolution of extras in the dependency tree. 🔍 It works by collecting extras from requirement specifiers (e.g. oauthlib[signedtoken]), evaluating PEP 508 markers with the correct extras context via marker.evaluate({"extra": name}), and iterating until no new extras are discovered. Each extra dependency is annotated with its extra name across all output formats — text shows extra: signedtoken, JSON includes an "extra" field, and graphviz/mermaid prefix edge labels with [signedtoken]. Cycle warnings also annotate extras with [extra: ...] so it's clear when a cycle involves optional vs required deps.

Only explicitly requested extras are resolved — those appearing in requirement specifiers like cyclonedx-python-lib[validation]. Root packages' own extras are not auto-included since pip doesn't record which extras were used during installation, and heuristics produce false positives for forwarding-style extras (e.g. meta-package-manager forwards its extras to click-extra). Validated against meta-package-manager[yaml,hjson] as a real-world test case.

The flag is off by default to preserve backward compatibility. The from_pkgs constructor gained a keyword-only include_extras parameter, and the extras resolution runs as a separate pass after the base DAG is built.

pipdeptree ignores extra/optional dependencies (PEP 508 extra markers), causing packages installed via `pip install pkg[extra]` to appear as unrelated top-level packages instead of children. This has been a long-standing request (issue tox-dev#107). The new `--extras`/`-x` flag enables transitive resolution of extras in the dependency tree. It collects extras from requirement specifiers (e.g. `oauthlib[signedtoken]`), evaluates markers with extras context, and includes root packages' provided extras where deps are installed. Extra dependencies are annotated in all output formats (text, JSON, graphviz, mermaid) for clarity.

kemzeb · 2026-03-03T18:01:51Z

This is great, I'll review it as soon as I can

kemzeb · 2026-03-04T06:27:16Z

From a couple of passthroughs of the changes, this looks great. All the rendering options appear fine when testing. Tomorrow I plan to review and test it some more just to see if I can catch any issues.

kemzeb

After further review, I wasn't able to find any glaring issues.

The only thing I did find was with the cyclic dependency validation check:

It maybe confusing as at first glance it looks like the dependency cycle is between required dependencies, when it is actually is between required and optional deps:

To get the changes above I made changes to render_cycles_text using the following:

# . . .
        for idx, pkg in enumerate(cycle):
            name = pkg.project_name
            if isinstance(pkg, ReqPackage):
                name = name + f" [extra: {pkg.extra}]" if pkg.extra else ""
            if idx == size:
                print(name, end="", file=sys.stderr)  # noqa: T201
            else:
                print(f"{name} =>", end=" ", file=sys.stderr)  # noqa: T201
# . . .

You could use this or find a better alternative if you wish.

Cycle warnings didn't distinguish between required and optional deps, making it look like a required-dependency cycle when the cycle actually involves an extras edge. Annotate ReqPackage nodes that carry an extra with `[extra: ...]` in the cycle output.

gaborbernat · 2026-03-05T05:26:45Z

Thanks for the thorough review @kemzeb! Great catch on the cycle output — applied your suggestion in 48721ad. Cycle warnings now annotate extras with [extra: ...] so it's clear when a cycle involves optional vs required deps.

kemzeb · 2026-03-05T05:47:17Z

Yep no problem!

I did decide to mess around with projects with some complex extras, and found meta-package-manager.

Did find some things:

It does appear that it would duplicate the click-extra dependency, but click-extra is defined both as a required dependency + as an optional dependency associated to multiple extras. Maybe we can iterate on this in a newer version? Though I need to investigate why we are seeing some of these as they appear to not be explicitly requested (e.g. I see yaml and xml here, but meta-package-manager does not explicitly request these extras for click-extra)
xmltodict is defined as both a required dependency and an optional dependency. Should we be including the extra: xml text for these?
Not sure why PyYAML is being included in these sub-trees as it's not a required dependency

kemzeb · 2026-03-05T05:48:04Z

I'll need to do some more digging into meta-package-manager's metadata file to look for answers

kemzeb · 2026-03-05T06:28:44Z

Not sure why PyYAML is being included in these sub-trees as it's not a required dependency

Interesting, so it appears that though the yaml extra wasn't requested by meta-package-manager, I have a package called spdx-tools that requires PyYAML so since it would exist as a DistPackage my guess is the current changes see this as the yaml extra being requested so it will be included under click-extra:

├── spdx-tools [required: >=0.8.2, installed: 0.8.3]
│   ├── click [required: Any, installed: 8.3.1]
│   ├── PyYAML [required: Any, installed: 6.0.3]  <------ Installed since it is required
│   ├── xmltodict [required: Any, installed: 1.0.2]
│   ├── rdflib [required: Any, installed: 7.6.0]
│   ├── beartype [required: Any, installed: 0.22.9]
│   ├── uritools [required: Any, installed: 6.0.1]
│   ├── license-expression [required: Any, installed: 30.4.4]
│   ├── ply [required: Any, installed: 3.11]
│   └── semantic-version [required: Any, installed: 2.10.0]
├── tomli [required: >=2.3.0, installed: 2.4.0]
├── tomli_w [required: >=1.1.0, installed: 1.2.0]
├── xmltodict [required: >=1.0.0, installed: 1.0.2]
├── click-extra [required: Any, installed: 7.6.4, extra: hjson]
│   ├── boltons [required: >=25, installed: 25.0.0]
│   ├── click [required: >=8.3.1, installed: 8.3.1]
│   ├── cloup [required: >=3.0.7, installed: 3.0.8]
│   ├── deepmerge [required: >=2, installed: 2.0]
│   ├── extra-platforms [required: >=8, installed: 9.0.0]
│   ├── requests [required: >=2.32.5, installed: 2.32.5]
│   ├── tabulate [required: >=0.9, installed: 0.9.0]
│   ├── tomli [required: >=2.3, installed: 2.4.0]
│   ├── wcmatch [required: >=10, installed: 10.1]
│   ├── xmltodict [required: >=1, installed: 1.0.2, extra: xml]
│   └── PyYAML [required: >=6.0.3, installed: 6.0.3, extra: yaml]    <------ Added, most likely because it exist as a DistPackage

Though I need to investigate why we are seeing some of these as they appear to not be explicitly requested (e.g. I see yaml and xml here, but meta-package-manager does not explicitly request these extras for click-extra)

I think this issue and the issue above have the same problem. I believe it's because we are trying to grab all extras when we should be grabbing only extras that were explicitly requested (so a requirement specifier like cryptography[security,validation]; the security and validation extras are explicit, we shouldn't take a non-explicit extra from cryptography such as test since it was never asked for).

If you take a look at some of the optional versions of click-extra like jsonc, you would see the text render does not include the package it is requesting for (so for jsonc, the json-with-comments package should have been there). This further provides reason to only accept explicit extras.

xmltodict is defined as both a required dependency and an optional dependency. Should we be including the extra: xml text for these?

Forget about this, it's an optional dependency.

kemzeb · 2026-03-05T06:40:17Z

So what I think we need to figure out now is how to consider only explicit extras

Extras are resolved from two sources: explicit requirement specifiers (e.g. pkg[extra]) and satisfied extras where all dependencies of an extra group are already installed in the environment.

gaborbernat · 2026-03-05T16:33:30Z

So what I think we need to figure out now is how to consider only explicit extras

I think we should show an extra as installed if all its dependencies are available. This is how packages handle features, if a given extra package is importable is considered feature enabled, independent if was installed directly or through an additional dependency/install.

kemzeb

This is good, I believe your approach makes sense. It's a shame there's currently no officially supported way to detect what non-explicit extras have been requested in the environment, but I can see why it is a complicated problem.

src/pipdeptree/_models/dag.py

kemzeb

Not able to find any further issues.

However, I'm not sure if it would be worth it to account for optional dependencies in our DAG validation phase. For example:

Warning!!! Possibly conflicting dependencies found:
* graphviz==0.21
 - pytest [required: >=7,<8.1, installed: 9.0.2, extra: test]
* pyproject-api==1.10.0
 - setuptools [required: >=80.9, installed: 78.1.1, extra: testing]
* rdflib==7.6.0
 - lxml [required: >=4.3,<6.0, installed: 6.0.2, extra: lxml]
------------------------------------------------------------------------
Warning!!! Cyclic dependencies found:
* pytest => exceptiongroup => pytest [extra: test]
* tox => pluggy => tox [extra: dev]
* pytest-cov => pluggy => tox [extra: dev] => pyproject-api => pytest-cov [extra: testing]
* pytest-mock => pytest => pluggy => tox [extra: dev] => pyproject-api => pytest-mock [extra: testing]
* setuptools => pytest-cov [extra: cover] => pluggy => tox [extra: dev] => pyproject-api => setuptools [extra: testing]
* exceptiongroup => pytest [extra: test] => exceptiongroup
* pyproject-api => pytest-cov [extra: testing] => pluggy => tox [extra: dev] => pyproject-api
* pluggy => tox [extra: dev] => pluggy
------------------------------------------------------------------------

Yes this logic for both detecting conflicting and cyclic dependencies is correct, but is it useful to include optional dependencies? Let's look at this line for example:

* pluggy => tox [extra: dev] => pluggy

pluggy uses tox in it's development workflow and tox uses pluggy for its plugin system, so technically it is a cyclic dependency but is this useful information? I won't block this since it is still technically correct

gaborbernat added the enhancement label Mar 3, 2026

gaborbernat requested a review from kemzeb March 3, 2026 06:08

kemzeb requested changes Mar 5, 2026

View reviewed changes

gaborbernat enabled auto-merge (squash) March 5, 2026 05:27

♻️ refactor(extras): resolve explicit and satisfied extras

8f56475

Extras are resolved from two sources: explicit requirement specifiers (e.g. pkg[extra]) and satisfied extras where all dependencies of an extra group are already installed in the environment.

gaborbernat force-pushed the fix/extras-support branch from 63199eb to 8f56475 Compare March 5, 2026 16:32

gaborbernat requested a review from kemzeb March 5, 2026 16:34

kemzeb requested changes Mar 6, 2026

View reviewed changes

src/pipdeptree/_models/dag.py Show resolved Hide resolved

gaborbernat force-pushed the fix/extras-support branch 4 times, most recently from 54df85e to 494514e Compare March 7, 2026 00:19

gaborbernat requested a review from kemzeb March 7, 2026 00:20

✨ feat(extras): verify sub-extras satisfaction recursively

7fc0b63

gaborbernat force-pushed the fix/extras-support branch from 494514e to 7fc0b63 Compare March 7, 2026 00:23

kemzeb approved these changes Mar 7, 2026

View reviewed changes

gaborbernat merged commit 6d2d24b into tox-dev:main Mar 7, 2026
10 checks passed

kemzeb mentioned this pull request Mar 7, 2026

Doesn't work with extras_require and tests_require #107

Closed

gaborbernat deleted the fix/extras-support branch March 9, 2026 14:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

✨ feat(extras): add --extras flag for optional deps#549

✨ feat(extras): add --extras flag for optional deps#549
gaborbernat merged 4 commits intotox-dev:mainfrom
gaborbernat:fix/extras-support

gaborbernat commented Mar 3, 2026 •

edited

Loading

Uh oh!

kemzeb commented Mar 3, 2026

Uh oh!

kemzeb commented Mar 4, 2026

Uh oh!

kemzeb left a comment

Uh oh!

gaborbernat commented Mar 5, 2026

Uh oh!

kemzeb commented Mar 5, 2026 •

edited

Loading

Uh oh!

kemzeb commented Mar 5, 2026

Uh oh!

kemzeb commented Mar 5, 2026 •

edited

Loading

Uh oh!

kemzeb commented Mar 5, 2026

Uh oh!

gaborbernat commented Mar 5, 2026

Uh oh!

kemzeb left a comment

Uh oh!

Uh oh!

kemzeb left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

gaborbernat commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kemzeb commented Mar 3, 2026

Uh oh!

kemzeb commented Mar 4, 2026

Uh oh!

kemzeb left a comment

Choose a reason for hiding this comment

Uh oh!

gaborbernat commented Mar 5, 2026

Uh oh!

kemzeb commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kemzeb commented Mar 5, 2026

Uh oh!

kemzeb commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kemzeb commented Mar 5, 2026

Uh oh!

gaborbernat commented Mar 5, 2026

Uh oh!

kemzeb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kemzeb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gaborbernat commented Mar 3, 2026 •

edited

Loading

kemzeb commented Mar 5, 2026 •

edited

Loading

kemzeb commented Mar 5, 2026 •

edited

Loading