[PD] Bump Nixl version by NickLucche · Pull Request #39797 · vllm-project/vllm

NickLucche · 2026-04-14T13:24:46Z

Address #39521.

Mind that we're pinning cu13 on vllm right now.

gemini-code-assist

Code Review

This pull request updates the version constraint for the nixl[cu13] package in requirements/kv_connectors.txt from < 0.10.0 to <= 1.0.0 and adds a documentation note regarding CUDA 12 compatibility. I have no feedback to provide.

markmc · 2026-04-14T14:35:59Z

@@ -1,3 +1,3 @@
 lmcache >= 0.3.9
-nixl[cu13] >= 0.7.1, < 0.10.0 # Required for disaggregated prefill
+nixl[cu13] >= 0.7.1, <= 1.0.0 # Required for disaggregated prefill. Use nixl[cu12] for CUDA 12.


Why the upper bound? If 1.0.1 comes out tomorrow, you don't want to adopt it without another PR?

I see #35495 but the reason back then feels very temporary

Only 11 out of the 56 deps in requirements/common.txt have an upper bound ... so I think we'd only apply an upper bound for nixl if we had some specific reason to be wary of new releases

@markmc tbh I don't have a strong opinion here, we're quite happy with nixl perf. Let me re-purposes PR

actually let me hold on this, I don't get why nixl[cu13] is pulling in cu12

uv pip install nixl[cu13]==1.0 Using Python 3.12.9 environment at: /home/NickLucche/llmd/.venv Resolved 30 packages in 48ms Installed 3 packages in 217ms + nixl==1.0.0 + nixl-cu12==1.0.1 <== why does this need to be pulled? + nixl-cu13==1.0.0 cat ../.venv/lib/python3.12/site-packages/nixl-1.0.0.dist-info/METADATA | grep Requires-Dist Requires-Dist: nixl-cu12>=1.0.0 <== Requires-Dist: nixl-cu12==1.0.0; extra == "cu12" Requires-Dist: nixl-cu13==1.0.0; extra == "cu13"

Well I think that is the fundamental packaging issue in nixl wheel files... They should have packed the wheel files with +cu12 and +cu13 local version tags like pytorch, and the Requires-Dist should be simply itself (can be removed entirely if cuda runtime were in local version).

Signed-off-by: NickLucche <nlucches@redhat.com>

mergify · 2026-04-15T07:53:44Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @NickLucche.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

NickLucche · 2026-04-15T13:41:54Z

I want to figure out 2 things before merging here.

If there's a better way to install nixl so that we can avoid pulling in both -cu12 and -cu13 for systems that only need 13. This will require changes to nixl package so we'll eventually pick it up (if a better way to do it is found)

 uv pip install nixl[cu13]==1.0
Using Python 3.12.9 environment at: /home/NickLucche/llmd/.venv
Resolved 30 packages in 48ms
Installed 3 packages in 217ms
 + nixl==1.0.0
 + nixl-cu12==1.0.1 <== why does this need to be pulled?
 + nixl-cu13==1.0.0

cat ../.venv/lib/python3.12/site-packages/nixl-1.0.0.dist-info/METADATA | grep Requires-Dist

Requires-Dist: nixl-cu12>=1.0.0 <==
Requires-Dist: nixl-cu12==1.0.0; extra == "cu12"
Requires-Dist: nixl-cu13==1.0.0; extra == "cu13"

part2: I'd like that nixl-cu12 dependency to be an exact == not >= or we're always going to have a moving target

there's some issue with new nixl 1.0.0 version [Tracking] NIXL >= 1.0.0 Support for NIXL KV Connector #39521 (comment) that needs to be addressed before we consider bumping

cjackal · 2026-04-15T14:36:44Z

If we do not pursue a mental health, we can just leave the nixl_cu1x variant in the kv_connectors.txt requirement file and post-install the nixl metapackage with --no-deps option like:

In `kv_connectors.txt`

 ...
-nixl[cu13] >= 0.7.1, <= 1.0.0
-nixl-cu12 >= 0.7.1, <= 1.0.0
 nixl-cu13 >= 0.7.1, <= 1.0.0

In Dockerfile

             apt-get update -y && \
             apt-get install -y --no-install-recommends --allow-change-held-packages ${BUILD_PKGS} && \
             uv pip install --system -r /tmp/kv_connectors.txt --no-build-isolation && \
+            uv pip install --system 'nixl>= 0.7.1,<= 1.0.0' --no-deps && \
             apt-get purge -y ${BUILD_PKGS} && \

As @NickLucche showed in the comment above, nixl has some packaging issues. vllm does not put nixl into the package dependency, so nixl packaging issue only matters in docker build stage. While a dirty hotfix, it can save ~200Mi of grand total container image size by removing duplicated nixl-packaged shared objects.

NickLucche · 2026-04-15T15:55:49Z

@cjackal thanks for taking a look. I think that solution works for Dockerfile, but not for local installation with uv pip install requirements/kv_connector.txt (as the api isn't pulled) which might leave some people confused.

Should we just patch the requirements file with sed in the Dockerfile as I believe you also initially suggested?

cjackal · 2026-04-16T06:15:38Z

@cjackal thanks for taking a look. I think that solution works for Dockerfile, but not for local installation with uv pip install requirements/kv_connector.txt (as the api isn't pulled) which might leave some people confused.

Should we just patch the requirements file with sed in the Dockerfile as I believe you also initially suggested?

I can't come up with a better solution yet; an excuse for the sed yoga is that there already exists several if branches wrt cuda runtime (e.g. https://github.com/vllm-project/vllm/blob/main/docker/Dockerfile#L735-L752 ) so why not another?

Or we may vender vllm fork of nixl, though I think it doesn't sound tempting in terms of ROI.

alec-flowers · 2026-04-23T05:19:08Z

I want to figure out 2 things before merging here.

If there's a better way to install nixl so that we can avoid pulling in both -cu12 and -cu13 for systems that only need 13. This will require changes to nixl package so we'll eventually pick it up (if a better way to do it is found)
 uv pip install nixl[cu13]==1.0
Using Python 3.12.9 environment at: /home/NickLucche/llmd/.venv
Resolved 30 packages in 48ms
Installed 3 packages in 217ms
 + nixl==1.0.0
 + nixl-cu12==1.0.1 <== why does this need to be pulled?
 + nixl-cu13==1.0.0

cat ../.venv/lib/python3.12/site-packages/nixl-1.0.0.dist-info/METADATA | grep Requires-Dist

Requires-Dist: nixl-cu12>=1.0.0 <==
Requires-Dist: nixl-cu12==1.0.0; extra == "cu12"
Requires-Dist: nixl-cu13==1.0.0; extra == "cu13"
part2: I'd like that nixl-cu12 dependency to be an exact == not >= or we're always going to have a moving target

there's some issue with new nixl 1.0.0 version [Tracking] NIXL >= 1.0.0 Support for NIXL KV Connector #39521 (comment) that needs to be addressed before we consider bumping

We can work with the NIXL team here. I have highlighted both these issues to them. We will do our best to resolve quickly.

ovidiusm · 2026-05-12T13:55:39Z

NIXL 1.1.0 has been released, with this you can drop the explicit cu12/cu13 dependencies and use directly nixl.

mergify · 2026-05-23T08:40:59Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @NickLucche.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

NickLucche · 2026-05-29T12:28:10Z

we landed 1.1.0

NickLucche changed the title ~~[Misc] Bump Nixl version~~ [PD] Bump Nixl version Apr 14, 2026

NickLucche added the ready ONLY add when PR is ready to merge/full CI is needed label Apr 14, 2026

mergify Bot added ci/build kv-connector labels Apr 14, 2026

gemini-code-assist Bot reviewed Apr 14, 2026

View reviewed changes

markmc reviewed Apr 14, 2026

View reviewed changes

ZhanqiuHu mentioned this pull request Apr 15, 2026

[CI][NIXL] Fix PD CI breakage: pin nixl-cu{12,13} versions #39851

Merged

init

96eebca

Signed-off-by: NickLucche <nlucches@redhat.com>

mergify Bot added the needs-rebase label Apr 15, 2026

NickLucche force-pushed the nixl-1.0 branch from eb1d150 to 96eebca Compare April 15, 2026 08:07

mergify Bot removed the needs-rebase label Apr 15, 2026

NickLucche mentioned this pull request Apr 15, 2026

[Nixl] Bump Nixl version to 0.10.1 #39922

Merged

This was referenced May 12, 2026

[PD] Bump NIXL connector dependency to 1.x #42364

Merged

[Tracking] NIXL >= 1.0.0 Support for NIXL KV Connector #39521

Closed

mergify Bot added the needs-rebase label May 23, 2026

NickLucche closed this May 29, 2026

Uh oh!

Conversation

NickLucche commented Apr 14, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

markmc Apr 14, 2026

Choose a reason for hiding this comment

Uh oh!

NickLucche Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

NickLucche Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

NickLucche Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

NickLucche Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

cjackal Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mergify Bot commented Apr 15, 2026

Uh oh!

NickLucche commented Apr 15, 2026

Uh oh!

cjackal commented Apr 15, 2026

In kv_connectors.txt

In Dockerfile

Uh oh!

NickLucche commented Apr 15, 2026

Uh oh!

cjackal commented Apr 16, 2026

Uh oh!

alec-flowers commented Apr 23, 2026

Uh oh!

ovidiusm commented May 12, 2026

Uh oh!

mergify Bot commented May 23, 2026

Uh oh!

NickLucche commented May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

cjackal Apr 15, 2026 •

edited

Loading

In `kv_connectors.txt`