python312Packages.triton*: 3.0.0 -> 3.1.0 by GaetanLepage · Pull Request #349159 · NixOS/nixpkgs

GaetanLepage · 2024-10-16T23:02:04Z

Things done

Diff: triton-lang/triton@91f24d8...cf34004

Changelog: one day maybe

cc @SomeoneSerge

Add a 👍 reaction to pull requests you find important.

Diff: triton-lang/triton@91f24d8...cf34004 Changelog: NONE LOL

DerDennisOP · 2024-10-16T23:22:43Z

lgtm

GaetanLepage · 2024-10-17T07:13:03Z

It builds lol. First try X)

SomeoneSerge · 2024-10-17T07:39:55Z

❯ nom build -I nixpkgs=https://github.com/GaetanLepage/nixpkgs/archive/openai-triton.tar.gz -f "<nixpkgs>" python3Packages.triton.gpuCheck
...
triton-pytest> FAILED language/test_line_info.py::test_line_info[dot_combine] - subprocess.CalledProcessError: Command '['/nix/store/97ba8v5ffpm7r7z8grpaqj...
triton-pytest> FAILED language/test_subprocess.py::test_print[device_print_large-int32] - assert False
triton-pytest> FAILED operators/test_flash_attention.py::test_op[True-True-dtype0-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
triton-pytest> FAILED operators/test_flash_attention.py::test_op[True-True-dtype1-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
triton-pytest> FAILED operators/test_flash_attention.py::test_op[True-False-dtype0-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
triton-pytest> FAILED operators/test_flash_attention.py::test_op[True-False-dtype1-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
triton-pytest> FAILED operators/test_flash_attention.py::test_op[False-True-dtype0-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
triton-pytest> FAILED operators/test_flash_attention.py::test_op[False-True-dtype1-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
triton-pytest> FAILED operators/test_flash_attention.py::test_op[False-False-dtype0-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
triton-pytest> FAILED operators/test_flash_attention.py::test_op[False-False-dtype1-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
triton-pytest> FAILED tools/test_aot.py::test_compile_link_matmul_no_specialization - subprocess.CalledProcessError: Command '['./test', '/build/tmpd7xmo9c2/a.cs...
triton-pytest> FAILED tools/test_aot.py::test_compile_link_matmul - subprocess.CalledProcessError: Command '['./test', '/build/tmph4nb393y/a.cs...
triton-pytest> FAILED tools/test_aot.py::test_launcher_has_no_available_kernel - AssertionError: assert -11 == -6
triton-pytest> FAILED tools/test_aot.py::test_compile_link_autotune_matmul - subprocess.CalledProcessError: Command '['./test_0', '/build/tmp0g352yw6/a....
triton-pytest> === 23 failed, 9134 passed, 2196 skipped, 181 warnings in 1224.10s (0:20:24) ===
error: builder for '/nix/store/gvcldv6vz16xkr2cpjrwp3a76isg00ph-triton-pytest-3.1.0.drv' failed with exit code 1;
       last 10 log lines:
       > FAILED operators/test_flash_attention.py::test_op[True-False-dtype1-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
       > FAILED operators/test_flash_attention.py::test_op[False-True-dtype0-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
       > FAILED operators/test_flash_attention.py::test_op[False-True-dtype1-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
       > FAILED operators/test_flash_attention.py::test_op[False-False-dtype0-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
       > FAILED operators/test_flash_attention.py::test_op[False-False-dtype1-2-4-512-128] - triton.runtime.errors.OutOfResources: out of resource: shared memory, Requi...
       > FAILED tools/test_aot.py::test_compile_link_matmul_no_specialization - subprocess.CalledProcessError: Command '['./test', '/build/tmpd7xmo9c2/a.cs...
       > FAILED tools/test_aot.py::test_compile_link_matmul - subprocess.CalledProcessError: Command '['./test', '/build/tmph4nb393y/a.cs...
       > FAILED tools/test_aot.py::test_launcher_has_no_available_kernel - AssertionError: assert -11 == -6
       > FAILED tools/test_aot.py::test_compile_link_autotune_matmul - subprocess.CalledProcessError: Command '['./test_0', '/build/tmp0g352yw6/a....
       > === 23 failed, 9134 passed, 2196 skipped, 181 warnings in 1224.10s (0:20:24) ===
       For full logs, run 'nix log /nix/store/gvcldv6vz16xkr2cpjrwp3a76isg00ph-triton-pytest-3.1.0.drv'.
┏━ 1 Errors: 
┃ error: builder for '/nix/store/gvcldv6vz16xkr2cpjrwp3a76isg00ph-triton-pytest-3.1.0.drv' failed with exit code 1;
┣━ Dependency Graph showing 1 of 2 roots:
┃          ┌─ ✔ triton-llvm-19.1.0-rc1 ⏱ 55m57s
┃       ┌─ ✔ python3.12-triton-3.1.0 ⏱ 5m0s
┃       ├─ ✔ magma-2.7.2 ⏱ 1h10m27s
┃    ┌─ ✔ python3.12-torch-2.4.1 ⏱ 1h30m28s
┃ ┌─ ✔ python3-3.12.6-env ⏱ 2s
┃ ├─ ⏸ source
┃ ⚠ triton-pytest-3.1.0 failed with exit code 1 after ⏱ 20m27s in checkPhase

No regression compares to the previous PR

GaetanLepage · 2024-10-17T07:40:56Z

No regression compares to the previous PR

Is it sarcastic or was this test really already broken ?

SomeoneSerge · 2024-10-17T07:48:13Z

Is it sarcastic or was this test really already broken ?

An understandable confusion! Yes it was broken, which I just took as a given since we've never run the pytest suite before ("pytorch is our test"):

... #328247 (comment)
I currently observe about 20 tests failing and spitting out junk: https://gist.github.com/SomeoneSerge/f9bd9ececc3a16438bd087edadf0fef4

The "OutOfResource" parts can be probably put in disabledTests, but iirc a few tests spat out outright gibberish

DerDennisOP · 2024-10-17T07:53:50Z

Is it sarcastic or was this test really already broken ?

An understandable confusion! Yes it was broken, which I just took as a given since we've never run the pytest suite before ("pytorch is our test"):

... #328247 (comment)
I currently observe about 20 tests failing and spitting out junk: https://gist.github.com/SomeoneSerge/f9bd9ececc3a16438bd087edadf0fef4

The "OutOfResource" parts can be probably put in disabledTests, but iirc a few tests spat out outright gibberish

The "OutOfResource" Tests work for me, I guess you need more resources. If I build it on 64C/128T CPU with a GPU piped into the nix builder it works.

SomeoneSerge · 2024-10-17T08:00:19Z

The "OutOfResource" Tests work for me, I guess you need more resources. If I build it on 64C/128T CPU with a GPU piped into the nix builder it works.

🥲 24G VRAM too little, rtx 3090 too old

May I ask what your GPU is?

DerDennisOP · 2024-10-17T08:01:37Z

The "OutOfResource" Tests work for me, I guess you need more resources. If I build it on 64C/128T CPU with a GPU piped into the nix builder it works.

🥲 24G VRAM too little, rtx 3090 too old

May I ask what your GPU is?

Specs:
CPU: AMD Epyc 7702P (64C/128T)
RAM: 512 GB (ECC)
STORAGE: 38 TB
GPU: 1x H100
NETWORKING: 600 Mbit/s

GaetanLepage · 2024-10-17T08:29:46Z

Specs:
CPU: AMD Epyc 7702P (64C/128T)
RAM: 512 GB (ECC)
STORAGE: 38 TB
GPU: 1x H100
NETWORKING: 600 Mbit/s

Not bad X)

SomeoneSerge · 2024-10-17T08:43:06Z

GPU: 1x H100

Maybe we should consider marking tests with a grading of "system features", e.g. based on vram size (e.g. "cuda-80gb"). We're not doing that yet because there would be many orthogonal dimensions, like cuda capabilities. An alternative is to hard-code GPU names for derivations where it matters

DerDennisOP · 2024-10-17T08:48:46Z

GPU: 1x H100

Maybe we should consider marking tests with a grading of "system features", e.g. based on vram size (e.g. "cuda-80gb"). We're not doing that yet because there would be many orthogonal dimensions, like cuda capabilities. An alternative is to hard-code GPU names for derivations where it matters

in my nix-ai project, I'm using hard-coded GPU names. Its not optimal but the best solution I could find.

python312Packages.triton: 3.0.0 -> 3.1.0

611373f

Diff: triton-lang/triton@91f24d8...cf34004 Changelog: NONE LOL

github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Oct 16, 2024

GaetanLepage mentioned this pull request Oct 16, 2024

python3Packages.triton*: 2.1.0 -> 3.0.0 #328247

Merged

13 tasks

SomeoneSerge mentioned this pull request Oct 16, 2024

Update request: python311Packages.triton 2.1.0 → 3.1.0 #343218

Closed

1 task

ofborg bot requested review from DerDennisOP, Madouura and SomeoneSerge October 17, 2024 01:29

ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. labels Oct 17, 2024

DerDennisOP approved these changes Oct 17, 2024

View reviewed changes

SomeoneSerge approved these changes Oct 17, 2024

View reviewed changes

GaetanLepage merged commit baaa9d5 into NixOS:master Oct 17, 2024

GaetanLepage deleted the openai-triton branch October 17, 2024 21:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

python312Packages.triton*: 3.0.0 -> 3.1.0#349159

python312Packages.triton*: 3.0.0 -> 3.1.0#349159
GaetanLepage merged 1 commit intoNixOS:masterfrom
GaetanLepage:openai-triton

GaetanLepage commented Oct 16, 2024 •

edited

Loading

Uh oh!

DerDennisOP commented Oct 16, 2024

Uh oh!

GaetanLepage commented Oct 17, 2024

Uh oh!

SomeoneSerge commented Oct 17, 2024

Uh oh!

GaetanLepage commented Oct 17, 2024

Uh oh!

SomeoneSerge commented Oct 17, 2024

Uh oh!

DerDennisOP commented Oct 17, 2024 •

edited

Loading

Uh oh!

SomeoneSerge commented Oct 17, 2024

Uh oh!

DerDennisOP commented Oct 17, 2024

Uh oh!

GaetanLepage commented Oct 17, 2024

Uh oh!

SomeoneSerge commented Oct 17, 2024

Uh oh!

DerDennisOP commented Oct 17, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

GaetanLepage commented Oct 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Things done

Uh oh!

DerDennisOP commented Oct 16, 2024

Uh oh!

GaetanLepage commented Oct 17, 2024

Uh oh!

SomeoneSerge commented Oct 17, 2024

Uh oh!

GaetanLepage commented Oct 17, 2024

Uh oh!

SomeoneSerge commented Oct 17, 2024

Uh oh!

DerDennisOP commented Oct 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SomeoneSerge commented Oct 17, 2024

Uh oh!

DerDennisOP commented Oct 17, 2024

Uh oh!

GaetanLepage commented Oct 17, 2024

Uh oh!

SomeoneSerge commented Oct 17, 2024

Uh oh!

DerDennisOP commented Oct 17, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

GaetanLepage commented Oct 16, 2024 •

edited

Loading

DerDennisOP commented Oct 17, 2024 •

edited

Loading