Skip to content

[main] Don't override numpy versions coming from requirements.txt#82

Merged
jithunnair-amd merged 1 commit intomainfrom
retain_numpy_versions
Jun 11, 2025
Merged

[main] Don't override numpy versions coming from requirements.txt#82
jithunnair-amd merged 1 commit intomainfrom
retain_numpy_versions

Conversation

@jithunnair-amd
Copy link
Collaborator

Relates to ROCm/pytorch#2253

This change is required to avoid overriding the numpy2.x version with numpy1.x.

@jithunnair-amd jithunnair-amd changed the title Don't override numpy versions coming from requirements.txt [main] Don't override numpy versions coming from requirements.txt Jun 10, 2025
jithunnair-amd added a commit to ROCm/pytorch that referenced this pull request Jun 10, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch_internal-manylinux-wheels/98/

Using
`registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16180_ubuntu24.04_py3.12_pytorch_lw_rocm7.0_IT_upgrade_numpy_452f3df6`:
```
root@d92befdbb2a6:/# pip list | egrep "numpy|pandas"
numpy                   2.1.2
pandas                  2.2.3
root@d92befdbb2a6:/# python3
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import torch
>>> import numpy
>>> exit()
root@d92befdbb2a6:/data/pytorch-micro-benchmarking# HIP_VISIBLE_DEVICES=1 python3 micro_benchmarking_pytorch.py --network resnet50
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.11369450092315674
Throughput [img/sec] : 562.9120096428937
```

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
jithunnair-amd added a commit to ROCm/pytorch that referenced this pull request Jun 10, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch2.7-manylinux-wheels/80/

Ran the microbenchmark, import pandas, torch, numpy in 

**`registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16180_ubuntu24.04_py3.12_pytorch_lw_upgrade_numpy_versions_546fe04d`**
```
root@ubb4-rack-22:/var/lib/jenkins/pytorch-micro-benchmarking# pip list | egrep "numpy|pandas"
numpy                   2.1.2
pandas                  2.2.3
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import numpy
>>> import torch
root@ubb4-rack-22:/var/lib/jenkins/pytorch-micro-benchmarking# python3 micro_benchmarking_pytorch.py --network resnet50
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.10239996910095214
Throughput [img/sec] : 625.0001885928783
```

---------

Co-authored-by: Jithun Nair <jithun.nair@amd.com>
@jithunnair-amd jithunnair-amd marked this pull request as ready for review June 11, 2025 17:18
@jithunnair-amd jithunnair-amd merged commit 96cd1e2 into main Jun 11, 2025
1 check failed
@jithunnair-amd jithunnair-amd deleted the retain_numpy_versions branch June 11, 2025 17:18
jithunnair-amd added a commit that referenced this pull request Jun 11, 2025
…83)

Copy of #82 for rocm6.4 branch

This change is required to avoid overriding the numpy2.x version with
numpy1.x.
jithunnair-amd added a commit to ROCm/pytorch that referenced this pull request Jun 11, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch2.6-manylinux-wheels/120/

Ran import torch, import numpy, import pandas and the microbenchmark in
image called

_**registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16180_ubuntu24.04_py3.12_pytorch_lw_release2.6_upgrade_numpy_57e5941d**_

```
root@ubb4-rack-22:/var/lib/jenkins/pytorch-micro-benchmarking# pip list | grep pandas
pandas                  2.2.3
root@ubb4-rack-22:/var/lib/jenkins/pytorch-micro-benchmarking# pip list | grep numpy
numpy                   2.1.2

Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import numpy
>>> import torch
>>> exit()

root@ubb4-rack-22:/var/lib/jenkins# cd pytorch-micro-benchmarking/
root@ubb4-rack-22:/var/lib/jenkins/pytorch-micro-benchmarking# python3 micro_benchmarking_pytorch.py --network resnet50
/opt/venv/lib/python3.12/site-packages/redis/connection.py:77: UserWarning: redis-py works best with hiredis. Please consider installing
  warnings.warn(msg)
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.1023630142211914
Throughput [img/sec] : 625.2258248443663
root@ubb4-rack-22:/var/lib/jenkins/pytorch-micro-benchmarking#

```

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
jithunnair-amd added a commit to ROCm/pytorch that referenced this pull request Jun 12, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch2.5-manylinux-wheels/394/

From
`registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16153_ubuntu22.04_py3.10_pytorch_lw_release2.5_upgrade_numpy_2e3346a5`:

```
root@87bf0db55e11:/# pip list | egrep "pandas|numpy"
numpy               2.2.6
root@87bf0db55e11:/# python
Python 3.10.12 (main, Feb  4 2025, 14:57:36) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import numpy
root@87bf0db55e11:/data/pytorch-micro-benchmarking# HIP_VISIBLE_DEVICES=1 python3 micro_benchmarking_pytorch.py --network resnet50
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.11416447162628174
Throughput [img/sec] : 560.5947199537216
root@87bf0db55e11:/data/pytorch-micro-benchmarking# 
```

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
jithunnair-amd added a commit to ROCm/pytorch that referenced this pull request Jul 7, 2025
Relates to: ROCm/builder#82

Moves torchvision commit to unpin numpy dependency:
https://github.com/ROCm/vision/commits/bbf1b41e2b24a46f69f3ceba7576d67d56459647

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch2.4-manylinux-wheels/272/
http://rocm-ci.amd.com/job/mainline-pytorch2.4-manylinux-wheels/282/
From:
**registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16281_ubuntu22.04_py3.11_pytorch_lw_release2.4_upgrade_numpy_134c11da**

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
pragupta pushed a commit to ROCm/pytorch that referenced this pull request Oct 29, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch_internal-manylinux-wheels/98/

Using
`registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16180_ubuntu24.04_py3.12_pytorch_lw_rocm7.0_IT_upgrade_numpy_452f3df6`:
```
root@d92befdbb2a6:/# pip list | egrep "numpy|pandas"
numpy                   2.1.2
pandas                  2.2.3
root@d92befdbb2a6:/# python3
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import torch
>>> import numpy
>>> exit()
root@d92befdbb2a6:/data/pytorch-micro-benchmarking# HIP_VISIBLE_DEVICES=1 python3 micro_benchmarking_pytorch.py --network resnet50
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.11369450092315674
Throughput [img/sec] : 562.9120096428937
```

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
(cherry picked from commit cf32479)
pragupta pushed a commit to ROCm/pytorch that referenced this pull request Oct 29, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch_internal-manylinux-wheels/98/

Using
`registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16180_ubuntu24.04_py3.12_pytorch_lw_rocm7.0_IT_upgrade_numpy_452f3df6`:
```
root@d92befdbb2a6:/# pip list | egrep "numpy|pandas"
numpy                   2.1.2
pandas                  2.2.3
root@d92befdbb2a6:/# python3
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import torch
>>> import numpy
>>> exit()
root@d92befdbb2a6:/data/pytorch-micro-benchmarking# HIP_VISIBLE_DEVICES=1 python3 micro_benchmarking_pytorch.py --network resnet50
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.11369450092315674
Throughput [img/sec] : 562.9120096428937
```

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
(cherry picked from commit cf32479)
pragupta pushed a commit to ROCm/pytorch that referenced this pull request Oct 29, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch_internal-manylinux-wheels/98/

Using
`registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16180_ubuntu24.04_py3.12_pytorch_lw_rocm7.0_IT_upgrade_numpy_452f3df6`:
```
root@d92befdbb2a6:/# pip list | egrep "numpy|pandas"
numpy                   2.1.2
pandas                  2.2.3
root@d92befdbb2a6:/# python3
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import torch
>>> import numpy
>>> exit()
root@d92befdbb2a6:/data/pytorch-micro-benchmarking# HIP_VISIBLE_DEVICES=1 python3 micro_benchmarking_pytorch.py --network resnet50
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.11369450092315674
Throughput [img/sec] : 562.9120096428937
```

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
(cherry picked from commit cf32479)
pragupta pushed a commit to ROCm/pytorch that referenced this pull request Oct 31, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch_internal-manylinux-wheels/98/

Using
`registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16180_ubuntu24.04_py3.12_pytorch_lw_rocm7.0_IT_upgrade_numpy_452f3df6`:
```
root@d92befdbb2a6:/# pip list | egrep "numpy|pandas"
numpy                   2.1.2
pandas                  2.2.3
root@d92befdbb2a6:/# python3
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import torch
>>> import numpy
>>> exit()
root@d92befdbb2a6:/data/pytorch-micro-benchmarking# HIP_VISIBLE_DEVICES=1 python3 micro_benchmarking_pytorch.py --network resnet50
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.11369450092315674
Throughput [img/sec] : 562.9120096428937
```

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
(cherry picked from commit cf32479)
jithunnair-amd added a commit to ROCm/pytorch that referenced this pull request Nov 3, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch_internal-manylinux-wheels/98/

Using
`registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16180_ubuntu24.04_py3.12_pytorch_lw_rocm7.0_IT_upgrade_numpy_452f3df6`:
```
root@d92befdbb2a6:/# pip list | egrep "numpy|pandas"
numpy                   2.1.2
pandas                  2.2.3
root@d92befdbb2a6:/# python3
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import torch
>>> import numpy
>>> exit()
root@d92befdbb2a6:/data/pytorch-micro-benchmarking# HIP_VISIBLE_DEVICES=1 python3 micro_benchmarking_pytorch.py --network resnet50
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.11369450092315674
Throughput [img/sec] : 562.9120096428937
```

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
(cherry picked from commit cf32479)
pragupta pushed a commit to ROCm/pytorch that referenced this pull request Nov 25, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch_internal-manylinux-wheels/98/

Using
`registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16180_ubuntu24.04_py3.12_pytorch_lw_rocm7.0_IT_upgrade_numpy_452f3df6`:
```
root@d92befdbb2a6:/# pip list | egrep "numpy|pandas"
numpy                   2.1.2
pandas                  2.2.3
root@d92befdbb2a6:/# python3
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import torch
>>> import numpy
>>> exit()
root@d92befdbb2a6:/data/pytorch-micro-benchmarking# HIP_VISIBLE_DEVICES=1 python3 micro_benchmarking_pytorch.py --network resnet50
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.11369450092315674
Throughput [img/sec] : 562.9120096428937
```

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
(cherry picked from commit cf32479)
(cherry picked from commit b296de5)
pragupta pushed a commit to ROCm/pytorch that referenced this pull request Nov 25, 2025
Relates to: ROCm/builder#82

Validation:
http://rocm-ci.amd.com/job/mainline-pytorch_internal-manylinux-wheels/98/

Using
`registry-sc-harbor.amd.com/framework/compute-rocm-dkms-no-npi-hipclang:16180_ubuntu24.04_py3.12_pytorch_lw_rocm7.0_IT_upgrade_numpy_452f3df6`:
```
root@d92befdbb2a6:/# pip list | egrep "numpy|pandas"
numpy                   2.1.2
pandas                  2.2.3
root@d92befdbb2a6:/# python3
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> import torch
>>> import numpy
>>> exit()
root@d92befdbb2a6:/data/pytorch-micro-benchmarking# HIP_VISIBLE_DEVICES=1 python3 micro_benchmarking_pytorch.py --network resnet50
INFO: running forward and backward for warmup.
INFO: running the benchmark..
OK: finished running benchmark..
--------------------SUMMARY--------------------------
Microbenchmark for network : resnet50
Num devices: 1
Dtype: FP32
Mini batch size [img] : 64
Time per mini-batch : 0.11369450092315674
Throughput [img/sec] : 562.9120096428937
```

---------

Co-authored-by: Jithun Nair <37884920+jithunnair-amd@users.noreply.github.com>
(cherry picked from commit cf32479)
(cherry picked from commit b296de5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant