Skip to content

[WIP] Add cupy.complex32#4454

Closed
leofang wants to merge 9 commits intocupy:mainfrom
leofang:complex32
Closed

[WIP] Add cupy.complex32#4454
leofang wants to merge 9 commits intocupy:mainfrom
leofang:complex32

Conversation

@leofang
Copy link
Member

@leofang leofang commented Dec 17, 2020 •
edited
Loading

First step toward #3370. Related to #4407. DECISION NEEDED BEFORE ANY CODE REVIEW

This is a working prototype to demonstrate that we can define a new Python type (cupy.complex32) and the associated NumPy dtype (np.dtype(cupy.complex32)) so that this type object can be used to hint that the input array consists of half-precision complex numbers, to be used internally wherever its siblings cupy.complex64 and cupy.complex128 are applicable. The subsequent GPU operations is out of scope in this PR and will be addressed when this is merged.

Currently it works for:

  • Type identification
  • Correct inheritance (ex: cupy.complex32 inherits from numpy.complexfloating, same as other complex types)
  • Correct itemsize (4 bytes) for the purpose of array allocation and indexing
  • Correct type kind ('c' for complex)
  • Meaningful type character (uppercase 'E' for 2 float16 as complex, as oppose to lowercase 'e' for 1 float16 as real)
  • 4-byte alignment
  • Very limited functionalities (see below)

Critical decision to make before moving on: I incline to consider it acceptable that it has very limited functionality when used as a dtype. In particular, I do not wish to support all kinds of CPU ufuncs or array funcs (as defined in this section) when manipulated as a NumPy complex32 array, because it would then be more suitable for a PR to NumPy, not to CuPy, and as far as I know NumPy folks do not want to add more new built-in types (numpy/numpy#14753), nor do I have the bandwidth to do so. I think having this type object defined is already enough for GPU needs. If it's acceptable then I will work on polishing this.

@leofang leofang mentioned this pull request Dec 17, 2020
Open
@kmaehashi kmaehashi added cat:enhancement Improvements to existing features st:needs-discussion labels Dec 18, 2020
@leofang leofang mentioned this pull request Dec 18, 2020
Closed
@leofang leofang mentioned this pull request Jan 3, 2021
Open
@leofang
Copy link
Member Author

leofang commented Jan 4, 2021

The discussion is ongoing in numpy/numpy#14753!

@emcastillo
Copy link
Member

emcastillo commented Jan 12, 2021 •
edited
Loading

We have been talking offline, maybe it's better to postpone this until NumPy implements complex32.
We don't want to do it ourselves first as it may lead to some incompatibilities 😁

@leofang
Copy link
Member Author

leofang commented Jan 18, 2021 •
edited
Loading

So let me quickly summarize the discussion in numpy/numpy#14753 so far. It's doable and the Numpy core devs generally support it. We agree that this dtype will be called numpy.complex32 with the single character representation 'E', as chosen in #4407 and this PR. NumPy's new dtype system will be finalized hopefully in a few months, and the questions left are how to add complex32 to their type hierarchy, and whether or not to make it as "odd" as float16 (see the discussion therein to know what they meant by "odd"; as far as CuPy is concerned, it's just another dtype).

@leofang
Copy link
Member Author

leofang commented Jan 18, 2021 •
edited
Loading

I'd like to note, though, that if CuPy would just alias numpy.complex32, it means this feature won't be available in CuPy for at least 1 year or more, until the new NumPy version (the earliest one in which complex32 is added) becomes our lowest supported NumPy...

@emcastillo
Copy link
Member

emcastillo commented Feb 5, 2021

I'd like to note, though, that if CuPy would just alias numpy.complex32, it means this feature won't be available in CuPy for at least 1 year or more, until the new NumPy version (the earliest one in which complex32 is added) becomes our lowest supported NumPy...

We discussed this, we can implement it once it is released in NumPy master branch, as we always support the latest released NumPy version during development. However, we will need to disable it for older NumPy versions.

@leofang
Copy link
Member Author

leofang commented Feb 25, 2021

Let me close this to clean up my PR queue 😁 I'll leave the branch alive for reference. This need is already tracked in #3370.

@leofang leofang closed this Feb 25, 2021
@asi1024 asi1024 added this to the Closed PRs milestone Feb 25, 2021
@kmaehashi kmaehashi mentioned this pull request May 11, 2023
Closed
21 tasks
@leofang
Copy link
Member Author

leofang commented May 11, 2023 •
edited
Loading

Hmmm I can't reopen because the target branch (master) is gone. @kmaehashi or @emcastillo, can you try changing the target branch to main? It seems don't have access to do that. If it doesn't work, I'll open a new PR. Thanks!

@kmaehashi kmaehashi reopened this May 11, 2023
@kmaehashi kmaehashi changed the base branch from master to main May 11, 2023 09:31
@kmaehashi kmaehashi removed this from the Closed PRs milestone May 17, 2023
@leofang
Copy link
Member Author

leofang commented Jul 11, 2023

Sorry for delay. @seberg has kindly taken a look at this PR, and agreed I was on the right track. What I'll do is to clean up/polish the code, hook it up with the build system, and add tests. I'll ping Sebastian and the team once it's ready.

@kmaehashi kmaehashi added this to the v13.0.0rc1 milestone Aug 10, 2023
@kmaehashi kmaehashi removed this from the v13.0.0rc1 milestone Nov 27, 2023
@kmaehashi
Copy link
Member

kmaehashi commented Nov 27, 2023

Milestone removed.

@leofang
Copy link
Member Author

leofang commented Dec 1, 2023

Yeah unfortunately I can't make it for v13.

@emcastillo emcastillo assigned kmaehashi and unassigned emcastillo Feb 6, 2024
@ashvardanian ashvardanian mentioned this pull request Mar 13, 2024
Merged
@leofang leofang added this to the v14 milestone Mar 31, 2024
@asi1024 asi1024 mentioned this pull request Apr 3, 2024
Closed
12 tasks
@leofang leofang mentioned this pull request Jun 13, 2024
Closed
@jakirkham
Copy link
Member

jakirkham commented Aug 7, 2024

It looks like this is in the v14 milestone

Though most of the CuPy 14 items are under the v14.0.0a1 milestone

Should we move this under the v14.0.0a1 milestone? Or do we think this won't make CuPy 14, in which case maybe the milestone should be dropped?

@leofang
Copy link
Member Author

leofang commented Aug 8, 2024

I am trying to find someone from NVIDIA to take over this PR and include it in ml_dtypes (jax-ml/ml_dtypes#110 (comment)). I doubt this can be done in v14.0.0.a1, but might be a stretching goal for the v14 a2 or b1 releases, so I think keeping it in the v14 milestone sounds appropriate.

@kmaehashi kmaehashi removed this from the v14 milestone Oct 2, 2025
@leofang
Copy link
Member Author

leofang commented Jan 15, 2026

I am trying to find someone from NVIDIA to take over this PR and include it in ml_dtypes (jax-ml/ml_dtypes#110 (comment)).

@seberg will have something ready to upstream soon (jax-ml/ml_dtypes#110 (comment)), but it may not land by v14.0.0. In the meanwhile, the scalar infra is being set up in #9494 and once complex32 is ready, it should be possible to extend the infra to accept it. This PR is moot, then, with all these ongoing works!

@leofang leofang closed this Jan 15, 2026
@leofang leofang deleted the complex32 branch January 15, 2026 05:21
@kmaehashi kmaehashi added this to the Closed PRs milestone Feb 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@kmaehashi kmaehashi

Labels

cat:enhancement Improvements to existing features st:needs-discussion

Projects

None yet

Milestone

Closed PRs

Development

Successfully merging this pull request may close these issues.

5 participants

@leofang @emcastillo @kmaehashi @jakirkham @asi1024