Several things to be done after merging this PR:

1. Should add comments above the line of code to indicate the reason for the change.
2. CI of PyTorch 1.10.

Originally posted by @ZwwWayne in #7074 (comment)