[WIP] Allow autocast for 1.6 by mcarilli · Pull Request #2384 · pytorch/vision

mcarilli · 2020-07-02T23:58:31Z

This PR introduces a minimal set of diffs that enable torchvision models to work with torch.cuda.amp.autocast. Custom C++ ops (roi_align and nms) require the most attention.

In later modifications to Pytorch, I'll allow external libs to use Pytorch's internal autocast utilities, after which this code can be made cleaner. For 1.6, however, copy pasting some utilities is the best we can do.

Should close pytorch/pytorch#37735.

fmassa

Thanks for the PR!

Can you add some full model-forward test in a similar location as

vision/test/test_models.py

Lines 273 to 293 in 5247f7b

    
           @unittest.skipIf(not torch.cuda.is_available(), 'needs GPU') 
        
           def test_fasterrcnn_switch_devices(self): 
        
               model = models.detection.fasterrcnn_resnet50_fpn(num_classes=50, pretrained_backbone=False) 
        
               model.cuda() 
        
               model.eval() 
        
               input_shape = (3, 300, 300) 
        
               x = torch.rand(input_shape, device='cuda') 
        
               model_input = [x] 
        
               out = model(model_input) 
        
               self.assertIs(model_input[0], x) 
        
               self.assertEqual(len(out), 1) 
        
               self.assertTrue("boxes" in out[0]) 
        
               self.assertTrue("scores" in out[0]) 
        
               self.assertTrue("labels" in out[0]) 
        
               # now switch to cpu and make sure it works 
        
               model.cpu() 
        
               x = x.cpu() 
        
               out_cpu = model([x]) 
        
               self.assertTrue("boxes" in out_cpu[0]) 
        
               self.assertTrue("scores" in out_cpu[0]) 
        
               self.assertTrue("labels" in out_cpu[0])

and also add a forward-backward test for roi_align after

vision/test/test_ops.py

Lines 290 to 291 in 5247f7b

    
           def _test_boxes_shape(self): 
        
               self._helper_boxes_shape(ops.roi_align)

(and we will clean it up later on when adding support for the other ops to autocast)

torchvision/ops/poolers.py

codecov · 2020-07-03T08:55:15Z

Codecov Report

Merging #2384 into master will increase coverage by 0.70%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #2384      +/-   ##
==========================================
+ Coverage   70.65%   71.36%   +0.70%     
==========================================
  Files          94       94              
  Lines        7897     8328     +431     
  Branches     1241     1385     +144     
==========================================
+ Hits         5580     5943     +363     
- Misses       1934     1972      +38     
- Partials      383      413      +30

Impacted Files	Coverage Δ
torchvision/ops/poolers.py	`97.02% <100.00%> (ø)`
torchvision/io/image.py	`71.73% <0.00%> (+0.77%)`	⬆️
torchvision/transforms/functional_tensor.py	`65.41% <0.00%> (+0.83%)`	⬆️
torchvision/transforms/functional.py	`81.91% <0.00%> (+2.09%)`	⬆️
torchvision/transforms/transforms.py	`78.25% <0.00%> (+2.23%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 86b6c3e...f3451c9. Read the comment docs.

torchvision/csrc/ROIAlign.h

fmassa · 2020-07-07T10:20:15Z

Test failures seem related

torchvision/csrc/cuda/nms_cuda.cu

…e still passes

…merics

fmassa

Thanks a lot Michael!

* Fixes Xiao's repro * Ports nms to use full dispatcher * Move HIPGuard to nms_cuda * clang-format * run models in test_models.py on GPU if available * Francisco's comment, also disable cuda model tests to see if CPU alone still passes * cuda tests now pass locally, although still not comparing to saved numerics * add note for thing to ask francisco * Allow cuda and cpu tests to share a data file * ignore suffix if unneeded * Skip autocast numerics checks for a few models * Add roi_align test Co-authored-by: Michael Carilli <mcarilli@nvidia.com>

* Fixes Xiao's repro * Ports nms to use full dispatcher * Move HIPGuard to nms_cuda * clang-format * run models in test_models.py on GPU if available * Francisco's comment, also disable cuda model tests to see if CPU alone still passes * cuda tests now pass locally, although still not comparing to saved numerics * add note for thing to ask francisco * Allow cuda and cpu tests to share a data file * ignore suffix if unneeded * Skip autocast numerics checks for a few models * Add roi_align test Co-authored-by: Michael Carilli <mcarilli@nvidia.com> Co-authored-by: mcarilli <mcarilli@gmail.com> Co-authored-by: Michael Carilli <mcarilli@nvidia.com>

Fixes Xiao's repro

6ea1840

mcarilli changed the title ~~Allow autocast for 1.6~~ [WIP] Allow autocast for 1.6 Jul 3, 2020

fmassa reviewed Jul 3, 2020

View reviewed changes

torchvision/ops/poolers.py Show resolved Hide resolved

fmassa reviewed Jul 3, 2020

View reviewed changes

torchvision/csrc/ROIAlign.h Show resolved Hide resolved

definitelynotmcarilli added 2 commits July 6, 2020 17:04

Ports nms to use full dispatcher

1175bbc

Merge remote-tracking branch 'upstream/master' into allow_autocast

4f2725f

fmassa reviewed Jul 7, 2020

View reviewed changes

torchvision/csrc/ROIAlign.h Outdated Show resolved Hide resolved

Move HIPGuard to nms_cuda

4e8deeb

fmassa reviewed Jul 7, 2020

View reviewed changes

torchvision/csrc/cuda/nms_cuda.cu Show resolved Hide resolved

definitelynotmcarilli added 9 commits July 7, 2020 14:11

clang-format

4d33617

run models in test_models.py on GPU if available

d961dfc

Francisco's comment, also disable cuda model tests to see if CPU alon…

4bc38d9

…e still passes

cuda tests now pass locally, although still not comparing to saved nu…

3e84c5c

…merics

add note for thing to ask francisco

85c32d4

Allow cuda and cpu tests to share a data file

df60fdb

ignore suffix if unneeded

1b6ebbb

Skip autocast numerics checks for a few models

ac75f8f

Add roi_align test

f3451c9

fmassa approved these changes Jul 9, 2020

View reviewed changes

fmassa merged commit 0a8586c into pytorch:master Jul 9, 2020

fmassa mentioned this pull request Sep 1, 2020

ROCM version doesn't work #2621

Closed

fmassa mentioned this pull request Oct 13, 2020

Enable autocast for all ops #2797

Closed

6 tasks

This was referenced Oct 21, 2020

Make R-CNN models support Automatic Mixed Precision (AMP) #2222

Closed

MaskRCNN model doesn't work with torch.cuda.amp.autocast RuntimeError: Unrecognized tensor type ID: Autocast #2172

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Allow autocast for 1.6#2384

[WIP] Allow autocast for 1.6#2384
fmassa merged 13 commits intopytorch:masterfrom
mcarilli:allow_autocast

mcarilli commented Jul 2, 2020 •

edited

Loading

Uh oh!

fmassa left a comment

Uh oh!

Uh oh!

codecov bot commented Jul 3, 2020 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

fmassa commented Jul 7, 2020

Uh oh!

Uh oh!

fmassa left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	@unittest.skipIf(not torch.cuda.is_available(), 'needs GPU')
	def test_fasterrcnn_switch_devices(self):
	model = models.detection.fasterrcnn_resnet50_fpn(num_classes=50, pretrained_backbone=False)
	model.cuda()
	model.eval()
	input_shape = (3, 300, 300)
	x = torch.rand(input_shape, device='cuda')
	model_input = [x]
	out = model(model_input)
	self.assertIs(model_input[0], x)
	self.assertEqual(len(out), 1)
	self.assertTrue("boxes" in out[0])
	self.assertTrue("scores" in out[0])
	self.assertTrue("labels" in out[0])
	# now switch to cpu and make sure it works
	model.cpu()
	x = x.cpu()
	out_cpu = model([x])
	self.assertTrue("boxes" in out_cpu[0])
	self.assertTrue("scores" in out_cpu[0])
	self.assertTrue("labels" in out_cpu[0])

	def _test_boxes_shape(self):
	self._helper_boxes_shape(ops.roi_align)

Conversation

mcarilli commented Jul 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fmassa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Jul 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

fmassa commented Jul 7, 2020

Uh oh!

Uh oh!

fmassa left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mcarilli commented Jul 2, 2020 •

edited

Loading

codecov bot commented Jul 3, 2020 •

edited

Loading