Skip to content

Add mixed precision#40

Merged
andersonic merged 8 commits intomasterfrom
add-mixed-precision
Aug 14, 2020
Merged

Add mixed precision#40
andersonic merged 8 commits intomasterfrom
add-mixed-precision

Conversation

@andersonic
Copy link
Copy Markdown
Contributor

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
  • Did you read the contributor guideline?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

What does this PR do?

Add mixed precision training to Adam. Update benchmark.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 14, 2020
@codecov
Copy link
Copy Markdown

codecov bot commented Aug 14, 2020

Codecov Report

Merging #40 into master will increase coverage by 0.01%.
The diff coverage is 96.07%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #40      +/-   ##
==========================================
+ Coverage   94.10%   94.12%   +0.01%     
==========================================
  Files          35       35              
  Lines        2003     2043      +40     
==========================================
+ Hits         1885     1923      +38     
- Misses        118      120       +2     
Flag Coverage Δ
#Python 94.12% <96.07%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
fairscale/optim/adam.py 91.48% <96.07%> (+2.60%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3427a03...4bd562f. Read the comment docs.

# Assert that words per second is within 3 standard deviations of the average
# of six golden runs
assert wps > 20052.1 - (3 * 359)
assert wps > 27799.2 - (3 * 522.145)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice speedups and memory reduction (below) !

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link
Copy Markdown
Contributor

@sidgoyal78 sidgoyal78 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Looks goods overall. I left a few minor comments.

assert torch.cuda.memory_stats(1)["allocated_bytes.all.peak"] < 1281024 * 1.1
assert torch.cuda.memory_stats(2)["allocated_bytes.all.peak"] < 2788864 * 1.1
assert torch.cuda.memory_stats(3)["allocated_bytes.all.peak"] < 190724608 * 1.1
assert torch.cuda.memory_stats(0)["allocated_bytes.all.peak"] < 210479616 * 1.1
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, i'm curious what tool did you use get these exact numbers?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the values printed by the above four lines of code

float* m = (float *)tl.addresses[1][tensor_loc];
m += chunk_idx*chunk_size;
T* v = (T *)tl.addresses[2][tensor_loc];
float* v = (float *)tl.addresses[2][tensor_loc];
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

naive question: why do we have types of m and v as float?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are momentum and velocity! And right now, we require them to be floats; in the Python code, when they are instantiated, they are always dtype=torch.float32. Next pull request will add in the option for them to be fp16

(adamMode_t) mode,
decay
);
} else { // tl_sz == 4
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add an explicit comment for the setting here, similar to "mix precision case" for the above :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea! Done!

@andersonic andersonic requested a review from sidgoyal78 August 14, 2020 19:59
Copy link
Copy Markdown
Contributor

@sidgoyal78 sidgoyal78 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@andersonic andersonic merged commit e2d8f57 into master Aug 14, 2020
@andersonic andersonic deleted the add-mixed-precision branch August 14, 2020 20:58
myleott pushed a commit that referenced this pull request Feb 22, 2021
* Test CPU offload

* remove dead code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants