Skip to content

TPU gradient clipping. #963

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Feb 27, 2020
Merged

TPU gradient clipping. #963

merged 6 commits into from
Feb 27, 2020

Conversation

@srush
Copy link
Contributor

@srush srush commented Feb 27, 2020 •
edited
Loading

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
  • Did you read the contributor guideline?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
  • If you made a notable change (that affects users), did you update the CHANGELOG?
    What does this PR do?
    Switches gradient clipping to TPU friendly style.

PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?
Make sure you had fun coding

Borda
Borda approved these changes Feb 27, 2020
View reviewed changes
Copy link
Collaborator

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@Borda Borda added this to the 0.6.1 milestone Feb 27, 2020
@Borda Borda added feature Is an improvement or enhancement ready PRs ready to be merged labels Feb 27, 2020
@Borda Borda mentioned this pull request Feb 27, 2020
Closed
3 tasks
srush and others added 4 commits February 27, 2020 09:16
@srush @Borda
Update pytorch_lightning/trainer/training_tricks.py
5d14129 
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
@srush @Borda
Update pytorch_lightning/trainer/training_tricks.py
41ff747 
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
pull out epsilon
aa1b50c
add fp16 case
a3ef8f2
Borda
Borda reviewed Feb 27, 2020
View reviewed changes
@srush @Borda
Update pytorch_lightning/trainer/training_tricks.py
cd86179 
Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>
@williamFalcon williamFalcon merged commit 27a3be0 into Lightning-AI:master Feb 27, 2020
tullie pushed a commit to tullie/pytorch-lightning that referenced this pull request Apr 3, 2020
@srush @Borda @tullie
TPU gradient clipping. (Lightning-AI#963)
a4871c7 
* clip

* Update pytorch_lightning/trainer/training_tricks.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/trainer/training_tricks.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

* pull out epsilon

* add fp16 case

* Update pytorch_lightning/trainer/training_tricks.py

Co-Authored-By: Jirka Borovec <Borda@users.noreply.github.com>

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
@awaelchli awaelchli mentioned this pull request Apr 19, 2021
Merged
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@williamFalcon williamFalcon williamFalcon left review comments

@Borda Borda Borda approved these changes

Assignees

No one assigned

Labels

feature Is an improvement or enhancement ready PRs ready to be merged

Projects

None yet

Milestone

pl:0.7

Development

Successfully merging this pull request may close these issues.

3 participants

@srush @williamFalcon @Borda