Skip to content

Pre-trained weights for raft-large, fine-tuned on sintel#5079

Merged
NicolasHug merged 9 commits intopytorch:mainfrom
NicolasHug:raft_large_sintel
Dec 9, 2021
Merged

Pre-trained weights for raft-large, fine-tuned on sintel#5079
NicolasHug merged 9 commits intopytorch:mainfrom
NicolasHug:raft_large_sintel

Conversation

@NicolasHug
Copy link
Copy Markdown
Member

@NicolasHug NicolasHug commented Dec 9, 2021 •
edited
Loading

Towards #4644

This PR adds our own pre-trained weights as well as the original weights from the paper.

I evaluated these on Sintel test set, and we do slightly better than the original weights.

There is a limited amount of submissions we can make to sintel test, so I decided to just submit C_T_V2 after fine-tuning it on sintel with the "S+K+H" schedule (--train-dataset sintel_skh).

Results below are on sintel train, just to demonstrate that it works. They're not incredibly relevant since we trained on sintel train.

(raft) ➜  vision git:(raft_large_sintel) ✗ torchrun --nproc_per_node 8 --nnodes 1 references/optical_flow/train.py --val-dataset sintel --batch-size 10 --model raft_large --dataset-root ../downloads --weights Raft_Large_Weights.C_T_SKHT_V1

Sintel val clean epe: 0.7647	1px: 0.9233	3px: 0.9687	5px: 0.9793	per_image_epe: 0.7626	f1: 2.8538
Sintel val final epe: 1.2172	1px: 0.8862	3px: 0.9461	5px: 0.9628	per_image_epe: 1.2111	f1: 4.9317


(raft) ➜  vision git:(raft_large_sintel) ✗ torchrun --nproc_per_node 8 --nnodes 1 references/optical_flow/train.py --val-dataset sintel --batch-size 10 --model raft_large --dataset-root ../downloads --weights Raft_Large_Weights.C_T_SKHT_V2

Sintel val clean epe: 0.6482	1px: 0.9255	3px: 0.9701	5px: 0.9805	per_image_epe: 0.6470	f1: 2.6793
Sintel val final epe: 1.0197	1px: 0.8898	3px: 0.9493	5px: 0.9660	per_image_epe: 1.0151	f1: 4.5754

cc @datumbox

@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Dec 9, 2021 •
edited
Loading

💊 CI failures summary and remediations

As of commit 9b8db17 (more details on the Dr. CI page):

✅ None of the CI failures appear to be your fault 💚

🚧 1 ongoing upstream failure:

These were probably caused by upstream breakages that are not fixed yet.

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@NicolasHug NicolasHug added module: models other if you have no clue or if you will manually handle the PR in the release notes labels Dec 9, 2021
@NicolasHug NicolasHug mentioned this pull request Dec 9, 2021
Closed
12 tasks
datumbox
datumbox approved these changes Dec 9, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@datumbox datumbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@NicolasHug NicolasHug merged commit 954a117 into pytorch:main Dec 9, 2021
facebook-github-bot pushed a commit that referenced this pull request Dec 17, 2021
@NicolasHug @facebook-github-bot
[fbsync] Pre-trained weights for raft-large, fine-tuned on sintel (#5079
db9bd9a 
)

Reviewed By: fmassa

Differential Revision: D33184996

fbshipit-source-id: c19e25fd262529ae9059d94a51df1b3f688a8c78
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@datumbox datumbox datumbox approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

ciflow/default cla signed module: models other if you have no clue or if you will manually handle the PR in the release notes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@NicolasHug @facebook-github-bot @datumbox