Skip to content

Add raft_large weights fined-tuned on Kitti#5081

Merged
NicolasHug merged 2 commits intopytorch:mainfrom
NicolasHug:raft_kitti_weights
Dec 9, 2021
Merged

Add raft_large weights fined-tuned on Kitti#5081
NicolasHug merged 2 commits intopytorch:mainfrom
NicolasHug:raft_kitti_weights

Conversation

@NicolasHug
Copy link
Copy Markdown
Member

@NicolasHug NicolasHug commented Dec 9, 2021 •
edited
Loading

towards #4644

This PR adds pre-trained weights for raft-large, fine-tuned on Kitti.

I'm publishing both our weights and the original weights. The Kitti authors kindly allowed us to submit our code for evaluation on Kitti-test.

Our f1-epe on Kitti-test is 5.19 wheras the original is 5.10. We're a tiny bit higher (i.e. worse), but still significantly better than the other baselines that are compared against in the paper (next best one is 6.10).

There are submission restrictions, so to keep consistency with the C_T and C_T_SKHT weights, I just submitted the model that we already used in both (after kitti-specific fine-tuning). In other words these weights are litterally the current C_T_SKHT_V with the last fine-tuning step on Kitti.

Evaluated on kitti train:

(raft) ➜  vision git:(raft_kitti_weights) ✗ torchrun --nproc_per_node 8 --nnodes 1 references/optical_flow/train.py --val-dataset kitti --batch-size 10 --model raft_large --dataset-root ../downloads --weights Raft_Large_Weights.C_T_SKHT_K_V1

Kitti val epe: 0.6262	1px: 0.8732	3px: 0.9696	5px: 0.9859	per_image_epe: 0.6303	f1: 1.4712

(raft) ➜  vision git:(raft_kitti_weights) ✗ torchrun --nproc_per_node 8 --nnodes 1 references/optical_flow/train.py --val-dataset kitti --batch-size 10 --model raft_large --dataset-root ../downloads --weights Raft_Large_Weights.C_T_SKHT_K_V2

Kitti val epe: 0.6283	1px: 0.8752	3px: 0.9689	5px: 0.9852	per_image_epe: 0.6347	f1: 1.5217

kitti test submission:

Screenshot 2021-12-09 at 13-22-36 The KITTI Vision Benchmark Suite

cc @datumbox

@NicolasHug NicolasHug added module: models other if you have no clue or if you will manually handle the PR in the release notes labels Dec 9, 2021
@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Dec 9, 2021 •
edited
Loading

💊 CI failures summary and remediations

As of commit 2c14700 (more details on the Dr. CI page):

  • 2/2 failures introduced in this PR

🕵️ 2 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build unittest_linux_cpu_py3.8 (1/2)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

/root/project/torchvision/io/video.py:399: Runt...log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found 

test/test_internet.py::TestDatasetUtils::test_download_url_dont_exist
  /root/project/env/lib/python3.8/site-packages/_pytest/fixtures.py:928: ResourceWarning: unclosed <ssl.SSLSocket [closed] fd=13, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>
    finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)

test/test_internet.py::TestDatasetUtils::test_download_url_dont_exist
  /root/project/env/lib/python3.8/site-packages/_pytest/fixtures.py:928: ResourceWarning: unclosed <ssl.SSLSocket [closed] fd=14, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>
    finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)

test/test_io.py::TestVideo::test_read_video_timestamps_corrupted_file
  /root/project/torchvision/io/video.py:399: RuntimeWarning: Failed to open container for /tmp/tmpd5lueftl.mp4; Caught error: [Errno 1094995529] Invalid data found when processing input: '/tmp/tmpd5lueftl.mp4'; last error log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found
    warnings.warn(msg, RuntimeWarning)

test/test_models.py::test_memory_efficient_densenet[densenet121]
test/test_models.py::test_memory_efficient_densenet[densenet169]
test/test_models.py::test_memory_efficient_densenet[densenet201]
test/test_models.py::test_memory_efficient_densenet[densenet161]
  /root/project/env/lib/python3.8/site-packages/torch/autocast_mode.py:162: UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling
    warnings.warn('User provided device_type of \'cuda\', but CUDA is not available. Disabling')

test/test_models.py::test_inception_v3_eval

See CircleCI build unittest_linux_cpu_py3.7 (2/2)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

/root/project/torchvision/io/video.py:399: Runt...log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found 

test/test_internet.py::TestDatasetUtils::test_download_url_dispatch_download_from_google_drive
  /root/project/env/lib/python3.7/unittest/mock.py:1966: ResourceWarning: unclosed <ssl.SSLSocket [closed] fd=13, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>
    self.name = name

test/test_internet.py::TestDatasetUtils::test_download_url_dispatch_download_from_google_drive
  /root/project/env/lib/python3.7/unittest/mock.py:1966: ResourceWarning: unclosed <ssl.SSLSocket [closed] fd=14, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6>
    self.name = name

test/test_io.py::TestVideo::test_read_video_timestamps_corrupted_file
  /root/project/torchvision/io/video.py:399: RuntimeWarning: Failed to open container for /tmp/tmp34p85xv2.mp4; Caught error: [Errno 1094995529] Invalid data found when processing input: '/tmp/tmp34p85xv2.mp4'; last error log: [mov,mp4,m4a,3gp,3g2,mj2] moov atom not found
    warnings.warn(msg, RuntimeWarning)

test/test_models.py::test_memory_efficient_densenet[densenet121]
test/test_models.py::test_memory_efficient_densenet[densenet169]
test/test_models.py::test_memory_efficient_densenet[densenet201]
test/test_models.py::test_memory_efficient_densenet[densenet161]
  /root/project/env/lib/python3.7/site-packages/torch/autocast_mode.py:162: UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling
    warnings.warn('User provided device_type of \'cuda\', but CUDA is not available. Disabling')

test/test_models.py::test_inception_v3_eval
This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

datumbox
datumbox approved these changes Dec 9, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@datumbox datumbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@NicolasHug NicolasHug merged commit e0c5cc4 into pytorch:main Dec 9, 2021
@NicolasHug NicolasHug mentioned this pull request Dec 9, 2021
Closed
12 tasks
facebook-github-bot pushed a commit that referenced this pull request Dec 17, 2021
@NicolasHug @facebook-github-bot
[fbsync] Add raft_large weights fined-tuned on Kitti (#5081)
be1edfd 
Reviewed By: fmassa

Differential Revision: D33185012

fbshipit-source-id: 0cde871c6054416b86634865ec758034ec51519e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@datumbox datumbox datumbox approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

ciflow/default cla signed module: models other if you have no clue or if you will manually handle the PR in the release notes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@NicolasHug @facebook-github-bot @datumbox