add FCOS by xiaohu2015 · Pull Request #4961 · pytorch/vision

xiaohu2015 · 2021-11-18T13:06:47Z

The pr is about #2707
We (@xiaohu2015 and @zhiqwang ) implemented torchvision-style FCOS by referencing to detectron2 and AdelaiDet repo.

if train with the retinanet training command:

torchrun --nproc_per_node=8 train.py\
    --dataset coco --model fcos_resnet50_fpn --epochs 26 \
    --lr-steps 16 22 --aspect-ratio-group-factor 3 --lr 0.01 --amp

you can get:

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.390
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.585
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.415
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.221
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.419
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.511
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.325
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.526
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.562
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.359
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.603
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.717

the AP result is better to the paper (38.6).

if train with training command (1x, 12 epoch, the dafult setting in the FCOS paper):

torchrun --nproc_per_node=8 train.py\
    --dataset coco --model fcos_resnet50_fpn --epochs 12 \
    --lr-steps 8 11 --aspect-ratio-group-factor 3 --lr 0.01 --amp

you can get:

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.385
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.583
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.411
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.225
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.424
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.490
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.325
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.533
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.572
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.384
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.619
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.709

the AP result is smilar to the paper (38.6).

cc @jdsgomes

facebook-github-bot · 2021-11-18T13:06:54Z

💊 CI failures summary and remediations

As of commit 6840a7d (more details on the Dr. CI page):

1/2 failures introduced in this PR
1/2 broken upstream at merge base fe65d37 since Jan 14

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

unittest_linux_gpu_py3.8 (1/1)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

test/test_utils.py::test_draw_keypoints_errors �[33mSKIPPED�[0m (We're in a C...)�[31m [ 99%]�[0m

test/test_utils.py::test_draw_segmentation_masks[1-blue] �[33mSKIPPED�[0m (We...)�[31m [ 99%]�[0m

test/test_utils.py::test_draw_segmentation_masks[1-#FF00FF] �[33mSKIPPED�[0m�[31m      [ 99%]�[0m

test/test_utils.py::test_draw_segmentation_masks[1-colors3] �[33mSKIPPED�[0m�[31m      [ 99%]�[0m

test/test_utils.py::test_draw_segmentation_masks[1-colors4] �[33mSKIPPED�[0m�[31m      [ 99%]�[0m

test/test_utils.py::test_draw_segmentation_masks[1-colors5] �[33mSKIPPED�[0m�[31m      [ 99%]�[0m

test/test_utils.py::test_draw_segmentation_masks_errors �[33mSKIPPED�[0m (We'...)�[31m [ 99%]�[0m

test/test_utils.py::test_draw_keypoints_vanilla �[33mSKIPPED�[0m (We're in a ...)�[31m [ 99%]�[0m

test/test_utils.py::test_draw_keypoints_colored[red] �[33mSKIPPED�[0m (We're ...)�[31m [ 99%]�[0m

test/test_utils.py::test_draw_keypoints_colored[#FF00FF] �[33mSKIPPED�[0m (We...)�[31m [ 99%]�[0m

test/test_utils.py::test_draw_keypoints_colored[colors2] �[33mSKIPPED�[0m (We...)�[31m [ 99%]�[0m

test/test_utils.py::test_draw_keypoints_errors �[33mSKIPPED�[0m (We're in a C...)�[31m [ 99%]�[0m

test/test_video_gpu_decoder.py::TestVideoGPUDecoder::test_frame_reading �[33mSKIPPED�[0m�[31m [ 99%]�[0m

test/test_video_reader.py::TestVideoReader::test_stress_test_read_video_from_file �[33mSKIPPED�[0m�[31m [ 99%]�[0m

test/test_video_reader.py::TestVideoReader::test_read_video_from_file �[33mSKIPPED�[0m�[31m [ 99%]�[0m

test/test_video_reader.py::TestVideoReader::test_read_video_from_file_read_single_stream_only �[33mSKIPPED�[0m�[31m [ 99%]�[0m

test/test_video_reader.py::TestVideoReader::test_read_video_from_file_rescale_min_dimension �[33mSKIPPED�[0m�[31m [ 99%]�[0m

test/test_video_reader.py::TestVideoReader::test_read_video_from_file_rescale_max_dimension �[33mSKIPPED�[0m�[31m [ 99%]�[0m

test/test_video_reader.py::TestVideoReader::test_read_video_from_file_rescale_both_min_max_dimension �[33mSKIPPED�[0m�[31m [ 99%]�[0m

test/test_video_reader.py::TestVideoReader::test_read_video_from_file_rescale_width �[33mSKIPPED�[0m�[31m [ 99%]�[0m

test/test_video_reader.py::TestVideoReader::test_read_video_from_file_rescale_height �[33mSKIPPED�[0m�[31m [ 99%]�[0m

test/test_video_reader.py::TestVideoReader::test_read_video_from_file_rescale_width_and_height �[33mSKIPPED�[0m�[31m [ 99%]�[0m

🚧 1 ongoing upstream failure:

These were probably caused by upstream breakages that are not fixed yet.

unittest_prototype since Jan 14 (adf8466)
- 🔁 rerun

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Fixing lint

Fixing lint in giou_loss.py

Fixing python lint, docstrings and add typing annotations

Apply ufmt format

Fixing docs indentation

Fixing docs format in GIoU Loss

torchvision/prototype/models/detection/fcos.py

torchvision/models/detection/fcos.py

torchvision/prototype/models/detection/fcos.py

Minor fixes for the docs

Co-authored-by: Zhiqiang Wang <zhiqwang@foxmail.com>

jdsgomes

Thank you for all your patience and commitment. Looks great to me now and I think its ready to merge, but I will just wait for @datumbox approval to make sure I didn't miss any changes suggested by him.

torchvision/models/detection/fcos.py

torchvision/prototype/models/detection/fcos.py

datumbox

LGTM! Thanks a lot @xiaohu2015 and @zhiqwang for your amazing work.

We are going to base our new model contribution process on this collaboration. @jdsgomes is currently leading this and we will ping you once this is drafted to get your input.

Looking forward working with you again, soon. Perhaps next target is EfficientNetV2? 😃

github-actions · 2022-01-21T15:02:56Z

Hey @jdsgomes!

You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py

jdsgomes · 2022-01-25T11:26:00Z

I have opened a ticket to add FCOS model in Pytorch Hub: pytorch/hub#265

Summary: * add fcos * update fcos * add giou_loss * add BoxLinearCoder for FCOS * add full code for FCOS * add giou loss * add fcos * add __all__ * Fixing lint * Fixing lint in giou_loss.py * Add typing annotation to fcos * Add trained checkpoints * Use partial to replace lambda * Minor fixes to docstrings * Apply ufmt format * Fixing docstrings * Fixing jit scripting * Minor fixes to docstrings * Fixing jit scripting * Ignore mypy in fcos * Fixing trained checkpoints * Fixing unit-test of jit script * Fixing docstrings * Add test/expect/ModelTester.test_fcos_resnet50_fpn_expect.pkl * Fixing test_detection_model_trainable_backbone_layers * Update test_fcos_resnet50_fpn_expect.pkl * rename stride to box size * remove TODO and fix some typo * merge some code for better * impove the comments * remove decode and encode of BoxLinearCoder * remove some unnecessary hints * use the default value in detectron2. * update doc * Add unittest for BoxLinearCoder * Add types in FCOS * Add docstring for BoxLinearCoder * Minor fix for the docstring * update doc * Update fcos_resnet50_fpn_coco pretained weights url * Update torchvision/models/detection/fcos.py * Update torchvision/models/detection/fcos.py * Update torchvision/models/detection/fcos.py * Update torchvision/models/detection/fcos.py * Add FCOS model documentation * Fix typo in FCOS documentation * Add fcos to the prototype builder * Capitalize COCO_V1 * Fix params of fcos * fix bug for partial * Fixing docs indentation * Fixing docs format in giou_loss * Adopt Reference for GIoU Loss * Rename giou_loss to generalized_box_iou_loss * remove overwrite_eps * Update AP test values * Minor fixes for the docs * Minor fixes for the docs * Update torchvision/models/detection/fcos.py * Update torchvision/prototype/models/detection/fcos.py Reviewed By: jdsgomes, prabhat00155 Differential Revision: D33739385 fbshipit-source-id: 7dab616adfd0c34fe21f0153c1da51f97ef43b95 Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com> Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com> Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com> Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com> Co-authored-by: Zhiqiang Wang <zhiqwang@foxmail.com> Co-authored-by: Zhiqiang Wang <zhiqwang@foxmail.com> Co-authored-by: zhiqiang <zhiqwang@foxmail.com> Co-authored-by: Joao Gomes <jdsgomes@fb.com> Co-authored-by: Vasilis Vryniotis <datumbox@users.noreply.github.com> Co-authored-by: Joao Gomes <joaopsgomes@gmail.com>

xiaohu2015 added 9 commits November 16, 2021 19:49

add fcos

791d938

update fcos

c9a00c1

Merge branch 'pytorch:main' into main

6c31577

add giou_loss

e82f825

add BoxLinearCoder for FCOS

391b8c9

add full code for FCOS

b01f145

add giou loss

a4eeb59

add fcos

61e946d

add __all__

997f189

pytorch-probot bot added the ciflow/default label Nov 18, 2021

facebook-github-bot added the cla signed label Nov 18, 2021

Merge branch 'main' into main

19ede68

xiaohu2015 marked this pull request as draft November 18, 2021 13:15

zhiqwang and others added 16 commits November 18, 2021 21:33

Fixing lint

793b1db

Merge pull request #1 from o295/main

a683060

Fixing lint

Fixing lint in giou_loss.py

3cf91e3

Merge pull request #2 from o295/main

1f67950

Fixing lint in giou_loss.py

Merge branch 'pytorch:main' into main

5d75049

Add typing annotation to fcos

c7a7d52

Add trained checkpoints

42b147d

Use partial to replace lambda

ff2d78a

Minor fixes to docstrings

d4c08d3

Merge pull request #3 from o295/main

edb2a1a

Fixing python lint, docstrings and add typing annotations

Apply ufmt format

c464249

Merge pull request #4 from o295/main

8955e77

Apply ufmt format

Fixing docstrings

293e6b9

Fixing jit scripting

4cbce22

Minor fixes to docstrings

b444d21

Fixing jit scripting

ac8d062

zhiqwang and others added 5 commits January 20, 2022 14:44

Fixing docs indentation

9085461

Merge pull request #14 from zhiqwang/fix-doc-indentation

9b25135

Fixing docs indentation

Fixing docs format in giou_loss

ebbd772

Adopt Reference for GIoU Loss

d8ed195

Merge pull request #15 from zhiqwang/fix-lint

b125f66

Fixing docs format in GIoU Loss

datumbox reviewed Jan 20, 2022

View reviewed changes

torchvision/prototype/models/detection/fcos.py Outdated Show resolved Hide resolved

torchvision/models/detection/fcos.py Outdated Show resolved Hide resolved

torchvision/models/detection/fcos.py Show resolved Hide resolved

jdsgomes and others added 5 commits January 20, 2022 11:43

Rename giou_loss to generalized_box_iou_loss

b0b0892

remove overwrite_eps

8c24ac3

Update AP test values

bd3e262

Minor fixes for the docs

e452cfe

Minor fixes for the docs

e0af329

zhiqwang reviewed Jan 20, 2022

View reviewed changes

torchvision/models/detection/fcos.py Outdated Show resolved Hide resolved

torchvision/prototype/models/detection/fcos.py Outdated Show resolved Hide resolved

xiaohu2015 and others added 4 commits January 20, 2022 20:06

Merge pull request #17 from zhiqwang/remove_overwrite_eps

c17a6e8

Minor fixes for the docs

Update torchvision/models/detection/fcos.py

10be35d

Co-authored-by: Zhiqiang Wang <zhiqwang@foxmail.com>

Update torchvision/prototype/models/detection/fcos.py

4922635

Co-authored-by: Zhiqiang Wang <zhiqwang@foxmail.com>

Merge branch 'main' into main

6bae6c1

jdsgomes approved these changes Jan 21, 2022

View reviewed changes

torchvision/models/detection/fcos.py Outdated Show resolved Hide resolved

torchvision/prototype/models/detection/fcos.py Outdated Show resolved Hide resolved

datumbox approved these changes Jan 21, 2022

View reviewed changes

Merge branch 'main' into main

6840a7d

jdsgomes merged commit 7d4bdd4 into pytorch:main Jan 21, 2022

jdsgomes added module: models topic: object detection labels Jan 21, 2022

xiaohu2015 mentioned this pull request Jan 24, 2022

fix bug when the target is empty in FCOS #5267

Merged

jdsgomes mentioned this pull request Jan 25, 2022

Add FCOS pytorch/hub#265

Closed

datumbox mentioned this pull request Feb 11, 2022

[RFC] Batteries Included - Phase 2 #5410

Closed

24 tasks

pmeier mentioned this pull request Apr 25, 2022

Google Colab Cuda 11.1 module 'torchvision.models.detection' has no attribute 'fcos_resnet50_fpn' pytorch/pytorch#76280

Closed

datumbox mentioned this pull request Nov 7, 2022

[RFC] Batteries Included - Phase 3 #6323

Open

16 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add FCOS#4961

add FCOS#4961
jdsgomes merged 94 commits intopytorch:mainfrom
xiaohu2015:main

xiaohu2015 commented Nov 18, 2021 •

edited by jdsgomes

Loading

Uh oh!

facebook-github-bot commented Nov 18, 2021 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jdsgomes left a comment

Uh oh!

Uh oh!

Uh oh!

datumbox left a comment

Uh oh!

github-actions bot commented Jan 21, 2022

Uh oh!

jdsgomes commented Jan 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

xiaohu2015 commented Nov 18, 2021 • edited by jdsgomes Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Nov 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

unittest_linux_gpu_py3.8 (1/1)

🚧 1 ongoing upstream failure:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jdsgomes left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

datumbox left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 21, 2022

Uh oh!

jdsgomes commented Jan 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

xiaohu2015 commented Nov 18, 2021 •

edited by jdsgomes

Loading

facebook-github-bot commented Nov 18, 2021 •

edited

Loading