Add coco dataset download script#6715
Closed
triple-mu wants to merge 8 commits intoopen-mmlab:devfrom
Closed
Conversation
You can use command "python tools/download.py --win --unzip" to download coco dataset. Linux for using command "python tools/download.py --unzip"
|
|
ZwwWayne
reviewed
Dec 13, 2021
tools/download.py
Outdated
| type=str, | ||
| help='the dir to save dataset', | ||
| default='data/coco') | ||
|
|
Collaborator
There was a problem hiding this comment.
The empty line in line 77,80,82,85 may not be necessary.
ZwwWayne
reviewed
Dec 13, 2021
tools/download.py
Outdated
| parser.add_argument( | ||
| '--unzip', action='store_true', help='whether unzip dataset or not') | ||
|
|
||
| parser.add_argument('--delete', action='store_true', help='del dataset') |
Collaborator
There was a problem hiding this comment.
- Does this mean the zipped files will be deleted or the who dataset?
- del -> delete. We usually use complete word in help messages.
ZwwWayne
reviewed
Dec 13, 2021
tools/download.py
Outdated
| if not path.exists(): | ||
| path.mkdir(parents=True, exist_ok=True) | ||
|
|
||
| urls = [ |
Collaborator
There was a problem hiding this comment.
Suggest putting the urls to a dict because we will also download VOC/LVIS dataset in the future, you can put them in a dict like below:
data2url = dict(coco2017=[
URLs...
], lvis=[], voc2007=[], )
ZwwWayne
reviewed
Dec 13, 2021
Czm369
reviewed
Dec 22, 2021
| help='delete the download zipped files') | ||
| parser.add_argument( | ||
| '--threads', type=int, help='number of threading', default=4) | ||
| parser.add_argument( |
Collaborator
There was a problem hiding this comment.
The argument --win may be unnecessary, for
win = Flase if sys.platform == 'linux' else win = True.
| elif not f.exists(): | ||
| print('Downloading {} to {}'.format(url, f)) | ||
| if curl: | ||
| os.system('curl -L {} -o {} --retry 9 -C -'.format(url, f)) |
Collaborator
There was a problem hiding this comment.
Sometimes, the command curl does not exit in a conda env, so whether check it or not ?
Collaborator
|
Kindly ping @q3394101 |
Member
|
@q3394101 Hi, we will take over this PR and support more dataset. This PR will be closed and your commits will be kept in #7015 |
ZwwWayne
added a commit
that referenced
this pull request
Jan 19, 2022
…6715 (#7015) * Add coco dataset download script You can use command "python tools/download.py --win --unzip" to download coco dataset. Linux for using command "python tools/download.py --unzip" * Add coco dataset download script * Add coco dataset download script * Add coco dataset download script * add some notes and modify dataset urls * add some notes and modify dataset urls * remove some useless lines and modify urls list to dict * add urls of lvis and voc, and delete --win * add parse_args() * Add documentation of this tool in docs/en/1_exist_data_model.md, docs/zh_cn/1_exist_data_model.md and docs/en/useful_tools.md. * add a link * Download files regardless of system。 * Use get() of dict * add empty line above the code block * Update useful_tools.md Co-authored-by: q3394101 <92794867+q3394101@users.noreply.github.com> Co-authored-by: q3394101 <3394101@qq.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>
chhluo
pushed a commit
to chhluo/mmdetection
that referenced
this pull request
Feb 21, 2022
…pen-mmlab#6715 (open-mmlab#7015) * Add coco dataset download script You can use command "python tools/download.py --win --unzip" to download coco dataset. Linux for using command "python tools/download.py --unzip" * Add coco dataset download script * Add coco dataset download script * Add coco dataset download script * add some notes and modify dataset urls * add some notes and modify dataset urls * remove some useless lines and modify urls list to dict * add urls of lvis and voc, and delete --win * add parse_args() * Add documentation of this tool in docs/en/1_exist_data_model.md, docs/zh_cn/1_exist_data_model.md and docs/en/useful_tools.md. * add a link * Download files regardless of system。 * Use get() of dict * add empty line above the code block * Update useful_tools.md Co-authored-by: q3394101 <92794867+q3394101@users.noreply.github.com> Co-authored-by: q3394101 <3394101@qq.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>
ZwwWayne
added a commit
that referenced
this pull request
Jul 18, 2022
…6715 (#7015) * Add coco dataset download script You can use command "python tools/download.py --win --unzip" to download coco dataset. Linux for using command "python tools/download.py --unzip" * Add coco dataset download script * Add coco dataset download script * Add coco dataset download script * add some notes and modify dataset urls * add some notes and modify dataset urls * remove some useless lines and modify urls list to dict * add urls of lvis and voc, and delete --win * add parse_args() * Add documentation of this tool in docs/en/1_exist_data_model.md, docs/zh_cn/1_exist_data_model.md and docs/en/useful_tools.md. * add a link * Download files regardless of system。 * Use get() of dict * add empty line above the code block * Update useful_tools.md Co-authored-by: q3394101 <92794867+q3394101@users.noreply.github.com> Co-authored-by: q3394101 <3394101@qq.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>
ZwwWayne
added a commit
to ZwwWayne/mmdetection
that referenced
this pull request
Jul 19, 2022
…pen-mmlab#6715 (open-mmlab#7015) * Add coco dataset download script You can use command "python tools/download.py --win --unzip" to download coco dataset. Linux for using command "python tools/download.py --unzip" * Add coco dataset download script * Add coco dataset download script * Add coco dataset download script * add some notes and modify dataset urls * add some notes and modify dataset urls * remove some useless lines and modify urls list to dict * add urls of lvis and voc, and delete --win * add parse_args() * Add documentation of this tool in docs/en/1_exist_data_model.md, docs/zh_cn/1_exist_data_model.md and docs/en/useful_tools.md. * add a link * Download files regardless of system。 * Use get() of dict * add empty line above the code block * Update useful_tools.md Co-authored-by: q3394101 <92794867+q3394101@users.noreply.github.com> Co-authored-by: q3394101 <3394101@qq.com> Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Add a script to download coco2017 dataset
Modification
Add a python file in tools
BC-breaking (Optional)
This script do not link any other files
Use cases (Optional)
You can use command "python tools/download.py --win" to download coco2017 dataset.
Linux for using command "python tools/download.py
You can alse use flag --unzip to unzip the zipfiles
such as python tools/download.py --win --unzip
Checklist