de-singleton tile extract in graphreader#3281
Merged
kevinkreiser merged 49 commits intomasterfrom Oct 27, 2021
Merged
Conversation
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
e656389 to
7bcc7b9
Compare
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
…er for graphreader
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
nilsnolde
commented
Aug 23, 2021
…ve different tar file sizes with predicted traffic
…so we don't get a race condition with the test which actually creates the tar
Member
Author
|
I removed setting the |
Member
|
@nilsnolde ill review this today, might be tonight by the time i have time but ill get to it. thanks for hanging in there! |
Member
|
ok i made a couple requests for minor changes otherwise looks good! |
nilsnolde
commented
Oct 26, 2021
Member
Author
|
thanks @kevinkreiser for final review. I changed all the things you requested:
finally done 🥳 next up: integrate the ability to reload tiles to python bindings (what else?) |
kevinkreiser
approved these changes
Oct 27, 2021
This was referenced Oct 29, 2021
4 tasks
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #3117
I started with this on the side, working on it here and there. to re-iterate: we want to be able to load other tile extracts while a consumer application is running (e.g.
valhalla_service). in #3117 @kevinkreiser proposed to use a fixed-width index file storing the tile_id and byte offset in the final tar file which the graphreader can read first (if available) to really fast initialize the tile_extract_t (faster because it does not have to scan through the entire tar file thanks to the index file).tasks:
this (currently) does not work for existing extracts, so far it only builds an extract from-> not necessary IMOmjolnir.tile_dir. is that fine with you? we could also add that ability and it sorta was what @kevinkreiser initially suggestedwhere to add this functionality: python bindings for sure IMO, what about-> subsequent PRsvalhalla_service?@kevinkreiser I didn't do much yet on the c++ side, will comment a bit on the files, a few pointers would be cool:)
example commandline:
python scripts/valhalla_build_extract --config site/valhalla.json