Skip to content

colossus: do not serve content from pending folder#4989

Merged
mnaamani merged 1 commit intoJoystream:masterfrom
mnaamani:colossus-dont-serve-from-pending
Dec 5, 2023
Merged

colossus: do not serve content from pending folder#4989
mnaamani merged 1 commit intoJoystream:masterfrom
mnaamani:colossus-dont-serve-from-pending

Conversation

@mnaamani
Copy link
Copy Markdown
Member

@mnaamani mnaamani commented Dec 4, 2023

It is not clearly obvious how the node will behave, if while service a file from temp pending, the file is moved/renamed into the uploads folder. Will the move fail or will the http request handler to getFile fail. So it seems safer and more predictable behavior to only serve objects in the main uploads folder.

This PR reverts change added in #4971 which allows serving objects in pending folder.

Copy link
Copy Markdown
Contributor

@zeeshanakram3 zeeshanakram3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, the only implication of this change is that even the uploader won't able to access the file till it's has been accepted.

The other possible solution (and might not be trivial to implement) is that whenever the conflict happens we let the HTTP request fail and the move should always succeed. WDYT?

@mnaamani
Copy link
Copy Markdown
Member Author

mnaamani commented Dec 4, 2023

Looks good to me, the only implication of this change is that even the uploader won't able to access the file till it's has been accepted.

Yes that is unfortunate, best case scenario is they would have to wait up to 6s for file to become accessible.

The other possible solution (and might not be trivial to implement) is that whenever the conflict happens we let the HTTP request fail and the move should always succeed. WDYT?

Yes I can think of a few ways, but it does seem complex. The root issue is of course having separate location for pending folder. Would it be less complex if we eliminated the pending folder, and track the objects that need to be "confirmed" with onchain tx in a local db, or simply a JSON file?

@zeeshanakram3
Copy link
Copy Markdown
Contributor

The root issue is of course having separate location for pending folder. Would it be less complex if we eliminated the pending folder, and track the objects that need to be "confirmed" with onchain tx in a local db, or simply a JSON file?

Sounds good. Maybe this can be implemented as a feature in storage-squid (i.e. as a mutation that will store relevant info in storage-squid db)

@mnaamani
Copy link
Copy Markdown
Member Author

mnaamani commented Dec 5, 2023

Sounds good. Maybe this can be implemented as a feature in storage-squid (i.e. as a mutation that will store relevant info in storage-squid db)

Worth considering.

@mnaamani mnaamani merged commit 5d911ce into Joystream:master Dec 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants