Skip to content

Delay the preCollection remove after upload to workaround sporadic timing issue. Fixes #324#331

Merged
dr-dimitru merged 1 commit intoveliovgroup:devfrom
derwok:dev
Jan 19, 2017
Merged

Delay the preCollection remove after upload to workaround sporadic timing issue. Fixes #324#331
dr-dimitru merged 1 commit intoveliovgroup:devfrom
derwok:dev

Conversation

@derwok
Copy link
Copy Markdown
Contributor

@derwok derwok commented Jan 19, 2017

Though the call to Collection.update should be blocking (server side) according to metoer API:
https://docs.meteor.com/api/collections.html#Mongo-Collection-update
In fact the update sometimes is not completed before the remove kicks in, which in turn triggers the preCollection.obeserve("remove") code. This leads to the situation that the oberver code sees an unfinished upload while the upload actually finished correctly.

…ming issue. Fixes veliovgroup#324

Though the call to Collection.update should be blocking (server side) according to metoer API:
https://docs.meteor.com/api/collections.html#Mongo-Collection-update
In fact the update sometimes is not completed before the remove kicks in, which in turn triggers the preCollection.obeserve("remove") code. This leads to the situation that the oberver code sees an unfinished upload while the upload actually finished correctly.
@dr-dimitru dr-dimitru merged commit 4dc0229 into veliovgroup:dev Jan 19, 2017
@dr-dimitru
Copy link
Copy Markdown
Member

Hi @derwok ,

I this issue solved on your end? I'm still experiencing it on heavy MongoDB load (when response is slightly delayed). And often on setup with remote MongoDB, like on mongolab (mlab.com).

@derwok
Copy link
Copy Markdown
Contributor Author

derwok commented Jan 23, 2017

Hm. For me it was fine...
But I agree with you: if the problem still shows up: maybe the root cause is not yet really understood.

I had one more idea (where my coffescript knowledge is not good enough to implement it)...

Could we remove this part here:

        # we delay the remove to resolve timing issues (details see #324)
        Meteor.setTimeout ->
          self._preCollection.remove {_id: opts.fileId}, (error) ->
            if error
              cb and cb error
              console.error '[FilesCollection] [Upload] [_finishUpload] Error:', error if self.debug
            else
              result._id = _id
              console.info "[FilesCollection] [Upload] [finish(ed)Upload] -> #{result.path}" if self.debug
              self.onAfterUpload and self.onAfterUpload.call self, result
              self.emit 'afterUpload', result
              cb and cb null, result
            return  # .remove
          return  # .setTimeout
        , 25

and move it in an observer changed() block like this:

      _preCollectionCursor = @_preCollection.find {}

      _preCollectionCursor.observe changed: (newDoc, oldDoc) ->
                  if newDoc.isFinished
                      Meteor.setTimeout ->
                         self._preCollection.remove {_id: newDoc._id}, (error) ->
                         [...]

So we would decouple the $set {isFinished: true} from the remove() operation and we would wait for the observer-changed callback when MongoDB signals the doc has changed. Now if any preCollection doc changed and newDoc.isFinished==true then this could be our real signal to remove this doc from the preCollection...?

Just an idea...
What do you think?

dr-dimitru added a commit that referenced this pull request Jan 24, 2017
 - See discussion at #331 , thanks to @derwok
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants