Skip to content

fix append bug and overwrite/truncate bug#1164

Merged
jbenet merged 1 commit intomasterfrom
fix/ipns-append
May 5, 2015
Merged

fix append bug and overwrite/truncate bug#1164
jbenet merged 1 commit intomasterfrom
fix/ipns-append

Conversation

@whyrusleeping
Copy link
Member

No description provided.

@whyrusleeping whyrusleeping added the status/in-progress In progress label Apr 29, 2015
@whyrusleeping
Copy link
Member Author

was hoping that fuse was just being buggy like always on my machine, looks like tests are actually failing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pretty sure this line is what's causing tests to fail

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but that fixed other things... :(

computers are hard

@jbenet
Copy link
Member

jbenet commented Apr 29, 2015

was hoping that fuse was just being buggy like always on my machine, looks like tests are actually failing.

fuse doesn't run on travis, so its the dagmodifier

@whyrusleeping
Copy link
Member Author

@jbenet this one is good now!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trust me, this is right.
Maybe it deserves a comment...

an alternative fix to the truncate/append problem

Update ipns_unix.go
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure about the underlying structure--

  • why isn't this happening for all opens?
  • how come the same fi *File can be opened by multiple things and set the seek position to different spots.

my impression is that an open call should allocate a handle/descriptor of some sort with its own seek offset.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fuse kinda handles this for us, and now that I think about it, this seek(0) should really just go at the top of the function.

All reads at 'ReadAt' and all writes are 'WriteAt', so you dont have to worry about what offset your file descriptor is at. The seek(0) resets the cache buffer for the ipnsfs file, I should probably just have a Reset method on it instead of doing this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so it's not threadsafe. right? two simultaneous reads will read at different locations?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats the weird part, I've thrown a concurrent reads/writes stress test over this code, with the race detector on, and got nothing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with the race detector

did you check all the reads got what you expected? interleaved writes could happen, without meaning a golang race.

Thats the weird part

idk, sounds to me like this should have a separate handle per open with its own offest. should ask @tv42

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aall the actual reads and writes really are ReadAt / WriteAt style. Kernel keeps file offsets, doesn't even communicate them through FUSE. You seem to be doing that right.

The real thing you have to worry about is reads and writes happening at the same time. How does your fi.fi there like two Write calls at the same time?

Separate Handles is mostly useful for e.g. keeping track whether the handle was opened O_RDONLY or not, for example if your backing store is cheaper to open read-only than read-write.

The OpenTruncate logic there belongs in Setattr, the kernel will tell you when it wants to change the length of the file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tv42 does fuse allow concurrent writes to the same file? or concurrent reads?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, yeah. Ipnsfs.File is mutex protected. everything here is fine

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@whyrusleeping FUSE in itself has very few real guarantees about what cannot be concurrent. Different Linux versions, OSXFUSE, FreeBSD etc may enforce some of their own mutual exclusions, but I wouldn't rely on that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, i realized after asking that I already do all my locking in the ipnsfs.File methods

@whyrusleeping
Copy link
Member Author

@jbenet RFM

jbenet added a commit that referenced this pull request May 5, 2015
fix append bug and overwrite/truncate bug
@jbenet jbenet merged commit 688275b into master May 5, 2015
@jbenet jbenet removed the status/in-progress In progress label May 5, 2015
@jbenet jbenet deleted the fix/ipns-append branch May 5, 2015 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants