Skip to content

Implement local lock cache, support querying it#1760

Merged
sinbad merged 11 commits into
locking-masterfrom
locking-wip/cache
Dec 14, 2016
Merged

Implement local lock cache, support querying it#1760
sinbad merged 11 commits into
locking-masterfrom
locking-wip/cache

Conversation

@sinbad

@sinbad sinbad commented Dec 12, 2016

Copy link
Copy Markdown
Contributor

The next incremental rewrite of the locking work; this time we're using our key value store to locally cache locks for the current user for more efficient access. This will be used in the next PR to provide efficient support for marking files read-only on checkout if not locked.

Also adds some minor API improvements such as returning a full Lock structure on acquiring a lock rather than just an Id.

@sinbad

sinbad commented Dec 12, 2016

Copy link
Copy Markdown
Contributor Author

I think with the merging of this PR, locking-master can be merged into master because it's a stable mid-stage. There will then be a few further staging posts between here and when the locking is completely finished.

@ttaylorr ttaylorr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, I left some thoughts about a potential further abstraction you could make along with some nit-picks.

Comment thread locking/locks.go
// path must be relative to the root of the repository
// Returns the lock id if successful, or an error
func (c *Client) LockFile(path string) (id string, e error) {
func (c *Client) LockFile(path string) (Lock, error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think returning a *Lock would be appropriate here, since this method doesn't guarantee that locking a file will succeed all of the time, and the zero-value Lock{} doesn't make sense.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I remember this was deliberately by value now. Returning a non-pointer is important because I don't want to allow the client to modify the original lock, because it's cached and therefore they could corrupt the cache by doing that. This outweighs the zero value Lock concern IMO. Yes I could return a pointer and copy it but it's unnecessary complexity. A zero value in the event of an error is idiomatic anyway (nil is just a zero value pointer after all).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't mind implementing a (l *Lock) Copy() *Lock function, but I don't feel strongly.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That depends on callers doing the right thing (not modifying return values from the locking client. I think cache corruption is a good enough reason to return a value.

Comment thread locking/locks.go Outdated
// that this lock was created. For most server implementations, this
// should be set to the instant at which the lock was initially
// received.
// LockedAt tells you when this lock was acquired.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/tells you/is the/

Comment thread locking/locks.go
// If localOnly = true, don't query the server & report only own local locks
func (c *Client) SearchLocks(filter map[string]string, limit int, localOnly bool) (locks []Lock, err error) {

if localOnly {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😻

Comment thread locking/locks.go
locks := make([]Lock, 0, len(cachedlocks))
for _, l := range cachedlocks {
// Manually filter by Path/Id
if (filterByPath && path != l.Path) ||

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I follow this logic, but I think we can make it clearer:

for _, l := range cachedLocks {
        matchesPath := path == l.Path
        matchesId := id == l.Id

        if (filterByPath && matchesPath) || (filterById && matchesId) {
                // ...
        }
}

This does add a level of indentation to the body of the loop, but I think the extra clarity is worth it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hate to be the optimisation nitpicker here but doing it my way avoids comparison operations in the case of !filterByPath or !filterById whereas your way requires the comparison all the time, so I prefer my way 😄 I think it's clear enough.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that the original is clear enough.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment thread locking/locks.go Outdated
// that won't be in a path)
const idKeyPrefix = "*id*://"

func (c *Client) encodeIdKey(id string) string {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if (c *Client) is the right receiver for this method. What do you think about wrapping tools/kv in this package with functions that understand *Lock and etc, and putting this method there?

package locking

const (
       idPrefix string = "*id*://"
)

type LockCache struct {
        kv *kv.Store
}

func (c *LockCache) CacheLock(l Lock) error {
        // ...
}

func (c *LockCache) encodeId(id string) string {
       if !c.isId(id) {
               return idPrefix+id
        }

       return id
}

func (c *LockCache) isId(key string) bool {
       return strings.HasPrefix(key, idPrefix)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this approach a lot. I think it's a nice, clear line of abstraction that we can draw. The *locking.Client would own a *LockCache underneath and delegate into the cache's public methods from there.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's a good shout.

Comment thread locking/locks_test.go Outdated
config.LocalGitStorageDir = oldStore
}()

client, err := NewClient(config.NewFrom(config.Values{}))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's another place where I think the approach I proposed above would yield a win. This test is testing the behavior of caching a lock, but creates a locking client to do so. Moving that behavior down into a more isolated part of the system would mean that this test can be closer to it's subject.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, it made it better :)

Comment thread locking/locks.go Outdated
func (c *Client) cacheUnlockByPath(filePath string) error {
ilock := c.cache.Get(filePath)
if ilock != nil {
lock := ilock.(*Lock)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that this is likely behavior, but I think that using the two-tuple variant of this cast would be safer in that it'd prevent an unlikely panic:

func (c *Client) cacheUnlockByPath(filePath string) error {
        ilock := c.cache.Get(filePath)
        if ilock == nil { // <- needed to prevent `(*Lock)(nil)`
                return nil
        }
        if l, ok := ilock.(*Lock); ok {
                // ...
        }

        return nil
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I've just collapsed this into:

if lock, ok := ilock.(*Lock); ok && lock != nil {

Since the actual casting of nil is safe anyway so long as we check.

Comment thread locking/locks.go Outdated
c.cache.Remove(idkey)
c.cache.Remove(lock.Path)
}
return nil

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should return an error if we couldn't find the lock? I think a sentinel value here would be fine:

package locking

var ErrMissingLock error = errors.New("locking: could not find lock")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always think it's better to fail silently when requesting to remove a non-existent item from any collection; this is idiomatic with core collections and what I would expect from an API.

Comment thread locking/locks.go Outdated
return nil
func (c *Client) cachedLocks() []Lock {
var locks []Lock
c.cache.Visit(func(key string, val interface{}) bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This API is awesome 👍

@technoweenie technoweenie left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some naming concerns, but that's it. If you feel strongly about your names, go ahead and merge. Or we can discuss it in the locking-master -> master PR (our last chance to finalize exported names) :)

Comment thread locking/cache.go Outdated
}

// Cache a successful lock for faster local lookup later
func (c *LockCache) CacheLock(l Lock) error {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having functions with a Cache prefix is redundant. Especially in the *Client usage: c.cache.CacheUnlockById(id). What do you think about method names like this:

  • CacheLock -> Set
  • CacheUnlockByPath -> DeleteByPath
  • CacheUnlockById -> DeleteById
  • CachedLocks -> GetAll or just All

@sinbad sinbad Dec 14, 2016

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point; to keep it distinct from kv I'll make it Add / RemoveByBlah / Locks.

Comment thread locking/locks.go
locks := make([]Lock, 0, len(cachedlocks))
for _, l := range cachedlocks {
// Manually filter by Path/Id
if (filterByPath && path != l.Path) ||

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that the original is clear enough.

Comment thread locking/locks.go Outdated
}
// Fetch locked files for the current committer and cache them locally
// This can be used to sync up locked files when moving machines
func (c *Client) fetchLocksToCache() error {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think something like refreshLocksCache() would be a better name.

@ttaylorr ttaylorr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sinbad - thanks for addressing my comments 😄. I'm with @technoweenie on this one: everything looks good with some naming changes, unless you feel strongly.

@sinbad sinbad merged commit 086779a into locking-master Dec 14, 2016
@sinbad sinbad deleted the locking-wip/cache branch December 14, 2016 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants