Skip to content

Accessing previous versions of tree leaks memory #68

@silasdavis

Description

@silasdavis

When a version is saved SaveBranch is called: https://github.com/tendermint/iavl/blob/master/nodedb.go#L133

SaveBranch both saves the nodes to disk and unloads the nodes from memory. This is nice/necessary because it means all we store in memory as our versioned history grows is a pointer (hash) two the root of each older version.

However if we access values from a previous tree via GetVersioned (btw we could really do with a IterateVersioned or just GetTree(version) because currently hanging on to a previous tree is awkward but possible by calling Tree() the right time) then the necessary nodes to access our value are lazy-loaded. However once loaded they are never unloaded which is a memory leak. We cannot call SaveBranch to unload the tree because the nodeDB is private.

Context: we hang on to the previous tree and use it as a read-only tree that can run long iterations on without interfering with writes. It's possible that we load large portions of the tree of many previous versions. We also currently store unlimited versions, so these things together mean we will leak memory unless we can unload.

Allow SaveBranch would do the job it doesn't seem quite right for this purpose. Can we expose the ability to unload a Tree.

On a more general note, it feels like some useful functionality is hidden by being unexported. For example access to previous trees or operations on the NodeDB. It would be nice to see a slightly more permissive policy to exporting types and functions that would allow IAVL to be used in ways that might not have been anticipated by the user. I realise it is a balance, but currently it feels a little closed off to reasonable extension.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions