Skip to content

Refactor estimatedSize of table#1524

Merged
animesh2049 merged 5 commits intomasterfrom
animesh2049/refactor-table-size
Sep 23, 2020
Merged

Refactor estimatedSize of table#1524
animesh2049 merged 5 commits intomasterfrom
animesh2049/refactor-table-size

Conversation

@animesh2049
Copy link
Contributor

@animesh2049 animesh2049 commented Sep 17, 2020

Users reported mismatch between size of data on disk and size reported by dgraph. Due to compression, the real size of data on disk is much smaller than what we estimate from index.EstimatedSize.


This change is Reviewable

@CLAassistant
Copy link

CLAassistant commented Sep 17, 2020

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

@jarifibrahim jarifibrahim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @animesh2049, @ashish-goswami, @jarifibrahim, and @manishrjain)


table/table.go, line 451 at r1 (raw file):

	// Due to compression the real size on disk is much
	// smaller than what we estimate from index.EstimatedSize.
	t.estimatedSize = uint64(t.tableSize)

This should happen only if compression is enabled. If compression is diabled, the estimated size of the file is smaller than the on-disk size.


table/table_test.go, line 927 at r2 (raw file):

func TestOpenKVSize(t *testing.T) {
	opts := getTestTableOptions()
	table, err := OpenTable(buildTestTable(t, "foo", 1, opts), opts)

Add a seperate test for no-compression mode

Add the case when compression is off. In that scenario
`index.EstimatedSize` is more precise.
Copy link
Contributor Author

@animesh2049 animesh2049 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @ashish-goswami, @jarifibrahim, and @manishrjain)


table/table.go, line 451 at r1 (raw file):

Previously, jarifibrahim (Ibrahim Jarif) wrote…

This should happen only if compression is enabled. If compression is diabled, the estimated size of the file is smaller than the on-disk size.

Done.


table/table_test.go, line 927 at r2 (raw file):

Previously, jarifibrahim (Ibrahim Jarif) wrote…

Add a seperate test for no-compression mode

Done.

@animesh2049 animesh2049 merged commit 9b43ebe into master Sep 23, 2020
@animesh2049 animesh2049 deleted the animesh2049/refactor-table-size branch September 23, 2020 10:04
animesh2049 added a commit that referenced this pull request Sep 23, 2020
jarifibrahim pushed a commit that referenced this pull request Oct 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants