Skip to content

Buffer: Use 256GB mmap size instead of MaxInt64#198

Merged
martinmr merged 3 commits intomasterfrom
ibrahim/buffer-fix
Sep 30, 2020
Merged

Buffer: Use 256GB mmap size instead of MaxInt64#198
martinmr merged 3 commits intomasterfrom
ibrahim/buffer-fix

Conversation

@jarifibrahim
Copy link
Contributor

@jarifibrahim jarifibrahim commented Sep 28, 2020

MaxInt64 is 9.2 Exabyte and the test fails with cannot allocate memory on my computer.
This PR also fixes the build (it is failing on master).


This change is Reviewable

Copy link
Contributor

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r1.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @anurags92, @jarifibrahim, @karlmcguire, and @manishrjain)


z/buffer.go, line 73 at r1 (raw file):

// Newbuffer is a helper utility, which creates a virtually unlimited Buffer in UseCalloc mode.
func NewBuffer(sz int) *Buffer {
	buf, err := NewBufferWith(sz, math.MaxInt32, UseCalloc)

so this is now 2GB. not virtually unlimited anymore. Is that ok?

Copy link
Contributor Author

@jarifibrahim jarifibrahim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @anurags92, @karlmcguire, @manishrjain, and @martinmr)


z/buffer.go, line 73 at r1 (raw file):

Previously, martinmr (Martin Martinez Rivera) wrote…

so this is now 2GB. not virtually unlimited anymore. Is that ok?

I can make it 100*math.maxInt32 or something. math.MaxInt64 crashes on my computer.

@jarifibrahim jarifibrahim changed the title Buffer: Use MaxInt32 instead of MaxInt64 Buffer: Use 100xMaxInt32 instead of MaxInt64 Sep 28, 2020
@jarifibrahim jarifibrahim changed the title Buffer: Use 100xMaxInt32 instead of MaxInt64 Buffer: Use 256GB mmap size instead of MaxInt64 Sep 29, 2020
@martinmr martinmr merged commit 163c5d4 into master Sep 30, 2020
@martinmr martinmr deleted the ibrahim/buffer-fix branch September 30, 2020 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants