Skip to content

We now pool ContentManager scratch buffers#5921

Merged
harry-cpp merged 3 commits intoMonoGame:developfrom
Jjagg:scratchpool
Mar 27, 2020
Merged

We now pool ContentManager scratch buffers#5921
harry-cpp merged 3 commits intoMonoGame:developfrom
Jjagg:scratchpool

Conversation

@Jjagg
Copy link
Copy Markdown
Contributor

@Jjagg Jjagg commented Sep 3, 2017

As discussed in #5341. (edit for autoclose: fixes #5341)

I don't think this is right as is. Consider the case where a user loads incrementally larger assets. For each asset a new scratch buffer will be created and returned to the pool, staying in memory until game exit.

Maybe it would be better to use a slightly modified version of ByteBufferPool that deletes the smallest buffer from the pool whenever Get is called and there are no buffers that are large enough. That way the pool size won't grow beyond the number of threads that use a buffer from it simultaneously (most often 1).

cc @nkast

@nkast
Copy link
Copy Markdown
Contributor

nkast commented Sep 3, 2017

I agree 100%.

Here is a bufferPool I'am testing and works exactly like that.
https://github.com/nkast/MonoGame/blob/beac8d7e7ae87877d1d282ecd3208594ae00c2cb/MonoGame.Framework/Content/ContentManager.cs#L490-L532

The only danger is if the user hit it with dozens or hundreds of threads at once. Should we use a Semaphore or something there?

@Jjagg
Copy link
Copy Markdown
Contributor Author

Jjagg commented Sep 3, 2017

You mean to allow a maximum of N threads to use a buffer from the pool at once? We could do that, it would certainly make this more foolproof :)

@nkast
Copy link
Copy Markdown
Contributor

nkast commented Sep 3, 2017

yes, it's an extreme case of course.
or we can drop returned buffers when the pool already has reached a maximum number.

Edit
Any number of threads is bound to hit the disk bottleneck. I think it's best to halt them until a buffer is freed rather than keep allocating more buffers and then drop them.

@Jjagg
Copy link
Copy Markdown
Contributor Author

Jjagg commented Sep 4, 2017

I prefer the second option! I'll close this since you already implemented the better pool.

@nkast
Copy link
Copy Markdown
Contributor

nkast commented Sep 4, 2017

I kind of stuck on this issue and then forgot about it.

@nkast
Copy link
Copy Markdown
Contributor

nkast commented Sep 4, 2017

no, I'am ok with it.


internal void ReturnScratchBuffer(byte[] buffer)
{
_scratchBufferPool.Return(buffer);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just making _scratchBufferPool internal and let people call it directly? Just to avoid these redirect methods. Just make ByteBufferPool ctor take an optional minimum size.

Copy link
Copy Markdown
Contributor

@nkast nkast Sep 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if it's best to modify ByteBufferPool or use a second pool and let the two take different roads for contentManager & soundEffects. In every case I prefer having a layer that allows us to be more flexible with the internal implementation. Maybe having an IBufferPool with Get()/Return() or something like that.

@nkast
Copy link
Copy Markdown
Contributor

nkast commented Sep 4, 2017

@Jjagg
My branch is not ready to submit, I just took a pool class I had, paste it in ContentManager.cs and modify it to get the 1 buffer per thread as you described. Ideally, I was hopping to have 1 buffer per Core - max.

@Jjagg
Copy link
Copy Markdown
Contributor Author

Jjagg commented Sep 4, 2017

@nkast We can modify BBP to have a maximum number of items in the pool and a minimal buffer size. Maximum pool size for ContentManager can be set to Environment.ProcessorCount. That way we only need one implementation.

@Jjagg
Copy link
Copy Markdown
Contributor Author

Jjagg commented Sep 4, 2017

ContentManager.ScratchBufferPool is now exposed so ContentReaders can use it directly. I added a minimum buffer size and maximum pool size option and set the max pool size for the scratchbuffers to Environment.ProcessorCount.

Anything left to do? This seems like a good solution to me :)
EDIT: Maybe we should still also remove buffers from the pool, because there's still unnecessary amounts of memory usage when users load incrementally larger assets. For example with 8 logical cores, even if users only use 1 ContentManager they can get 8 large scratchbuffers stuck in memory as long as the game is running.

@Jjagg
Copy link
Copy Markdown
Contributor Author

Jjagg commented Sep 4, 2017

I modified the ByteBufferPool to remove a buffer when it does not have a large enough buffer for a Get call as explained in my first post. This ensures the pool does not grow larger than the number of buffers in use simultaneously.

@nkast, @tomspilman Sounds good?

@nkast
Copy link
Copy Markdown
Contributor

nkast commented Sep 4, 2017

This ensures the pool does not grow larger than the number of buffers in use simultaneously.

Is that desirable for SoundEffectInstances ?
maybe we can adjust that with a _minBuffers.

Other than that those changes will work very well for the ContentManager.

@Jjagg
Copy link
Copy Markdown
Contributor Author

Jjagg commented Sep 4, 2017

I think it is. We use the ByteBufferPool for DynamicSoundEffect instances. The buffers there are not immediately returned like the ScratchBufferPool buffers, but when the DSFX is stopped. So the pool will grow to the maximum number of DynamicSoundEffect instances playing at the same time and no larger.

@Jjagg
Copy link
Copy Markdown
Contributor Author

Jjagg commented Oct 6, 2017

Bump! IMO this is good to go like this :)

@Jjagg Jjagg requested review from dellis1972 and tomspilman April 19, 2018 01:45
@Jjagg
Copy link
Copy Markdown
Contributor Author

Jjagg commented Apr 19, 2018

Bumping because I think someone might've ran into the same issue as #5341 (which this fixes): http://community.monogame.net/t/contentmanger-not-unloading-properly/10550

@Jjagg
Copy link
Copy Markdown
Contributor Author

Jjagg commented Mar 27, 2020

@harry-cpp This is good to go!

@harry-cpp harry-cpp merged commit 1d179e0 into MonoGame:develop Mar 27, 2020
@tomspilman tomspilman changed the title Pool ContentManager scratch buffers We now pool ContentManager scratch buffers Aug 1, 2020
@tomspilman tomspilman added this to the 3.8 Release milestone Aug 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v3.5.1+ ContentManager.Unload() doesn't seem to free up memory

4 participants