Conversation
A cursory review of Box2D and our use of it lead me to figure out that the leaked bodies contain references to our objects (creatures, projectiles, food, etc.) that are set using `setUserData(this)`. Rather than hack the library to fix its leaks, instead, in each class's respective `killBody()` function, I now `setUserData(null)` to remove the reference to our objects. Since the bodies are still leaked and sometimes even come into "contact," I had to check if either body is null in our `endContact()` function and simply return if that is the case. This has dropped our memory usage significantly: less than 50 MB after 16 generations.
|
@BIOEvolveTD I believe this demonstrates that my hypothesis was correct, and the presentation is a pretty detailed write-up. I can link to it in our wiki if you'd like. Where would it go? @tsoule88 This ended up being a very elegant solution that does not, as far as I can tell, introduce any new issues, nor did it require anything very "hacky." We simply remove the references to our objects when we're done with them. @higles Please test this for me on your machine, as it's a different platform than mine. |
|
Oh and @dstreett, like you surmised, it was simple once I figured it out. |
|
Awesome work! Not sure where it would go on the wiki, but I feel strongly that it should be on there somewhere. Terry? Is this merged with master now? From: Andrew Schwartzmeyer notifications@github.com @BIOEvolveTDhttps://github.com/BIOEvolveTD I believe this demonstrates that my hypothesis was correct, and the presentation is a pretty detailed write-up. I can link to it in our wiki if you'd like. Where would it go? @tsoule88https://github.com/tsoule88 This ended up being a very elegant solution that does not, as far as I can tell, introduce any new issues, nor did it require anything very "hacky." We simply remove the references to our objects when we're done with them. @higleshttps://github.com/higles Please test this for me on your machine, as it's a different platform than mine. Reply to this email directly or view it on GitHubhttps://github.com//pull/203#issuecomment-93155247. |
|
Ha! I thought that usage looked funky. I knew I needed to test more, but I'm excited! The memory leak is still fixed, but the usage is actually ~500 MB on average. I would like some more testing to be absolutely certain this fixes it. Granted, the bodies still leak, so there will always be a slight memory leak, but without objects being cleaned up, I think we can leave it. |
|
I'm running it now, and it looks like 1.8Gb of usage at Gen 12 for me, but not crashing yet. From: Andrew Schwartzmeyer notifications@github.com Ha! I thought that usage looked funky. I knew I needed to test more, but I'm excited! The memory leak is still fixed, but the usage is actually ~500 MB on average. I would like some more testing to be absolutely certain this fixes it. Granted, the bodies still leak, so there will always be a slight memory leak, but with out objects being cleaned up, I think we can leave it. Reply to this email directly or view it on GitHubhttps://github.com//pull/203#issuecomment-93164113. |
|
Generation 28: Sadly this does not fix it 100% because the library itself still leaks body objects. But I believe it takes care of, say, 90% of the leak. |
|
Generation 29, 2.16 Gb, still running. From: Andrew Schwartzmeyer notifications@github.com Generation 28: Sadly this does not fix it 100% because the library itself still leaks body objects. But I believe it takes care of, say, 90% of the leak. Reply to this email directly or view it on GitHubhttps://github.com//pull/203#issuecomment-93167537. |
|
@BIOEvolveTD My guess is that you're seeing the allocated heap size (orange in my graph) and not the used size (blue), which would line up with what I'm seeing. We don't really want to have to fix the library. And I think replacing the world each wave just to clear the leaked bodies may be a bit drastic; but if it comes to it, we know what to do. |
|
I'm positively giddy that I can run this thing to generation 30! From: Andrew Schwartzmeyer notifications@github.com @BIOEvolveTDhttps://github.com/BIOEvolveTD My guess is that you're seeing the allocated heap size (orange in my graph) and not the used size (blue), which would line up with what I'm seeing. We don't really want to have to fix the library. And I think replacing the world each wave just to clear the leaked bodies may be a bit drastic; but if it comes to it, we know what to do. Reply to this email directly or view it on GitHubhttps://github.com//pull/203#issuecomment-93168646. |
|
finally crashed at generation 37. A record run for me. From: Andrew Schwartzmeyer notifications@github.com @BIOEvolveTDhttps://github.com/BIOEvolveTD My guess is that you're seeing the allocated heap size (orange in my graph) and not the used size (blue), which would line up with what I'm seeing. We don't really want to have to fix the library. And I think replacing the world each wave just to clear the leaked bodies may be a bit drastic; but if it comes to it, we know what to do. Reply to this email directly or view it on GitHubhttps://github.com//pull/203#issuecomment-93168646. |
|
Ah! Damn it Box2D, why-you-leak-so-much. If you only got to 37, although a massive improvement, we may still need to take measures to eliminate the leaked bodies. I am looking into it now. |
|
I'm at 30+ minutes and 50 generations, which is long enough to clearly see the memory leak of the Box2D bodies themselves. I think this fix should definitely be merged now so people can get on with Tuesday's assignment without as many issues, with a better fix forthcoming. |
|
agreed. The good news? In one run I clearly see evidence of stabilizing selection on armor, with an optimum armor value of 11. From: Andrew Schwartzmeyer notifications@github.com I'm at 30+ minutes and 50 generations, which is long enough to clearly see the memory leak of the Box2D bodies themselves. I think this fix should definitely be merged now so people can get on with Tuesday's assignment without as many issues, with a better fix forthcoming. Reply to this email directly or view it on GitHubhttps://github.com//pull/203#issuecomment-93172084. |
Fix Box2D memory leak (mostly)
|
Fantastic. There’s probably a note (or several) on-line somewhere reminding users to set userdata to null that I missed. A link from the wiki is a good idea. What about a new page, from the main page, on the coding process in general (I have some comments that could go there including a brief discussion of our choices to use Processing, github, etc.) and a link from that page to this particular issue? |
|
@tsoule88 Ha, I don't think they intend for this to have to be done. From the library for
It is clear that there shouldn't be a leak of these bodies (further research showed that Box2D waits until the next step to actually destroy them), but there is. So no worries, this was a workaround we had to find ourselves. I am all for a coding process page! Great idea. |
A cursory review of Box2D and our use of it lead me to figure out that the leaked bodies contain references to our objects (creatures, projectiles, food, etc.) that are set using
setUserData(this).Rather than hack the library to fix its leaks, instead, in each class's respective
killBody()function, I nowsetUserData(null)to remove the reference to our objects.Since the bodies are still leaked and sometimes even come into "contact," I had to check if either body is null in our
endContact()function and simply return if that is the case.This has dropped our memory usage significantly: less than 50 MB after 16 generations.See the bottom of my presentation for screenshots of this fix in action.
This finishes #200 and #186.
Any additional body types added to the game must follow the same pattern detailed here.