Skip to content

Avoid PermGen leak#49

Closed
elrodro83 wants to merge 1 commit intocglib:masterfrom
elrodro83:patch-1
Closed

Avoid PermGen leak#49
elrodro83 wants to merge 1 commit intocglib:masterfrom
elrodro83:patch-1

Conversation

@elrodro83
Copy link
Contributor

The key object put in the cache may be an instance of a dynamically generated class that references an application classloader. That reference causes said classloader to not be available for garbage collection, thus leaking the PermGen with old classes when that classloader is not needed anymore. This is mostly experienced in application restart scenarios, where each application has its own classloader managed in a long-lived container.

Attached is an exceprt from a heap dump that shoes the reference to the classloader that causes the issue:

screen shot 2015-11-10 at 11 00 50 am

@sameb
Copy link
Contributor

sameb commented Nov 10, 2015

Any way you can add a test that would have failed before?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is key always of java.lang.Class type?
If it is always a java.lang.Class, can you please use #getName() instead of #toString() as it would avoid producing java garbage?

Can you please add relevant comment explaining why you avoid using key as a, well, key?
That would make the intention clear for other developers who happen to touch this code later.

@elrodro83
Copy link
Contributor Author

Added a test case and improved the fix.

@elrodro83
Copy link
Contributor Author

Went back to the key.toString approach. The way keys are being generated should produce unique string representations.

The problem comes because the key holds references to the callbacks of the enhanced class that may hold references to whatever the callback needs to do. That reference in the callback, being referenced by the key is what pevents those classes from being unloaded. By getting the string from the key and not using it for anything else prevents these unwanted references from staying in the cache.

Also, with the previous WeakHashMap approach, the cache entry would be wiped out by any GC run, defeating its purpose, that's why i went back to the key.toString approach.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be faster if you just test that a single classloader gets unloaded via PhantomReference<ClassLoader> -> ReferenceQueue.
No need to wait for OOM, you just need to test that a single classloader can be unloaded.

@vlsi
Copy link
Contributor

vlsi commented Nov 11, 2015

Went back to the key.toString approach

Why not .getName?

@vlsi
Copy link
Contributor

vlsi commented Nov 11, 2015

Went back to the key.toString approach

Let me put that in another way: what will happen if two keys would accidentally happen to have the same #toString() representation?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use spaces, not tabs for indent

@elrodro83
Copy link
Contributor Author

From what i could see in the code, the toString in the keys are build with the same information that is used for equals and hashCode, all referenced classes being put as fully qualified.
Also, the key received by AbstractClassGenerator#create() is never a class. It may be a key form a KEY_FACTORY (for which applies the above statement) or a string/primitive type. So the only thing available that would fit all cases is #toString().

Anyway, I'm open to suggestions as long as the classloader leak issue can be fixed.

Thanks,
Rodro

@vlsi
Copy link
Contributor

vlsi commented Nov 11, 2015

What if usecache2.put(new WeakKey(key), new WeakReference(gen)); where WeakKey extends WeakReference and WeakKey defines equals(Object that) as return get() == that?

@elrodro83
Copy link
Contributor Author

I tried that approach, but the problem there would be that a CG run will remove the cache entry since the key is a "normal" object and not something that goes in PermGen. That would happen to often for the cache to be effective as a cache.

@vlsi
Copy link
Contributor

vlsi commented Nov 11, 2015

What if the key itself would hold a weak reference to the class?

@elrodro83
Copy link
Contributor Author

The problem is that the reference from the key is not to a class, it is (in this case) to the aspected object. If that were done weak, garbage collector could remove it anytime and that would reduce the efficiency of the cache.

@vlsi
Copy link
Contributor

vlsi commented Nov 11, 2015

Does this aspected object hold a reference to the class?
Does it need that reference? Can it be weak?

@elrodro83
Copy link
Contributor Author

The aspected object is an instance of the class loaded by the same classloader that the enhaced class is being generated in. I don't think a weak reference there would work, since it will eventually need to use the aspected object from the aspect generated with cglib.

@elrodro83
Copy link
Contributor Author

I found a way, as Vladimir pointed out before, to use the generated class name of the key instead of #toSting(). That change is pushed. I had to handle some cases where the key was a String already.
Please let me know if you have any other concerns with this change.

@vlsi
Copy link
Contributor

vlsi commented Nov 12, 2015

I'll have a look in a couple of days. It is somewhat hard to validate without opening the code in IDE.

@vlsi
Copy link
Contributor

vlsi commented Nov 13, 2015

@elrodro83 , I'm afraid, your fix relies on the fact "hashCodes never collide" which is apparently false.

@vlsi
Copy link
Contributor

vlsi commented Nov 13, 2015

@elrodro83 , see my approach to tackle the problem: #50

@elrodro83
Copy link
Contributor Author

@vlsi looking at your code, and testing with the stressHashCode property, i see that calls to getClassName may not be idempotent.

@elrodro83
Copy link
Contributor Author

Just looked at it, that's fine by me it it gets the fix for the leak. I'll close this PR and continue in #50

@elrodro83 elrodro83 closed this Nov 13, 2015
@elrodro83 elrodro83 mentioned this pull request Nov 14, 2015
@elrodro83
Copy link
Contributor Author

Have redone this solution in #54. This seems to have less impact than #50, but anyway, suggestions are accepted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants