I am trying to determine if a SNMP packet is TLS or plain text TCP and have a trap packet that starts with these bytes: 30 82 0c 48 02 01 01 04

Expected behavior

SSLHandler.isEncrypted should return false as trap is not encrypted

Actual behavior

It returns true. This is because the getEncryptedPacketLength() method returns what it calculated as the packetLength.
Initially the packet is identified as not tls because the first byte is not an SSL_CONTENT_TYPE.
Later on in the if (!tls) stanza the following variables are set.
headerLength = 3
majorVersion = 2
packetLength=12422

And because the packetLength <= headerLength, the packetLength is returned rather then NOT_ENCRYPTED. A possible solution would be to add an optional parameter to specify if checking for SSLv2 is disabled.

If you assume that the incoming TCP stream cannot be SSLv2 and omit this check then the method returns NOT_ENCRYPTED as desired.

Steps to reproduce

Create a ByteBuf with the above bytes and call SSLHanlder.isEncypted()

Minimal yet complete reproducer code (or URL to code)

import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.handler.ssl.SslHandler;

public class ShortTest {

public static void main(String[] args) {
	
	ByteBuf buf = Unpooled.buffer();

	byte[] data = new byte[] {0x30, (byte)0x82, 0x0c, 0x48, 0x02, 0x01, 0x01, 0x01};
	buf.writeBytes(data);
	
	if (SslHandler.isEncrypted(buf)) {
		System.out.println("data is ssl or tls");
	} else {
		System.out.println("data is not ssl or tls");
	}
}

}### Netty version
Version 4.1.109

JVM version (e.g. java -version)

openjdk version "1.8.0_412"
OpenJDK Runtime Environment (build 1.8.0_412-b08)
OpenJDK 64-Bit Server VM (build 25.412-b08, mixed mode)

OS version (e.g. uname -a)

[14:35:06 elise@joy3 bin]$ uname -a
Linux joy3.tavve.com 3.10.0-327.36.2.el7.x86_64 #1 SMP Mon Oct 10 23:08:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux