Skip to content

Commit 61d478c

Browse files
tirantiran
authored
bpo-31399: Let OpenSSL verify hostname and IP address (#3462)
bpo-31399: Let OpenSSL verify hostname and IP The ssl module now uses OpenSSL's X509_VERIFY_PARAM_set1_host() and X509_VERIFY_PARAM_set1_ip() API to verify hostname and IP addresses. * Remove match_hostname calls * Check for libssl with set1_host, libssl must provide X509_VERIFY_PARAM_set1_host() * Add documentation for OpenSSL 1.0.2 requirement * Don't support OpenSSL special mode with a leading dot, e.g. ".example.org" matches "www.example.org". It's not standard conform. * Add hostname_checks_common_name Signed-off-by: Christian Heimes <christian@python.org>
1 parent 746cc75 commit 61d478c

File tree

15 files changed

+302
-73
lines changed

15 files changed

+302
-73
lines changed

‎Doc/library/ssl.rst‎

Lines changed: 40 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -146,9 +146,10 @@ Functions, Constants, and Exceptions
146146

147147
.. exception:: CertificateError
148148

149-
Raised to signal an error with a certificate (such as mismatching
150-
hostname). Certificate errors detected by OpenSSL, though, raise
151-
an :exc:`SSLCertVerificationError`.
149+
An alias for :exc:`SSLCertVerificationError`.
150+
151+
.. versionchanged:: 3.7
152+
The exception is now an alias for :exc:`SSLCertVerificationError`.
152153

153154

154155
Socket creation
@@ -430,8 +431,14 @@ Certificate handling
430431
of the certificate, is now supported.
431432

432433
.. versionchanged:: 3.7
434+
The function is no longer used to TLS connections. Hostname matching
435+
is now performed by OpenSSL.
436+
433437
Allow wildcard when it is the leftmost and the only character
434-
in that segment.
438+
in that segment. Partial wildcards like ``www*.example.com`` are no
439+
longer supported.
440+
441+
.. deprecated:: 3.7
435442

436443
.. function:: cert_time_to_seconds(cert_time)
437444

@@ -850,6 +857,14 @@ Constants
850857

851858
.. versionadded:: 3.5
852859

860+
.. data:: HAS_NEVER_CHECK_COMMON_NAME
861+
862+
Whether the OpenSSL library has built-in support not checking subject
863+
common name and :attr:`SSLContext.hostname_checks_common_name` is
864+
writeable.
865+
866+
.. versionadded:: 3.7
867+
853868
.. data:: HAS_ECDH
854869

855870
Whether the OpenSSL library has built-in support for Elliptic Curve-based
@@ -1075,6 +1090,12 @@ SSL sockets also have the following additional methods and attributes:
10751090
The socket timeout is no more reset each time bytes are received or sent.
10761091
The socket timeout is now to maximum total duration of the handshake.
10771092

1093+
.. versionchanged:: 3.7
1094+
Hostname or IP address is matched by OpenSSL during handshake. The
1095+
function :func:`match_hostname` is no longer used. In case OpenSSL
1096+
refuses a hostname or IP address, the handshake is aborted early and
1097+
a TLS alert message is send to the peer.
1098+
10781099
.. method:: SSLSocket.getpeercert(binary_form=False)
10791100

10801101
If there is no certificate for the peer on the other end of the connection,
@@ -1730,6 +1751,17 @@ to speed up repeated connections from the same clients.
17301751
The protocol version chosen when constructing the context. This attribute
17311752
is read-only.
17321753

1754+
.. attribute:: SSLContext.hostname_checks_common_name
1755+
1756+
Whether :attr:`~SSLContext.check_hostname` falls back to verify the cert's
1757+
subject common name in the absence of a subject alternative name
1758+
extension (default: true).
1759+
1760+
.. versionadded:: 3.7
1761+
1762+
.. note::
1763+
Only writeable with OpenSSL 1.1.0 or higher.
1764+
17331765
.. attribute:: SSLContext.verify_flags
17341766

17351767
The flags for certificate verification operations. You can set flags like
@@ -2324,6 +2356,10 @@ in this case, the :func:`match_hostname` function can be used. This common
23242356
check is automatically performed when :attr:`SSLContext.check_hostname` is
23252357
enabled.
23262358

2359+
.. versionchanged:: 3.7
2360+
Hostname matchings is now performed by OpenSSL. Python no longer uses
2361+
:func:`match_hostname`.
2362+
23272363
In server mode, if you want to authenticate your clients using the SSL layer
23282364
(rather than using a higher-level authentication mechanism), you'll also have
23292365
to specify :const:`CERT_REQUIRED` and similarly check the client certificate.

‎Doc/whatsnew/3.7.rst‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -568,6 +568,32 @@ can be set within the scope of a group.
568568
``'^$'`` or ``(?=-)`` that matches an empty string.
569569
(Contributed by Serhiy Storchaka in :issue:`25054`.)
570570

571+
ssl
572+
---
573+
574+
The ssl module now uses OpenSSL's builtin API instead of
575+
:func:`~ssl.match_hostname` to check host name or IP address. Values
576+
are validated during TLS handshake. Any cert validation error including
577+
a failing host name match now raises :exc:`~ssl.SSLCertVerificationError` and
578+
aborts the handshake with a proper TLS Alert message. The new exception
579+
contains additional information. Host name validation can be customized
580+
with :attr:`~ssl.SSLContext.host_flags`.
581+
(Contributed by Christian Heimes in :issue:`31399`.)
582+
583+
.. note::
584+
The improved host name check requires an OpenSSL 1.0.2 or 1.1 compatible
585+
libssl. OpenSSL 0.9.8 and 1.0.1 are no longer supported. LibreSSL is
586+
temporarily not supported until it gains the necessary OpenSSL 1.0.2 APIs.
587+
588+
The ssl module no longer sends IP addresses in SNI TLS extension.
589+
(Contributed by Christian Heimes in :issue:`32185`.)
590+
591+
:func:`~ssl.match_hostname` no longer supports partial wildcards like
592+
``www*.example.org``. :attr:`~ssl.SSLContext.host_flags` has partial
593+
wildcard matching disabled by default.
594+
(Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in
595+
:issue:`31399`.)
596+
571597
string
572598
------
573599

@@ -1120,6 +1146,12 @@ Other CPython implementation changes
11201146
emitted in the first place), and an explicit ``error::BytesWarning``
11211147
warnings filter added to convert them to exceptions.
11221148

1149+
* CPython' :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl.
1150+
OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer
1151+
supported. LibreSSL is temporarily not supported as well. LibreSSL releases
1152+
up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs.
1153+
1154+
11231155
Documentation
11241156
=============
11251157

‎Lib/asyncio/sslproto.py‎

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -590,12 +590,6 @@ def _on_handshake_complete(self, handshake_exc):
590590
raise handshake_exc
591591

592592
peercert = sslobj.getpeercert()
593-
if not hasattr(self._sslcontext, 'check_hostname'):
594-
# Verify hostname if requested, Python 3.4+ uses check_hostname
595-
# and checks the hostname in do_handshake()
596-
if (self._server_hostname and
597-
self._sslcontext.verify_mode != ssl.CERT_NONE):
598-
ssl.match_hostname(peercert, self._server_hostname)
599593
except BaseException as exc:
600594
if self._loop.get_debug():
601595
if isinstance(exc, ssl.CertificateError):

‎Lib/http/client.py‎

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1375,7 +1375,8 @@ def __init__(self, host, port=None, key_file=None, cert_file=None,
13751375
if key_file or cert_file:
13761376
context.load_cert_chain(cert_file, key_file)
13771377
self._context = context
1378-
self._check_hostname = check_hostname
1378+
if check_hostname is not None:
1379+
self._context.check_hostname = check_hostname
13791380

13801381
def connect(self):
13811382
"Connect to a host on a given (SSL) port."
@@ -1389,13 +1390,6 @@ def connect(self):
13891390

13901391
self.sock = self._context.wrap_socket(self.sock,
13911392
server_hostname=server_hostname)
1392-
if not self._context.check_hostname and self._check_hostname:
1393-
try:
1394-
ssl.match_hostname(self.sock.getpeercert(), server_hostname)
1395-
except Exception:
1396-
self.sock.shutdown(socket.SHUT_RDWR)
1397-
self.sock.close()
1398-
raise
13991393

14001394
__all__.append("HTTPSConnection")
14011395

‎Lib/ssl.py‎

Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,6 @@
148148
lambda name: name.startswith('CERT_'),
149149
source=_ssl)
150150

151-
152151
PROTOCOL_SSLv23 = _SSLMethod.PROTOCOL_SSLv23 = _SSLMethod.PROTOCOL_TLS
153152
_PROTOCOL_NAMES = {value: name for name, value in _SSLMethod.__members__.items()}
154153

@@ -172,6 +171,8 @@
172171
else:
173172
CHANNEL_BINDING_TYPES = []
174173

174+
HAS_NEVER_CHECK_COMMON_NAME = hasattr(_ssl, 'HOSTFLAG_NEVER_CHECK_SUBJECT')
175+
175176

176177
# Disable weak or insecure ciphers by default
177178
# (OpenSSL's default setting is 'DEFAULT:!aNULL:!eNULL')
@@ -216,9 +217,7 @@
216217
'!aNULL:!eNULL:!MD5:!DSS:!RC4:!3DES'
217218
)
218219

219-
220-
class CertificateError(ValueError):
221-
pass
220+
CertificateError = SSLCertVerificationError
222221

223222

224223
def _dnsname_match(dn, hostname):
@@ -473,6 +472,23 @@ def options(self):
473472
def options(self, value):
474473
super(SSLContext, SSLContext).options.__set__(self, value)
475474

475+
if hasattr(_ssl, 'HOSTFLAG_NEVER_CHECK_SUBJECT'):
476+
@property
477+
def hostname_checks_common_name(self):
478+
ncs = self._host_flags & _ssl.HOSTFLAG_NEVER_CHECK_SUBJECT
479+
return ncs != _ssl.HOSTFLAG_NEVER_CHECK_SUBJECT
480+
481+
@hostname_checks_common_name.setter
482+
def hostname_checks_common_name(self, value):
483+
if value:
484+
self._host_flags &= ~_ssl.HOSTFLAG_NEVER_CHECK_SUBJECT
485+
else:
486+
self._host_flags |= _ssl.HOSTFLAG_NEVER_CHECK_SUBJECT
487+
else:
488+
@property
489+
def hostname_checks_common_name(self):
490+
return True
491+
476492
@property
477493
def verify_flags(self):
478494
return VerifyFlags(super().verify_flags)
@@ -699,11 +715,6 @@ def pending(self):
699715
def do_handshake(self):
700716
"""Start the SSL/TLS handshake."""
701717
self._sslobj.do_handshake()
702-
if self.context.check_hostname:
703-
if not self.server_hostname:
704-
raise ValueError("check_hostname needs server_hostname "
705-
"argument")
706-
match_hostname(self.getpeercert(), self.server_hostname)
707718

708719
def unwrap(self):
709720
"""Start the SSL shutdown handshake."""

‎Lib/test/test_asyncio/test_events.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1148,11 +1148,13 @@ def test_create_server_ssl_match_failed(self):
11481148
with test_utils.disable_logger():
11491149
with self.assertRaisesRegex(
11501150
ssl.CertificateError,
1151-
"hostname '127.0.0.1' doesn't match 'localhost'"):
1151+
"IP address mismatch, certificate is not valid for "
1152+
"'127.0.0.1'"):
11521153
self.loop.run_until_complete(f_c)
11531154

11541155
# close connection
1155-
proto.transport.close()
1156+
# transport is None because TLS ALERT aborted the handshake
1157+
self.assertIsNone(proto.transport)
11561158
server.close()
11571159

11581160
@support.skip_unless_bind_unix_socket

‎Lib/test/test_ftplib.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -330,6 +330,9 @@ def _do_ssl_handshake(self):
330330
return
331331
elif err.args[0] == ssl.SSL_ERROR_EOF:
332332
return self.handle_close()
333+
# TODO: SSLError does not expose alert information
334+
elif "SSLV3_ALERT_BAD_CERTIFICATE" in err.args[1]:
335+
return self.handle_close()
333336
raise
334337
except OSError as err:
335338
if err.args[0] == errno.ECONNABORTED:

‎Lib/test/test_imaplib.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -485,7 +485,8 @@ def test_ssl_raises(self):
485485
ssl_context.load_verify_locations(CAFILE)
486486

487487
with self.assertRaisesRegex(ssl.CertificateError,
488-
"hostname '127.0.0.1' doesn't match 'localhost'"):
488+
"IP address mismatch, certificate is not valid for "
489+
"'127.0.0.1'"):
489490
_, server = self._setup(SimpleIMAPHandler)
490491
client = self.imap_class(*server.server_address,
491492
ssl_context=ssl_context)
@@ -874,7 +875,8 @@ def test_ssl_verified(self):
874875

875876
with self.assertRaisesRegex(
876877
ssl.CertificateError,
877-
"hostname '127.0.0.1' doesn't match 'localhost'"):
878+
"IP address mismatch, certificate is not valid for "
879+
"'127.0.0.1'"):
878880
with self.reaped_server(SimpleIMAPHandler) as server:
879881
client = self.imap_class(*server.server_address,
880882
ssl_context=ssl_context)

‎Lib/test/test_poplib.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,9 @@ def _do_tls_handshake(self):
176176
return
177177
elif err.args[0] == ssl.SSL_ERROR_EOF:
178178
return self.handle_close()
179+
# TODO: SSLError does not expose alert information
180+
elif "SSLV3_ALERT_BAD_CERTIFICATE" in err.args[1]:
181+
return self.handle_close()
179182
raise
180183
except OSError as err:
181184
if err.args[0] == errno.ECONNABORTED:

‎Lib/test/test_ssl.py‎

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -988,6 +988,19 @@ def test_verify_mode_protocol(self):
988988
self.assertEqual(ctx.verify_mode, ssl.CERT_REQUIRED)
989989
self.assertTrue(ctx.check_hostname)
990990

991+
def test_hostname_checks_common_name(self):
992+
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
993+
self.assertTrue(ctx.hostname_checks_common_name)
994+
if ssl.HAS_NEVER_CHECK_COMMON_NAME:
995+
ctx.hostname_checks_common_name = True
996+
self.assertTrue(ctx.hostname_checks_common_name)
997+
ctx.hostname_checks_common_name = False
998+
self.assertFalse(ctx.hostname_checks_common_name)
999+
ctx.hostname_checks_common_name = True
1000+
self.assertTrue(ctx.hostname_checks_common_name)
1001+
else:
1002+
with self.assertRaises(AttributeError):
1003+
ctx.hostname_checks_common_name = True
9911004

9921005
@unittest.skipUnless(have_verify_flags(),
9931006
"verify_flags need OpenSSL > 0.9.8")
@@ -1511,6 +1524,16 @@ def test_bad_idna_in_server_hostname(self):
15111524
ctx.wrap_bio(ssl.MemoryBIO(), ssl.MemoryBIO(),
15121525
server_hostname="xn--.com")
15131526

1527+
def test_bad_server_hostname(self):
1528+
ctx = ssl.create_default_context()
1529+
with self.assertRaises(ValueError):
1530+
ctx.wrap_bio(ssl.MemoryBIO(), ssl.MemoryBIO(),
1531+
server_hostname="")
1532+
with self.assertRaises(ValueError):
1533+
ctx.wrap_bio(ssl.MemoryBIO(), ssl.MemoryBIO(),
1534+
server_hostname=".example.org")
1535+
1536+
15141537
class MemoryBIOTests(unittest.TestCase):
15151538

15161539
def test_read_write(self):
@@ -2536,8 +2559,9 @@ def test_check_hostname(self):
25362559
with server:
25372560
with client_context.wrap_socket(socket.socket(),
25382561
server_hostname="invalid") as s:
2539-
with self.assertRaisesRegex(ssl.CertificateError,
2540-
"hostname 'invalid' doesn't match 'localhost'"):
2562+
with self.assertRaisesRegex(
2563+
ssl.CertificateError,
2564+
"Hostname mismatch, certificate is not valid for 'invalid'."):
25412565
s.connect((HOST, server.port))
25422566

25432567
# missing server_hostname arg should cause an exception, too

0 commit comments

Comments
 (0)