-
Notifications
You must be signed in to change notification settings - Fork 771
Integ test bloat #4375
Description
Security issue notifications
If you discover a potential security issue in s2n we ask that you notify
AWS Security via our vulnerability reporting page. Please do not create a public github issue.
Problem:
Our integ tests are bloated, especially in the test_happy_path file. Having so many parameters can lead us to combinatorially increasing the amount of tests running, but not necessarily increasing actual coverage.
For example, we often test s2n against itself in the integ tests. However, the command line of s2nc/d won't change for some of the parameters we have, like protocol. The user doesn't have much control over what protocol gets negotiated ---which is a good thing-- but it means that any test between s2n and s2n that purports to negotiate sslv3, tls10, tls11 and tls12 will be running the same test over and over.
Solution:
I don't know what the solution here is. I kind of want to say "don't use s2n as a provider against itself" but that's probably not a good solution. We can minimally remove some of the tests that don't actually increase test coverage.
It would be kind of cool if we had some type of solution that checked that each parameter combo produced unique command line output. That might be a little over-engineered though.