Skip to content

Integ test bloat #4375

@maddeleine

Description

@maddeleine

Security issue notifications

If you discover a potential security issue in s2n we ask that you notify
AWS Security via our vulnerability reporting page. Please do not create a public github issue.

Problem:

Our integ tests are bloated, especially in the test_happy_path file. Having so many parameters can lead us to combinatorially increasing the amount of tests running, but not necessarily increasing actual coverage.
For example, we often test s2n against itself in the integ tests. However, the command line of s2nc/d won't change for some of the parameters we have, like protocol. The user doesn't have much control over what protocol gets negotiated ---which is a good thing-- but it means that any test between s2n and s2n that purports to negotiate sslv3, tls10, tls11 and tls12 will be running the same test over and over.

Solution:

I don't know what the solution here is. I kind of want to say "don't use s2n as a provider against itself" but that's probably not a good solution. We can minimally remove some of the tests that don't actually increase test coverage.

It would be kind of cool if we had some type of solution that checked that each parameter combo produced unique command line output. That might be a little over-engineered though.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions