Skip to content

Implementation-specific tests#981

Merged
neilalexander merged 11 commits intodevelopfrom
neilalexander/implspecific
Nov 17, 2020
Merged

Implementation-specific tests#981
neilalexander merged 11 commits intodevelopfrom
neilalexander/implspecific

Conversation

@neilalexander
Copy link
Contributor

This PR adds a test parameter implementation_specific, which instructs Sytest to run a test for a specific implementation only. It also marks the admin tests as Synapse-specific.

@neilalexander neilalexander requested a review from a team November 16, 2020 16:34
@anoadragon453
Copy link
Member

This generally looks fine to me. I'm a little wary of adding yet another way of excluding tests (on top of the test whitelist/blacklist), and generally the right solution to me sounds like we should just rewrite these tests in Synapse's unit test suite.

Then again, I don't particularly want to do the latter... so this seems like a good alternative. Though I'm not entirely sure if it could be useful in any other case.

@richvdh curious for your thoughts, but otherwise lgtm.

Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks nice otherwise.

run-tests.pl Outdated
Comment on lines +626 to +627
my @tokens = split /::/, $SERVER_IMPL;
if ( $tokens[0] ne $params{implementation_specific} ) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels slightly too magical and fragile; could we give the Homeserver classes an implementation_name method (see server_name as an existing example) or something instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added the implementation_name method to the HomeserverFactory, since that seems to be easily accessible from run_tests.pl.

Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a stub impl to HomeserverFactory itself, with some comments to document the interface?

(ideally HomeserverFactory would have better documentation in general)

Comment on lines +111 to +115
sub implementation_name
{
return "synapse";
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be unnecessary, as it will be inherited from SyTest::HomeserverFactory::Synapse ?

Comment on lines +73 to +76
sub implementation_name
{
return "dendrite";
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Member

@richvdh richvdh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@neilalexander neilalexander merged commit 3914ed6 into develop Nov 17, 2020
@neilalexander neilalexander deleted the neilalexander/implspecific branch November 17, 2020 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants