Skip to content

Scan stable plugins for named components upon install#92528

Merged
pgomulka merged 37 commits intoelastic:mainfrom
pgomulka:plugin_install_scanning
Jan 18, 2023
Merged

Scan stable plugins for named components upon install#92528
pgomulka merged 37 commits intoelastic:mainfrom
pgomulka:plugin_install_scanning

Conversation

@pgomulka
Copy link
Copy Markdown
Contributor

stable plugins not build with ES's gradle plugin will not have named_components.json file.
To allow these plugins to expose their named components, a scan can be performed upon install.

relates #88980

@pgomulka pgomulka added >enhancement :Core/Infra/CLI CLI utilities, scripts, and infrastructure v8.7.0 labels Dec 22, 2022
@pgomulka pgomulka self-assigned this Dec 22, 2022
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Hi @pgomulka, I've created a changelog YAML for you.

@pgomulka pgomulka marked this pull request as ready for review January 12, 2023 12:56
@elasticsearchmachine elasticsearchmachine added the Team:Core/Infra Meta label for core/infra team label Jan 12, 2023
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@rjernst rjernst self-requested a review January 12, 2023 15:27
@pgomulka pgomulka mentioned this pull request Jan 13, 2023
25 tasks
Copy link
Copy Markdown
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a general comment about how testing is done. I think we should avoid mocks (mockito), and then we shouldn't need to have instances of the utility classes or pass them into the actions.


@SuppressForbidden(reason = "sets java.io.tmpdir")
public InstallPluginActionTests(FileSystem fs, Function<String, Path> temp) {
assert "false".equals(System.getProperty("tests.security.manager")) : "-Dtests.security.manager=false has to be set";
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use assumeTrue or assumeFalse?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using just assert looks better. this is not a real assertion but a hint when running from IDE
with assumeTrue it would look confusing
assumeTrue("-Dtests.security.manager=false has to be set", "false".equals(System.getProperty("tests.security.manager"));

public class ClassReadersProvider {
private static final String MODULE_INFO = "module-info.class";

public List<ClassReader> ofClassPath() throws IOException {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What in this class requires some of these methods to be instance methods? I don't see a ctor or any instance member vars. I suspect this was for mocking; see my other comment. Note that it is especially odd to make some methods here instance methods, but still also have static utility methods.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this was only made an instance to allow mocking this

import org.elasticsearch.test.PosixPermissionsResetter;
import org.junit.After;
import org.junit.Before;
import org.mockito.Mockito;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should use mocks here. The rest of the tests here use fakes (we call them mocks, but they are custom mocks/filesystem impls). Ultimately what we are trying to test is just a couple cases, only one or two that need an actual components file, to test the integration.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only way to avoid the use of mockito is to use a "fake jar with no content" so that it relies on nothing being scanned, and nothing being written. (updated the PR)
The classes are being used and file is being created. And this is what that test is mainly trying to show, so perhaps it is ok.

if (hasNamedComponentFile) {
PluginTestUtil.writeNamedComponentsFile(structure, namedComponentsJSON());
}
writeJar(structure.resolve("plugin.jar"));// empty jar so that there is nothing to scan
Copy link
Copy Markdown
Member

@rjernst rjernst Jan 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we want something in the jar so that the named component file is not empty?

installPlugins(List.of(stablePluginZip), env.v1());

assertPlugin("stable1", pluginDir, env.v2());
// the test was using a jar with no classes, so the scanner did not find anything, but this is fine. that logic is tested elsewhere
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we care about slightly more than that. Is there a legitimate case for an empty file? I think we want to see that the generation was called and a real file was created, with at least one actual component. I would do that in writeStablePlugin using the InMemoryJavaCompiler.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair point, InMemoryJavaCompiler might help with test here and it helped me realise that plugin-cli has to have a dependency on plugin-api. I incorrectly assumed that it might be shipped with plugin, but it will not. Plugins only have 'implementation/compileOnly' scope on it.

@pgomulka pgomulka requested a review from rjernst January 18, 2023 06:31
Copy link
Copy Markdown
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one last cleanup

}

// pkg private for testing
public void setNamedComponentScanner(NamedComponentScanner scanner) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems leftover, it is no longer used?

@pgomulka pgomulka merged commit 2a7f61f into elastic:main Jan 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Core/Infra/CLI CLI utilities, scripts, and infrastructure >enhancement Team:Core/Infra Meta label for core/infra team v8.7.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants