Skip to content

Adding system test files for Apache package#263

Merged
ycombinator merged 17 commits intoelastic:masterfrom
ycombinator:system-test-apache
Oct 6, 2020
Merged

Adding system test files for Apache package#263
ycombinator merged 17 commits intoelastic:masterfrom
ycombinator:system-test-apache

Conversation

@ycombinator
Copy link
Copy Markdown
Contributor

@ycombinator ycombinator commented Sep 1, 2020

What does this PR do?

Adds system test files for the apache package.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all datasets collect metrics or logs.

TODO

  • Add config file for apache/access dataset.
  • Add config file for apache/error dataset.
  • Add config file for apache/status dataset.

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Sep 1, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #263 event]

  • Start Time: 2020-10-05T19:59:07.837+0000

  • Duration: 9 min 53 sec

@ycombinator
Copy link
Copy Markdown
Contributor Author

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case you assume that the image exists somewhere in the registry. Do you think we can add a build definition here?

I'd like to enable error and access logging to log files, but unfortunately it's not possible.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mtojek I was keeping this for later because it brings up a few questions related to the spec. Right now the spec for the _dev/deploy folder only allows specific files and folders under it. It does not allow for having any arbitrary additional files or folders.

Adding a build definition to the Docker Compose file and a Dockerfile to go with it is not a problem. We can add the Dockerfile to the spec file referenced above. However, sometimes a Dockerfile may reference other files, e.g. configuration files that must be copied into the image while building it. How do we account for this nicely in the spec? The simplest way would be to set additionalContents: true. More involved but slightly stricter would be to allow a specific sub-folder under _dev/deploy for arbitrary files. Do you have any thoughts?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid we have to enable this option (additionalContents: true) for whole deploy or defining a specific type of deploy: docker and then enable it. This way we are safe if we want to introduce another deployment model (terraform?)

Copy link
Copy Markdown
Contributor Author

@ycombinator ycombinator Sep 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing we could do is expect a specific folder name under _dev/deploy corresponding to a supported deployment model. So a package could have this:

_dev/
  deploy/
   variants.yml (optional)
    docker/
      docker-compose.yml    (required)
      Dockerfile   (optional)
      (other, optional files as needed via `additionalContents: true`)

or:

_dev/
  deploy/
    terraform/
      TBD

or:

_dev/
  deploy/
    SOME_NEW_MODEL/
      TBD

And further we can probably enforce in the spec (in the future) that there must be exactly one sub-folder under _dev/deploy since a package should only specify one deployment model.

WDYT?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this model, but let's first focus on the docker runtime as this is the only one we have :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spec is adjusted in elastic/package-spec#49. Will do the dependency updates dance shortly.

@ycombinator
Copy link
Copy Markdown
Contributor Author

Blocked on elastic/kibana#77613.

@mtojek
Copy link
Copy Markdown
Contributor

mtojek commented Sep 18, 2020

BTW You can switch with development of system tests for Apache to elastic-package now and keep it there as a test package. Introducing potential fixes to elastic-package can be easier.

@mtojek
Copy link
Copy Markdown
Contributor

mtojek commented Oct 2, 2020

Please adjust the PR accordingly to changes introduced in #280 .

Main changes:

  1. Rebase against master.
  2. Rename "dataset" folder to "data_stream".
  3. Rename "config_templates" to "policy_templates" in package manifest file.

@ycombinator
Copy link
Copy Markdown
Contributor Author

Updated this PR based on changes in elastic/elastic-package#111. Ready for your review now, @mtojek. Thanks!

@ycombinator ycombinator marked this pull request as ready for review October 5, 2020 17:50
@elasticmachine
Copy link
Copy Markdown

Pinging @elastic/integrations (Team:Integrations)

@ycombinator ycombinator requested a review from mtojek October 5, 2020 17:51
@ycombinator
Copy link
Copy Markdown
Contributor Author

System tests are now running as part of CI! 🎉

Screen Shot 2020-10-05 at 12 58 14 PM

Copy link
Copy Markdown
Contributor

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job!

@ycombinator ycombinator merged commit ab5affd into elastic:master Oct 6, 2020
@ycombinator ycombinator deleted the system-test-apache branch October 6, 2020 05:34
@ph
Copy link
Copy Markdown
Contributor

ph commented Oct 6, 2020

I am super excited by this! Good job!

@EricDavisX @ruflin FYI

@ycombinator
Copy link
Copy Markdown
Contributor Author

FYI, I'm working on a HOWTO guide as well. Hoping to land that PR in this repo this week 🤞. Will notify Elastic package developers via the mailing list once that is done.

@ycombinator
Copy link
Copy Markdown
Contributor Author

HOWTO guide for system testing packages with elastic-package is now up: https://github.com/elastic/elastic-package/blob/master/docs/howto/system_testing.md#howto-writing-system-tests-for-a-package

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Team:Integrations Label for the Integrations team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants