List of feature to add to testlusters:

Now that we have a fair amount of features in testclusters and module and most plugin tests are using it, time to make a list of what's left.

• support security with username and password - by convention [1]
• support security with https [2]
• DSL for adding users [3]
• add support (and tests) for all versions newer then 6.0 [4]
• add support for upgrading clusters or individual nodes [5]

List of projects to convert [with required features]:

• client/rest-high-level, (elasticsearch.rest-test) [1]
• docs (docs-test)
• plugins
  • repository-hdfs ( elasticsearch-rest-test , elasticsearch.esplugin )
  • examples (elasticsearch.esplugin)
    • custom-settings
    • custom-suggester
    • painless-whitelist
    • rescore
    • rest-handler
    • script-expert-scrolling
    • security-authorization-engine [3]
• qa
  • ccs-unavailable-clusters ( elasticsearch-standalone-rest-test )
  • die-with-dignity (elasticsearch.esplugin) -- requires exposing the pid of the ElasticsearchNode
  • :[x] imp: evil-tests (elasticsearch.standalone-test)
  • full-cluster-restart [4, 5]
  • mixed-cluster [4, 5]
  • multi-cluster-search
  • rolling-upgrade [4, 5]
  • smoke-test-client
  • smoke-test-http
  • smoke-test-ingest-disabled
  • smoke-test-ingest-with-all-dependencies
  • smoke-test-multinode
  • smoke-test-plugins
  • unconfigured-node-name - might be better to move it to a packaging test as it's a snowflake that requires defaults such as discoery and cluster name to be changed
  • wildfly
• x-pack
  • docs [1]
  • plugin
    • ccr/qa
    • data-frame/qa/single-node-tests [1]
    • graph/qa/with-security [3]
    • ilm/qa
    • ml/qa/
    • sql/qa
  • qa
    • core-rest-tests-with-security [1]
    • full-cluster-restart [1, 4, 5]
    • kerberos-tests [1]
    • multi-cluster-search-security [1]
    • multi-node [3]
    • reindex-tests-with-security [1]
    • rolling -uograde [1 , 4, 5]
    • rolling-upgrade-basic [1, 4 ,5]
    • rolling-upgrade-multi-cluster [4, 5]
    • saml-idp-tests [1]
    • security-client-tests [3]
    • security-example-spi-extension [1]
    • security-migrate-tests [3]
    • security-setup-password-tests [1]
    • smoke-test-monitoring-with-watcher
    • smoke-test-plugins [1]
    • smoke-test-plugins-ssl [3, 6]
    • smoke-test-security-with-mustache [1]
    • smoke-test-watcher
    • smoke-test-watcher-with-security [3]
    • third-party/jira
    • third-party/pagerduty
    • third-party/slack

We are starting to see the light at the end of the tunnel.