Shard time is calculated by summing all test durations together. FTL doesn't report orchestrator setup/teardown. That means individual test times and overall shard times used by flank are incorrect.
- Work with FTL to get valid time data from the API
- Update sharding algorithm to use real time values
Example JUnit XML result with 3s per test overhead:
<testsuite name="" tests="2" failures="0" errors="0" skipped="0" time="8s" timestamp="2019-05-13T16:03:42" hostname="localhost">
<testcase name="test1" classname="com.example.ExampleTest" time="1.0"/>
<testcase name="test2" classname="com.example.ExampleTest" time="1.0"/>
</testsuite>
Total time in the JUnit XML is valid <testsuite time="8s". Individual test times don't include overhead.
Shard time is calculated by summing all test durations together. FTL doesn't report orchestrator setup/teardown. That means individual test times and overall shard times used by flank are incorrect.
Example JUnit XML result with 3s per test overhead:
Total time in the JUnit XML is valid
<testsuite time="8s". Individual test times don't include overhead.