Issue overview

When same measure is run multiple times output various if all calls for model objects are not explicitly sorted. While oder of IDF file does not matter, complex measures can have different behavior based on order changes which in turn changes the IDF and simulation results. This results in results in non deterministic results from CI.

Current Behavior

Below is the output from two measure test runs that makes and then returns 10 spaces.
https://github.com/DavidGoldwasser/measure_sandbox/tree/osm_object_return_order/create_and_report_model_objects

Run A:
The building finished with 10 spaces.
INFO MESSAGES
Space 1 was added.
Space 2 was added.
Space 3 was added.
Space 4 was added.
Space 5 was added.
Space 6 was added.
Space 7 was added.
Space 8 was added.
Space 9 was added.
Space 10 was added.
Space 9 is in the model.
Space 6 is in the model.
Space 5 is in the model.
Space 4 is in the model.
Space 7 is in the model.
Space 3 is in the model.
Space 10 is in the model.
Space 8 is in the model.
Space 2 is in the model.
Space 1 is in the model.

Run B:
INFO MESSAGES
Space 1 was added.
Space 2 was added.
Space 3 was added.
Space 4 was added.
Space 5 was added.
Space 6 was added.
Space 7 was added.
Space 8 was added.
Space 9 was added.
Space 10 was added.
Space 9 is in the model.
Space 8 is in the model.
Space 6 is in the model.
Space 4 is in the model.
Space 5 is in the model.
Space 10 is in the model.
Space 7 is in the model.
Space 3 is in the model.
Space 2 is in the model.
Space 1 is in the model.

Expected Behavior

I don't necessarily expect the spaces to be returned in order 1-10 but it would be nice if they were returned in the same order in multiple runs. Without this it may be necessary to always sort when requesting any objects to avoid non-deterministic testing and simulation results. I demonstrated this with spaces but I believe happens with any object type. I have not but can test seed model that already has spaces instead of adding them in the measure.

Steps to Reproduce

Run measure test
https://github.com/DavidGoldwasser/measure_sandbox/blob/osm_object_return_order/create_and_report_model_objects/tests/create_and_report_model_objects_test.rb

Possible Solution

Always sort when object is requested, or find another way to assure order is deterministic. For example using UUID of object would not be good as that isn't deterministic.

Details

Environment

Some additional details about your environment for this issue (if relevant):

• Platform (Operating system, version):
• Version of OpenStudio (Confirmed with 2.9.1 and 3.0

Context

Creating complications in variation of simulation results and complications in CI.