Add tests to check that SLX file versions do not exceed R2020b #919

H0R5E · 2022-07-25T12:35:14Z

This PR adds some new tests to check whether the versions of any SLX files stored in the repository exceed R2020b.

Having these tests will make sure that incompatible versions are not merged accidentally.

H0R5E · 2022-07-25T13:15:14Z

The tests are revealing 5 files that exceed the maximum version:

examples/OSWEC/OSWEC.slx
examples/RM3/RM3.slx
examples/RM3FromSimulink/RM3FromSimulink.slx
source/lib/WEC-Sim/WECSim_Lib_Body_Elements.slx
source/lib/WEC-Sim/WECSim_Lib_Frames.slx

================================================================================
  Verification failed in SLXVersionTest/testSLXVersion(Paths=p__home_runner_work_WEC_Sim_WEC_Sim_examples_OSWEC_OSWEC_slx).
      ----------------
      Test Diagnostic:
      ----------------
      /home/runner/work/WEC-Sim/WEC-Sim/examples/OSWEC/OSWEC.slx model version R2021a exceeds R2020b
      ---------------------
      Framework Diagnostic:
      ---------------------
      verifyTrue failed.
      --> The value must evaluate to "true".
  
      Actual Value:
        logical
  
         0
      ------------------
      Stack Information:
      ------------------
      In /home/runner/work/WEC-Sim/WEC-Sim/tests/devTests/SLXVersionTest.m (SLXVersionTest.testSLXVersion) at 16
  ================================================================================
  .
  ================================================================================
  Verification failed in SLXVersionTest/testSLXVersion(Paths=p__home_runner_work_WEC_Sim_WEC_Sim_examples_RM3_RM3_slx).
      ----------------
      Test Diagnostic:
      ----------------
      /home/runner/work/WEC-Sim/WEC-Sim/examples/RM3/RM3.slx model version R2021a exceeds R2020b
      ---------------------
      Framework Diagnostic:
      ---------------------
      verifyTrue failed.
      --> The value must evaluate to "true".
  
      Actual Value:
        logical
  
         0
      ------------------
      Stack Information:
      ------------------
      In /home/runner/work/WEC-Sim/WEC-Sim/tests/devTests/SLXVersionTest.m (SLXVersionTest.testSLXVersion) at 16
  ================================================================================
  .
  ================================================================================
  Verification failed in SLXVersionTest/testSLXVersion(Paths=p__home_runner_work_WEC_Sim_WEC_Sim_examples_RM3FromSimulink_RM).
      ----------------
      Test Diagnostic:
      ----------------
      /home/runner/work/WEC-Sim/WEC-Sim/examples/RM3FromSimulink/RM3FromSimulink.slx model version R2021a exceeds R2020b
      ---------------------
      Framework Diagnostic:
      ---------------------
      verifyTrue failed.
      --> The value must evaluate to "true".
  
      Actual Value:
        logical
  
         0
      ------------------
      Stack Information:
      ------------------
      In /home/runner/work/WEC-Sim/WEC-Sim/tests/devTests/SLXVersionTest.m (SLXVersionTest.testSLXVersion) at 16
  ================================================================================
  ...
  ================================================================================
  Verification failed in SLXVersionTest/testSLXVersion(Paths=p__home_runner_work_WEC_Sim_WEC_Sim_source_lib_WEC_Sim_WECSim_1).
      ----------------
      Test Diagnostic:
      ----------------
      /home/runner/work/WEC-Sim/WEC-Sim/source/lib/WEC-Sim/WECSim_Lib_Body_Elements.slx model version R2021b exceeds R2020b
      ---------------------
      Framework Diagnostic:
      ---------------------
      verifyTrue failed.
      --> The value must evaluate to "true".
  
      Actual Value:
        logical
  
         0
      ------------------
      Stack Information:
      ------------------
      In /home/runner/work/WEC-Sim/WEC-Sim/tests/devTests/SLXVersionTest.m (SLXVersionTest.testSLXVersion) at 16
  ================================================================================
  ...
  ================================================================================
  Verification failed in SLXVersionTest/testSLXVersion(Paths=p__home_runner_work_WEC_Sim_WEC_Sim_source_lib_WEC_Sim_WECSim_4).
      ----------------
      Test Diagnostic:
      ----------------
      /home/runner/work/WEC-Sim/WEC-Sim/source/lib/WEC-Sim/WECSim_Lib_Frames.slx model version R2021b exceeds R2020b
      ---------------------
      Framework Diagnostic:
      ---------------------
      verifyTrue failed.
      --> The value must evaluate to "true".
  
      Actual Value:
        logical
  
         0
      ------------------
      Stack Information:
      ------------------
      In /home/runner/work/WEC-Sim/WEC-Sim/tests/devTests/SLXVersionTest.m (SLXVersionTest.testSLXVersion) at 16
  ================================================================================

H0R5E · 2022-07-25T13:27:50Z

I added #920 to track converting the SLX files with the wrong versions.

H0R5E · 2022-07-25T14:14:42Z

@kmruehl, do you think it's worth adding something to the WEC-Sim Library page about how to save the files to the correct version?

kmruehl · 2022-07-27T18:39:03Z

@H0R5E we require the WEC-Sim Library to be saved in 2020b, but we've never required model files to also be saved in a prior version of MATLAB. There have been (rare) instances when a model is saved in a newer version of MATLAB (e.g. 2021b), and I'm unable to open it in an older version (e.g. 2020a). However, this is not a common issue, and I do not want to require all WEC-Sim models to be saved in prior versions of MATLAB. I'd prefer to just update the cases causing issues.

In this instance, it looks like two of the library files were not properly saved in 2020b. @jtgrasb will update these files:

source/lib/WEC-Sim/WECSim_Lib_Body_Elements.slx
source/lib/WEC-Sim/WECSim_Lib_Frames.slx

kmruehl · 2022-07-27T18:50:04Z

@H0R5E, @jtgrasb is going to save the WEC-Sim library blocks on dev to 2020b. I don't want to require the WEC-Sim model files to be saved to 2020b. But this PR check would be helpful to run on the library files, on the WEC-Sim\source\lib directory.

Can you update this PR so that it only checks the library files? Thanks!

H0R5E · 2022-07-28T10:05:58Z

@H0R5E we require the WEC-Sim Library to be saved in 2020b, but we've never required model files to also be saved in a prior version of MATLAB.

Sorry, I'm going to have to disagree with this approach. If we are supporting users running R2020b, then they should be able to open the example files. For example, when I try to open \examples\RM3\RM3.slx, I get this error:

It's very low effort to save the files in the correct format, so I don't understand why it's not worth doing to ensure compatibility for all users.

At a minimum, we should put a warning in the docs saying that the example files can only be opened with a specific version.

H0R5E · 2022-07-28T10:27:35Z

So, another option is to get our users to turn off the compatibility check when opening Simulink files saved in newer versions. RM3.slx still throws this warning after being opened:

=== Model Load (Elapsed: 0.176 sec) ===
    Warning:Model 'RM3' was created with a newer version (R2021a) of Simulink
    To create a model that is compatible with this version of Simulink, load the model in Simulink R2021a and select Save > Export Model to > Previous Version.
    Warning:Warning: Cannot load version 'R2021a_202009111111' of family 'sltp_mm'.

In this case, we should document the process of setting the option in the Simulink preferences.

kmruehl · 2022-08-25T16:01:30Z

@H0R5E do you know is there is a way to turn off that option programmatically? The WEC-Sim team discussed this issue. We only way to require the library to be saved in a prior version of MATLAB, but would like users to be able to open model files saved in a newer version of MATLAB.

H0R5E · 2022-08-30T10:47:09Z

@H0R5E do you know is there is a way to turn off that option programmatically?

Yes, the command is set_param(0,'ErrorIfLoadNewModel','off')

EDIT: Note the above command is one-and-done. It's a global setting, not one associated to the model.

kmruehl · 2022-08-31T14:30:48Z

set_param(0,'ErrorIfLoadNewModel','off') is a global setting. It only needs to be run once.

There are 3 options of how to implement this (not listed in order of preference):

add a note in the documentation with instructions on how to change the global setting (this would be run optionally)
add it to wec-sim install (e.g. startup.m) with a comment about it being a global setting (this would be run once)
add it to simulation class (whether other simulink settings are changed), and add a warning noting the change to a global setting (this would be run multiple times unless there's an if/else statement)

H0R5E · 2022-08-31T14:32:35Z

So, the command above is already called here in initializeWecSim.m.

I really don't think this is the best place for it, as you need to run WEC-Sim before you can open the model.

🛒 before 🐴.

H0R5E · 2022-08-31T15:06:58Z

This command has been inside the wecSim script for a very long time. It was added by @yuyihsiang on 2017/08/16.

The problem is that if we removed this command and it was not run before calling WEC-Sim then WEC-Sim would fail if we were trying to run a newer model. Therefore, you could argue it's a necessary part of the installation process.

So I think putting it into startup.m makes the most sense to me. You can then open newer model files without running WEC-Sim first and WEC-Sim won't break when running newer model files if you haven't set the preference manually.

H0R5E · 2022-08-31T17:49:02Z

So, I've gone ahead and moved set_param(0,'ErrorIfLoadNewModel','off') from initializeWecSim.m to AddWecSimSource.m and reduced the Simulink file version testing to the source folder.

Happy to change again if not what you want. I just thought it would be useful to see what it looked like in this configuration.

kmruehl · 2022-09-21T14:22:41Z

@H0R5E do you know why these tests are failing?

H0R5E · 2022-09-22T08:51:08Z

Hi, the test failure being generated is:

 ================================================================================
  Verification failed in SLXVersionTest/testSLXVersion(Paths=p__home_runner_work_WEC_Sim_WEC_Sim_source_lib_WEC_Sim_WECSim_2).
      ----------------
      Test Diagnostic:
      ----------------
      /home/runner/work/WEC-Sim/WEC-Sim/source/lib/WEC-Sim/WECSim_Lib_Cables.slx model version R2021b exceeds R2020b
      ---------------------
      Framework Diagnostic:
      ---------------------
      verifyTrue failed.
      --> The value must evaluate to "true".
      
      Actual Value:
        logical
      
         0
      ------------------
      Stack Information:
      ------------------
      In /home/runner/work/WEC-Sim/WEC-Sim/tests/devTests/SLXVersionTest.m (SLXVersionTest.testSLXVersion) at 16

The most important part being source/lib/WEC-Sim/WECSim_Lib_Cables.slx model version R2021b exceeds R2020b, indicating that WECSim_Lib_Cables.slx is not saved in a R2020b version.

kmruehl · 2022-11-16T15:16:02Z

@kmruehl will save the cable library block to 2020b

kmruehl

@H0R5E I'm going to make this minor change and then merge the PR once the tests pass

kmruehl · 2022-12-07T18:13:36Z

source/functions/initializeWecSim.m

 warning('off','sm:sli:setup:compile:SteadyStateStartNotSupported')
-set_param(0, 'ErrorIfLoadNewModel', 'off')
+
 % Load parameters to Simulink model


My only concern about removing it here, is that not everyone uses the addWecSimSource script each time they open MATLAB. I think we should keep it in both places. I'm going to add it back here, additioanlly.

kmruehl · 2022-12-07T18:14:21Z

addWecSimSource.m

-clear wecSimSource
+% Allow opening of Simulink models saved in a newer version
+set_param(0, 'ErrorIfLoadNewModel', 'off')
+


This solution only works if users execute this script, which they may not do. For example, I have the source always added and never run this script. I think we should keep it in both places, and will make this revision.

kmruehl · 2022-12-07T19:11:39Z

@H0R5E you can delete your branch if you like

H0R5E changed the title ~~Add tests to check that the SLX file version do not exceed R2020b~~ Add tests to check that SLX file versions do not exceed R2020b Jul 25, 2022

H0R5E mentioned this pull request Jul 25, 2022

[BUG] SLX files with versions incompatible with R2020b #920

Closed

kmruehl self-requested a review July 27, 2022 18:52

kmruehl added Tests/CI related WEC-Sim tests or Continuous Integration Library updates to the WEC-Sim Library labels Jul 27, 2022

kmruehl self-assigned this Jul 27, 2022

kmruehl mentioned this pull request Jul 27, 2022

[BUG] Unable to resolve the name 'clock_out.time' (Desalination) #918

Closed

H0R5E force-pushed the test_slx_versions branch from 95e30c6 to c8da02d Compare August 31, 2022 15:18

H0R5E force-pushed the test_slx_versions branch from 537ccd0 to ef955ae Compare September 1, 2022 16:34

H0R5E added 5 commits September 22, 2022 09:13

Add tests to check that the SLX file version do not exceed R2020b

3455b65

Move setting ErrorIfLoadNewModel to AddWecSimSource.m

276c6d7

Only test simulink version numbers for files in the source directory

a1cf017

Add set_param(0, 'ErrorIfLoadNewModel', 'off') to workflow

c8b0cbc

Move set_param into testing step

dc32cfb

H0R5E force-pushed the test_slx_versions branch from ef955ae to dc32cfb Compare September 22, 2022 08:13

kmruehl added 2 commits December 7, 2022 11:08

saving cable lib to 2020b

7faebc9

added Simulink compatability param setting to initializeWecSim

237b0c1

kmruehl approved these changes Dec 7, 2022

View reviewed changes

kmruehl merged commit 3fdaec8 into WEC-Sim:dev Dec 7, 2022

H0R5E deleted the test_slx_versions branch January 11, 2023 12:35

This was referenced Apr 24, 2023

Save mooring library to R2020b #1040

Merged

[BUG] WECSim_Lib_Moorings incompatible matlab 2020b #1037

Closed

ttran18 mentioned this pull request Jan 13, 2026

[Support] error examples and mooring Applications #1607

Open

Add tests to check that SLX file versions do not exceed R2020b #919

Add tests to check that SLX file versions do not exceed R2020b #919

Uh oh!

Conversation

H0R5E commented Jul 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

H0R5E commented Jul 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

H0R5E commented Jul 25, 2022

Uh oh!

H0R5E commented Jul 25, 2022

Uh oh!

kmruehl commented Jul 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kmruehl commented Jul 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

H0R5E commented Jul 28, 2022

Uh oh!

H0R5E commented Jul 28, 2022

Uh oh!

kmruehl commented Aug 25, 2022

Uh oh!

H0R5E commented Aug 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kmruehl commented Aug 31, 2022

Uh oh!

H0R5E commented Aug 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

H0R5E commented Aug 31, 2022

Uh oh!

H0R5E commented Aug 31, 2022

Uh oh!

kmruehl commented Sep 21, 2022

Uh oh!

H0R5E commented Sep 22, 2022

Uh oh!

kmruehl commented Nov 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kmruehl left a comment

Choose a reason for hiding this comment

Uh oh!

kmruehl Dec 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kmruehl Dec 7, 2022

Choose a reason for hiding this comment

Uh oh!

kmruehl commented Dec 7, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

H0R5E commented Jul 25, 2022 •

edited

Loading

H0R5E commented Jul 25, 2022 •

edited

Loading

kmruehl commented Jul 27, 2022 •

edited

Loading

kmruehl commented Jul 27, 2022 •

edited

Loading

H0R5E commented Aug 30, 2022 •

edited

Loading

H0R5E commented Aug 31, 2022 •

edited

Loading

kmruehl commented Nov 16, 2022 •

edited

Loading

kmruehl Dec 7, 2022 •

edited

Loading