New formatting and units options for table outputs (Tabular, SQLite, and JSON) #11210

mjwitte · 2025-09-12T18:37:17Z

Pull request overview

Fixes Add switch to disable rounding or truncating digits for tabular reports #10896
Fixes Allow different unit conversions for JSON tabular output file #10489
Add a new field to OutputControl:Table:Style:

   A3; \field Format Numeric Values
       \memo If No, all digits are shown after the decimal point without any rounding (23.238769213).
       \memo If Yes, values are rounded for readability (23.24).
       \type choice
       \key Yes
       \key No
       \default Yes

Add a new field to Output:SQLite:

   A3; \field Format Numeric Values for Tabular Data
       \memo If No, all digits are shown after the decimal point without any rounding (23.238769213).
       \memo If Yes, values are rounded for readability (23.24).
       \type choice
       \key Yes
       \key No
       \default Yes

Add two new fields to Output:JSON:

  A5 , \field Unit Conversion for Tabular Data
       \type choice
       \note Unit conversion option used when writing JSON Tabular Data
       \note This option applies to TabularData and TabularDatawithString in the JSON file(s)
       \key UseOutputControlTableStyle
       \key None
       \key JtoKWH
       \key JtoMJ
       \key JtoGJ
       \key InchPound
       \key InchPoundExceptElectricity
       \default UseOutputControlTableStyle
  A6 ; \field Format Numeric Values for Tabular Data
       \memo If No, all digits are shown after the decimal point without any rounding (23.238769213).
       \memo If Yes, values are rounded for readability (23.24).
       \type choice
       \key Yes
       \key No
       \default Yes

Fixes Output:Table:Annual not writing to JSON output.

Pull Request Author

Reviewer

github-actions · 2025-09-12T19:07:32Z

⚠️ Regressions detected on ubuntu-24.04 for commit `ad696cc`

Regression Summary

EIO: 143
Table Big Diffs: 80

github-actions · 2025-09-12T19:10:38Z

⚠️ Regressions detected on macos-14 for commit `ad696cc`

Regression Summary

EIO: 143
Table Big Diffs: 80

github-actions · 2025-09-15T17:47:39Z

⚠️ Regressions detected on macos-14 for commit `7c4dc21`

Regression Summary

EIO: 786
Table Big Diffs: 80

github-actions · 2025-09-15T17:55:20Z

⚠️ Regressions detected on ubuntu-24.04 for commit `7c4dc21`

Regression Summary

EIO: 789
Table Big Diffs: 80

mjwitte · 2025-09-15T21:28:48Z

@JasonGlazer Here are some sample outputs with rounding turned off.
10896-TableRounding.zip

github-actions · 2025-09-15T21:30:04Z

⚠️ Regressions detected on ubuntu-24.04 for commit `d5036a9`

Regression Summary

EIO: 789
Table Big Diffs: 80

github-actions · 2025-09-15T21:32:28Z

⚠️ Regressions detected on macos-14 for commit `d5036a9`

Regression Summary

EIO: 785
Table Big Diffs: 80

mjwitte · 2025-09-15T21:35:41Z

And some json outputs with and without rounding.
5ZoneAirCooledConvCoef-SI-NoRounding.json
5ZoneAirCooledConvCoef-SI-2Rounding.json

JasonGlazer · 2025-09-16T11:45:34Z

5ZoneAirCooledConvCoef-SI-2Rounding.json

I noticed some values that are not being rounded in this file.

JasonGlazer · 2025-09-16T12:01:22Z

Here are some sample outputs with rounding turned off.

Originally, I wasn't envisioning the HTML file to be changed, just the JSON files, but it is ok that it is a general change.

Some values aren't shown as rounded in the 2Rounding case. Maybe not every tabular output uses RealToStr?

See "Outdoor Air Details", "Standard 62.1 Summary", "Initialization Summary"

Maybe not that important since we are really just after the 0 case without any rounding.

mjwitte · 2025-09-16T14:48:45Z

5ZoneAirCooledConvCoef-SI-2Rounding.json

I noticed some values that are not being rounded in this file.

This is a first pass. At this point, for tables that use the predefined report method, rounding values >0 are only touching entries that don't include the last argument. Any table entries where the number of places are specified in the call are using the input value. e.g.:
PreDefTableEntry(state, state.dataOutRptPredefined->pdchOaOccBzNatVent, thisZone.Name, natVent, 4);

I was concerned that certain values that are expected to be small numbers would end up showing 0.00, but I didn't test that. It's an easy change.

JasonGlazer · 2025-09-16T14:59:51Z

was concerned that certain values that are expected to be small numbers would end up showing 0.00

That is a very legitimate concern. 0.00 implies zero even if the value is supposed to be small. Maybe the current behaviour is better, but it would need to be documented.

mjwitte · 2025-09-16T16:43:45Z

was concerned that certain values that are expected to be small numbers would end up showing 0.00

That is a very legitimate concern. 0.00 implies zero even if the value is supposed to be small. Maybe the current behaviour is better, but it would need to be documented.

While we're in here, we could add logic to switch to an exponent format whenever the value is smaller than the specified number of sig digits.

Also regarding rounding for HTML vs JSON (and SQL). My understanding of the code is that the current approach sends the tablebody to the results framework for the various formats, so it's already been rounded or units converted, and it's all strings at that point. For predefined reports, we have the .origRealEntry value so we could access that. But for the other reports that are using realToString the original value is not being saved (which can impact unit conversions, too, if I'm following correctly.

JasonGlazer · 2025-09-16T18:27:03Z

When I was thinking of this, I was thinking of extending the current loop that does different unit systems to be more general and basically reproduce the tables multiple times. Not very efficient. Your approach seems fine.

…Rounding

JasonGlazer · 2025-09-30T19:32:03Z

The SQL file in "TabularData" seems to be not getting rounded even when is set to Yes

…Rounding

mjwitte · 2025-10-03T18:30:31Z

No diffs here as expected, but there are some api unit test failures. I think they are unrelated? @mitchute ?
https://myoldmopar.github.io/EnergyPlusBuildResults/EnergyPlus-5979ad801c6ede9b14538a97ead96b5aaf94f26e-x86_64-Linux-Ubuntu-24.04-gcc-13.3-UnitTestsCoverage-RelWithDebInfo.html

Next step will be a revised example file which should generate some JSON diffs.

mjwitte · 2025-10-03T20:04:09Z

Here's an example of Table (html) output with and without numeric formatting.

mjwitte · 2025-10-03T20:10:20Z

Here's an example of JSON output, format numbers vs not:

And JSON output IP units vs SI (no format for both):

mjwitte · 2025-10-03T20:16:31Z

And here's SQLite output formatted vs not:

And SQLite IP vs SI (just to confirm it still works):

mjwitte · 2025-10-03T20:25:02Z

idd/Energy+.idd.in

      \min-fields 4
      \memo This object defines the connection between two nodes and a component.
 A1 , \field Name
+      \reference AirflowNetworkDistributionLinkageNames


Unrelated fix to add missing reference that caused a pop-up warning in IDFEditor (see #10887 ).

mitchute · 2025-10-03T20:52:31Z

No diffs here as expected, but there are some api unit test failures. I think they are unrelated? @mitchute ? https://myoldmopar.github.io/EnergyPlusBuildResults/EnergyPlus-5979ad801c6ede9b14538a97ead96b5aaf94f26e-x86_64-Linux-Ubuntu-24.04-gcc-13.3-UnitTestsCoverage-RelWithDebInfo.html

Next step will be a revised example file which should generate some JSON diffs.

@mjwitte those are old results. The latest ones are passing. https://myoldmopar.github.io/EnergyPlusBuildResults/EnergyPlus-bcbe5cda8682a986b16b1076d51170e58e577ded-x86_64-Linux-Ubuntu-24.04-gcc-13.3-UnitTestsCoverage-RelWithDebInfo.html

mjwitte

Code walkthru

mjwitte · 2025-10-03T20:25:40Z

idd/Energy+.idd.in

       \key InchPound
       \key InchPoundExceptElectricity
       \default None
+   A3; \field Format Numeric Values


New format option for tabular output (html, tab, or csv)

mjwitte · 2025-10-03T20:26:34Z

idd/Energy+.idd.in

       \key No
       \default No
-
+  A5 , \field Unit Conversion for Tabular Data


New units and format options for JSON output (JSON, CBOR, MessagePack).

mjwitte · 2025-10-03T20:27:03Z

idd/Energy+.idd.in

       \key InchPound
       \key InchPoundExceptElectricity
       \default UseOutputControlTableStyle
+   A3; \field Format Numeric Values for Tabular Data


New format option for SQLite output (already had a units option).

mjwitte · 2025-10-03T20:29:46Z

src/EnergyPlus/OutputReportTabular.hh

+    struct tabularReportStyle
+    {
+        UnitsStyle unitsStyle = UnitsStyle::None;
+        bool formatReals = true;
+        bool produceTabular = true;
+        bool produceSQLite = true;
+        bool produceJSON = true;


New tabularReportStyle struct/vector. Tracks units and formatting for table reports in one of three output formats:
Tabular, JSON, and SQLite. By default, all three output types (if active) will have the same units and numerics will be formatted and there will only be on instance of this.

If different options are specified for JSON and/or SQLite, then another instance will be created for that file type, up to three instances max.

mjwitte · 2025-10-03T20:36:21Z

src/EnergyPlus/OutputReportTabular.cc

        AlphArray(2) = "None";
+        ort->formatReals_Tabular = getYesNoValue(AlphArray(3)) == BooleanSwitch::Yes;


Set new format option for tabular (html etc.)

mjwitte · 2025-10-03T21:09:46Z

src/EnergyPlus/OutputReportTabular.hh

-    void writeVeriSumSpaceTables(EnergyPlusData &state, bool produceTabular, bool produceSQLite);
+    void writeVeriSumSpaceTables(EnergyPlusData &state, const tabularReportStyle &style);


Various sub functions for the report writing need the style information passed in (more than before).

mjwitte · 2025-10-03T21:13:47Z

testfiles/5ZoneCAV_MaxTemp_JSON_Outputs.idf

    Yes,                     !- Output JSON
    No,                      !- Output CBOR
-    No;                      !- Output MessagePack
+    No,                      !- Output MessagePack
+    None,                    !- Unit Conversion for Tabular Data
+    No;                      !- Format Numeric Values for Tabular Data


Add new options to existing example file.

Should we flip this to Yes so we exercise this via the ~~unit~~ integration tests?

mjwitte · 2025-10-03T21:14:34Z

tst/EnergyPlus/unit/OutputReportTabular.unit.cc


    state->dataOutRptTab->WriteTabularFiles = true;

+    // activate JSON output


Extend existing unit test.

mjwitte · 2025-10-03T21:15:43Z

tst/EnergyPlus/unit/OutputReportTabular.unit.cc

+    }
+}
+
+TEST_F(SQLiteFixture, ORT_EndUseBySubcategorySQL_DualUnits2)


Clone the unit test with different options.

mjwitte · 2025-10-03T21:16:20Z

tst/EnergyPlus/unit/OutputReportTabularAnnual.unit.cc


    OutputReportTabularAnnual::GetInputTabularAnnual(*state);
    EXPECT_EQ(state->dataOutputReportTabularAnnual->annualTables.size(), 1u);
+    OutputReportTabular::setTabularReportStyles(*state);


And several unit tests need this new call to set things up.

mjwitte · 2025-10-03T21:21:09Z

@mitchute This should be ready for review.

mitchute

Minor question, but the rest looks good.

mitchute · 2025-10-06T18:34:44Z

testfiles/5ZoneCAV_MaxTemp_JSON_Outputs.idf

    Yes,                     !- Output JSON
    No,                      !- Output CBOR
-    No;                      !- Output MessagePack
+    No,                      !- Output MessagePack
+    None,                    !- Unit Conversion for Tabular Data
+    No;                      !- Format Numeric Values for Tabular Data


Should we flip this to Yes so we exercise this via the ~~unit~~ integration tests?

mjwitte · 2025-10-06T18:41:16Z

None,                    !- Unit Conversion for Tabular Data
No;                      !- Format Numeric Values for Tabular Data

Should we flip this to Yes so we exercise this via the unit integration tests?

Since the prior (now default) behavior was Yes, my thought was to set this to No to force something different. But since no diffs were reported, apparently, we don't compare JSON outputs for regressions? But this will at least cause more than one reporting pass so it will help make sure nothing breaks the regular table outputs.

mitchute · 2025-10-06T18:51:46Z

Should we flip this to Yes so we exercise this via the unit integration tests?

Since the prior (now default) behavior was Yes, my thought was to set this to No to force something different. But since no diffs were reported, apparently, we don't compare JSON outputs for regressions? But this will at least cause more than one reporting pass so it will help make sure nothing breaks the regular table outputs.

Hmm.... I thought it was defaulted to No. In that case, I guess I would expect some diffs. Checking on whether we're comparing the JSON during regressions...

mitchute · 2025-10-06T19:26:19Z

OK, so eplusout_hourly.json is compared with a math diff (these values don't change here), and eplusout.json is not compared at all; however, I am seeing the rounding diffs there. Let's merge this and then I'll update the regression tool to catch the eplusout.json diffs.

New sig digits field for tables - implement for predefined

94ca68e

mjwitte added NewFeature Includes code to add a new feature to EnergyPlus IDDChange Code changes impact the IDD file (cannot be merged after IO freeze) RPD labels Sep 12, 2025

mjwitte added the OutputChange Code changes output in such a way that it cannot be merged after IO freeze label Sep 15, 2025

mjwitte added 2 commits September 15, 2025 12:08

Shorten field name

57fbb16

Add sig digits and always write tabular style to eio

ded91c1

mjwitte added the Defect Includes code to repair a defect in EnergyPlus label Sep 15, 2025

mjwitte added 3 commits September 15, 2025 14:42

format

fcdfbe4

Fix unit conversion eio output

5eaa4ad

Extend no rounding option to RealToStr function

cdde3d5

mjwitte added 2 commits September 16, 2025 17:45

Fix output:table:annual to JSON

165ffa7

Add inputs for units and real format to JSON and SQLite

bde6e6a

mjwitte changed the title ~~New significant digits option for table outputs~~ New formatting and units options for table outputs (Tabular, SQLite, and JSON) Sep 17, 2025

mjwitte added 2 commits September 17, 2025 17:20

Plan B - Units and formatting options for tabular, SQLite, and JSON

36b9697

Merge remote-tracking branch 'remotes/origin/develop' into 10896Table…

1470211

…Rounding

mjwitte added 13 commits September 30, 2025 15:50

Merge remote-tracking branch 'remotes/origin/develop' into 10896Table…

36a4842

…Rounding

Fix sql rounding input processing

084def2

typo

a7e72b9

Merge remote-tracking branch 'remotes/origin/develop' into 10896Table…

81501cf

…Rounding

Implement format options for economics

ba6b96b

format

e3d5266

unit test cleanup

c1438fb

Merge remote-tracking branch 'remotes/origin/develop' into 10896Table…

d56493d

…Rounding

Fix missing reference from AFN Duct Model PR

ec4e901

Fix duplicate SQLite tables

5979ad8

Revised example file

3ddb135

Fix economic tariff variable reporting if multiple table passes

ebf2a47

Fix component sizing reporting if multiple table passes

bcbe5cd

mjwitte commented Oct 3, 2025

View reviewed changes

Minor cleanup

8f31c74

mitchute reviewed Oct 6, 2025

View reviewed changes

mitchute merged commit 86cc706 into develop Oct 6, 2025
11 checks passed

mitchute deleted the 10896TableRounding branch October 6, 2025 21:12

mjwitte mentioned this pull request Oct 8, 2025

Fix eio output for table style #11260

Merged

20 tasks

		AlphArray(2) = "None";
		ort->formatReals_Tabular = getYesNoValue(AlphArray(3)) == BooleanSwitch::Yes;

		void writeVeriSumSpaceTables(EnergyPlusData &state, bool produceTabular, bool produceSQLite);
		void writeVeriSumSpaceTables(EnergyPlusData &state, const tabularReportStyle &style);


		state->dataOutRptTab->WriteTabularFiles = true;

		// activate JSON output

New formatting and units options for table outputs (Tabular, SQLite, and JSON) #11210

New formatting and units options for table outputs (Tabular, SQLite, and JSON) #11210

Uh oh!

Conversation

mjwitte commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request overview

Pull Request Author

Reviewer

Uh oh!

github-actions bot commented Sep 12, 2025

⚠️ Regressions detected on ubuntu-24.04 for commit ad696cc

Uh oh!

github-actions bot commented Sep 12, 2025

⚠️ Regressions detected on macos-14 for commit ad696cc

Uh oh!

github-actions bot commented Sep 15, 2025

⚠️ Regressions detected on macos-14 for commit 7c4dc21

Uh oh!

github-actions bot commented Sep 15, 2025

⚠️ Regressions detected on ubuntu-24.04 for commit 7c4dc21

Uh oh!

mjwitte commented Sep 15, 2025

Uh oh!

github-actions bot commented Sep 15, 2025

⚠️ Regressions detected on ubuntu-24.04 for commit d5036a9

Uh oh!

github-actions bot commented Sep 15, 2025

⚠️ Regressions detected on macos-14 for commit d5036a9

Uh oh!

mjwitte commented Sep 15, 2025

Uh oh!

JasonGlazer commented Sep 16, 2025

Uh oh!

JasonGlazer commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mjwitte commented Sep 16, 2025

Uh oh!

JasonGlazer commented Sep 16, 2025

Uh oh!

mjwitte commented Sep 16, 2025

Uh oh!

JasonGlazer commented Sep 16, 2025

Uh oh!

JasonGlazer commented Sep 30, 2025

Uh oh!

mjwitte commented Oct 3, 2025

Uh oh!

mjwitte commented Oct 3, 2025

Uh oh!

mjwitte commented Oct 3, 2025

Uh oh!

mjwitte commented Oct 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mitchute commented Oct 3, 2025

Uh oh!

mjwitte left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mitchute Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mjwitte commented Sep 12, 2025 •

edited

Loading

⚠️ Regressions detected on ubuntu-24.04 for commit `ad696cc`

⚠️ Regressions detected on macos-14 for commit `ad696cc`

⚠️ Regressions detected on macos-14 for commit `7c4dc21`

⚠️ Regressions detected on ubuntu-24.04 for commit `7c4dc21`

⚠️ Regressions detected on ubuntu-24.04 for commit `d5036a9`

⚠️ Regressions detected on macos-14 for commit `d5036a9`

JasonGlazer commented Sep 16, 2025 •

edited

Loading

mitchute Oct 6, 2025 •

edited

Loading

mjwitte Oct 3, 2025 •

edited

Loading

mitchute Oct 6, 2025 •

edited

Loading