Skip to content

Escape.cs Improvements #185

Merged
milkshakeuk merged 1 commit intomasterfrom
escape-changes
Mar 8, 2021
Merged

Escape.cs Improvements #185
milkshakeuk merged 1 commit intomasterfrom
escape-changes

Conversation

@milkshakeuk
Copy link
Copy Markdown
Member

@milkshakeuk milkshakeuk commented Feb 27, 2021

What's in this pull request

  1. Removed the use of Trim() from Escape.cs fixes HL7 TX type, obx.5 leading space being trimmed PipeParser #103.
  2. Changed behaviour of Escape to (mostly) match hapi implementation.
  3. Adds unit tests (as there were none before) to ensure Escape behaves correctly (some borrowed from hapi) others not.
  4. Change the default hexadecimal escape sequences for carriage return \r to \X0D\ from \X000d\.
  5. Make line feed / carriage return hexadecimal escape sequences configurable so as to allow people to go back to the previous escape sequences which was \X000d\ for carriage returns.
  6. Add .gitattributes file to prevent the line endings of text files being changed on checkout (as this was breaking some tests when running on linux).

Could potentially be a breaking change for those relying on existing behaviour but this implementation should be more "correct" in line with the hl7 standards, plus the next release is a Major one anyway so breaking changes are expected.

Hexadecimal line feed and carriage return escape sequence configuration

The escape sequence for line feed \n and carriage return \r are now configurable to 1 of 3 values each respectively.

example App.config configuration:

<configSections>
  <section name="EscapingConfiguration" type="NHapi.Base.Model.Configuration.EscapingConfigurationSection, NHapi.Base"/>
</configSections>
...
<EscapingConfiguration>
  <HexadecimalEscaping lineFeedHexadecimal="X0A" carriageReturnHexadecimal="X0D"/>
</EscapingConfiguration>

Possible values for lineFeedHexadecimal are X0A, X00A and X000a (they map to an enum)
Possible values for carriageReturnHexadecimal are X0D, X00D and X000d (they map to an enum)

If the configuration isn't specified in the App.config the default values are X0A for line feed and X0D for carriage return.

@github-actions

This comment has been minimized.

@milkshakeuk
Copy link
Copy Markdown
Member Author

Will look into the failing tests tomorrow.

@milkshakeuk
Copy link
Copy Markdown
Member Author

milkshakeuk commented Feb 28, 2021

Ok one of the tests that is failing is failing because I changed Escape.cs.

X TestValidHl7Data [1s 6ms]
Error Message:
   Expected string length 39 but was 31. Strings differ at index 8.
Expected: "Th\\T\\is\\E\\.br\\E\\is\\E\\.br\\E\\A T\\F\\est\\E\\"
But was:  "Th\\T\\is\\.br\\is\\.br\\A T\\F\\est\\E\\"
----------------------^

Stack Trace:
   at NHapi.NUnit.ParserTest.TestValidHl7Data() in D:\a\nHapi\nHapi\tests\NHapi.NUnit\ParserTest.cs:line 147

However, I beleive the actual result i.e. Th\\T\\is\\.br\\is\\.br\\A T\\F\\est\\E\\ is correct since the starting value Th&is\.br\is\.br\A T|est\ and contains formatted text which should not be re-escaped i.e. \.br\ should be left intact as it is indicating a carriage return, the escape character \ either side in this instance should not be escaped to \E\.

So ill update the test and push those changes along with the StyleCop warning fixes.

The second failing test...

X UnEscapesData [5ms]
Error Message:
   Expected string length 122 but was 128. Strings differ at index 32.
Expected: "...39;Thirty days have September,\rApril\nJune,\nand November.\nW..."
But was:  "...39;Thirty days have September,\\X000d\\April\nJune,\nand Novem..."
---------------------------------------------^

Stack Trace:
   at NHapi.NUnit.ParserTest.UnEscapesData() in D:\a\nHapi\nHapi\tests\NHapi.NUnit\ParserTest.cs:line 174

This one requires a discussion since its failing because of a Unicode hexadecimal representation of a carriage return i.e. \X000d\ this is one of the escape sequences that is applied when someone adds a \r (carriage return) to their text value since this has a special meaning in hl7 indicating the end of a segment.

This is what the hl7 v24 standard says at section 2.10.5

2.10.5 Hexadecimal
When the hexadecimal escape sequence (\Xdddd...) is used the X should be followed by 1 or more pairs of
hexadecimal digits (0, 1, . . . , 9, A, . . . , F). Consecutive pairs of the hexadecimal digits represent 8-bit
binary values. The interpretation of the data is entirely left to an agreement between the sending and receiving applications that is beyond the scope of this Standard.

Both Hapi and nHapi currently use the Unicode hexadecimal representation of the carriage return i.e. \X000d\ rather than the utf-8 hexadecimal representation i.e. \X0D\.

This Pull Request changes what nHapi uses from \X000d\ to \X0D\ hense the failing test here.

Other hl7 libraries such as HL7-dotnetcore which is based from the java library hl7inspector use the latter.

Also, Intersystems who are the company behind the integration engine software ensemble also suggest to use the latter.

I would like to use \X0D\ also but that would mean a breaking change, this should be fine since this will be in a new major version of nHapi.

@duaneedwards @ryankelley @AMCN41R what are your opinions on this one?

@github-actions

This comment has been minimized.

@duaneedwards
Copy link
Copy Markdown
Collaborator

Was a bit concerned about the first change but if I understand this right, you are basically looking to undo the changes made in this commit?

521934d#diff-16e92736eac2a191a20e92f6a74ee9ac812c2c31ba196ec1427d69ca4dda37e7

It looks like this was previously set to expect the value you are proposing to change it to now?

Otherwise the changes look fine, do agree with the idea of putting these behind a major release as it's highly likely someone might run into issues due to some particular system encoding / decoding things some particular way.

@milkshakeuk
Copy link
Copy Markdown
Member Author

milkshakeuk commented Mar 1, 2021

Was a bit concerned about the first change but if I understand this right, you are basically looking to undo the changes made in this commit?

521934d#diff-16e92736eac2a191a20e92f6a74ee9ac812c2c31ba196ec1427d69ca4dda37e7

Yes however that change was only made to make the tests pass, since I just assumed all the tests from the project should already be passing (that was done a long time ago 😁)

It looks like this was previously set to expect the value you are proposing to change it to now?

Yes since this is the correct behaviour (from what I have come to understand anyway, plus it is in line with hapi)

Otherwise the changes look fine, do agree with the idea of putting these behind a major release as it's highly likely someone might run into issues due to some particular system encoding / decoding things some particular way.

I'm still looking into the second one, just in case, as I don't want to introduce anything which isn't correct as far as the standards are concerned.

@milkshakeuk
Copy link
Copy Markdown
Member Author

@duaneedwards @AMCN41R having looked further into HL7-dotnetcore and hl7inspector they actually uses \X00D\ for \r (carriage return) and \X00A\ for \n (line feeds) which is different to hapi which uses \X000d\ and Intersystems which says \XOD\ and \X0A\ respectively so... the question is which one should nHapi use?

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@milkshakeuk
Copy link
Copy Markdown
Member Author

@duaneedwards @AMCN41R I asked the question in the hl7.org zulip chat and here are the responses:
image
It looks like it's mostly up to us but sounds like \X0D\ and \X0A\ might be the way to go.

In the future we can make it configurable instead.

@duaneedwards
Copy link
Copy Markdown
Collaborator

Interesting, would it be much work to make it configurable?

I'm thinking that ideally if it were going to be changed from \X000d\ to \X0D\ then perhaps providing the escape hatch back to the previous behaviour at the same time would also be worth the effort if it's within reason.

@milkshakeuk
Copy link
Copy Markdown
Member Author

@duaneedwards

Interesting, would it be much work to make it configurable?

I'm thinking that ideally if it were going to be changed from \X000d\ to \X0D\ then perhaps providing the escape hatch back to the previous behaviour at the same time would also be worth the effort if it's within reason.

I'm not sure how difficult it would be to make it configurable but I'll look into it, there was one final reply on the hl7 zulip chat:

image

@milkshakeuk
Copy link
Copy Markdown
Member Author

@duaneedwards ok the escape sequence for line feed \n and carriage return \r are now configurable to 1 of 3 values each respectively.

<configSections>
  <section name="EscapingConfiguration" type="NHapi.Base.Model.Configuration.EscapingConfigurationSection, NHapi.Base"/>
</configSections>
...
<EscapingConfiguration>
  <HexadecimalEscaping lineFeedHexadecimal="X0A" carriageReturnHexadecimal="X0D"/>
</EscapingConfiguration>

Possible values for lineFeedHexadecimal are X0A, X00A and X000a (they map to an enum)
Possible values for carriageReturnHexadecimalare are X0D, X00D and X000d (they map to an enum)

If the configuration isn't specified in the App.config the default values are X0A for line feed and X0D for carriage return.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Changed behaviour of Escape to (mostly) match hapi implementation
Also added unit test to ensure Escape behaves correctly (some borrowed from hapi)
Make line feed / carriage return hexadecimal escape sequences configurable
Add .gitattributes to prevent the line endings of txt files being changed on checkout (was breaking tests)
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 7, 2021

Unit Test Results

    5 files    54 suites   7s ⏱️
406 tests 404 ✔️ 2 💤 0 ❌
810 runs  807 ✔️ 3 💤 0 ❌

Results for commit f293ba1.

@milkshakeuk milkshakeuk merged commit 44187f3 into master Mar 8, 2021
@milkshakeuk milkshakeuk deleted the escape-changes branch March 8, 2021 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HL7 TX type, obx.5 leading space being trimmed PipeParser

3 participants