Describe the bug
When I am reading Mainframe file (EBCDIC_037) in Databricks Notebook with cobrix library,for one of the character(i.e broken bar - ¦ ) is getting changed to unbroken bar (|).
Actually, when I set debug ='hex' option while reading the file , debug dataframe column value giving correct hexa decimal value(6A) but value on the normal df column giving incorrect hexadecimal value (7C).
Due to this we are seeing data mismatch between source vs cobrix read data.
Anyone can help to fix this? and can you please clarify why this change happening in cobrix ?
Broken - \x6a
Unbroken Bar - \x7c
Expected behaviour
I should get same value as I see on the source.
Describe the bug
When I am reading Mainframe file (EBCDIC_037) in Databricks Notebook with cobrix library,for one of the character(i.e broken bar - ¦ ) is getting changed to unbroken bar (|).
Actually, when I set debug ='hex' option while reading the file , debug dataframe column value giving correct hexa decimal value(6A) but value on the normal df column giving incorrect hexadecimal value (7C).
Due to this we are seeing data mismatch between source vs cobrix read data.
Anyone can help to fix this? and can you please clarify why this change happening in cobrix ?
Broken - \x6a
Unbroken Bar - \x7c
Expected behaviour
I should get same value as I see on the source.