sent_id = answers-20111107175720AAlb2TB_ans-0015
text = The Irish weather works on the ‘four seasons in a day’ principle, which basic<U+00AD>ally means that you can’t predict a thing when it comes to the behavior of the sky.
contains an encoding error in the token basically. I double-checked and the error is in the original file in the https://catalog.ldc.upenn.edu/LDC2012T13.
The question is: does it should be fixed? Or encoding errors should be preserved to make parsers robust to them? I would prefer to fix them.
sent_id =
answers-20111107175720AAlb2TB_ans-0015text = The Irish weather works on the ‘four seasons in a day’ principle, which basic<U+00AD>ally means that you can’t predict a thing when it comes to the behavior of the sky.
contains an encoding error in the token
basically. I double-checked and the error is in the original file in the https://catalog.ldc.upenn.edu/LDC2012T13.The question is: does it should be fixed? Or encoding errors should be preserved to make parsers robust to them? I would prefer to fix them.