Brief summary of issue / Description of requested feature:
When the "fix unnecessary linebreaks on GA servers" option is enabled additional processing is done in cTelnet::gotPrompt() that removes a linebreak from the first character of visible output if one is there. If a linebreak is not the first visible character in the output, nothing is removed.
This usually works as intended but if the output from the game becomes fragmented in certain ways at the network level linebreaks could be removed that shouldn't be (because cTelnet::processSocketData() may only process one packet (fragment) of data at a time and gotPrompt() is not aware of previously processed packets).
An example of the bug manifesting is when a large chunk of game output is split into two packets where the first packet does not contain GA (gotPrompt() does not fire to apply the fix) and the second packet does contain GA + starts with a \n character. gotPrompt() strips the \n character from the beginning of the 2nd packet (undesirable because the end of the first packet was not a prompt/GA).
Background
After adding the GA signal to TFE I noticed that in certain situations line breaks the game was sending were being removed that should not have been when the "fix unnecessary linebreaks on GA servers" option is enabled. The following image is an example showing movement between two rooms. In the second room the top header where the room exits are shown becomes garbled because of a removed line break:
I repeated the test setting a breakpoint in cTelnet::gotPrompt() and running Wireshark. In this screenshot we can see some of the output already in the display after giving the "s" command (starting with "You leave south."). This block of output corresponds to a packet in Wireshark that ends with the long line below the room name (starting with +=---). The breakpoint in gotPrompt() did not fire here because this block of output does not end with the prompt/GA signal.
The next block of output contains the rest of the room header, room description, and prompt/GA signal. The breakpoint in gotPrompt() fired at this point. In the screenshot you can see the variable mud_data starting with a "\n" linebreak character (but note that it is in the middle of the room header because you see the "Exits: N" information.)
The following is a screenshot from Wireshark capturing during the above scenario. Line 35 is the garbled block of output and the data for that is shown at the bottom (note the same line starting with a linebreak and "| Exits"). Lines 31 and 32 above are the beginning of this block of output from the game which includes the output shown above and some GMCP data.
I circled "length 1514" on line 32 above, remembering 1500 bytes being a limit on packet size (MTU) and wondering if from a networking perspective the amount of data coming from the game hits this limit and is broken up into multiple tcp packets (Lines 31, 32, and 35 from the Wireshark screenshot). Lines 31-32 get processed in full by Mudlet first (and no prompt/GA signal in those, so gotPrompt() isn't called and no linebreaks removed) and then finally we get the GA in line 35 but we're mid-stream and the "fix unncessary linebreaks" does what it does and chops that "\n" at the beginning of that block (line 35 - it doesn't know any better that we're mid stream).
What is the purpose of the "fix unnecessary linebreaks" option?
This was discussed on the Mudlet Discord server -- Vadi found a few people to comment and show example screenshots. We believe the intention behind this option is to remove an extra linebreak that IRE games send after a prompt/GA and before the next output from the game. It was shown that the option does (usually) remove this extra linebreak when it is enabled.
Additional testing
I changed the MTU value on my computer's network software to 600, then connected to Aetolia (an IRE game) with compression OFF and the "fix unnecessary linebreaks" option ON. In this scenario Mudlet did not always remove the linebreaks after the prompt/GA signal as intended. (see the following screenshot for more description)
Path forward
While we encountered this bug frequently with TFE I think overall this is a rarely occurring bug for most GA games. I think in a typical scenario people have a normal MTU (like 1500) and output coming from games that need this fix comes in at under that size per send, especially with compression enabled, so the bug doesn't manifest.
This bug can be fixed but it is a change deep in the telnet/networking code in ctelnet.cpp and should not be taken lightly. Given the above it may not be worth the effort to test if a fix is introduced.
To that end, in the Discord discussion Vadi proposed turning this option OFF by default for any new profiles being created in Mudlet (so the option would only be on for IRE muds and for those profiles that exist already).
If there is an appetite to fix this issue I have a commit here showing an idea of a possible way to deal with the scenarios this issue describes: mrocke@2e2c4eb
Brief summary of issue / Description of requested feature:
When the "fix unnecessary linebreaks on GA servers" option is enabled additional processing is done in cTelnet::gotPrompt() that removes a linebreak from the first character of visible output if one is there. If a linebreak is not the first visible character in the output, nothing is removed.
This usually works as intended but if the output from the game becomes fragmented in certain ways at the network level linebreaks could be removed that shouldn't be (because cTelnet::processSocketData() may only process one packet (fragment) of data at a time and gotPrompt() is not aware of previously processed packets).
An example of the bug manifesting is when a large chunk of game output is split into two packets where the first packet does not contain GA (gotPrompt() does not fire to apply the fix) and the second packet does contain GA + starts with a \n character. gotPrompt() strips the \n character from the beginning of the 2nd packet (undesirable because the end of the first packet was not a prompt/GA).
Background
After adding the GA signal to TFE I noticed that in certain situations line breaks the game was sending were being removed that should not have been when the "fix unnecessary linebreaks on GA servers" option is enabled. The following image is an example showing movement between two rooms. In the second room the top header where the room exits are shown becomes garbled because of a removed line break:
I repeated the test setting a breakpoint in cTelnet::gotPrompt() and running Wireshark. In this screenshot we can see some of the output already in the display after giving the "s" command (starting with "You leave south."). This block of output corresponds to a packet in Wireshark that ends with the long line below the room name (starting with +=---). The breakpoint in gotPrompt() did not fire here because this block of output does not end with the prompt/GA signal.
The next block of output contains the rest of the room header, room description, and prompt/GA signal. The breakpoint in gotPrompt() fired at this point. In the screenshot you can see the variable mud_data starting with a "\n" linebreak character (but note that it is in the middle of the room header because you see the "Exits: N" information.)
The following is a screenshot from Wireshark capturing during the above scenario. Line 35 is the garbled block of output and the data for that is shown at the bottom (note the same line starting with a linebreak and "| Exits"). Lines 31 and 32 above are the beginning of this block of output from the game which includes the output shown above and some GMCP data.
I circled "length 1514" on line 32 above, remembering 1500 bytes being a limit on packet size (MTU) and wondering if from a networking perspective the amount of data coming from the game hits this limit and is broken up into multiple tcp packets (Lines 31, 32, and 35 from the Wireshark screenshot). Lines 31-32 get processed in full by Mudlet first (and no prompt/GA signal in those, so gotPrompt() isn't called and no linebreaks removed) and then finally we get the GA in line 35 but we're mid-stream and the "fix unncessary linebreaks" does what it does and chops that "\n" at the beginning of that block (line 35 - it doesn't know any better that we're mid stream).
What is the purpose of the "fix unnecessary linebreaks" option?
This was discussed on the Mudlet Discord server -- Vadi found a few people to comment and show example screenshots. We believe the intention behind this option is to remove an extra linebreak that IRE games send after a prompt/GA and before the next output from the game. It was shown that the option does (usually) remove this extra linebreak when it is enabled.
Additional testing
I changed the MTU value on my computer's network software to 600, then connected to Aetolia (an IRE game) with compression OFF and the "fix unnecessary linebreaks" option ON. In this scenario Mudlet did not always remove the linebreaks after the prompt/GA signal as intended. (see the following screenshot for more description)
Path forward
While we encountered this bug frequently with TFE I think overall this is a rarely occurring bug for most GA games. I think in a typical scenario people have a normal MTU (like 1500) and output coming from games that need this fix comes in at under that size per send, especially with compression enabled, so the bug doesn't manifest.
This bug can be fixed but it is a change deep in the telnet/networking code in ctelnet.cpp and should not be taken lightly. Given the above it may not be worth the effort to test if a fix is introduced.
To that end, in the Discord discussion Vadi proposed turning this option OFF by default for any new profiles being created in Mudlet (so the option would only be on for IRE muds and for those profiles that exist already).
If there is an appetite to fix this issue I have a commit here showing an idea of a possible way to deal with the scenarios this issue describes: mrocke@2e2c4eb