With just me connected to my server running on my laptop, with the client on the same machine, I observe a Ping time of ~2-3 ms and an Overall Delay of ~22 ms.

However, when the display "sleeps", the lag time increases to over 50 ms and the ping time also increases. If then I wake the display, I can observe that the overall delay will drift back down to ~22 ms. This happens whether a screensaver is set or not.

This could be at the heart of why sound quality, initially very good, has deteriorated over the length of a session.