The deadlock may happen when we try to insert messages inside message callback and the request queue happen to be full.
During the above mentioned pattern, the following happened in sequence:

1. RequestCallback triggered in connectionWorker.java
2. appendResult.setException is triggered
3. user's provided callback triggered, triggering append
4. inflight queue happen to be full, stuck in waiting in flight queue back in normal
   however inflight queue can't be progressed until step 1 finish, as there could be only one requestCallback triggered each time
   The fix was to fix the example code by puting the append at another thread, it solved issue but not good enough. The true fix is to put step 2 to another thread directly so that step 1 is unblocked. This will fix any user on the original example code directly.

The example code can also be upgraded to remove the unnecessary thread pool

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

• Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
• Ensure the tests and linter pass
• Code coverage does not decrease (if any source code was changed)
• Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

If you write sample code, please follow the samples format.