We would like to have something like a side-channel for arbitrary messaging between nodes. Such messages should not influence consensus/mempool. One solution is to write a custom p2p layer on top of our ABCI app(s) but why not use the existing p2p network established by the nodes?
We could have a subscribe endpoint (like provided for WSRPC)
Could you implement such a thing? If yes, how difficult would that be?

something like /broadcast_sidechannel?msg=“Hi There!”