Inspired by a conversation in #1204 is this proposal to improve the information carried in every log line. In essence each log line should provide enough context so that an operator can infer origin and meainng, even when looked at in isolation in a remote place. From the original conversation:

Usually thinking about in what remote ends logs can end up in typical infrastructure I try to imagine what vital info is needed to make sense of it when consumed in some log infrastructure remote from the origin. Additionally how each log line can interpreted by itself without context of the lines before or after. As the logger is passed from the outside to the inside in the program we can enrich the keyvals with information as well which subsystems don't need to care about. A standard set of fields I try to surface, most of them which can easily be achieved with Dynamic Contextual Values from the go-kit log package:

Propsed set of standard fields:

• caller: https://godoc.org/github.com/go-kit/kit/log#Caller
• timestamp: https://godoc.org/github.com/go-kit/kit/log#pkg-variables
• revision: compile flag: https://github.com/tendermint/tendermint/blob/master/Makefile#L10
• job: tendermint in our case
• task: nodefor the node command
• service: what we pass to cmn.NewBaseService(...) e.g.: privValidatorSocketServer
• err: set if error present

All field names are up for discussion and can be adjusted according to some sane strategy.
Proper error propagation and messages are important for the overall goal but exceeds the scope of this issue