Skip to content

integration/clientv3/examples test flakes frequently #12983

Closed
Closed
integration/clientv3/examples test flakes frequently#12983
Labels
backport/v3.5priority/important-soonMust be staffed and worked on either currently, or very soon, ideally in time for the next release.staletype/flake
Milestone
etcd-v3.5
@ptabor

Description

@ptabor
ptabor
opened on May 17, 2021

ExampleCluster_memberAddAsLearner

I managed to repro this with:

for i in `seq 1 100`; do (cd tests && 'env' 'go' 'test' '-timeout=15m' '--race=false' '--cpu=4' './integration/clientv3/examples' --count=1 -v -run ExampleCluster_memberAddAsLearner| tee log.log); done

but it does not always flake.

Flake on Actions:
https://github.com/etcd-io/etcd/pull/12981/checks?check_run_id=2595306923

Uploaded log: logs_1206.zip

2021/05/16 18:49:34 2021-05-16T18:49:34.400Z	INFO	m2	started stream writer with remote peer	{"member": "m2", "local-member-id": "685c5c71ce9a9328", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.400Z	INFO	m2	started stream writer with remote peer	{"member": "m2", "local-member-id": "685c5c71ce9a9328", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.400Z	INFO	m2	started stream reader with remote peer	{"member": "m2", "stream-reader-type": "stream MsgApp v2", "local-member-id": "685c5c71ce9a9328", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.400Z	INFO	m0	started remote peer	{"member": "m0", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.400Z	INFO	m0	added remote peer	{"member": "m0", "local-member-id": "c5c5a20ca4073d6a", "remote-peer-id": "3b13bbeaeef551eb", "remote-peer-urls": ["http://localhost:32381"]}
2021/05/16 18:49:34 2021-05-16T18:49:34.400Z	INFO	m0	started stream writer with remote peer	{"member": "m0", "local-member-id": "c5c5a20ca4073d6a", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.400Z	INFO	m0	started stream writer with remote peer	{"member": "m0", "local-member-id": "c5c5a20ca4073d6a", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.400Z	INFO	m0	started stream reader with remote peer	{"member": "m0", "stream-reader-type": "stream MsgApp v2", "local-member-id": "c5c5a20ca4073d6a", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.400Z	INFO	m0	started stream reader with remote peer	{"member": "m0", "stream-reader-type": "stream Message", "local-member-id": "c5c5a20ca4073d6a", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.400Z	INFO	m2	started stream reader with remote peer	{"member": "m2", "stream-reader-type": "stream Message", "local-member-id": "685c5c71ce9a9328", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.401Z	INFO	m0	applied a configuration change through raft	{"member": "m0", "local-member-id": "c5c5a20ca4073d6a", "raft-conf-change": "ConfChangeAddLearnerNode", "raft-conf-change-node-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.401Z	INFO	m2.raft	685c5c71ce9a9328 ignoring conf change {ConfChangeRemoveNode 4256952690501046763 [] 4425483133757641240} at config voters=(7519987121669182248 13966091899056808041 14250974771059047786) learners=(4256952690501046763): possible unapplied conf change at index 25 (applied to 24)	{"member": "m2"}
2021/05/16 18:49:34 2021-05-16T18:49:34.401Z	INFO	m1.raft	c1d186962fbde069 switched to configuration voters=(7519987121669182248 13966091899056808041 14250974771059047786) learners=(4256952690501046763)	{"member": "m1"}
2021/05/16 18:49:34 2021-05-16T18:49:34.402Z	INFO	m1	added member	{"member": "m1", "cluster-id": "1cfa2497ce0563d4", "local-member-id": "c1d186962fbde069", "added-peer-id": "3b13bbeaeef551eb", "added-peer-peer-urls": ["http://localhost:32381"]}
2021/05/16 18:49:34 2021-05-16T18:49:34.402Z	INFO	m1	starting remote peer	{"member": "m1", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.402Z	INFO	m1	started HTTP pipelining with remote peer	{"member": "m1", "local-member-id": "c1d186962fbde069", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.402Z	WARN	m2	failed to reach the peer URL	{"member": "m2", "address": "http://localhost:32381/version", "remote-member-id": "3b13bbeaeef551eb", "error": "Get \"http://localhost:32381/version\": dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 18:49:34 2021-05-16T18:49:34.402Z	WARN	m2	failed to get version	{"member": "m2", "remote-member-id": "3b13bbeaeef551eb", "error": "Get \"http://localhost:32381/version\": dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 18:49:34 2021-05-16T18:49:34.402Z	INFO	m1	started stream writer with remote peer	{"member": "m1", "local-member-id": "c1d186962fbde069", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.402Z	INFO	m1	started stream writer with remote peer	{"member": "m1", "local-member-id": "c1d186962fbde069", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.402Z	INFO	m1	started remote peer	{"member": "m1", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.402Z	INFO	m1	added remote peer	{"member": "m1", "local-member-id": "c1d186962fbde069", "remote-peer-id": "3b13bbeaeef551eb", "remote-peer-urls": ["http://localhost:32381"]}
2021/05/16 18:49:34 2021-05-16T18:49:34.402Z	INFO	m1	started stream reader with remote peer	{"member": "m1", "stream-reader-type": "stream MsgApp v2", "local-member-id": "c1d186962fbde069", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:34 2021-05-16T18:49:34.403Z	INFO	m1	started stream reader with remote peer	{"member": "m1", "stream-reader-type": "stream Message", "local-member-id": "c1d186962fbde069", "remote-peer-id": "3b13bbeaeef551eb"}
2021/05/16 18:49:38 2021-05-16T18:49:38.404Z	WARN	m2	failed to reach the peer URL	{"member": "m2", "address": "http://localhost:32381/version", "remote-member-id": "3b13bbeaeef551eb", "error": "Get \"http://localhost:32381/version\": dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 18:49:38 2021-05-16T18:49:38.404Z	WARN	m2	failed to get version	{"member": "m2", "remote-member-id": "3b13bbeaeef551eb", "error": "Get \"http://localhost:32381/version\": dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 18:49:39 2021-05-16T18:49:39.400Z	WARN	m2	prober detected unhealthy status	{"member": "m2", "round-tripper-name": "ROUND_TRIPPER_RAFT_MESSAGE", "remote-peer-id": "3b13bbeaeef551eb", "rtt": "0s", "error": "dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 18:49:39 2021-05-16T18:49:39.401Z	WARN	m0	prober detected unhealthy status	{"member": "m0", "round-tripper-name": "ROUND_TRIPPER_RAFT_MESSAGE", "remote-peer-id": "3b13bbeaeef551eb", "rtt": "0s", "error": "dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 18:49:39 2021-05-16T18:49:39.401Z	WARN	m0	prober detected unhealthy status	{"member": "m0", "round-tripper-name": "ROUND_TRIPPER_SNAPSHOT", "remote-peer-id": "3b13bbeaeef551eb", "rtt": "0s", "error": "dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 18:49:39 2021-05-16T18:49:39.401Z	WARN	m2	prober detected unhealthy status	{"member": "m2", "round-tripper-name": "ROUND_TRIPPER_SNAPSHOT", "remote-peer-id": "3b13bbeaeef551eb", "rtt": "0s", "error": "dial tcp [::1]:32381: connect: connection refused"}

This line looks suspecious

2021/05/16 18:49:34 2021-05-16T18:49:34.401Z	INFO	m2.raft	685c5c71ce9a9328 ignoring conf change {ConfChangeRemoveNode 4256952690501046763 [] 4425483133757641240} at config voters=(7519987121669182248 13966091899056808041 14250974771059047786) learners=(4256952690501046763): possible unapplied conf change at index 25 (applied to 24)	{"member": "m2"}

goroutine 1 [select, 14 minutes]:
google.golang.org/grpc/internal/transport.(*Stream).waitOnHeader(0xc0003d70e0)
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.37.0/internal/transport/transport.go:322 +0x99
google.golang.org/grpc/internal/transport.(*Stream).RecvCompress(...)
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.37.0/internal/transport/transport.go:337
google.golang.org/grpc.(*csAttempt).recvMsg(0xc003a3ef00, 0x1025000, 0xc000284440, 0x0, 0x0, 0x0)
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.37.0/stream.go:937 +0x731
google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc003a3ef00, 0xc0037fcbe0, 0xa)
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.37.0/stream.go:802 +0x46
google.golang.org/grpc.(*clientStream).withRetry(0xc0003d6ea0, 0xc000379298, 0xc000379268, 0xc0037fcbea, 0x9c7a49)
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.37.0/stream.go:660 +0x9f
google.golang.org/grpc.(*clientStream).RecvMsg(0xc0003d6ea0, 0x1025000, 0xc000284440, 0x0, 0x0)
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.37.0/stream.go:801 +0x105
google.golang.org/grpc.invoke(0x11d7738, 0xc0038c1cb0, 0x109b415, 0x22, 0x101aea0, 0xc0038c1c50, 0x1025000, 0xc000284440, 0xc002d5e000, 0xc0002844c0, ...)
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.37.0/call.go:73 +0x142
go.etcd.io/etcd/client/v3.(*Client).unaryClientInterceptor.func1(0x11d76c8, 0xc0038c1cb0, 0x109b415, 0x22, 0x101aea0, 0xc0038c1c50, 0x1025000, 0xc000284440, 0xc002d5e000, 0x10c6d98, ...)
	/home/runner/work/etcd/etcd/client/v3/retry_interceptor.go:58 +0x46a
google.golang.org/grpc.(*ClientConn).Invoke(0xc002d5e000, 0x11d76c8, 0xc00011e010, 0x109b415, 0x22, 0x101aea0, 0xc0038c1c50, 0x1025000, 0xc000284440, 0x1823240, ...)
	/home/runner/go/pkg/mod/google.golang.org/grpc@v1.37.0/call.go:35 +0x109
go.etcd.io/etcd/api/v3/etcdserverpb.(*clusterClient).MemberRemove(0xc00018e108, 0x11d76c8, 0xc00011e010, 0xc0038c1c50, 0x1823240, 0x3, 0x3, 0xfc4be0, 0x1, 0xc0038c1c50)
	/home/runner/work/etcd/etcd/api/etcdserverpb/rpc.pb.go:7083 +0xcf
go.etcd.io/etcd/client/v3.(*retryClusterClient).MemberRemove(0xc0039fd310, 0x11d76c8, 0xc00011e010, 0xc0038c1c50, 0x1823240, 0x3, 0x3, 0xc0032fc4b0, 0x0, 0x0)
	/home/runner/work/etcd/etcd/client/v3/retry.go:175 +0x7c
go.etcd.io/etcd/client/v3.(*cluster).MemberRemove(0xc003a42db0, 0x11d76c8, 0xc00011e010, 0x3b13bbeaeef551eb, 0x1, 0x1, 0xc0032fc4b0)
	/home/runner/work/etcd/etcd/client/v3/cluster.go:103 +0x88
go.etcd.io/etcd/tests/v3/integration/clientv3/examples_test.ExampleCluster_memberAddAsLearner.func1()
	/home/runner/work/etcd/etcd/tests/integration/clientv3/examples/example_cluster_test.go:106 +0x247
go.etcd.io/etcd/tests/v3/integration/clientv3/examples_test.forUnitTestsRunInMockedContext(...)
	/home/runner/work/etcd/etcd/tests/integration/clientv3/examples/main_test.go:40
go.etcd.io/etcd/tests/v3/integration/clientv3/examples_test.ExampleCluster_memberAddAsLearner()
	/home/runner/work/etcd/etcd/tests/integration/clientv3/examples/example_cluster_test.go:90 +0x2b
testing.runExample(0x1099ea7, 0x21, 0x10c6868, 0x10a96f3, 0x2e, 0x0, 0x0)
	/opt/hostedtoolcache/go/1.16.4/x64/src/testing/run_example.go:63 +0x222
testing.runExamples(0xc000379e58, 0x1816a80, 0x18, 0x18, 0xc02079108fcf076d)
	/opt/hostedtoolcache/go/1.16.4/x64/src/testing/example.go:44 +0x17a
testing.(*M).Run(0xc0001f0380, 0x0)
	/opt/hostedtoolcache/go/1.16.4/x64/src/testing/testing.go:1418 +0x273
go.etcd.io/etcd/tests/v3/integration/clientv3/examples_test.TestMain(0xc0001f0380)
	/home/runner/work/etcd/etcd/tests/integration/clientv3/examples/main_test.go:46 +0x48
main.main()
	_testmain.go:91 +0x165

ptab@ptab ~/corp/etcd% (cd tests && 'env' 'go' 'test' '-timeout=15m' '--race=false' '--cpu=4' './integration/clientv3/examples' --count=1 -v -run ExampleCluster_memberAddAsLearner| tee log.log)
=== RUN   ExampleCluster_memberAddAsLearner
2021/05/16 22:58:56 Working directory '/home/ptab/corp/etcd/tests/integration/clientv3/examples' expected to be in temp-dir ('/tmp').Have you executed integration.BeforeTest(t) ?
2021/05/16 22:58:56 2021-05-16T22:58:56.587+0200	INFO	m0	LISTEN GRPC	{"member": "m0", "m.grpcAddr": "localhost:m0", "m.Name": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.587+0200	INFO	m1	LISTEN GRPC	{"member": "m1", "m.grpcAddr": "localhost:m1", "m.Name": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.587+0200	INFO	m2	LISTEN GRPC	{"member": "m2", "m.grpcAddr": "localhost:m2", "m.Name": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.587+0200	INFO	m2	launching a member	{"member": "m2", "name": "m2", "advertise-peer-urls": ["unix://127.0.0.1:210053101996"], "listen-client-urls": ["unix://127.0.0.1:210063101996"], "grpc-address": "unix://localhost:m20"}
2021/05/16 22:58:56 2021-05-16T22:58:56.587+0200	INFO	m1	launching a member	{"member": "m1", "name": "m1", "advertise-peer-urls": ["unix://127.0.0.1:210033101996"], "listen-client-urls": ["unix://127.0.0.1:210043101996"], "grpc-address": "unix://localhost:m10"}
2021/05/16 22:58:56 2021-05-16T22:58:56.587+0200	INFO	m0	launching a member	{"member": "m0", "name": "m0", "advertise-peer-urls": ["unix://127.0.0.1:210013101996"], "listen-client-urls": ["unix://127.0.0.1:210023101996"], "grpc-address": "unix://localhost:m00"}
2021/05/16 22:58:56 2021-05-16T22:58:56.605+0200	INFO	m1	opened backend db	{"member": "m1", "path": "/tmp/lazy_cluster813018228/etcd819471171/member/snap/db", "took": "17.295908ms"}
2021/05/16 22:58:56 2021-05-16T22:58:56.605+0200	INFO	m0	opened backend db	{"member": "m0", "path": "/tmp/lazy_cluster668773490/etcd777733929/member/snap/db", "took": "17.274012ms"}
2021/05/16 22:58:56 2021-05-16T22:58:56.605+0200	INFO	m2	opened backend db	{"member": "m2", "path": "/tmp/lazy_cluster025711558/etcd404414061/member/snap/db", "took": "17.357534ms"}
2021/05/16 22:58:56 2021-05-16T22:58:56.637+0200	INFO	m2	starting local member	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "cluster-id": "3c5b384f03b0f420"}
2021/05/16 22:58:56 2021-05-16T22:58:56.637+0200	INFO	m2.raft	f22d0a0f9f1b431b switched to configuration voters=()	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.637+0200	INFO	m2.raft	f22d0a0f9f1b431b became follower at term 0	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.637+0200	INFO	m2.raft	newRaft f22d0a0f9f1b431b [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.637+0200	INFO	m2.raft	f22d0a0f9f1b431b became follower at term 1	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.637+0200	INFO	m2.raft	f22d0a0f9f1b431b switched to configuration voters=(11609101907503085289)	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.637+0200	INFO	m2.raft	f22d0a0f9f1b431b switched to configuration voters=(11609101907503085289 16575215055615236278)	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.637+0200	INFO	m2.raft	f22d0a0f9f1b431b switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m1	starting local member	{"member": "m1", "local-member-id": "e606fe4619a718b6", "cluster-id": "3c5b384f03b0f420"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m1.raft	e606fe4619a718b6 switched to configuration voters=()	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m1.raft	e606fe4619a718b6 became follower at term 0	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m1.raft	newRaft e606fe4619a718b6 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m1.raft	e606fe4619a718b6 became follower at term 1	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m1.raft	e606fe4619a718b6 switched to configuration voters=(11609101907503085289)	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m1.raft	e606fe4619a718b6 switched to configuration voters=(11609101907503085289 16575215055615236278)	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m1.raft	e606fe4619a718b6 switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m0	starting local member	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "cluster-id": "3c5b384f03b0f420"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m0.raft	a11bd0ad5e07f6e9 switched to configuration voters=()	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.640+0200	INFO	m0.raft	a11bd0ad5e07f6e9 became follower at term 0	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.641+0200	INFO	m0.raft	newRaft a11bd0ad5e07f6e9 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.641+0200	INFO	m0.raft	a11bd0ad5e07f6e9 became follower at term 1	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.641+0200	INFO	m0.raft	a11bd0ad5e07f6e9 switched to configuration voters=(11609101907503085289)	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.641+0200	INFO	m0.raft	a11bd0ad5e07f6e9 switched to configuration voters=(11609101907503085289 16575215055615236278)	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.641+0200	INFO	m0.raft	a11bd0ad5e07f6e9 switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.648+0200	WARN	m2	simple token is not cryptographically signed	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.651+0200	WARN	m1	simple token is not cryptographically signed	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.651+0200	WARN	m0	simple token is not cryptographically signed	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.652+0200	INFO	m2	kvstore restored	{"member": "m2", "current-rev": 1}
2021/05/16 22:58:56 2021-05-16T22:58:56.655+0200	INFO	m1	kvstore restored	{"member": "m1", "current-rev": 1}
2021/05/16 22:58:56 2021-05-16T22:58:56.658+0200	INFO	m0	kvstore restored	{"member": "m0", "current-rev": 1}
2021/05/16 22:58:56 2021-05-16T22:58:56.658+0200	INFO	m2	enabled backend quota with default value	{"member": "m2", "quota-name": "v3-applier", "quota-size-bytes": 2147483648, "quota-size": "2.1 GB"}
2021/05/16 22:58:56 2021-05-16T22:58:56.661+0200	INFO	m2	starting remote peer	{"member": "m2", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.661+0200	INFO	m2	started HTTP pipelining with remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.662+0200	INFO	m2	started stream writer with remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.663+0200	INFO	m2	started stream writer with remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.663+0200	INFO	m1	starting remote peer	{"member": "m1", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.663+0200	INFO	m1	started HTTP pipelining with remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.664+0200	INFO	m1	started stream writer with remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.665+0200	INFO	m2	started remote peer	{"member": "m2", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.665+0200	INFO	m2	added remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "a11bd0ad5e07f6e9", "remote-peer-urls": ["unix://127.0.0.1:210013101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.665+0200	INFO	m2	starting remote peer	{"member": "m2", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.665+0200	INFO	m2	started HTTP pipelining with remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.665+0200	INFO	m2	started stream reader with remote peer	{"member": "m2", "stream-reader-type": "stream Message", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.666+0200	INFO	m2	started stream reader with remote peer	{"member": "m2", "stream-reader-type": "stream MsgApp v2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.666+0200	INFO	m0	starting remote peer	{"member": "m0", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.666+0200	INFO	m0	started HTTP pipelining with remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.669+0200	INFO	m2	started stream writer with remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.670+0200	INFO	m1	started remote peer	{"member": "m1", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.670+0200	INFO	m1	added remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "a11bd0ad5e07f6e9", "remote-peer-urls": ["unix://127.0.0.1:210013101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.670+0200	INFO	m1	starting remote peer	{"member": "m1", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.670+0200	INFO	m1	started HTTP pipelining with remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.671+0200	INFO	m0	started remote peer	{"member": "m0", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.672+0200	INFO	m0	added remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "e606fe4619a718b6", "remote-peer-urls": ["unix://127.0.0.1:210033101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.672+0200	INFO	m0	starting remote peer	{"member": "m0", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.672+0200	INFO	m0	started HTTP pipelining with remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.673+0200	INFO	m1	started stream writer with remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.673+0200	INFO	m1	started stream writer with remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.673+0200	INFO	m0	started stream reader with remote peer	{"member": "m0", "stream-reader-type": "stream MsgApp v2", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.673+0200	INFO	m1	started stream reader with remote peer	{"member": "m1", "stream-reader-type": "stream Message", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.673+0200	INFO	m0	started stream reader with remote peer	{"member": "m0", "stream-reader-type": "stream Message", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.672+0200	INFO	m1	started stream reader with remote peer	{"member": "m1", "stream-reader-type": "stream MsgApp v2", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.673+0200	INFO	m2	started stream writer with remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.673+0200	INFO	m1	started stream writer with remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.673+0200	INFO	m0	started stream writer with remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m0	started stream writer with remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m2	started remote peer	{"member": "m2", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m2	added remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "e606fe4619a718b6", "remote-peer-urls": ["unix://127.0.0.1:210033101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m2	starting etcd server	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "local-server-version": "3.5.0-alpha.0", "cluster-version": "to_be_decided"}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m1	started remote peer	{"member": "m1", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m1	added remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "f22d0a0f9f1b431b", "remote-peer-urls": ["unix://127.0.0.1:210053101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m1	starting etcd server	{"member": "m1", "local-member-id": "e606fe4619a718b6", "local-server-version": "3.5.0-alpha.0", "cluster-version": "to_be_decided"}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m1	started stream reader with remote peer	{"member": "m1", "stream-reader-type": "stream MsgApp v2", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m1	started stream reader with remote peer	{"member": "m1", "stream-reader-type": "stream Message", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m0	started remote peer	{"member": "m0", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m0	added remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "f22d0a0f9f1b431b", "remote-peer-urls": ["unix://127.0.0.1:210053101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m0	starting etcd server	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "local-server-version": "3.5.0-alpha.0", "cluster-version": "to_be_decided"}
2021/05/16 22:58:56 2021-05-16T22:58:56.674+0200	INFO	m0	started stream writer with remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.675+0200	INFO	m0	started stream writer with remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.675+0200	INFO	m0	started stream reader with remote peer	{"member": "m0", "stream-reader-type": "stream MsgApp v2", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.675+0200	INFO	m0	started stream reader with remote peer	{"member": "m0", "stream-reader-type": "stream Message", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.675+0200	INFO	m1	starting initial election tick advance	{"member": "m1", "election-ticks": 10}
2021/05/16 22:58:56 2021-05-16T22:58:56.676+0200	INFO	m1.raft	e606fe4619a718b6 switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.676+0200	INFO	m1	added member	{"member": "m1", "cluster-id": "3c5b384f03b0f420", "local-member-id": "e606fe4619a718b6", "added-peer-id": "a11bd0ad5e07f6e9", "added-peer-peer-urls": ["unix://127.0.0.1:210013101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.676+0200	INFO	m0	starting initial election tick advance	{"member": "m0", "election-ticks": 10}
2021/05/16 22:58:56 2021-05-16T22:58:56.676+0200	INFO	m1.raft	e606fe4619a718b6 switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.677+0200	INFO	m1	added member	{"member": "m1", "cluster-id": "3c5b384f03b0f420", "local-member-id": "e606fe4619a718b6", "added-peer-id": "e606fe4619a718b6", "added-peer-peer-urls": ["unix://127.0.0.1:210033101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.677+0200	INFO	m1.raft	e606fe4619a718b6 switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.677+0200	INFO	m1	added member	{"member": "m1", "cluster-id": "3c5b384f03b0f420", "local-member-id": "e606fe4619a718b6", "added-peer-id": "f22d0a0f9f1b431b", "added-peer-peer-urls": ["unix://127.0.0.1:210053101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.677+0200	INFO	m0.raft	a11bd0ad5e07f6e9 switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.677+0200	INFO	m2	starting initial election tick advance	{"member": "m2", "election-ticks": 10}
2021/05/16 22:58:56 2021-05-16T22:58:56.677+0200	INFO	m2	started stream reader with remote peer	{"member": "m2", "stream-reader-type": "stream Message", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.677+0200	INFO	m2.raft	f22d0a0f9f1b431b switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.677+0200	INFO	m0	added member	{"member": "m0", "cluster-id": "3c5b384f03b0f420", "local-member-id": "a11bd0ad5e07f6e9", "added-peer-id": "a11bd0ad5e07f6e9", "added-peer-peer-urls": ["unix://127.0.0.1:210013101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.678+0200	INFO	m2	added member	{"member": "m2", "cluster-id": "3c5b384f03b0f420", "local-member-id": "f22d0a0f9f1b431b", "added-peer-id": "a11bd0ad5e07f6e9", "added-peer-peer-urls": ["unix://127.0.0.1:210013101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.678+0200	INFO	m0.raft	a11bd0ad5e07f6e9 switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.678+0200	INFO	m2.raft	f22d0a0f9f1b431b switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.678+0200	INFO	m0	added member	{"member": "m0", "cluster-id": "3c5b384f03b0f420", "local-member-id": "a11bd0ad5e07f6e9", "added-peer-id": "e606fe4619a718b6", "added-peer-peer-urls": ["unix://127.0.0.1:210033101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.678+0200	INFO	m2	added member	{"member": "m2", "cluster-id": "3c5b384f03b0f420", "local-member-id": "f22d0a0f9f1b431b", "added-peer-id": "e606fe4619a718b6", "added-peer-peer-urls": ["unix://127.0.0.1:210033101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.678+0200	INFO	m0.raft	a11bd0ad5e07f6e9 switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.678+0200	INFO	m2.raft	f22d0a0f9f1b431b switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227)	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.678+0200	INFO	m0	added member	{"member": "m0", "cluster-id": "3c5b384f03b0f420", "local-member-id": "a11bd0ad5e07f6e9", "added-peer-id": "f22d0a0f9f1b431b", "added-peer-peer-urls": ["unix://127.0.0.1:210053101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.678+0200	INFO	m2	added member	{"member": "m2", "cluster-id": "3c5b384f03b0f420", "local-member-id": "f22d0a0f9f1b431b", "added-peer-id": "f22d0a0f9f1b431b", "added-peer-peer-urls": ["unix://127.0.0.1:210053101996"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.677+0200	INFO	m2	started stream reader with remote peer	{"member": "m2", "stream-reader-type": "stream MsgApp v2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.681+0200	INFO	m2	set message encoder	{"member": "m2", "from": "f22d0a0f9f1b431b", "to": "a11bd0ad5e07f6e9", "stream-type": "stream Message"}
2021/05/16 22:58:56 2021-05-16T22:58:56.681+0200	INFO	m0	set message encoder	{"member": "m0", "from": "a11bd0ad5e07f6e9", "to": "f22d0a0f9f1b431b", "stream-type": "stream Message"}
2021/05/16 22:58:56 2021-05-16T22:58:56.681+0200	INFO	m0	peer became active	{"member": "m0", "peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.681+0200	INFO	m0	established TCP streaming connection with remote peer	{"member": "m0", "stream-writer-type": "stream Message", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.681+0200	INFO	m2	peer became active	{"member": "m2", "peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.681+0200	INFO	m2	established TCP streaming connection with remote peer	{"member": "m2", "stream-writer-type": "stream Message", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.682+0200	INFO	m2	launched a member	{"member": "m2", "name": "m2", "advertise-peer-urls": ["unix://127.0.0.1:210053101996"], "listen-client-urls": ["unix://127.0.0.1:210063101996"], "grpc-address": "unix://localhost:m20"}
2021/05/16 22:58:56 2021-05-16T22:58:56.682+0200	INFO	m1	launched a member	{"member": "m1", "name": "m1", "advertise-peer-urls": ["unix://127.0.0.1:210033101996"], "listen-client-urls": ["unix://127.0.0.1:210043101996"], "grpc-address": "unix://localhost:m10"}
2021/05/16 22:58:56 2021-05-16T22:58:56.682+0200	INFO	m0	established TCP streaming connection with remote peer	{"member": "m0", "stream-reader-type": "stream Message", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.682+0200	INFO	m2	established TCP streaming connection with remote peer	{"member": "m2", "stream-reader-type": "stream Message", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.682+0200	INFO	m2	set message encoder	{"member": "m2", "from": "f22d0a0f9f1b431b", "to": "e606fe4619a718b6", "stream-type": "stream MsgApp v2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.683+0200	INFO	m2	peer became active	{"member": "m2", "peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.683+0200	INFO	m2	established TCP streaming connection with remote peer	{"member": "m2", "stream-writer-type": "stream MsgApp v2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.683+0200	INFO	m2	set message encoder	{"member": "m2", "from": "f22d0a0f9f1b431b", "to": "e606fe4619a718b6", "stream-type": "stream Message"}
2021/05/16 22:58:56 2021-05-16T22:58:56.683+0200	INFO	m2	established TCP streaming connection with remote peer	{"member": "m2", "stream-writer-type": "stream Message", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.685+0200	INFO	m2	set message encoder	{"member": "m2", "from": "f22d0a0f9f1b431b", "to": "a11bd0ad5e07f6e9", "stream-type": "stream MsgApp v2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.685+0200	INFO	m2	established TCP streaming connection with remote peer	{"member": "m2", "stream-writer-type": "stream MsgApp v2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.685+0200	INFO	m0	launched a member	{"member": "m0", "name": "m0", "advertise-peer-urls": ["unix://127.0.0.1:210013101996"], "listen-client-urls": ["unix://127.0.0.1:210023101996"], "grpc-address": "unix://localhost:m00"}
2021/05/16 22:58:56 2021-05-16T22:58:56.686+0200	INFO	m2	established TCP streaming connection with remote peer	{"member": "m2", "stream-reader-type": "stream MsgApp v2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.686+0200	INFO	m1	peer became active	{"member": "m1", "peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.686+0200	INFO	m1	established TCP streaming connection with remote peer	{"member": "m1", "stream-reader-type": "stream MsgApp v2", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.686+0200	INFO	m2	established TCP streaming connection with remote peer	{"member": "m2", "stream-reader-type": "stream MsgApp v2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m1	set message encoder	{"member": "m1", "from": "e606fe4619a718b6", "to": "a11bd0ad5e07f6e9", "stream-type": "stream MsgApp v2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m1	peer became active	{"member": "m1", "peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m1	established TCP streaming connection with remote peer	{"member": "m1", "stream-writer-type": "stream MsgApp v2", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m0	set message encoder	{"member": "m0", "from": "a11bd0ad5e07f6e9", "to": "f22d0a0f9f1b431b", "stream-type": "stream MsgApp v2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m0	established TCP streaming connection with remote peer	{"member": "m0", "stream-writer-type": "stream MsgApp v2", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m1	set message encoder	{"member": "m1", "from": "e606fe4619a718b6", "to": "f22d0a0f9f1b431b", "stream-type": "stream MsgApp v2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m1	established TCP streaming connection with remote peer	{"member": "m1", "stream-writer-type": "stream MsgApp v2", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m1	set message encoder	{"member": "m1", "from": "e606fe4619a718b6", "to": "a11bd0ad5e07f6e9", "stream-type": "stream Message"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m1	established TCP streaming connection with remote peer	{"member": "m1", "stream-writer-type": "stream Message", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m1	set message encoder	{"member": "m1", "from": "e606fe4619a718b6", "to": "f22d0a0f9f1b431b", "stream-type": "stream Message"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m1	established TCP streaming connection with remote peer	{"member": "m1", "stream-writer-type": "stream Message", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m2	established TCP streaming connection with remote peer	{"member": "m2", "stream-reader-type": "stream Message", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.687+0200	INFO	m1	established TCP streaming connection with remote peer	{"member": "m1", "stream-reader-type": "stream MsgApp v2", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.688+0200	INFO	m0	set message encoder	{"member": "m0", "from": "a11bd0ad5e07f6e9", "to": "e606fe4619a718b6", "stream-type": "stream MsgApp v2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.688+0200	INFO	m0	peer became active	{"member": "m0", "peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.688+0200	INFO	m0	established TCP streaming connection with remote peer	{"member": "m0", "stream-writer-type": "stream MsgApp v2", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.688+0200	INFO	m0	established TCP streaming connection with remote peer	{"member": "m0", "stream-reader-type": "stream MsgApp v2", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.688+0200	INFO	m0	established TCP streaming connection with remote peer	{"member": "m0", "stream-reader-type": "stream Message", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.689+0200	INFO	m0	established TCP streaming connection with remote peer	{"member": "m0", "stream-reader-type": "stream MsgApp v2", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.690+0200	INFO	m0	set message encoder	{"member": "m0", "from": "a11bd0ad5e07f6e9", "to": "e606fe4619a718b6", "stream-type": "stream Message"}
2021/05/16 22:58:56 2021-05-16T22:58:56.690+0200	INFO	m0	established TCP streaming connection with remote peer	{"member": "m0", "stream-writer-type": "stream Message", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "e606fe4619a718b6"}
2021/05/16 22:58:56 2021-05-16T22:58:56.690+0200	INFO	m1	established TCP streaming connection with remote peer	{"member": "m1", "stream-reader-type": "stream Message", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "a11bd0ad5e07f6e9"}
2021/05/16 22:58:56 2021-05-16T22:58:56.690+0200	INFO	m1	established TCP streaming connection with remote peer	{"member": "m1", "stream-reader-type": "stream Message", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "f22d0a0f9f1b431b"}
2021/05/16 22:58:56 2021-05-16T22:58:56.726+0200	INFO	m1	initialized peer connections; fast-forwarding election ticks	{"member": "m1", "local-member-id": "e606fe4619a718b6", "forward-ticks": 8, "forward-duration": "80ms", "election-ticks": 10, "election-timeout": "100ms", "active-remote-members": 2}
2021/05/16 22:58:56 2021-05-16T22:58:56.726+0200	INFO	m1.raft	e606fe4619a718b6 is starting a new election at term 1	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.726+0200	INFO	m1.raft	e606fe4619a718b6 became candidate at term 2	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.726+0200	INFO	m1.raft	e606fe4619a718b6 received MsgVoteResp from e606fe4619a718b6 at term 2	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.726+0200	INFO	m1.raft	e606fe4619a718b6 [logterm: 1, index: 3] sent MsgVote request to a11bd0ad5e07f6e9 at term 2	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.726+0200	INFO	m1.raft	e606fe4619a718b6 [logterm: 1, index: 3] sent MsgVote request to f22d0a0f9f1b431b at term 2	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.726+0200	INFO	m0	initialized peer connections; fast-forwarding election ticks	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "forward-ticks": 8, "forward-duration": "80ms", "election-ticks": 10, "election-timeout": "100ms", "active-remote-members": 2}
2021/05/16 22:58:56 2021-05-16T22:58:56.728+0200	INFO	m2	initialized peer connections; fast-forwarding election ticks	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "forward-ticks": 8, "forward-duration": "80ms", "election-ticks": 10, "election-timeout": "100ms", "active-remote-members": 2}
2021/05/16 22:58:56 2021-05-16T22:58:56.728+0200	INFO	m2.raft	f22d0a0f9f1b431b [term: 1] received a MsgVote message with higher term from e606fe4619a718b6 [term: 2]	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.728+0200	INFO	m2.raft	f22d0a0f9f1b431b became follower at term 2	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.728+0200	INFO	m2.raft	f22d0a0f9f1b431b [logterm: 1, index: 3, vote: 0] cast MsgVote for e606fe4619a718b6 [logterm: 1, index: 3] at term 2	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.728+0200	INFO	m0.raft	a11bd0ad5e07f6e9 [term: 1] received a MsgVote message with higher term from e606fe4619a718b6 [term: 2]	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.728+0200	INFO	m0.raft	a11bd0ad5e07f6e9 became follower at term 2	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.728+0200	INFO	m0.raft	a11bd0ad5e07f6e9 [logterm: 1, index: 3, vote: 0] cast MsgVote for e606fe4619a718b6 [logterm: 1, index: 3] at term 2	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.730+0200	INFO	m1.raft	e606fe4619a718b6 received MsgVoteResp from a11bd0ad5e07f6e9 at term 2	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.730+0200	INFO	m1.raft	e606fe4619a718b6 has received 2 MsgVoteResp votes and 0 vote rejections	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.730+0200	INFO	m1.raft	e606fe4619a718b6 became leader at term 2	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.730+0200	INFO	m1.raft	raft.node: e606fe4619a718b6 elected leader e606fe4619a718b6 at term 2	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.730+0200	INFO	m2.raft	raft.node: f22d0a0f9f1b431b elected leader e606fe4619a718b6 at term 2	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.730+0200	INFO	m0.raft	raft.node: a11bd0ad5e07f6e9 elected leader e606fe4619a718b6 at term 2	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.735+0200	INFO	m1	setting up initial cluster version	{"member": "m1", "cluster-version": "3.5"}
2021/05/16 22:58:56 2021-05-16T22:58:56.738+0200	INFO	m1	set initial cluster version	{"member": "m1", "cluster-id": "3c5b384f03b0f420", "local-member-id": "e606fe4619a718b6", "cluster-version": "3.5"}
2021/05/16 22:58:56 2021-05-16T22:58:56.739+0200	INFO	m1	enabled capabilities for version	{"member": "m1", "cluster-version": "3.5"}
2021/05/16 22:58:56 2021-05-16T22:58:56.739+0200	INFO	m0	set initial cluster version	{"member": "m0", "cluster-id": "3c5b384f03b0f420", "local-member-id": "a11bd0ad5e07f6e9", "cluster-version": "3.5"}
2021/05/16 22:58:56 2021-05-16T22:58:56.739+0200	INFO	m2	set initial cluster version	{"member": "m2", "cluster-id": "3c5b384f03b0f420", "local-member-id": "f22d0a0f9f1b431b", "cluster-version": "3.5"}
2021/05/16 22:58:56 2021-05-16T22:58:56.739+0200	INFO	m0	published local member to cluster through raft	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "local-member-attributes": "{Name:m0 ClientURLs:[unix://127.0.0.1:210023101996]}", "request-path": "/0/members/a11bd0ad5e07f6e9/attributes", "cluster-id": "3c5b384f03b0f420", "publish-timeout": "5.2s"}
2021/05/16 22:58:56 2021-05-16T22:58:56.739+0200	INFO	m1	published local member to cluster through raft	{"member": "m1", "local-member-id": "e606fe4619a718b6", "local-member-attributes": "{Name:m1 ClientURLs:[unix://127.0.0.1:210043101996]}", "request-path": "/0/members/e606fe4619a718b6/attributes", "cluster-id": "3c5b384f03b0f420", "publish-timeout": "5.2s"}
2021/05/16 22:58:56 2021-05-16T22:58:56.739+0200	INFO	m2	published local member to cluster through raft	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "local-member-attributes": "{Name:m2 ClientURLs:[unix://127.0.0.1:210063101996]}", "request-path": "/0/members/f22d0a0f9f1b431b/attributes", "cluster-id": "3c5b384f03b0f420", "publish-timeout": "5.2s"}
2021/05/16 22:58:56  - m0 -> a11bd0ad5e07f6e9 (unix://localhost:m00)
2021/05/16 22:58:56  - m1 -> e606fe4619a718b6 (unix://localhost:m10)
2021/05/16 22:58:56  - m2 -> f22d0a0f9f1b431b (unix://localhost:m20)
2021/05/16 22:58:56 2021-05-16T22:58:56.782+0200	INFO	m1.raft	e606fe4619a718b6 switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227) learners=(9777699696455160727)	{"member": "m1"}
2021/05/16 22:58:56 2021-05-16T22:58:56.782+0200	INFO	m2.raft	f22d0a0f9f1b431b switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227) learners=(9777699696455160727)	{"member": "m2"}
2021/05/16 22:58:56 2021-05-16T22:58:56.782+0200	INFO	m1	added member	{"member": "m1", "cluster-id": "3c5b384f03b0f420", "local-member-id": "e606fe4619a718b6", "added-peer-id": "87b15e11c7a79397", "added-peer-peer-urls": ["http://localhost:32381"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.782+0200	INFO	m1	starting remote peer	{"member": "m1", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.782+0200	INFO	m1	started HTTP pipelining with remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.782+0200	INFO	m2	added member	{"member": "m2", "cluster-id": "3c5b384f03b0f420", "local-member-id": "f22d0a0f9f1b431b", "added-peer-id": "87b15e11c7a79397", "added-peer-peer-urls": ["http://localhost:32381"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.782+0200	INFO	m2	starting remote peer	{"member": "m2", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.782+0200	INFO	m2	started HTTP pipelining with remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.783+0200	INFO	m1	started remote peer	{"member": "m1", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.783+0200	INFO	m1	added remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "87b15e11c7a79397", "remote-peer-urls": ["http://localhost:32381"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.783+0200	INFO	m0.raft	a11bd0ad5e07f6e9 switched to configuration voters=(11609101907503085289 16575215055615236278 17450615193340691227) learners=(9777699696455160727)	{"member": "m0"}
2021/05/16 22:58:56 2021-05-16T22:58:56.784+0200	INFO	m1	started stream writer with remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.784+0200	INFO	m1	started stream writer with remote peer	{"member": "m1", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.784+0200	INFO	m1	started stream reader with remote peer	{"member": "m1", "stream-reader-type": "stream MsgApp v2", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.785+0200	INFO	m0	added member	{"member": "m0", "cluster-id": "3c5b384f03b0f420", "local-member-id": "a11bd0ad5e07f6e9", "added-peer-id": "87b15e11c7a79397", "added-peer-peer-urls": ["http://localhost:32381"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.785+0200	INFO	m0	starting remote peer	{"member": "m0", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.786+0200	INFO	m0	started HTTP pipelining with remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.786+0200	INFO	m2	started remote peer	{"member": "m2", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.786+0200	INFO	m2	added remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "87b15e11c7a79397", "remote-peer-urls": ["http://localhost:32381"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.786+0200	INFO	m2	started stream writer with remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.787+0200	INFO	m2	started stream writer with remote peer	{"member": "m2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.787+0200	INFO	m1	started stream reader with remote peer	{"member": "m1", "stream-reader-type": "stream Message", "local-member-id": "e606fe4619a718b6", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.787+0200	INFO	m0	started remote peer	{"member": "m0", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.787+0200	INFO	m2	started stream reader with remote peer	{"member": "m2", "stream-reader-type": "stream MsgApp v2", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.787+0200	INFO	m2	started stream reader with remote peer	{"member": "m2", "stream-reader-type": "stream Message", "local-member-id": "f22d0a0f9f1b431b", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.788+0200	INFO	m0	added remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "87b15e11c7a79397", "remote-peer-urls": ["http://localhost:32381"]}
2021/05/16 22:58:56 2021-05-16T22:58:56.788+0200	INFO	m0	started stream writer with remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.789+0200	INFO	m0	started stream writer with remote peer	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.789+0200	INFO	m0	started stream reader with remote peer	{"member": "m0", "stream-reader-type": "stream MsgApp v2", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.789+0200	INFO	m0	started stream reader with remote peer	{"member": "m0", "stream-reader-type": "stream Message", "local-member-id": "a11bd0ad5e07f6e9", "remote-peer-id": "87b15e11c7a79397"}
2021/05/16 22:58:56 2021-05-16T22:58:56.789+0200	INFO	m0	applied a configuration change through raft	{"member": "m0", "local-member-id": "a11bd0ad5e07f6e9", "raft-conf-change": "ConfChangeAddLearnerNode", "raft-conf-change-node-id": "87b15e11c7a79397"}

!!!  2021/05/16 22:58:56 2021-05-16T22:58:56.790+0200	INFO	m1.raft	e606fe4619a718b6 ignoring conf change {ConfChangeRemoveNode 9777699696455160727 [] 17791885354803724549} at config voters=(11609101907503085289 16575215055615236278 17450615193340691227) learners=(9777699696455160727): possible unapplied conf change at index 9 (applied to 8)	{"member": "m1"}

2021/05/16 22:58:56 2021-05-16T22:58:56.794+0200	WARN	m1	failed to reach the peer URL	{"member": "m1", "address": "http://localhost:32381/version", "remote-member-id": "87b15e11c7a79397", "error": "Get \"http://localhost:32381/version\": dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 22:58:56 2021-05-16T22:58:56.794+0200	WARN	m1	failed to get version	{"member": "m1", "remote-member-id": "87b15e11c7a79397", "error": "Get \"http://localhost:32381/version\": dial tcp [::1]:32381: connect: connection refused"}

2021/05/16 22:59:00 2021-05-16T22:59:00.797+0200	WARN	m1	failed to reach the peer URL	{"member": "m1", "address": "http://localhost:32381/version", "remote-member-id": "87b15e11c7a79397", "error": "Get \"http://localhost:32381/version\": dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 22:59:00 2021-05-16T22:59:00.798+0200	WARN	m1	failed to get version	{"member": "m1", "remote-member-id": "87b15e11c7a79397", "error": "Get \"http://localhost:32381/version\": dial tcp [::1]:32381: connect: connection refused"}


2021/05/16 22:59:01 2021-05-16T22:59:01.784+0200	WARN	m1	prober detected unhealthy status	{"member": "m1", "round-tripper-name": "ROUND_TRIPPER_RAFT_MESSAGE", "remote-peer-id": "87b15e11c7a79397", "rtt": "0s", "error": "dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 22:59:01 2021-05-16T22:59:01.787+0200	WARN	m1	prober detected unhealthy status	{"member": "m1", "round-tripper-name": "ROUND_TRIPPER_SNAPSHOT", "remote-peer-id": "87b15e11c7a79397", "rtt": "0s", "error": "dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 22:59:01 2021-05-16T22:59:01.788+0200	WARN	m2	prober detected unhealthy status	{"member": "m2", "round-tripper-name": "ROUND_TRIPPER_SNAPSHOT", "remote-peer-id": "87b15e11c7a79397", "rtt": "0s", "error": "dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 22:59:01 2021-05-16T22:59:01.788+0200	WARN	m2	prober detected unhealthy status	{"member": "m2", "round-tripper-name": "ROUND_TRIPPER_RAFT_MESSAGE", "remote-peer-id": "87b15e11c7a79397", "rtt": "0s", "error": "dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 22:59:01 2021-05-16T22:59:01.789+0200	WARN	m0	prober detected unhealthy status	{"member": "m0", "round-tripper-name": "ROUND_TRIPPER_SNAPSHOT", "remote-peer-id": "87b15e11c7a79397", "rtt": "0s", "error": "dial tcp [::1]:32381: connect: connection refused"}
2021/05/16 22:59:01 2021-05-16T22:59:01.790+0200	WARN	m0	prober detected unhealthy status	{"member": "m0", "round-tripper-name": "ROUND_TRIPPER_RAFT_MESSAGE", "remote-peer-id": "87b15e11c7a79397", "rtt": "0s", "error": "dial tcp [::1]:32381: connect: connection refused"}

Metadata

Metadata

Assignees

No one assigned

    Labels

    backport/v3.5priority/important-soonMust be staffed and worked on either currently, or very soon, ideally in time for the next release.staletype/flake

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions