Redis cluster bus port #10954
Description
Describe the bug
I am trying to run redis cluster in nomad. If I assign bus port increment by 10000 to the data port everything is fine.
However if I leave Nomad to assign the ports (it uses range 20000-32000) or I assign something that is not increment by 10000 the cluster create command is stuck on Waiting for the cluster to join and cluster cant be initialized. This is a problem because we can't really run more than one cluster. So it seems to me that cluster-bus in the config file does not overwrite the default behavior as described in the documentation. And the cluster expects the bus port to be +10000 of the data port.
Tested with official docker images, versions 7.0.2, 7.0, 6.2.7, 5.0.14
I have cluster-port defined in the configuration file.
Working with:
data port: 2164{0..6}
bus port: 3164{0..6}
Does not work:
data port: 2664{0..6}
bus port: 3164{0..6}
To reproduce
Create 6 redis instances in Nomad without setting static port and create cluster with them.
You can use the following Nomad file (needs to be run through Levant template)
Static port allocations are commented, if you remove the comments the cluster will work.
job "redis-cluster" {
datacenters = ["dc1"]
type = "service"
[[ range $idx := loop 6 ]]
group "redis-[[ $idx ]]" {
network {
port "db" {
//static = 638[[ $idx ]]
}
port "gossip" {
//static = 1638[[ $idx ]]
}
}
task "redis" {
driver = "docker"
config {
image = "redis:7.0.2"
ports = ["db", "gossip"]
args = [
"redis-server", "/etc/redis/redis.conf"
]
volumes = [
"local/redis.conf:/etc/redis/redis.conf"
]
}
template {
destination = "local/redis.conf"
change_mode = "restart"
data = <<EOF
#bind {{ env "NOMAD_IP_db" }}
bind 0.0.0.0
port {{ env "NOMAD_PORT_db"}}
########################## CLUSTER DOCKER/NAT support ########################
# In certain deployments, Redis Cluster nodes address discovery fails, because
# addresses are NAT-ted or because ports are forwarded (the typical case is
# Docker and other containers).
#
# In order to make Redis Cluster working in such environments, a static
# configuration where each node knows its public address is needed. The
# following two options are used for this scope, and are:
#
# * cluster-announce-ip
# * cluster-announce-port
# * cluster-announce-bus-port
cluster-announce-ip {{ env "NOMAD_IP_db" }}
cluster-announce-port {{ env "NOMAD_PORT_db" }}
cluster-announce-bus-port {{ env "NOMAD_PORT_gossip" }}
cluster-port {{ env "NOMAD_PORT_gossip"}}
cluster-node-timeout 15000
cluster-enabled yes
protected-mode no
cluster-config-file {{ env "NOMAD_TASK_DIR" }}/nodes.conf
cluster-node-timeout 5000
appendonly yes
EOF
}
service {
name = "redis"
port = "db"
check {
port = "db"
type = "tcp"
interval = "5s"
timeout = "3s"
#initial_status = "passing"
}
}
resources {
cpu = 100
memory = 100
}
}
}
[[ end ]]
}
Expected behavior
To be able to run cluster with any data and bus port range from 20000 to 32000