主题:
主机只有一台需要搭建三主三从的redis集群
开盘:
**第一步:**创建目录分发redis.conf; redis目录里创建data、conf目录,将redis.conf文件分别拷贝到conf目录里;修改redis.conf文件的port(从7001-7006)
redis.conf
#Redis configuration for testing. always-show-logo yes notify-keyspace-events KEA daemonize no pidfile /var/run/redis.pid port {按需添加端口} timeout 0 bind 0.0.0.0 loglevel verbose logfile '' databases 16 latency-monitor-threshold 1 save 900 1 save 300 10 save 60 10000 rdbcompression yes dbfilename dump.rdb dir ./ # 持久化配置 slave-serve-stale-data yes appendonly yes appendfilename appendonly.aof appendfsync everysec no-appendfsync-on-rewrite no activerehashing yes # 集群配置 cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 5000
**第二步,**便携docker-compose的yml文件;
docker-compose.yml
version: '3.1' services: redis-1: restart: always image: redis:latest container_name: redis-cluster-1 environment: - TZ=Asia/Shanghai network_mode: "host" volumes: - /data02/xxxx/middleware/redis/7001/data:/data:rw - /data02/xxxx/middleware/redis/7001/conf/redis.conf:/etc/redis/redis.conf:rw command: ["redis-server","/etc/redis/redis.conf"] redis-2: restart: always image: redis:latest container_name: redis-cluster-2 environment: - TZ=Asia/Shanghai network_mode: "host" volumes: - /data02/xxxx/middleware/redis/7002/data:/data:rw - /data02/xxxx/middleware/redis/7002/conf/redis.conf:/etc/redis/redis.conf:rw command: ["redis-server","/etc/redis/redis.conf"] redis-3: restart: always image: redis:latest container_name: redis-cluster-3 environment: - TZ=Asia/Shanghai network_mode: "host" volumes: - /data02/xxxx/middleware/redis/7003/data:/data:rw - /data02/xxxx/middleware/redis/7003/conf/redis.conf:/etc/redis/redis.conf:rw command: ["redis-server","/etc/redis/redis.conf"] redis-4: restart: always image: redis:latest container_name: redis-cluster-4 environment: - TZ=Asia/Shanghai network_mode: "host" volumes: - /data02/xxxx/middleware/redis/7004/data:/data:rw - /data02/xxxx/middleware/redis/7004/conf/redis.conf:/etc/redis/redis.conf:rw command: ["redis-server","/etc/redis/redis.conf"] redis-5: restart: always image: redis:latest container_name: redis-cluster-5 environment: - TZ=Asia/Shanghai network_mode: "host" volumes: - /data02/xxxx/middleware/redis/7005/data:/data:rw - /data02/xxxx/middleware/redis/7005/conf/redis.conf:/etc/redis/redis.conf:rw command: ["redis-server","/etc/redis/redis.conf"] redis-6: restart: always image: redis:latest container_name: redis-cluster-6 environment: - TZ=Asia/Shanghai network_mode: "host" volumes: - /data02/xxxx/middleware/redis/7006/data:/data:rw - /data02/xxxx/middleware/redis/7006/conf/redis.conf:/etc/redis/redis.conf:rw command: ["redis-server","/etc/redis/redis.conf"]
命令: docker-compose -f docker-compose.yml up -d #后台启动所有的容器
[root@host-192-168-0-4 redis]# docker-compose up -d Creating redis-1 ... done Creating redis-2 ... done Creating redis-3 ... done Creating redis-4 ... done Creating redis-6 ... done Creating redis-5 ... done
**第三步:**创建集群
docker exec -it xxxxxxxxxxxx /bin/bash redis-cli --cluster create 192.168.0.4:7001 192.168.0.4:7002 192.168.0.4:7003 192.168.0.4:7004 192.168.0.4:7005 192.168.0.4:7006 --cluster-replicas 1
中途有个yes
[root@host-192-168-0-4 redis]# docker exec -it xxxxxxxxxxxx /bin/bash redis-cli --cluster create 192.168.0.4:7001 192.168.0.4:7002 192.168.0.4:7003 192.168.0.4:7004 192.168.0.4:7005 192.168.0.4:7006 --cluster-replicas 1 ......... >>> Trying to optimize slaves allocation for anti-affinity ......... Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join .... ......... [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
有个问题需要注意和讨论:
我在第一次安装的时候,自建了网络使用bridge的模式;暴露端口7001->6379,17001->16379; 这样的话配置文件都指向同一个redis.conf即可;启动容器的时候可以正常启动,
默认的情况redis是在业务端口+10000,再启动一个端口用于redis集群的底层gossip协议通讯,按照理论来讲,按照宿主机地址创建集群即可;但是在添加的时候会卡在“Waiting for the cluster to join…”;网络规划指定了每个容器启动的ip地址,后将创建集群的ip地址改成容器地址,并制定6379端口;例如:
docker exec -it xxxxxxxxxxxx /bin/bash redis-cli --cluster create 172.17.0.4:7001 172.17.0.5:7002 172.17.0.6:7003 172.17.0.7:7004 172.17.0.8:7005 172.17.0.9:7006 --cluster-replicas 1
结果是集群顺利建立;但是在使用redis-ui测试的时候,出现了可以连接单机的redis,不能连接集群模式;至此问题暴露,即bridge模式下的redis不能建立集群模式;
经过一顿国内外搜索有两个观点:
1、redis的官网的说明,docker建立redis集群模式,需要配置网络模式为host,即yml里需要改成 network_mode: “host”
2、redis是通过gossip协议进行集群管理的,所有节点信息都将写入到node.conf里,如果配置了宿主机地址;docker容器不经修改防火墙规则,不能访问到宿主机的地址;即容器里不能访问到 192.168.0.4:17001、17002…17006
还有存有疑虑的事情,请路过的高手大侠施以援手:
1、当集群配置容器地址的时候,可以创建集群,但是业务系统不能访问以集群模式访问redis,这是为什么?
2、容器为什么不能访问到宿宿主机,我理解的是从容器发出来的包通过docker0网桥未NET地址,相当于内网访问,为什么不通呢 ?