简单明了搭建Redis哨兵模式

Redis镜像版本：docker.io/redis:6.2.5-alpine
docker版本：Docker version 1.13.1, build 7d71120/1.13.1
docker-compose版本：docker-compose version 1.24.1, build 4667896b
环境：CentOS7.9:2009

单机模式(开启AOF)

docker run -d -p 6379:6379 -v ~/redisdata:/data redis:6.2.5 redis-server --appendonly yes

主从模式

建立文件目录如下

~/docker
└── redis-master-slave
    └── redis-master-slave.yml

redis-master-slave.yml配置如下

version: '3'
services:
  master:
    environment: 
      - TZ=Asia/Shanghai
    image: docker.io/redis:6.2.5-alpine
    container_name: redis-master
    restart: always
    command: redis-server --requirepass root --masterauth root
    ports:
      - 6379:6379

  slave1:
    environment: 
      - TZ=Asia/Shanghai
    image: docker.io/redis:6.2.5-alpine
    container_name: redis-slave-1
    restart: always
    command: redis-server --slaveof redis-master 6379 --requirepass root --masterauth root 
    ports:
      - 6380:6379

  slave2:
    environment: 
      - TZ=Asia/Shanghai
    image: docker.io/redis:6.2.5-alpine
    container_name: redis-slave-2
    restart: always
    command: redis-server --slaveof redis-master 6379 --requirepass root --masterauth root 
    ports:
      - 6381:6379

启动命令

docker-compose -f redis-master-slave.yml -p redis up -d

查看实例中的redis角色的命令

docker exec redis-master redis-cli -a root info replication

不用进到容器里看,直接在shell下执行即可, 其中redis-master为容器名, -a指定redis密码

哨兵模式

哨兵模式是基于主从的, 建立文件目录如下

~/docker
└── redis-sentinel
    ├── redis-master-slave.yml
    ├── redis-sentinel-sentinel.yml
    ├── s1
    │   └── sentinel.conf
    ├── s2
    │   └── sentinel.conf
    └── s3
        └── sentinel.conf

只需要准备三件事

1. redis-master-slave.yml

redis-master-slave.yml配置与主从模式一样,可直接拷贝

2. redis-sentinel-sentinel.yml

redis-sentinel-sentinel.yml配置如下

version: '3'
services:
  sentinel1:
    environment: 
      - TZ=Asia/Shanghai
    image: docker.io/redis:6.2.5-alpine
    container_name: redis-sentinel-1
    restart: always
    command: redis-sentinel /usr/local/etc/redis/conf/sentinel.conf
    ports:
      - 26379:26379
    volumes:
      - ./s1/:/usr/local/etc/redis/conf/
    
  sentinel2:
    environment: 
      - TZ=Asia/Shanghai
    image: docker.io/redis:6.2.5-alpine
    container_name: redis-sentinel-2
    restart: always
    command: redis-sentinel /usr/local/etc/redis/conf/sentinel.conf
    ports:
      - 26380:26379
    volumes:
      - ./s2/:/usr/local/etc/redis/conf/
    
  sentinel3:
    environment: 
      - TZ=Asia/Shanghai
    image: docker.io/redis:6.2.5-alpine
    container_name: redis-sentinel-3
    restart: always
    command: redis-sentinel /usr/local/etc/redis/conf/sentinel.conf
    ports:
      - 26381:26379
    volumes:
      - ./s3/:/usr/local/etc/redis/conf/

networks:
  default:
    external:
      name: redis_default

注意networks配是主从模式的网络, 查看主从的任意一台实例可知网络, 查看命令

docker inspect redis-master |grep Networks -A 15

结果如下, 可知我的这台主服务器在网络redis_default上, ip为172.20.0.2

"Networks": {
                "redis_default": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "master",
                        "f609a0389d8d"
                    ],
                    "NetworkID": "6b0ded54960c5d9244c6b8ad4070082e1529cf35ea51ba05b436e2c23aca8e72",
                    "EndpointID": "26553cf90ca66458069b2f89500e535aecc9211c6923e36b0c552bf07597e0b3",
                    "Gateway": "172.20.0.1",
                    "IPAddress": "172.20.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,

3. sentinel.conf

以下的是s1下的sentinel.conf

port 26379
dir "/tmp"
sentinel monitor mymaster 172.20.0.2 6379 2
sentinel auth-pass mymaster root
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000
sentinel deny-scripts-reconfig yes

s2与s3下的配置文件的port分别改为26380, 26381, 其它一样

注意其中的172.20.0.2是master的docker网络的ip

启动命令

docker-compose -f redis-sentinel-sentinel.yml -p redis up -d

完成后查看容器运行情况

$ docker ps
CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS                                      NAMES
14307b04cf81        docker.io/redis:6.2.5-alpine   "docker-entrypoint..."   2 minutes ago       Up 2 minutes        6379/tcp, 0.0.0.0:26381->26379/tcp         redis-sentinel-3
c2230ea26e4c        docker.io/redis:6.2.5-alpine   "docker-entrypoint..."   2 minutes ago       Up 2 minutes        6379/tcp, 0.0.0.0:26380->26379/tcp         redis-sentinel-2
c64f3eb4c017        docker.io/redis:6.2.5-alpine   "docker-entrypoint..."   2 minutes ago       Up 2 minutes        6379/tcp, 0.0.0.0:26379->26379/tcp         redis-sentinel-1
7f603f09c7fe        docker.io/redis:6.2.5-alpine   "docker-entrypoint..."   46 minutes ago      Up 46 minutes       0.0.0.0:6381->6379/tcp                     redis-slave-2
c31c68676299        docker.io/redis:6.2.5-alpine   "docker-entrypoint..."   46 minutes ago      Up 46 minutes       0.0.0.0:6380->6379/tcp                     redis-slave-1
f609a0389d8d        docker.io/redis:6.2.5-alpine   "docker-entrypoint..."   46 minutes ago      Up 46 minutes       0.0.0.0:6379->6379/tcp                     redis-master

为方便试错, 将常用命令写成脚本

vim run

设想使用时方便程度如下

启动主从docker-compose: ./run up1
启动哨兵docker-compose: ./run up2
启动主从容器: ./run start1
启动哨兵容器: ./run start2
停止主从与哨兵所有容器: ./run stop
停止主从容器: ./run stop1
停止哨兵容器: ./run stop2
清除主从与哨兵所有容器: ./run clear
清除主从所有容器: ./run clear1
清除哨兵所有容器: ./run clear2

于是编写run文件内容如下(写完后用看需要使用chmod改可执行权限)

#!/bin/bash
if [ $# -gt 1 ]
then
  echo 'wrong arg numbers'
  exit
fi
op=$1
if test -z "$op"
then
  echo "null arg"
elif [ $op == 'stop' ]
then
  docker stop redis-master redis-slave-1 redis-slave-2 redis-sentinel-1 redis-sentinel-2 redis-sentinel-3
elif [ $op == 'stop1' ]
then
  docker stop redis-master redis-slave-1 redis-slave-2
elif [ $op == 'stop2' ]
then
  docker stop redis-sentinel-1 redis-sentinel-2 redis-sentinel-3
elif [ $op == 'clear' ]
then
  docker container rm redis-master redis-slave-1 redis-slave-2 redis-sentinel-1 redis-sentinel-2 redis-sentinel-3
elif [ $op == 'clear1' ]
then
  docker container rm redis-master redis-slave-1 redis-slave-2
elif [ $op == 'clear2' ]
then
  docker container rm redis-sentinel-1 redis-sentinel-2 redis-sentinel-3
elif [ $op == 'start1' ]
then
  docker start redis-master redis-slave-1 redis-slave-2
elif [ $op == 'start2' ]
then
  docker start redis-sentinel-1 redis-sentinel-2 redis-sentinel-3
elif [ $op == 'up1' ]
then
  docker-compose -f redis-master-slave.yml -p redis up -d
elif [ $op == 'up2' ]
then
  docker-compose -f redis-sentinel-sentinel.yml -p redis up -d
elif [ $op == 'info-master' ]
then
  docker exec redis-master redis-cli -a root info replication
elif [ $op == 'info-slave-1' ]
then
  docker exec redis-slave-1 redis-cli -a root info replication
elif [ $op == 'info-slave-2' ]
then
  docker exec redis-slave-2 redis-cli -a root info replication
elif [ $op == 'test' ]
then
  echo 'test pass'
else
  echo "arg invalid: $1"
fi

现在可以很方便地试错了~

搭建哨兵模式过程中的常见问题

指定sentinel.conf配置文件映射到容器内时直接使用文件映射, 这么做有可能导致哨兵没有写入配置文件的权限, 表现为WARNING: Sentinel was not able to save the new configuration on disk!!!: Device or resource busy. 解决方案:使用文件夹映射
看似搭建起来了, 可当stop掉master后哨兵却不会选举新的主节点, 可能是哨兵改写了sentinel.conf后使用了一样的myid. 解决方案: stop掉哨兵, 删掉myid那一行, 重新启动哨兵. myid会自动重新生成.
从库没有权限打开临时文件, 表现为同步时大量出现 Opening the temp file needed for MASTER <-> REPLICA synchronization: Permission denied 的log记录. 解决方案: sudo -u root ./redis-server [配置文件] 方式启动