redis集群方案

redis cluster + predixy + keepalived

redis使用官方的redis cluster集群方案。因为redis cluster是散列模型,需读对开发提供所有redis节点的访问地址,于是使用predixy对redis cluster节点进行代理。再使用keepalived对predixy进行高可用

  • 规划IP如下
IP 功能
192.168.1.10 VIP
192.168.1.11 redis-node-1
192.168.1.12 redis-node-2
192.168.1.13 redis-node-3

每台redis节点上运行两个redis实例
所有节点关闭防火墙和selinux

创建redis cluster

0x01 redis安装

1
2
3
4
5
yum install gcc -y
tar xf redis-4.0.9.tar.gz
cd redis-4.0.9
make MALLOC=libc
make install

0x02 创建redis配置文件

配置说明

1
2
3
4
5
6
7
8
9
port 7000      #监听的端口号
bind 0.0.0.0 #监听IP
cluster-enabled yes #开始cluster模式
cluster-config-file /opt/redis-master/nodes-7000.conf #cluster配置文件,这西写好路径自是生成
cluster-node-timeout 15000 #节点超时时间
cluster-require-full-coverage no #槽全覆盖,设置为no。避免因为节点宕机导致槽未全覆盖导致集大宕机
daemonize yes #后台运行
logfile /opt/redis-master/redis.log #日志文件记录地址
appendonly yes #数据写入硬盘,保证数据持久化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
mkdir /opt/redis-{master,slave}
#master的配置文件
vim /opt/redis-master/redis-master.conf
port 7000
bind 0.0.0.0
cluster-enabled yes
cluster-config-file /opt/redis-master/nodes-7000.conf
cluster-node-timeout 15000
cluster-require-full-coverage no
daemonize yes
logfile /opt/redis-master/redis.log
appendonly yes
dir /opt/redis-master

#slave的配置文件
vim /opt/redis-slave/redis-slave.conf
port 7001
bind 0.0.0.0
cluster-enabled yes
cluster-config-file /opt/redis-slave/nodes-7001.conf
cluster-node-timeout 15000
cluster-require-full-coverage no
daemonize yes
logfile /opt/redis-slave/redis.log
appendonly yes

0x03 开启redis

在三台redis node上创建配置文件后启动redis实例

1
2
redis-server /opt/redis-master/redis-master.conf
redis-server /opt/redis-slave/redis-slave.conf

0x04 创建redis cluster集群

1
2
3
4
5
6
7
8
9
10
11
12
13
#由于redis-trib.rb依赖ruby,而且redis 4.0以上需要ruby2.2.2以上版本。
#而yum仓库里的ruby是2.0本版,所以需要手工安装ruby
yum install rubygems
curl -sSL https://rvm.io/mpapis.asc | gpg2 --import -
curl -L get.rvm.io | bash -s stable
source /etc/profile.d/rvm.sh
rvm install 2.4.0
rvm use 2.4.0
gem install redis

#ruby基础环境安装好之后,创建redis集群
#参数说明 create 创建 replicas 分片模式 1 就是分片模式 1master 对应 1slave
./redis-4.0.9/src/redis-trib.rb create --replicas 1 192.168.1.11:7000 192.168.1.12:7000 192.168.1.13:7000 192.168.1.11:7001 192.168.1.12:7001 192.168.1.13:7001

创建predixy代理

0x01 安装predixy

1
2
3
4
git clone https://github.com/joyieldInc/predixy.git
cd predixy
make
cp src/predixy /bin

0x02 创建predixy配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
mkdir /opt/predixy
cp conf/* /opt/predixy

#修改主配置文件
vim /opt/predixy/predixy.conf
#配置如下
Name RedisCluster #修改集群名称
Bind 0.0.0.0:7003 #修改监听地址和IP
WorkerThreads 4 #开启的线程
ClientTimeout 300
LogVerbSample 0
LogDebugSample 0
LogInfoSample 10000
LogNoticeSample 1
LogWarnSample 1
LogErrorSample 1
Include auth.conf
Include cluster.conf #注释掉 include try.conf 开启 cluster.conf
Include latency.conf

#修改cluster配置文件
vim /opt/predixy/cluster.conf
#配置如下
ClusterServerPool {
MasterReadPriority 60
StaticSlaveReadPriority 50
DynamicSlaveReadPriority 50
RefreshInterval 1
ServerTimeout 1
ServerFailureLimit 10
ServerRetryTimeout 1
KeepAlive 120
Servers {
+ 192.168.1.11:7000
+ 192.168.1.12:7000
+ 192.168.1.13:7000
+ 192.168.1.11:7001
+ 192.168.1.12:7001
+ 192.168.1.13:7001
}
}

0x03 启动代理

代理部署到三个节点上,同样配置,并启动代理

1
setsid predixy /opt/predixy/predixy.conf >> /opt/predixy/predixy.log

部署keepalived

0x01 安装keepalived

1
yum insall keepalived -y

0x02 修改keepalived配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
vim /etc/keepalived/keepalived.conf
#配置如下
#拷贝配置文件到三个节点上,并且修改priority 建议改为每个节点IP第四位
! Configuration File for keepalived

global_defs {
router_id REDIS_PROXY
}

vrrp_instance VI_1 {
state BACKUP
interface ens160
virtual_router_id 10
priority 11
advert_int 1
nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.1.10
}
}

0x03 启动keepalived并且创建守护脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#启动keepalived
systemctl start keepalived

#编写守护脚本
mkdir /opt/scripts
vim /opt/scripts/pdwatch.sh
#脚本内容
#!/bin/bash
CMD="predixy /opt/predixy/predixy.conf"
while :
do
k=`ps aux | grep predixy | grep -v grep`
if [ -z "$k" ]
then
setsid $CMD &> /opt/predixy/predixy.log &
sleep 2
k=`ps aux | grep predixy | grep -v grep`
if [ -z "$k" ]
then
systemctl stop keepalived
fi
fi
done

#启动守护脚本
setsid /opt/scripts/pdwatch.sh &
0%