LVS-DR+keepalived实现负载均衡
配置环境:
名称 | 主机名 | IP地址 | 网关 | 需要的安装软件 | VIP地址 |
LVS 主调度器 | cdh1 | 192.168.10.34 | 192.168.10.254 | ipvsadm+keepalived | 192.168.10.29 |
LVS 从调度器 | cdh2 | 192.168.10.35 | 192.168.10.254 | ipvsadm+keepalived | 192.168.10.29 |
realserver1 | cdh6 | 192.168.10.39 | 192.168.10.254 | httpd或nginx | 192.168.10.29 |
realserver2 | cdh7 | 192.168.10.41 | 192.168.10.254 | httpd或nginx | 192.168.10.29 |
一、LVS(主调度器)
安装ipvsadm
[root@cdh1 ~]# yum -y install ipvsadm
安装keepalived的依赖包
[root@cdh1 ~]# yum -y install gcc openssl-devel libnfnetlink-devel libnl libnl3-devel
源码安装keepalived,keepalived下载地址:https://www.keepalived.org/download.html,下载完成后上传到主调度器和从调度器
[root@cdh1 ~]# tar zxf keepalived-1.4.5.tar.gz -C /usr/local/src/
[root@cdh1 ~]# cd /usr/local/src/keepalived-1.4.5/
[root@cdh1 ~]# ./configure --prefix=/usr/local/keepalived
[root@cdh1 ~]# make && make install
[root@cdh1 ~]# echo $?
生成启动脚本文件
[root@cdh1 keepalived-1.4.5]# cp keepalived/etc/init.d/keepalived /etc/init.d/
[root@cdh1 keepalived-1.4.5]# vim /etc/init.d/keepalived
. /usr/local/keepalived/etc/sysconfig/keepalived
//第15行
[root@cdh1 keepalived-1.4.5]# chmod +x /etc/init.d/keepalived
//做一个软链接给启动脚本文件使用
[root@cdh1 keepalived-1.4.5]# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin/
[root@cdh1 keepalived-1.4.5]# vim /usr/local/keepalived/etc/sysconfig/keepalived
//修改参数文件
KEEPALIVED_OPTIONS=
"-D -f /usr/local/keepalived/etc/keepalived/keepalived.conf"
//指向配置文件
[root@cdh1 keepalived-1.4.5]# vim /usr/local/keepalived/etc/keepalived/keepalived.conf
! Configuration File
for
keepalived
global_defs {
#邮件功能一般没用
notification_email { #指定当keepalived出现问题时,发送邮件给哪些用户
root@localhost
}
notification_email_from root@localhost #指定发件人
smtp_server localhost #指定SMTP服务器地址
smtp_connect_timeout 30 #指定SMTP连接超时时间
router_id youxi1 #标识当前节点名称,不允许重复
}
vrrp_instance VI_5 { #定义一个实例,一个实例就是一个集群,实例名称VI_1可以更改
state MASTER #指定该节点为主节点
interface
ens192 #指定VIP的网络接口
virtual_router_id 101 #指定VRRP组名,主节点和备用节点需要配置相同VRRP组名
priority 100 #优先级,范围1~254,数学数值比较,越大优先级越高。主节点优先级必须高于备用节点
advert_int 1 #组播信息发送间隔,单位秒,主节点备用节点必须设置一致
authentication { #设置验证信息,主节点备用节点必须设置一致
auth_type PASS #指定认证方法,PASS简单密码认证
auth_pass 1111 #指定认证所使用的密码,最多8位
}
virtual_ipaddress { #指定VIP,主节点备用节点必须一致
192.168.10.29/24
}
}
virtual_server 192.168.10.29 80 { #对VIP为192.168.10.29,端口号为80的服务器添加相关信息
delay_loop 6 #keepalived多长时间监测一次真实服务器,单位秒
lb_algo rr #LVS调度算法
lb_kind DR #LVS-DR模式
nat_mask 255.255.255.0
persistence_timeout 50 #同一个IP50秒内的请求都会发到同一个真实服务器,会影响rr算法调度,测试时可以注释掉
protocol TCP #4层协议
real_server 192.168.10.38 80 { #对IP为192.168.10.39,端口号为80的真实服务器添加相关信息
weight 1 #指定权重,默认为1
TCP_CHECK{
connect_timeout 3 #连接超时时间,默认5秒
nb_get_retry 3 #重试次数,默认1次
delay_before_retry 3 #重试时间间隔,默认1秒
connect_port 80 #监测端口号
}
}
real_server 192.168.10.41 80 { #对IP为192.168.10.41,端口号为80的真实服务器添加相关信息
weight 1
TCP_CHECK{
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
启动keepalived,并设置开机自启
[root@cdh1 keepalived-1.4.5]# keepalived-1.4.5]# systemctl start keepalived.service
[root@cdh1 keepalived-1.4.5]# keepalived-1.4.5]# systemctl enable keepalived.service
Created symlink
from
/etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.
[root@cdh1 keepalived-1.4.5]# keepalived-1.4.5]# systemctl status keepalived.service
如果防火墙是开启状态,请添加端口号
[root@cdh1 keepalived-1.4.5]# firewall-cmd --permanent --zone=
public
--add-port=80/tcp && firewall-cmd --reload
success
success
二、LVS(从调度器)
配置与主调度器一样,只需要修改keepalived.conf中的几个地主。
router_id cdh1
//修改
state BACKUP
//修改
priority 90
//修改
正常启动keepalived,并设置开机自启
[root@cdh1 keepalived-1.4.5]# systemctl start keepalived.service
[root@cdh1 keepalived-1.4.5]# systemctl enable keepalived.service
Created symlink
from
/etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.
防火墙如果是开启状态记得添加端口号。注意:备用节点比主节点多添加一个vrrp协议规则(如果是iptables,那么就是iptables -A INPUT -p VRRP -j ACCEPT)。
[root@cdh1 keepalived-1.4.5]# firewall-cmd --permanent --zone=
public
--add-protocol=vrrp
success
[root@cdh1 keepalived-1.4.5]# firewall-cmd --permanent --zone=
public
--add-port=80/tcp && firewall-cmd --reload
success
success
测试VIP漂移
查看主节点和备用节点的ip地址
[root@cdh1 ~]# ip a s ens192
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:0c:29:10:93:7c brd ff:ff:ff:ff:ff:ff
inet 192.168.10.34/24 brd 192.168.10.255 scope global noprefixroute ens192
valid_lft forever preferred_lft forever
inet 192.168.10.29/32 scope global ens192
valid_lft forever preferred_lft forever
inet6 fe80::3418:ac4b:b2f9:4957/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@cdh2 ~]# ip a s ens192
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:0c:29:95:a7:6a brd ff:ff:ff:ff:ff:ff
inet 192.168.10.35/24 brd 192.168.10.255 scope global noprefixroute ens192
valid_lft forever preferred_lft forever
inet6 fe80::cc97:cbe0:9d14:917c/64 scope link noprefixroute
valid_lft forever preferred_lft forever
停止主节点的keepalived,再查看主节点和备用节点IP
[root@cdh1 ~]# systemctl stop keepalived.service
[root@cdh1 ~]# ip a s ens192
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:0c:29:10:93:7c brd ff:ff:ff:ff:ff:ff
inet 192.168.10.34/24 brd 192.168.10.255 scope global noprefixroute ens192
valid_lft forever preferred_lft forever
inet6 fe80::3418:ac4b:b2f9:4957/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@cdh2 ~]# ip a s ens192
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:0c:29:95:a7:6a brd ff:ff:ff:ff:ff:ff
inet 192.168.10.35/24 brd 192.168.10.255 scope global noprefixroute ens192
valid_lft forever preferred_lft forever
inet 192.168.10.29/32 scope global ens192
valid_lft forever preferred_lft forever
inet6 fe80::cc97:cbe0:9d14:917c/64 scope link noprefixroute
valid_lft forever preferred_lft forever
测试完记得开启主调度器的keepalived
4、搭建真实服务器
配置回环口lo:1为VIP
[root@cdh6 ~]# cd /etc/sysconfig/network-scripts/
[root@cdh6 network-scripts]# cp ifcfg-lo{,:1}
[root@cdh6 network-scripts]# vim ifcfg-lo:1
DEVICE=lo:1
//修改
IPADDR=192.168.10.29
//修改
NETMASK=255.255.255.255
//修改
#iNETWORK=127.0.0.0 //注释
# If you're having problems with gated making 127.0.0.0/8 a martian,
# you can change this to something else (255.255.255.255, for example)
#BROADCAST=127.255.255.255 //注释
ONBOOT=yes
NAME=loopback
[root@cdh6 network-scripts]# systemctl restart network
[root@cdh6 network-scripts]# ip a sh dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
group
default
qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.5.100/32 brd 192.168.5.100 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
搭建一个简易的网页
[root@cdh6 network-scripts]# yum -y install httpd
[root@cdh6 network-scripts]# systemctl start httpd.service
[root@cdh6 network-scripts]# systemctl enable httpd.service
Created symlink
from
/etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[root@cdh6 network-scripts]# echo cdh6 192.168.10.39 > /
var
/www/html/index.html
如果防火墙是开启状态,记得添加端口号
[root@@cdh6 ~]# firewall-cmd --permanent --zone=
public
--add-port=80/tcp && firewall-cmd --reload
success
success
5、搭建真实服务器2
基本和搭建cdh6一样,只是index.html内容改为cdh7 192.168.10.41,方便测试。
6、测试
注释掉persistence_timeout参数,然后重启
三、创建LVS虚拟集群服务器(两台LVS调度器配置都一样)
[root@cdh1 ~]# ipvsadm -A -t 192.168.10.29:80 -s rr
[root@cdh1 ~]# ipvsadm -a -t 192.168.10.29:80 -r 192.168.10.39 -g
[root@cdh1 ~]# ipvsadm -a -t 192.168.10.29:80 -r 192.168.10.41 -g
保存配置:
[root@cdh1 ~]# ipvsadm -S -n
[root@cdh1 ~]# cat /etc/sysconfig/ipvsadm
-A -t 192.168.10.29:80 -s rr
-a -t 192.168.10.29:80 -r 192.168.10.39:80 -g -w 1
-a -t 192.168.10.29:80 -r 192.168.10.41:80 -g -w 1
配置两台LVS调度器的网络
[root@cdh1 ~]# ip addr add 192.168.10.29/24 dev ens192 label ens192:1
[root@cdh2 ~]# ip addr add 192.168.10.29/24 dev ens192 label ens192:1
也可以手动配置两台真实服务器的lo网络(上面步骤已经更改,此处可以忽略)
[root@cdh6 ~]# ip addr add 192.168.10.29/32 dev lo label lo:1
[root@cdh6 ~]# route add -host 192.168.10.29 dev lo (可省略)
[root@cdh7 ~]# ip addr add 192.168.10.29/32 dev lo label lo:1
[root@cdh7 ~]# route add -host 192.168.10.29 dev lo (可省略)
还有需要更改arp配置(两台真实服务器都需要改)
[root@cdh6 ~]# vim /etc/sysctl.conf
.. ..
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_announce = 2
刷新服务:
[root@cdh6 ~]#sysctl -p
整个过程搭建完成。