下文給大家?guī)?lái)實(shí)現(xiàn)keepalived雙主模型lvs高可用集群的實(shí)驗(yàn)總結(jié)，希望能夠給大家在實(shí)際運(yùn)用中帶來(lái)一定的幫助，負(fù)載均衡涉及的東西比較多，理論也不多，網(wǎng)上有很多書籍，今天我們就用創(chuàng)新互聯(lián)在行業(yè)內(nèi)累計(jì)的經(jīng)驗(yàn)來(lái)做一個(gè)解答。
實(shí)現(xiàn)keepalived雙主模型lvs高可用集群
一、拓?fù)鋱D
二、環(huán)境準(zhǔn)備
兩臺(tái)負(fù)載均衡調(diào)度器，兩臺(tái)web云服務(wù)器。
調(diào)度器A環(huán)境:
VS:一張網(wǎng)卡
DIP:192.168.0.7/24
VIP（主）：192.168.0.200
VIP(備)：192.168.0.201
軟件包：yum install -y keepalived ipvsadm nginx(作用：sorry-server服務(wù))
調(diào)度器B環(huán)境：
VS:一張網(wǎng)卡
DIP:192.168.0.8/24
VIP（主）：192.168.0.201
VIP(備)：192.168.0.200
軟件包：yum install -y keepalived ipvsadm nginx(作用：sorry-server服務(wù))
web服務(wù)器A環(huán)境：
RS1:192.168.0.3
lo:0:192.168.0.200
lo:1:192.168.0.201
軟件包：yum install -y httpd
web服務(wù)器B環(huán)境：
RS1:192.168.0.4
lo:0:192.168.0.200
lo:1:192.168.0.201
軟件包：yum install -y httpd
三、配置
調(diào)度器A配置
主要操作：ip配置，時(shí)間同步、hosts文件更改，修改主機(jī)名（確保主從節(jié)點(diǎn)能夠通信）、keepalived.conf配置文件更改，通知腳本配置(兩個(gè)腳本)，ipvsadm規(guī)則查看
ip配置
[root@node1keepalived]# ip a 1: lo:時(shí)間同步
ntpdate ntp-server-ip #時(shí)間服務(wù)器的iphosts文件
[root@node1 keepalived]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 node1 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 node1 192.168.0.8 node2 192.168.0.7 node1主機(jī)名修改
hostnamectl set-hostname node1keepalived.conf配置文件更改
[root@localhost keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 10 router_id node1 vrrp_mcast_group4 224.0.0.223 } vrrp_instance VI_1 { state MASTER interface ens33 virtual_router_id 51 priority 110 advert_int 1 authentication { auth_type PASS auth_pass JzvnmfkY } virtual_ipaddress { 192.168.0.200/24 dev ens33 label ens33:1 } notify_master "/etc/keepalived/notify1.sh master" notify_backup "/etc/keepalived/notify1.sh backup" notify_fault "/etc/keepalived/notify1.sh fault" } virtual_server 192.168.0.200 80 { delay_loop 3 lb_algo wrr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.0.3 80 { weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.0.4 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } } vrrp_instance VI_2 { state BACKUP interface ens33 virtual_router_id 52 priority 90 advert_int 1 authentication { auth_type PASS auth_pass JzvnmfkY } virtual_ipaddress { 192.168.0.201/24 dev ens33 label ens33:2 } notify_master "/etc/keepalived/notify2.sh master" notify_backup "/etc/keepalived/notify2.sh backup" notify_fault "/etc/keepalived/notify2.sh fault" } virtual_server 192.168.0.201 80 { delay_loop 3 lb_algo wrr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.0.3 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.0.4 80 { weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } }通知腳本配置
腳本1
[root@node1 keepalived]# cat notify1.sh #!/bin/bash #description: An example of notify script # vip=192.168.0.200 contact='root@localhost' notify(){ local mailsubject="$(hostname) to be $1:$vip floating" local mailbody="$(date +'%F %T'):vrrp1 transition,$(hostname) changed to be $1" echo "$mailbody" | mail -s "$mailsubject" $contact } case $1 in master) notify master ;; backup) notify backup ;; fault) notify fault ;; *) echo "Usage:$(basename $0) {master|backup|fault}" exit 1 ;; esac腳本2
[root@node1 keepalived]# cat notify2.sh #!/bin/bash #description: An example of notify script # vip=192.168.0.201 contact='root@localhost' notify(){ local mailsubject="$(hostname) to be $1:$vip floating" local mailbody="$(date +'%F %T'):vrrp2 transition,$(hostname) changed to be $1" echo "$mailbody" | mail -s "$mailsubject" $contact } case $1 in master) notify master ;; backup) notify backup ;; fault) notify fault ;; *) echo "Usage:$(basename $0) {master|backup|fault}" exit 1 ;; esacipvsadm規(guī)則查看
[root@node1 keepalived]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.0.200:80 wrr -> 192.168.0.3:80 Route 2 0 0 -> 192.168.0.4:80 Route 1 0 0 TCP 192.168.0.201:80 wrr -> 192.168.0.3:80 Route 1 0 0 -> 192.168.0.4:80 Route 2 0 0調(diào)度器B配置
主要操作：ip配置，時(shí)間同步、hosts文件更改，修改主機(jī)名（確保主從節(jié)點(diǎn)能夠通信）、keepalived.conf配置文件更改，通知腳本配置(兩個(gè)腳本)，ipvsadm規(guī)則查看
ip配置
[root@node2 keepalived]# ip a 1: lo:時(shí)間同步
ntpdate ntp-server-iphosts文件更改
[root@node2 keepalived]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 node2 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 node2 192.168.0.7 node1 192.168.0.8 node2主機(jī)名修改
hostnamectl set-hostname node2keepalived.conf配置文件更改
[root@localhost keepalived]# cat keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 10 router_id node2 vrrp_mcast_group4 224.0.0.223 } vrrp_instance VI_1 { state BACKUP interface ens33 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass JzvnmfkY } virtual_ipaddress { 192.168.0.200/24 dev ens33 label ens33:1 } notify_master "/etc/keepalived/notify1.sh master" notify_backup "/etc/keepalived/notify1.sh backup" notify_fault "/etc/keepalived/notify1.sh fault" } virtual_server 192.168.0.200 80 { delay_loop 3 lb_algo wrr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.0.3 80 { weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.0.4 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } } vrrp_instance VI_2 { state MASTER interface ens33 virtual_router_id 52 priority 100 advert_int 1 authentication { auth_type PASS auth_pass JzvnmfkY } virtual_ipaddress { 192.168.0.201/24 dev ens33 label ens33:2 } notify_master "/etc/keepalived/notify2.sh master" notify_backup "/etc/keepalived/notify2.sh backup" notify_fault "/etc/keepalived/notify2.sh fault" } virtual_server 192.168.0.201 80 { delay_loop 3 lb_algo wrr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.0.3 80 { weight 1 HTTP_GET { url { path / status_code 200 } status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.0.4 80 { weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } }通知腳本配置(同調(diào)度器A的腳本一致)
腳本1
[root@node2 keepalived]# cat notify1.sh #!/bin/bash #description: An example of notify script # vip=192.168.0.200 contact='root@localhost' notify(){ local mailsubject="$(hostname) to be $1:$vip floating" local mailbody="$(date +'%F %T'):vrrp1 transition,$(hostname) changed to be $1" echo "$mailbody" | mail -s "$mailsubject" $contact } case $1 in master) notify master ;; backup) notify backup ;; fault) notify fault ;; *) echo "Usage:$(basename $0) {master|backup|fault}" exit 1 ;; esac腳本2
[root@node2 keepalived]# cat notify2.sh #!/bin/bash #description: An example of notify script # vip=192.168.0.201 contact='root@localhost' notify(){ local mailsubject="$(hostname) to be $1:$vip floating" local mailbody="$(date +'%F %T'):vrrp2 transition,$(hostname) changed to be $1" echo "$mailbody" | mail -s "$mailsubject" $contact } case $1 in master) notify master ;; backup) notify backup ;; fault) notify fault ;; *) echo "Usage:$(basename $0) {master|backup|fault}" exit 1 ;; esacweb服務(wù)器A配置
ip配置
1: lo:lvs-dr-預(yù)配置腳本
腳本1
[root@localhost ~]# cat vip200-setrs.sh #!/bin/bash # vip=192.168.0.200 mask='255.255.255.255' case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig lo:0 $vip netmask $mask broadcast $vip up route add -host $vip dev lo:0 ;; stop) ifconfig lo:0 down echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce ;; *) echo "Usage $(basename $0) start|stop" exit 1 ;; esac腳本2
[root@localhost ~]# cat vip201-setrs.sh #!/bin/bash # vip=192.168.0.201 mask='255.255.255.255' case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig lo:1 $vip netmask $mask broadcast $vip up route add -host $vip dev lo:1 ;; stop) ifconfig lo:1 down echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce ;; *) echo "Usage $(basename $0) start|stop" exit 1 ;; esacweb服務(wù)器B配置
ip配置
[root@localhost ~]# ip a 1: lo:lvs-dr-預(yù)設(shè)腳本
腳本內(nèi)容同web服務(wù)A相同，拷貝過(guò)去即可
vip200-setrs.sh vip201-setrs.sh
客戶端測(cè)試
ip配置
[root@localhost ~]# ip a
1: lo:curl測(cè)試
[root@localhost ~]# for i in {1..10};do curl http://192.168.0.200;done backend server1 backend server1 backend server2 backend server1 backend server1 backend server2 backend server1 backend server1 backend server2 backend server1 [root@localhost ~]# for i in {1..10};do curl http://192.168.0.201;done backend server1 backend server2 backend server2 backend server1 backend server2 backend server2 backend server1 backend server2 backend server2 backend server1四、健康狀態(tài)查看以及故障排查參考方式
方式1：查看keepalived的切換過(guò)程
systemctl status keepalived
方式2：mail命令查看郵件
方式3：查看web服務(wù)器日志
tail /var/log/httpd/access_log
tail /var/log/httpd/error_log
五、模擬故障
1.模擬調(diào)度器B故障
調(diào)度器B上操作
停止keepalived服務(wù)
systemctl stop keepalived [root@node2 keepalived]# systemctl status keepalived #分析以下信息，VIP2已移除 Jul 02 10:18:53 node2 systemd[1]: Stopping LVS and VRRP High Availability Monitor... Jul 02 10:18:53 node2 Keepalived_vrrp[50773]: VRRP_Instance(VI_2) sending 0 priority Jul 02 10:18:53 node2 Keepalived_vrrp[50773]: VRRP_Instance(VI_2) removing protocol VIPs. Jul 02 10:18:53 node2 systemd[1]: Stopped LVS and VRRP High Availability Monitor.查看ip
[root@node2 keepalived]# ip a #VIP2移除 1: lo:調(diào)度器A操作
[root@node1 keepalived]# systemctl status keepalived #分析以下信息，可知VIP漂移到調(diào)度A ● keepalived.service - LVS and VRRP High Availability Monitor Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled) Active: active (running) since Sun 2017-07-02 10:07:36 CST; 17min ago Process: 20019 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS) Main PID: 20020 (keepalived) CGroup: /system.slice/keepalived.service ├─20020 /usr/sbin/keepalived -D ├─20021 /usr/sbin/keepalived -D └─20022 /usr/sbin/keepalived -D Jul 02 10:07:37 node1 Keepalived_vrrp[20022]: Opening script file /etc/keepalived/notify1.sh Jul 02 10:07:37 node1 Keepalived_healthcheckers[20021]: Netlink reflector reports IP 192.168.0.200 added Jul 02 10:07:42 node1 Keepalived_vrrp[20022]: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens33 for 192.168.0.200 Jul 02 10:18:54 node1 Keepalived_vrrp[20022]: VRRP_Instance(VI_2) Transition to MASTER STATE Jul 02 10:18:55 node1 Keepalived_vrrp[20022]: VRRP_Instance(VI_2) Entering MASTER STATE Jul 02 10:18:55 node1 Keepalived_vrrp[20022]: VRRP_Instance(VI_2) setting protocol VIPs. Jul 02 10:18:55 node1 Keepalived_vrrp[20022]: VRRP_Instance(VI_2) Sending gratuitous ARPs on ens33 for 192.168.0.201 Jul 02 10:18:55 node1 Keepalived_vrrp[20022]: Opening script file /etc/keepalived/notify2.sh Jul 02 10:18:55 node1 Keepalived_healthcheckers[20021]: Netlink reflector reports IP 192.168.0.201 added Jul 02 10:19:00 node1 Keepalived_vrrp[20022]: VRRP_Instance(VI_2) Sending gratuitous ARPs on ens33 for 192.168.0.201查看郵件
[root@node1 keepalived]# mail #node1成為VIP192.168.0.201的主節(jié)點(diǎn) Heirloom Mail version 12.5 7/5/10. Type ? for help. "/var/spool/mail/root": 1 message > 1 root Sun Jul 2 10:18 19/702 "node1 to be master:192.168.0.201 floating" & 1 Message 1: From root@node1.localdomain Sun Jul 2 10:18:55 2017 Return-Path:2.模擬web服務(wù)器A故障(RS1)
webA上操作
[root@localhost ~]# systemctl stop httpd調(diào)度器A上操作
查看郵件
[root@node1 keepalived]# mail #分析以下信息，可知RS1宕機(jī) Heirloom Mail version 12.5 7/5/10. Type ? for help. "/var/spool/mail/root": 4 messages 2 new 1 root Sun Jul 2 10:18 19/702 "node1 to be master:192.168.0.201 floating" 2 root Sun Jul 2 10:28 19/702 "node1 to be backup:192.168.0.201 floating" >N 3 keepalived@localhost Sun Jul 2 10:28 17/642 "[node1] Realserver [192.168.0.3]:80 - DOWN" N 4 keepalived@localhost Sun Jul 2 10:28 17/642 "[node1] Realserver [192.168.0.3]:80 - DOWN"客戶端請(qǐng)求查看
[root@localhost ~]# curl http://192.168.0.200 #只能訪問(wèn)到RS2 backend server2調(diào)度器B上操作
[root@node2 keepalived]# mail #分析以下信息，可知RS1宕機(jī) Heirloom Mail version 12.5 7/5/10. Type ? for help. "/var/spool/mail/root": 4 messages 2 new 1 root Sun Jul 2 10:28 19/702 "node2 to be backup:192.168.0.200 floating" 2 root Sun Jul 2 10:28 19/702 "node2 to be master:192.168.0.201 floating" >N 3 keepalived@localhost Sun Jul 2 10:28 17/642 "[node2] Realserver [192.168.0.3]:80 - DOWN" N 4 keepalived@localhost Sun Jul 2 10:28 17/642 "[node2] Realserver [192.168.0.3]:80 - DOWN"3.模擬web服務(wù)雙機(jī)故障
RS1和RS2停止web服務(wù)
調(diào)度器A上操作
查看郵件 #分析以下信息，可知雙機(jī)宕機(jī)
[root@node1 keepalived]# mail Heirloom Mail version 12.5 7/5/10. Type ? for help. "/var/spool/mail/root": 6 messages 2 new 1 root Sun Jul 2 10:18 19/702 "node1 to be master:192.168.0.201 floating" 2 root Sun Jul 2 10:28 19/702 "node1 to be backup:192.168.0.201 floating" 3 keepalived@localhost Sun Jul 2 10:28 18/653 "[node1] Realserver [192.168.0.3]:80 - DOWN" 4 keepalived@localhost Sun Jul 2 10:28 18/653 "[node1] Realserver [192.168.0.3]:80 - DOWN" >N 5 keepalived@localhost Sun Jul 2 10:36 17/642 "[node1] Realserver [192.168.0.4]:80 - DOWN" N 6 keepalived@localhost Sun Jul 2 10:36 17/642 "[node1] Realserver [192.168.0.4]:80 - DOWN"查看keepalived狀態(tài)
[root@node1 keepalived]# systemctl status keepalived #分析以下信息，雙機(jī)宕機(jī)，啟動(dòng)sorry服務(wù)器 ● keepalived.service - LVS and VRRP High Availability Monitor Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled) Active: active (running) since Sun 2017-07-02 10:07:36 CST; 30min ago Process: 20019 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS) Main PID: 20020 (keepalived) CGroup: /system.slice/keepalived.service ├─20020 /usr/sbin/keepalived -D ├─20021 /usr/sbin/keepalived -D └─20022 /usr/sbin/keepalived -D Jul 02 10:36:31 node1 Keepalived_healthcheckers[20021]: Removing alive servers from the pool for VS [192.168.0.200]:80 Jul 02 10:36:31 node1 Keepalived_healthcheckers[20021]: Remote SMTP server [127.0.0.1]:25 connected. Jul 02 10:36:31 node1 Keepalived_healthcheckers[20021]: SMTP alert successfully sent. Jul 02 10:36:32 node1 Keepalived_healthcheckers[20021]: Error connecting server [192.168.0.4]:80. Jul 02 10:36:32 node1 Keepalived_healthcheckers[20021]: Removing service [192.168.0.4]:80 from VS [192.168.0.201]:80 Jul 02 10:36:32 node1 Keepalived_healthcheckers[20021]: Lost quorum 1-0=1 > 0 for VS [192.168.0.201]:80 Jul 02 10:36:32 node1 Keepalived_healthcheckers[20021]: Adding sorry server [127.0.0.1]:80 to VS [192.168.0.201]:80 Jul 02 10:36:32 node1 Keepalived_healthcheckers[20021]: Removing alive servers from the pool for VS [192.168.0.201]:80 Jul 02 10:36:32 node1 Keepalived_healthcheckers[20021]: Remote SMTP server [127.0.0.1]:25 connected. Jul 02 10:36:32 node1 Keepalived_healthcheckers[20021]: SMTP alert successfully sent.客戶端請(qǐng)求
[root@localhost ~]# curl http://192.168.0.201 #請(qǐng)求至sorry服務(wù)器的web頁(yè)面 sorry server [root@localhost ~]# curl http://192.168.0.200 sorry server看了以上關(guān)于實(shí)現(xiàn)keepalived雙主模型lvs高可用集群的實(shí)驗(yàn)總結(jié)，如果大家還有什么地方需要了解的可以在創(chuàng)新互聯(lián)行業(yè)資訊里查找自己感興趣的或者找我們的專業(yè)技術(shù)工程師解答的，創(chuàng)新互聯(lián)技術(shù)工程師在行業(yè)內(nèi)擁有十幾年的經(jīng)驗(yàn)了。
另外有需要云服務(wù)器可以了解下創(chuàng)新互聯(lián)cdcxhl.cn，海內(nèi)外云服務(wù)器15元起步，三天無(wú)理由+7*72小時(shí)售后在線，公司持有idc許可證，提供“云服務(wù)器、裸金屬服務(wù)器、高防服務(wù)器、香港服務(wù)器、美國(guó)服務(wù)器、虛擬主機(jī)、免備案服務(wù)器”等云主機(jī)租用服務(wù)以及企業(yè)上云的綜合解決方案，具有“安全穩(wěn)定、簡(jiǎn)單易用、服務(wù)可用性高、性價(jià)比高”等特點(diǎn)與優(yōu)勢(shì)，專為企業(yè)上云打造定制，能夠滿足用戶豐富、多元化的應(yīng)用場(chǎng)景需求。
重慶分公司
重慶分公司
