本文共 6581 字，大约阅读时间需要 21 分钟。
组件及实现的功能
Keepalived：实现对Haproxy服务的高可用，并采用双主模型配置;
Haproxy：实现对Nginx的负载均衡和读写分离;
Nginx：实现对HTTP请求的高速处理;
架构设计图
重点概念
vrrp_script中节点权重改变算法
vrrp_script 里的script返回值为0时认为检测成功，其它值都会当成检测失败；
weight 为正时，脚本检测成功时此weight会加到priority上，检测失败时不加；
主失败:
主 priority < 从 priority + weight 时会切换。
主成功：
主 priority + weight > 从 priority + weight 时，主依然为主
weight 为负时，脚本检测成功时此weight不影响priority，检测失败时priority – abs(weight)
主失败:
主 priority – abs(weight) < 从priority 时会切换主从
主成功:
主 priority > 从priority 主依然为主
具体解释详见博文“”
部署配置
Keepalived部署
配置
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 | yum -y install keepalived # 两节点都需部署 # 172.16.25.109 # vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from admin@lnmmp.com smtp_connect_timeout 3 smtp_server 127.0.0.1 router_id LVS_DEVEL } vrrp_script chk_maintaince_down { script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0" interval 1 weight 2 } vrrp_script chk_haproxy { script "killall -0 haproxy" interval 1 weight 2 } vrrp_instance VI_1 { interface eth0 state MASTER priority 100 virtual_router_id 125 garp_master_delay 1 authentication { auth_type PASS auth_pass 1e3459f77aba4ded } track_interface { eth0 } virtual_ipaddress { 172.16.25.10 /16 dev eth0 label eth0:0 } track_script { chk_haproxy } notify_master "/etc/keepalived/notify.sh master 172.16.25.10" notify_backup "/etc/keepalived/notify.sh backup 172.16.25.10" notify_fault "/etc/keepalived/notify.sh fault 172.16.25.10" } vrrp_instance VI_2 { interface eth0 state BACKUP priority 99 virtual_router_id 126 garp_master_delay 1 authentication { auth_type PASS auth_pass 7615c4b7f518cede } track_interface { eth0 } virtual_ipaddress { 172.16.25.11 /16 dev eth0 label eth0:1 } track_script { chk_haproxy chk_maintaince_down } notify_master "/etc/keepalived/notify.sh master 172.16.25.11" notify_backup "/etc/keepalived/notify.sh backup 172.16.25.11" notify_fault "/etc/keepalived/notify.sh fault 172.16.25.11" } # 172.16.25.110 # vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from admin@lnmmp.com smtp_connect_timeout 3 smtp_server 127.0.0.1 router_id LVS_DEVEL } vrrp_script chk_maintaince_down { script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0" interval 1 weight 2 } vrrp_script chk_haproxy { script "killall -0 haproxy" interval 1 weight 2 } vrrp_instance VI_1 { interface eth0 state BACKUP priority 99 virtual_router_id 125 garp_master_delay 1 authentication { auth_type PASS auth_pass 1e3459f77aba4ded } track_interface { eth0 } virtual_ipaddress { 172.16.25.10 /16 dev eth0 label eth0:0 } track_script { chk_haproxy chk_maintaince_down } notify_master "/etc/keepalived/notify.sh master 172.16.25.10" notify_backup "/etc/keepalived/notify.sh backup 172.16.25.10" notify_fault "/etc/keepalived/notify.sh fault 172.16.25.10" } vrrp_instance VI_2 { interface eth0 state MASTER priority 100 virtual_router_id 126 garp_master_delay 1 authentication { auth_type PASS auth_pass 7615c4b7f518cede } track_interface { eth0 } virtual_ipaddress { 172.16.25.11 /16 dev eth0 label eth0:1 } track_script { chk_haproxy } notify_master "/etc/keepalived/notify.sh master 172.16.25.11" notify_backup "/etc/keepalived/notify.sh backup 172.16.25.11" notify_fault "/etc/keepalived/notify.sh fault 172.16.25.11" } # vi /etc/keepalived/notify.sh #!/bin/bash # Author: Jason.Yu <admin@lnmmp.com> # description: An example of notify script # contact= 'root@localhost' notify() { mailsubject= "`hostname` to be $1: $2 floating" mailbody= "`date '+%F %H:%M:%S'`: vrrp transition, `hostname` changed to be $1" echo $mailbody | mail -s "$mailsubject" $contact } case "$1" in master) notify master $2 /etc/rc .d /init .d /haproxy restart exit 0 ;; backup) notify backup $2 # 在节点切换成backup状态时，无需刻意停止haproxy服务，防止chk_maintaince和chk_haproxy多次对haproxy服务操作； exit 0 ;; fault) notify fault $2 # 同上 exit 0 ;; *) echo 'Usage: `basename $0` {master|backup|fault}' exit 1 ;; esac |
启动服务
| 1 | service keepalived start # 在两个节点上都需要启动 |
keepalived双主模型启动
Haproxy部署
安装配置
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 | yum -y install haproxy # 两节点都需部署 vi /etc/haproxy/haproxy .cfg # 两节点配置一致 global log 127.0.0.1 local2 chroot /var/lib/haproxy pidfile /var/run/haproxy .pid maxconn 4000 user haproxy group haproxy daemon # 以后台程序运行； defaults mode http # 选择HTTP模式，即可进行7层过滤； log global option httplog # 可以得到更加丰富的日志输出； option dontlognull option http-server-close # server端可关闭HTTP连接的功能； option forwardfor except 127.0.0.0 /8 # 传递client端的IP地址给server端，并写入“X-Forward_for”首部中； option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 30000 listen stats mode http bind 0.0.0.0:1080 # 统计页面绑定1080端口； stats enable # 开启统计页面功能； stats hide-version # 隐藏Haproxy版本号； stats uri /haproxyadmin ?stats # 自定义统计页面的访问uri； stats realm Haproxy\ Statistics # 统计页面密码验证时的提示信息； stats auth admin:admin # 为统计页面开启登录验证功能； stats admin if TRUE # 若登录用户验证通过，则赋予管理功能； frontend http- in bind *:80 mode http log global option httpclose option logasap option dontlognull capture request header Host len 20 capture request header Referer len 60 acl url_static path_beg -i /static /images /javascript /stylesheets acl url_static path_end -i .jpg .jpeg .gif .png .css .js .html use_backend static_servers if url_static # 符合ACL规则的，请求转入后端静态服务器 default_backend dynamic_servers # 默认请求转入后端动态服务器 backend static_servers balance roundrobin server imgsrv1 192.168.0.25:80 check maxconn 6000 # 静态服务器，可配置多台，还可设置权重weight； backend dynamic_servers balance source # 对于动态请求利用source调度算法，可一定程度上实现session保持；但最好利用cookie绑定的方式实现session保持 server websrv1 192.168.0.35:80 check maxconn 1000 # 动态服务器，可配置多台，还可设置权重weight； |
启动服务
| 1 | service haproxy start # 两节点都需要启动 |
Nginx部署
见博客“”
访问验证
Haproxy统计页面测试
动静分离测试
高可用测试