nginx与keepalived实现高可用性负载均衡

一、keepalived是什么?

1、keepalived是干嘛的?

keepalived是一种检测web服务器健康状态的软件。通俗的说,就是检测集群中的web服务器是正常运行还是挂掉了,如果某台web服务器挂掉了,则它会自动把它从集群中剔除掉,如果web服务器从挂掉的状态恢复到正常运行的状态,又会自动的把这台web服务器加入到集群中,这些过程都是自动的,不需要人工干预。(当然服务器挂掉了,让它重新恢复正常,则需要人工去干预)

更多的资料可以参考官方文档:http://www.keepalived.org/documentation.html

2、keepalived安装与配置

可以参考keepalived源码包中的INSTALL文件

  1. wget
  2. tar -zxvf keepalived-1.2.17.tar.gz
  3. cd keepalived-1.2.17
  4. ./configure
  5. make&&make install

将keepalived加入开机服务

  1. mkdir /etc/keepalived
  2. cp keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf
  3. cp keepalived/etc/init.d/keepalived.sysconfig /etc/sysconfig/keepalived
  4. cp keepalived/etc/init.d/keepalived.init /etc/init.d/keepalived
  5. ln -s /usr/local/sbin/keepalived /usr/sbin/keepalived
  6. chkconfig --add keepalived

二、nginx+keepalived实现nginx主备切换

1、如何检测一台机器或者某个服务挂掉或者正在运行呢?

我们这里以nginx服务为例,检测nginx服务是否正常,可以使用通过nginx的status模块,检测nginx的进程,检测nginx所在端口,或者访问nginx中某些页面来检测,我们这里以检测端口来实现,检测端口就不得不提一个软件叫nmap的软件。

注:nmap需要我们自己先用yum -y install nmap安装一下

比如我来扫描一下新浪的端口开放情况(只是例子)

  1. nmap www.sina.com.cn

执行过程

  1. [root@server200 src]# nmap www.sina.com.cn
  2. Starting Nmap 5.51 ( http://nmap.org ) at 2015-06-16 15:15 CST
  3. Nmap scan report for www.sina.com.cn (202.102.75.147)
  4. Host is up (0.017s latency).
  5. Not shown: 986 closed ports
  6. PORT STATE SERVICE
  7. 21/tcp filtered ftp
  8. 22/tcp filtered ssh
  9. 23/tcp filtered telnet
  10. 80/tcp open http
  11. 135/tcp filtered msrpc
  12. 139/tcp filtered netbios-ssn
  13. 161/tcp filtered snmp
  14. 389/tcp filtered ldap
  15. 445/tcp filtered microsoft-ds
  16. 873/tcp filtered rsync
  17. 3389/tcp filtered ms-term-serv
  18. 4444/tcp filtered krb524
  19. 5631/tcp filtered pcanywheredata
  20. 5900/tcp filtered vnc

查看本地某台机器的端口

  1. [root@server200 src]# nmap 192.168.18.201
  2. Starting Nmap 5.51 ( http://nmap.org ) at 2015-06-16 15:17 CST
  3. Nmap scan report for 192.168.18.201 (192.168.18.201)
  4. Host is up (0.00027s latency).
  5. Not shown: 995 closed ports
  6. PORT STATE SERVICE
  7. 22/tcp open ssh
  8. 80/tcp open http
  9. 139/tcp open netbios-ssn
  10. 445/tcp open microsoft-ds
  11. 3306/tcp open mysql
  12. MAC Address: 00:0C:29:AA:97:B7 (VMware)

查看本地某几台机器 的端口

  1. nmap 192.168.18.200-203

查看某台机器上某个端口(如80端口)

  1. [root@server200 src]# nmap -p 80 192.168.18.200
  2. Starting Nmap 5.51 ( http://nmap.org ) at 2015-06-16 15:19 CST
  3. Nmap scan report for 192.168.18.200 (192.168.18.200)
  4. Host is up (0.000034s latency).
  5. PORT STATE SERVICE
  6. 80/tcp open http
  7. Nmap done: 1 IP address (1 host up) scanned in 0.04 seconds
2、keepalived的配置

keepalived的配置文件与nginx类似,也是分模块的。具体配置参数详解,请参考https://github.com/acassen/keepalived/blob/master/doc/keepalived.conf.SYNOPSIS

(1)集群中主服务器的配置
  1. ! Configuration File for keepalived
  2. global_defs {
  3. router_id NGINX_CLUSTER
  4. }
  5. vrrp_script check_nginx_status {
  6. script "/usr/local/sysscripts/keepalived/check_nginx_status.sh"
  7. interval 3
  8. weight -3
  9. }
  10. vrrp_instance VI_1 {
  11. state MASTER
  12. interface eth1
  13. virtual_router_id 51
  14. priority 100
  15. advert_int 1
  16. authentication {
  17. auth_type PASS
  18. auth_pass 1111
  19. }
  20. virtual_ipaddress {
  21. 192.168.18.180
  22. }
  23. }
  24. track_script {
  25. check_nginx_status
  26. }

~

(2)集群中备份服务器的配置
  1. ! Configuration File for keepalived
  2. global_defs {
  3. router_id NGINX_CLUSTER
  4. }
  5. vrrp_script check_nginx_status {
  6. script "/usr/local/sysscripts/keepalived/check_nginx_status.sh"
  7. interval 3
  8. weight -3
  9. }
  10. vrrp_instance VI_1 {
  11. state BACKUP
  12. interface eth1
  13. virtual_router_id 51
  14. priority 101
  15. advert_int 1
  16. authentication {
  17. auth_type PASS
  18. auth_pass 1111
  19. }
  20. virtual_ipaddress {
  21. 192.168.18.180
  22. }
  23. }
  24. track_script {
  25. check_nginx_status
  26. }
3、nginx状态检测脚本的编写(主从服务器上都需要这个脚本)
  1. mkdir -p /usr/local/sysscripts/keepalived
  2. vim check_nginx_status.sh

脚本代码如下:

  1. #!/bin/bash
  2. # check nginx status
  3. NGINX=/usr/local/nginx/sbin/nginx
  4. PORT=80
  5. nmap localhost -p $PORT | grep "$PORT/tcp open"
  6. if [ $? -ne 0 ]; then
  7. $NGINX -s stop
  8. $NGINX
  9. sleep 3
  10. nmap localhost -p $PORT | grep "$PORT/tcp open"
  11. [ $? -ne 0 ] && /etc/init.d/keepalived stop
  12. fi
4、检测两台服务器是否已经拥有虚拟IP192.168.18.180
  1. ip addr
  2. 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
  3. link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  4. inet 127.0.0.1/8 scope host lo
  5. inet6 ::1/128 scope host
  6. valid_lft forever preferred_lft forever
  7. 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
  8. link/ether 00:0c:29:52:a1:cd brd ff:ff:ff:ff:ff:ff
  9. inet 192.168.18.200/24 brd 192.168.18.255 scope global eth0
  10. inet 192.168.18.180/32 scope global eth0
  11. inet6 fe80::20c:29ff:fe52:a1cd/64 scope link
  12. valid_lft forever preferred_lft forever
5、我们把其中一台机器的nginx停掉,看是否可以访问192.168.18.180这个虚拟IP

注:在同一时间只有一台机器可以使用这个虚拟IP

三、nginx+keepalived实现高可用性负载均衡

1、服务器情况:
  • 192.168.18.200 虚拟IP192.168.18.180

  • 192.168.18.201 虚拟IP192.168.18.180

  • 192.168.18.202

  • 192.168.18.203 这是一个负载均衡调度服务器

架构如下:

2、负载均衡服务器配置
  1. upstream backend {
  2. server 192.168.18.180 weight=3;
  3. server 192.168.18.202;
  4. }
  5. server {
  6. listen 80;
  7. server_name localhost;
  8. #charset koi8-r;
  9. #access_log logs/host.access.log main;
  10. location / {
  11. root /webdata/www;
  12. proxy_pass http://backend;
  13. index index.html index.htm index.php;
  14. }
  15. }

3、测试,关掉200或者201是的某一台服务器