雪重 Python Developer 知识汗水灵感机遇

Back Home

2023年5月15日 keepalived virtualip nginx

Keepalived+Virtual_ip+Nginx搭建高可用负载均衡器

在这个教程中，我们将介绍如何使用Nginx和Keepalived搭建高可用负载均衡器。以下是设置过程的简要说明：

环境准备

准备两台安装了Linux操作系统的服务器（如Ubuntu）。
为每台服务器分配一个独立IP地址，并确保它们处于相同的子网。
在每台服务器上安装Nginx和Keepalived。

Nginx 配置

此配置主要作用，通过公网ip->vip>nginx转发->ceph，实现打洞

user root;
worker_processes auto;
pid /run/nginx.pid;
worker_rlimit_nofile 1048576;
events {
    worker_connections 65535;
}

http {


    sendfile on;
    proxy_connect_timeout 30;
    proxy_send_timeout    600;
    proxy_read_timeout    600;

    # 设置单个缓冲区大小为 16k
    proxy_buffer_size 200k;
    # 设置每个缓冲区的数量和大小
    proxy_buffers 512 200k;

    default_type application/octet-stream;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;
    access_log off;
    error_log /var/log/nginx/error.log;
    # gzip on;


    server {
        listen 30000;
        location /test-upload {
            proxy_pass {ceph的endpoint};
            proxy_set_header Host $host:30000;
            proxy_redirect off;
            client_max_body_size 0;
            proxy_cache off;
       }    
    }

}

Keepalived 配置

安装 Keepalived:

apt install keepalived
创建并编辑 Keepalived 配置文件：

vim /etc/keepalived/keepalived.conf

在主节点上添加以下内容:

vrrp_script chk_nginx {
    script "/etc/keepalived/scripts/nginx_check.sh"
    interval 2
    weight -20
}
global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   #vrrp_skip_check_adv_addr
   #vrrp_strict
   script_user root
   enable_script_security
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 55
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.***
    }
    track_script {
        chk_nginx
    }

}

在备节点上添加以下内容：

vrrp_script chk_nginx {
    script "/etc/keepalived/scripts/nginx_check.sh"
    interval 2
    weight -20
}

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   #vrrp_skip_check_adv_addr
   #vrrp_strict
   script_user root
   enable_script_security
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 55
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.***
    }
    track_script {
        chk_nginx
    }

}

修改 interface、virtual_router_id 和 virtual_ipaddress 的值以适应你的环境。

注意：

同一网段中virtual_router_id的值不能重复，否则会出错

可以用这条命令来查看该网络中所存在的vrid： tcpdump -nn -i any net 224.0.0.0/8

选择未被占用的virtual_router_id

为 Nginx 配置 Keepalived 脚本：

vim /etc/keepalived/scripts/nginx_check.sh

添加以下内容：

#!/bin/bash
# Check if Nginx is running
if systemctl status nginx | grep -q 'running'; then
    exit 0 # Nginx is running, exit with success code
else
    exit 1 # Nginx is not running, exit with error code
fi

更改脚本权限：

chmod +x /etc/keepalived/scripts/nginx_check.sh

启动并检查 Keepalived 服务是否正常运行：

systemctl start keepalived
systemctl status keepalived

主节点输出：

Keepalived_vrrp[100681]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on eth0 for 10***

Keepalived_vrrp[100681]: Sending gratuitous ARP on eth0 for 10***

查看虚拟IP地址分配情况

输入命令ip addr

在网卡里会有 ”inet ｛virtual_ipaddress｝ scope global” 信息