一、upstream模块
nginx 的upstream 节点一般置于 http 节点大括号中,常规在 upstream 中配置需要被负载均衡的服务器列表。
#user nobody nobody。 #worker_processes 2; #pid /nginx/pid/nginx.pid; error_log log/error.log debug; events { …… } http { …… upstream testserver { server 192.168.1.5:8080; server 192.168.1.6:8080; …… keepalive 32; } server { …… location / { …… proxy_pass http://testserver; } } }
从结构可以看出,这种用的比较多的配置,主要是应对 http反向代理。
现实场景也有一些需求,如 mysql、redis 、mqtt等环境,我们也想对TCP进行代理,是否有解决方案?
其实 nginx 也是支持对 TCP/UDP 进行负载均衡的,下面要说到的就是 nginx 的 stream 模块,通过配置 stream 可以实现这样的需求。
二、stream模块
Nginx 的 TCP/UDP 负载均衡是应用 Stream 代理模块(ngx_stream_proxy_module)和 Stream 上游模块(ngx_stream_upstream_module)实现的。Nginx 的 TCP 负载均衡与 LVS 都是四层负载均衡的应用,所不同的是,LVS 是被置于 Linux 内核中的,而 Nginx 是运行于用户层的,基于 Nginx 的 TCP 负载可以实现更灵活的用户访问管理和控制。
下面来看一下具体的配置示例(在nginx.conf文件中):
stream { server { listen 3306; proxy_pass 172.17.0.4:3306; } } stream { log_format proxy '$remote_addr [$time_local] ' '$protocol $status $bytes_sent $bytes_received ' '$session_time "$upstream_addr"' '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"'; access_log /var/log/nginx/mqtt_access.log proxy; error_log /var/log/nginx/mqtt_error.log warn; open_log_file_cache off; upstream mqtt_tcp_server { server 127.0.0.1:3888; } server { listen 1488; #则使用mqtt://test.xx.com:1488 proxy_connect_timeout 150s; proxy_timeout 150s; proxy_pass mqtt_tcp_server; #反向代理地址 proxy_buffer_size 3M; tcp_nodelay on; } }
mqtt_access.log内容:
112.73.6.218 [11/Feb/2022:15:39:47 +0800] TCP 200 9 106 150.001 "10.88.88.88:3888""106" "9" "0.000"
服务端运行程序如果断开时,mqtt_error.log内容:
2022/02/11 15:24:55 [error] 13780#13780: *8133026 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: 0.0.0.0:1488, upstream: "10.88.88.88:3888", bytes from/to client:0/0, bytes from/to upstream:0/0 2022/02/11 15:24:55 [error] 13781#13781: *8133028 connect() failed (111: Connection refused) while connecting to upstream, client: 183.9.71.231, server: 0.0.0.0:1488, upstream: "10.88.88.88:3888", bytes from/to client:0/0, bytes from/to upstream:0/0 2022/02/11 15:24:55 [error] 13781#13781: *8133030 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: 0.0.0.0:1488, upstream: "10.88.88.88:3888", bytes from/to client:0/0, bytes from/to upstream:0/0