自建K8s上,如果部署了Nginx-Ingress,通常一些默认的参数有些可能需要优化下以便提升它的性能(阿里云之类的云厂商提供的Ingress是优化过的)。
我下面是自建的测试K8s上部署的ingress,这里贴下优化的地方:
kubectl get cm -n ingress-nginx
NAME DATA AGE ingress-controller-leader-nginx 0 6d19h ingress-nginx-controller 0 6d19h nginx-ingress-controller 22 5d20h
1、修改些nginx的常用参数
kubectl get cm -n ingress-nginx nginx-ingress-controller -oyaml 下面是data的内容,部分可以再根据实际情况修改(例如日志文件的路径)
apiVersion: v1 data: access-log-path: /var/log/nginx/access.log disable-access-log: "false" disable-ipv6: "true" disable-ipv6-dns: "true" enable-modsecurity: "false" enable-multi-accept: "true" enable-opentracing: "true" error-log-level: notice error-log-path: /var/log/nginx/error.log generate-request-id: "true" keep-alive: "60" keep-alive-requests: "10000" load-balance: round_robin log-format-escape-json: "true" max-worker-connections: "65535" max-worker-open-files: "10240" nginx-status-ipv4-whitelist: 0.0.0.0 reuse-port: "true" upstream-keepalive-connections: "200" upstream-keepalive-requests: "100" upstream-keepalive-timeout: "60" worker-processes: "4" kind: ConfigMap .... 其余部分忽略....
2、修改下nginx容器的内核参数
kubectl get deployments -n ingress-nginx -oyaml 注意这里加了个initContainers的配置,主要是修改一些内核参数,更适配Nginx的使用场景
dnsPolicy: ClusterFirst initContainers: - command: - /bin/sh - -c - | mount -o remount rw /proc/sys sysctl -w net.core.somaxconn=65535 sysctl -w net.ipv4.tcp_tw_reuse=1 sysctl -w net.ipv4.ip_local_port_range="1024 65535" sysctl -w fs.file-max=1048576 sysctl -w fs.inotify.max_user_instances=16384 sysctl -w fs.inotify.max_user_watches=524288 sysctl -w fs.inotify.max_queued_events=16384 image: busybox imagePullPolicy: IfNotPresent name: init-sysctl resources: {} securityContext: capabilities: add: - SYS_ADMIN drop: - ALL terminationMessagePath: /dev/termination-log terminationMessagePolicy: File nodeSelector: kubernetes.io/os: linux restartPolicy: Always schedulerName: default-scheduler
参考:
https://zhuanlan.zhihu.com/p/212620792 https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/ https://www.nginx.com/blog/tuning-nginx/ https://mp.weixin.qq.com/s/NAwz4dlsPuJnqfWYBHkfGg