通常情况下,service和pod的IP仅可在集群内部访问。集群外部的请求需要通过负载均衡转发到service在Node上暴露的NodePort上,然后再由kube-proxy将其转发给相关的Pod。
而Ingress就是为进入集群的请求提供路由规则的集合,如下图所示
internet | [ Ingress ] --|-----|-- [ Services ]
Ingress可以给service提供集群外部访问的URL、负载均衡、SSL终止、HTTP路由等。为了配置这些Ingress规则,集群管理员需要部署一个Ingress controller,它监听Ingress和service的变化,并根据规则配置负载均衡并提供访问入口。
一个 Ingress controller 是能将一个特定的负载均衡器和 Kubernetes 整合在一起的软件。开源的 NGINX 和 NGINX Plus 有相应的 Ingress controller,GitHub 仓库(https://github.com/nginxinc/kubernetes-ingress)。我自己理解Ingress Controller 实际上是将nginx 和 kubernetes做了一个整合,使得nginx 可以拿到kubernetes 的service 信息,我们启动一个ingressController 之后,在编写对应的rule 即可实现转发以及负载均衡。
可以理解为ingress 是对service进行代理,我们不需要知道service 对pods 暴露的端口等信息,pods 重启service 暴露的端口改变等对于应用无感,应用只和ingress 大交道。
ingress 和 pod 通过service 进行关联,ingress 作为统一入口,由service 关联一组pod。
ingress 作为统一的入口,根据请求路径不同转发到后端不同服务。
1. 部署ingress controller
2. 创建ingress 规则
选择官方维护nginx 控制器,实现部署。简单的实现基于不同的域名转发到不同的service。效果如下:
1. 创建nginx 应用,使用service 对外暴露端口
kubectl create deployment nginx --image=nginx kubectl expose deployment nginx --port=80 --type=NodePort
查看svc 以及 相关信息:
[root@k8smaster1 ~]# kubectl get svc -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 8d <none> nginx NodePort 10.109.175.64 <none> 80:32503/TCP 32m app=nginx
接下来从windows, 通过任何k8s 节点的32503 端口即可访问到nginx。
2. 创建tomcat服务, 并且使用service 对其暴露端口
[root@k8smaster1 ~]# kubectl create deployment tomcat --image=tomcat:8.0.15-jre8 deployment.apps/tomcat created [root@k8smaster1 ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-f89759699-vkf7d 1/1 Running 1 59m tomcat-58767d5b5-f5qwj 1/1 Running 0 8s [root@k8smaster1 ~]# kubectl expose deployment tomcat --port=8080 --type=NodePort service/tomcat exposed [root@k8smaster1 ~]# kubectl get svc -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 8d <none> nginx NodePort 10.109.175.64 <none> 80:32503/TCP 59m app=nginx tomcat NodePort 10.100.178.25 <none> 8080:30158/TCP 6s app=tomcat
3. 最后service 暴露后的效果如下: 可以从外部通过k8s任意的节点IP+端口访问到指定服务
4. 创建ingress controller
(1) 创建ingress-controller.yaml
apiVersion: v1 kind: Namespace metadata: name: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: nginx-configuration namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: nginx-ingress-clusterrole labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" - "networking.k8s.io" resources: - ingresses verbs: - get - list - watch - apiGroups: - "extensions" - "networking.k8s.io" resources: - ingresses/status verbs: - update --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: Role metadata: name: nginx-ingress-role namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "<election-id>-<ingress-class>" # Here: "<ingress-controller-leader>-<nginx>" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-role subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: nginx-ingress-clusterrole-nisa-binding labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrole subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: apps/v1 kind: Deployment metadata: name: nginx-ingress-controller namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: replicas: 1 selector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx template: metadata: labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx annotations: prometheus.io/port: "10254" prometheus.io/scrape: "true" spec: hostNetwork: true # wait up to five minutes for the drain of connections terminationGracePeriodSeconds: 300 serviceAccountName: nginx-ingress-serviceaccount nodeSelector: kubernetes.io/os: linux containers: - name: nginx-ingress-controller image: lizhenliang/nginx-ingress-controller:0.30.0 args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix=nginx.ingress.kubernetes.io securityContext: allowPrivilegeEscalation: true capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 101 runAsUser: 101 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 protocol: TCP - name: https containerPort: 443 protocol: TCP livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 lifecycle: preStop: exec: command: - /wait-shutdown --- apiVersion: v1 kind: LimitRange metadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: limits: - min: memory: 90Mi cpu: 100m type: ContainerView Code
(2) 创建相关资源
[root@k8smaster1 ~]# kubectl apply -f ingress-controller.yaml namespace/ingress-nginx created configmap/nginx-configuration created configmap/tcp-services created configmap/udp-services created serviceaccount/nginx-ingress-serviceaccount created clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created role.rbac.authorization.k8s.io/nginx-ingress-role created rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created deployment.apps/nginx-ingress-controller created limitrange/ingress-nginx created [root@k8smaster1 ~]# kubectl get all -o wide -n ingress-nginx NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES pod/nginx-ingress-controller-766fb9f77-tksqp 0/1 ContainerCreating 0 36s 192.168.13.105 k8snode2 <none> <none> NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR deployment.apps/nginx-ingress-controller 0/1 1 0 36s nginx-ingress-controller lizhenliang/nginx-ingress-controller:0.30.0 app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR replicaset.apps/nginx-ingress-controller-766fb9f77 1 1 0 36s nginx-ingress-controller lizhenliang/nginx-ingress-controller:0.30.0 app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx,pod-template-hash=766fb9f77
(3) 到k8snode2 节点查看nginx 监听的端口, nginx-controller 实际是启动了一个nginx
[root@k8snode2 ~]# netstat -nltp | grep 80 tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 5852/nginx: master tcp6 0 0 :::80 :::* LISTEN 5852/nginx: master [root@k8snode2 ~]# netstat -nltp | grep 443 tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 5852/nginx: master tcp6 0 0 :::443 :::* LISTEN 5852/nginx: master
5. 编写ingress 规则
(1) 编写 ingress-rule.yaml
apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: example-ingress spec: rules: - host: nginx.ingredemo.com http: paths: - path: / backend: serviceName: nginx servicePort: 80 - host: tomcat.ingredemo.com http: paths: - path: / backend: serviceName: tomcat servicePort: 8080
(2) 执行创建并且查看
[root@k8smaster1 ~]# kubectl apply -f ingress-rule.yaml ingress.networking.k8s.io/example-ingress configured [root@k8smaster1 ~]# kubectl get ing -o wide NAME CLASS HOSTS ADDRESS PORTS AGE example-ingress <none> nginx.ingredemo.com,tomcat.ingredemo.com 80 15h [root@k8smaster1 ~]# kubectl describe ing example-ingress Name: example-ingress Namespace: default Address: Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>) Rules: Host Path Backends ---- ---- -------- nginx.ingredemo.com / nginx:80 (10.244.1.89:80) tomcat.ingredemo.com / tomcat:8080 (10.244.1.92:8080) Annotations: Events: <none>
6. 测试:
(1) windows 下面配置虚拟的域名:
C:\Windows\System32\drivers\etc\hosts 文件增加如下配置, 注意IP 是自己nginx-controller 所在的节点
192.168.13.105 nginx.ingredemo.com 192.168.13.105 tomcat.ingredemo.com
(2) 访问测试:根据不同域名可以访问到不同的服务
补充: ingress 也可以根据不同的路径分配到不同的service
例如:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: test spec: rules: - host: foo.bar.com http: paths: - path: /foo backend: serviceName: s1 servicePort: 80 - path: /bar backend: serviceName: s2 servicePort: 80
上面表示如下结果:
foo.bar.com -> 178.91.123.132 -> / foo s1:80 / bar s2:80