Dashboard(仪表盘)是基于Web的Kubernetes用户界面。
可以使用仪表盘将容器化应用程序部署到Kubernetes集群,对容器化应用程序进行故障排除,并管理集群本身机器伴随资源。
可以使用仪表盘来概述集群上运行的应用程序,以及创建或修改单个Kubernetes资源(例如部署,作业,守护进程等)。
例如:可以使用部署向导扩展部署,启动滚动更新,重新启动Pod或部署新应用程序。仪表盘还提供有关集群中Kubernetes资源状态以及可能发生的任何错误的信息。
本次部署环境为k8s多Master节点
[root@master01 ~]# mkdir /opt/k8s/dashboard [root@master01 ~]# cd !$ cd /opt/k8s/dashboard [root@master01 dashboard]# rz -E #上传Dashboard.zip压缩包 rz waiting to receive. [root@master01 dashboard]# unzip Dashboard.zip #解压 Archive: Dashboard.zip inflating: dashboard-configmap.yaml inflating: dashboard-controller.yaml inflating: dashboard-rbac.yaml inflating: dashboard-secret.yaml inflating: dashboard-service.yaml inflating: k8s-admin.yaml inflating: dashboard-cert.sh [root@master01 dashboard]# ls dashboard-cert.sh dashboard-controller.yaml dashboard-secret.yaml Dashboard.zip dashboard-configmap.yaml dashboard-rbac.yaml dashboard-service.yaml k8s-admin.yaml
核心文件 | 说明 |
---|---|
dashboard-rbac.yaml | 用于访问控制设置,配置各种角色的访问控制权限及角色绑定(绑定角色和服务账户),内容中包含对应各种角色所配置的规则(rules) |
dashboard-secret.yaml | 提供令牌,访问API服务器所用(个人理解为一种安全认证机制) |
dashboard-configmap.yaml | 配置模板文件,负责设置Dashboard的文件,ConfigMap提供了将配置数据注入容器的方式,保证容器中的应用程序配置从Image内容中解耦 |
dashboard-controller.yaml | 负责控制器即服务账户的创建,来管理pod副本 |
dashboard-service.yaml | 负责将容器中的服务提供出去,供外部访问 |
Dashboard一共有7个文件,其中包含5个构建该界面的核心文件,一个k8s-admin.yaml文件是自己写的,用来生成待会在浏览器中登录时所用的令牌;一个dashboard-cert.sh,用来快速生成解决谷歌浏览器加密通信所需的证书文件。
核心文件官方下载资源地址:https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dashboard
[root@master01 dashboard]# cat dashboard-rbac.yaml kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: labels: k8s-app: kubernetes-dashboard addonmanager.kubernetes.io/mode: Reconcile name: kubernetes-dashboard-minimal namespace: kube-system rules: # Allow Dashboard to get, update and delete Dashboard exclusive secrets. - apiGroups: [""] resources: ["secrets"] resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"] verbs: ["get", "update", "delete"] # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map. - apiGroups: [""] resources: ["configmaps"] resourceNames: ["kubernetes-dashboard-settings"] verbs: ["get", "update"] # Allow Dashboard to get metrics from heapster. - apiGroups: [""] resources: ["services"] resourceNames: ["heapster"] verbs: ["proxy"] - apiGroups: [""] resources: ["services/proxy"] resourceNames: ["heapster", "http:heapster:", "https:heapster:"] verbs: ["get"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: kubernetes-dashboard-minimal namespace: kube-system labels: k8s-app: kubernetes-dashboard addonmanager.kubernetes.io/mode: Reconcile roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: kubernetes-dashboard-minimal subjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kube-system #主要参数说明: #kind: ServiceAccount 创建service用户,k8s中有两种用户,一种是ServiceAccount(给集群中的pod来访问集群用的),还有一种是具体的user(给咱们用户使用) #metadata 创建资源对象的一些元数据 #labels 标签信息 #name 资源对象名称 #namespace 命令空间 #kind: ClusterRoleBinding 创建用于集群绑定的角色,可以帮ServiceAccount绑定到具体的角色中、组中,使它有相应的访问权限 #kind: ClusterRole k8s中有两种角色,一种是ClusterRole(针对于整个集群的命名空间都起作用),还有一种是普通的角色(只对单个命名空间起作用)
[root@master01 dashboard]# cat dashboard-secret.yaml apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard # Allows editing resource and makes sure it is created first. addonmanager.kubernetes.io/mode: EnsureExists name: kubernetes-dashboard-certs namespace: kube-system type: Opaque --- apiVersion: v1 kind: Secret metadata: labels: k8s-app: kubernetes-dashboard # Allows editing resource and makes sure it is created first. addonmanager.kubernetes.io/mode: EnsureExists name: kubernetes-dashboard-key-holder namespace: kube-system type: Opaque
[root@master01 dashboard]# cat dashboard-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: labels: k8s-app: kubernetes-dashboard # Allows editing resource and makes sure it is created first. addonmanager.kubernetes.io/mode: EnsureExists name: kubernetes-dashboard-settings namespace: kube-system
[root@master01 dashboard]# cat dashboard-controller.yaml apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: kubernetes-dashboard addonmanager.kubernetes.io/mode: Reconcile name: kubernetes-dashboard namespace: kube-system --- apiVersion: apps/v1 kind: Deployment metadata: name: kubernetes-dashboard namespace: kube-system labels: k8s-app: kubernetes-dashboard kubernetes.io/cluster-service: "true" addonmanager.kubernetes.io/mode: Reconcile spec: selector: matchLabels: k8s-app: kubernetes-dashboard template: metadata: labels: k8s-app: kubernetes-dashboard annotations: scheduler.alpha.kubernetes.io/critical-pod: '' seccomp.security.alpha.kubernetes.io/pod: 'docker/default' spec: priorityClassName: system-cluster-critical containers: - name: kubernetes-dashboard image: siriuszg/kubernetes-dashboard-amd64:v1.8.3 resources: limits: cpu: 100m memory: 300Mi requests: cpu: 50m memory: 100Mi ports: - containerPort: 8443 protocol: TCP args: # PLATFORM-SPECIFIC ARGS HERE - --auto-generate-certificates volumeMounts: - name: kubernetes-dashboard-certs mountPath: /certs - name: tmp-volume mountPath: /tmp livenessProbe: httpGet: scheme: HTTPS path: / port: 8443 initialDelaySeconds: 30 timeoutSeconds: 30 volumes: - name: kubernetes-dashboard-certs secret: secretName: kubernetes-dashboard-certs - name: tmp-volume emptyDir: {} serviceAccountName: kubernetes-dashboard tolerations: - key: "CriticalAddonsOnly" operator: "Exists" #主要参数说明: #kind: Deployment 是整个集群中使用最频繁的对象,咱们应用服务一般都是使用Deployment来创建 #spec.selector.matchLabels 匹配某个标签 #spec.template.spec.serviceAccountName 指定创建的serviceAccount,使用该账户来访问集群 #spec.template.spec.containers.resources 对容器使用资源限制 #spec.template.spec.containers.ports.containerPort 指定暴露的端口 #spec.template.spec.containers.livenessProbe 健康检查 #spec.template.spec.containers.livenessProbe.initialDelaySeconds 检查间隔时间设置 #spec.template.spec.containers.livenessProbe.timeoutSeconds 检查超时设置
[root@master01 dashboard]# cat dashboard-service.yaml apiVersion: v1 kind: Service metadata: name: kubernetes-dashboard namespace: kube-system labels: k8s-app: kubernetes-dashboard kubernetes.io/cluster-service: "true" addonmanager.kubernetes.io/mode: Reconcile spec: type: NodePort selector: k8s-app: kubernetes-dashboard ports: - port: 443 targetPort: 8443 nodePort: 30001 #主要参数说明: #type: NodePort 可以通过在节点上使用nodeIP+端口访问服务 #spec.ports.port 为service在clusterIP暴露的端口 #spec.ports.targetPort 对应容器映射在pod上的端口 #spec.ports.nodePort 为nodeIP暴露的端口
[root@master01 dashboard]# cat k8s-admin.yaml apiVersion: v1 kind: ServiceAccount metadata: name: dashboard-admin namespace: kube-system --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: dashboard-admin subjects: - kind: ServiceAccount name: dashboard-admin namespace: kube-system roleRef: kind: ClusterRole name: cluster-admin apiGroup: rbac.authorization.k8s.io
[root@master01 dashboard]# cat dashboard-cert.sh #!/bin/bash #examle: ./dashboard-cert.sh /opt/k8s/k8s-cert/ cat > dashboard-csr.json <<EOF { "CN": "Dashboard", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing" } ] } EOF #定义一个变量,使用位置变量赋值,作用是指定你证书(依赖证书)的位置 K8S_CA=$1 #根据指定位置的证书进行创建和自签操作 cfssl gencert -ca=$K8S_CA/ca.pem -ca-key=$K8S_CA/ca-key.pem -config=$K8S_CA/ca-config.json -profile=kubernetes dashboard-csr.json | cfssljson -bare dashboard #生成的文件: #dashboard.csr:证书请求文件 #dashboard-key.pem:证书私钥 #dashboard.pem:数字签名证书 #清空命名空间中的认证 kubectl delete secret kubernetes-dashboard-certs -n kube-system #重新创建生成到指定的目录中(当前目录) kubectl create secret generic kubernetes-dashboard-certs --from-file=./ -n kube-system
规定kubernetes-dashboard-minimal该角色的权限:例如其中具备获取更新删除等不同的权限
[root@master01 dashboard]# kubectl create -f dashboard-rbac.yaml role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created #有几个kind就会有几个结果被创建,格式为kind+apiServer/name
查看类型为Role,RoleBinding的资源对象kubernetes-dashboard-minimal是否生成
[root@master01 dashboard]# kubectl get role,rolebinding -n kube-system #-n kube-system表示查看指定命名空间中的pod,缺省值为default NAME AGE role.rbac.authorization.k8s.io/extension-apiserver-authentication-reader 3d8h role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal 2m42s role.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager 3d8h role.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler 3d8h role.rbac.authorization.k8s.io/system:controller:bootstrap-signer 3d8h role.rbac.authorization.k8s.io/system:controller:cloud-provider 3d8h role.rbac.authorization.k8s.io/system:controller:token-cleaner 3d8h NAME AGE rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal 2m42s rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager 3d8h rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler 3d8h rolebinding.rbac.authorization.k8s.io/system:controller:bootstrap-signer 3d8h rolebinding.rbac.authorization.k8s.io/system:controller:cloud-provider 3d8h rolebinding.rbac.authorization.k8s.io/system:controller:token-cleaner 3d8h
证书和密钥创建
[root@master01 dashboard]# kubectl create -f dashboard-secret.yaml secret/kubernetes-dashboard-certs created secret/kubernetes-dashboard-key-holder created
查看类型为Secret的资源对象kubernetes-bashboard-crets,kubernetes-dashboard-key-holder是否生成
[root@master01 dashboard]# kubectl get secret -n kube-system NAME TYPE DATA AGE default-token-4nhtx kubernetes.io/service-account-token 3 3d8h kubernetes-dashboard-certs Opaque 0 107s kubernetes-dashboard-key-holder Opaque 0 107s
配置文件,对于集群dashboard设置的创建
[root@master01 dashboard]# kubectl create -f dashboard-configmap.yaml configmap/kubernetes-dashboard-settings created
查看类型为ConfigMap的资源对象kubernetes-dashboard-settings是否生成
[root@master01 dashboard]# kubectl get configmap -n kube-system NAME DATA AGE extension-apiserver-authentication 1 3d8h kubernetes-dashboard-settings 0 73s
创建容器需要的控制器以及服务账户
[root@master01 dashboard]# kubectl create -f dashboard-controller.yaml serviceaccount/kubernetes-dashboard created deployment.apps/kubernetes-dashboard created
查看类型为ServiceAccount,Deployment的资源对象kubernetes-dashboard-setting是否生成
[root@master01 dashboard]# kubectl get serviceaccount,deployment -n kube-system NAME SECRETS AGE serviceaccount/default 1 3d8h serviceaccount/kubernetes-dashboard 1 2m39s NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE deployment.extensions/kubernetes-dashboard 1 1 1 1 2m39s
将服务发布出去
[root@master01 dashboard]# kubectl create -f dashboard-service.yaml service/kubernetes-dashboard created
查看创建在指定的kube-system命名空间下的pod和service状态信息
[root@master01 dashboard]# kubectl get pods,svc -n kube-system -o wide #svc为service的缩写,可用 NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE pod/kubernetes-dashboard-65f974f565-nk4r8 1/1 Running 0 5m7s 172.17.97.3 192.168.122.12 <none> NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/kubernetes-dashboard NodePort 10.0.0.98 <none> 443:30001/TCP 3m37s k8s-app=kubernetes-dashboard
为提高速度,我这里已将siriuszg/kubernetes-dashboard-amd64:v1.8.3镜像压缩成tar包,在node节点释放该镜像。
该步骤也可省略,node节点会通过kubernetes公有仓库去自动拉取该镜像。
[root@node01 ~]# cd /opt [root@node01 opt]# rz -E #上传镜像包 rz waiting to receive. [root@node01 opt]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE nginx latest 87a94228f133 2 weeks ago 133MB centos 7 eeb6ee3f44bd 6 weeks ago 204MB nginx 1.14 295c7be07902 2 years ago 109MB registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64 3.0 99e59f495ffa 5 years ago 747kB [root@node01 opt]# docker load -i dashboard.tar #载入该镜像 23ddb8cbb75a: Loading layer [==================================================>] 102.8MB/102.8MB Loaded image: siriuszg/kubernetes-dashboard-amd64:v1.8.3 [root@node01 opt]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE nginx latest 87a94228f133 2 weeks ago 133MB centos 7 eeb6ee3f44bd 6 weeks ago 204MB nginx 1.14 295c7be07902 2 years ago 109MB siriuszg/kubernetes-dashboard-amd64 v1.8.3 784cf2722f44 3 years ago 102MB registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64 3.0 99e59f495ffa 5 years ago 747kB
不同浏览器的安全访问策略和防护级别是不同的,由于我们没有给dashboard做证书,因此使用不同的浏览器可能会出现不同的效果,本次使用我们最常用的Edge/Chrome、火狐以及360浏览器进行测试。
由于dashboard-service.yaml定义的nodePort: 30001,因此我们的测试地址应该是pod所属node的30001端口。
无法访问,由于Edge使用的是Chrome内核,因此Google的Chrome也是相同效果,可通过以下步骤查看问题。
chrome浏览器在Security项下查看
发现问题是缺少证书,那么我们为其制作证书即可。
[root@master01 dashboard]# cd /opt/k8s/dashboard/ [root@master01 dashboard]# vim dashboard-controller.yaml ...... args: # PLATFORM-SPECIFIC ARGS HERE ##在文件的第47行下面添加以下两行,指定加密(tls)的私钥和证书文件 - --auto-generate-certificates - --tls-key-file=dashboard-key.pem - --tls-cert-file=dashboard.pem
[root@master01 dashboard]# cd /opt/k8s/dashboard/ [root@master01 dashboard]# chmod +x dashboard-cert.sh [root@master01 dashboard]# ./dashboard-cert.sh /opt/k8s/k8s-cert/ 2021/11/01 02:35:39 [INFO] generate received request 2021/11/01 02:35:39 [INFO] received CSR 2021/11/01 02:35:39 [INFO] generating key: rsa-2048 2021/11/01 02:35:39 [INFO] encoded CSR 2021/11/01 02:35:39 [INFO] signed certificate with serial number 233541316653231246492121295508109281063386014227 2021/11/01 02:35:39 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for websites. For more information see the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org); specifically, section 10.2.3 ("Information Requirements"). secret "kubernetes-dashboard-certs" deleted secret/kubernetes-dashboard-certs created
[root@master01 dashboard]# ls *.pem dashboard-key.pem dashboard.pem
注意:当apply不生效时,先使用delete清除资源,再apply创建资源
[root@master01 dashboard]# kubectl apply -f dashboard-controller.yaml Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply serviceaccount/kubernetes-dashboard configured Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply deployment.apps/kubernetes-dashboard configured
由于可能会更换所分配的节点,所以要再次查看一下分配的节点服务器地址和端口号
[root@master01 dashboard]# kubectl get pods,svc -n kube-system -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE pod/kubernetes-dashboard-7dffbccd68-d8nzh 1/1 Running 0 2m39s 172.17.54.4 node01 <none> NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service/kubernetes-dashboard NodePort 10.0.0.98 <none> 443:30001/TCP 149m k8s-app=kubernetes-dashboard
查看发现node节点更换为node1,也就是192.168.122.11。
端口号仍是30001。
到此页面,说明可以访问,保持该页面,测试下一个浏览器。
进入该页面,说明可以访问,保持该页面,测试下一个浏览器。
360浏览器虽然显示证书风险,但未出现任何阻止浏览或风险提示窗口,直接可进入登录页面。
[root@master01 dashboard]# kubectl create -f k8s-admin.yaml serviceaccount/dashboard-admin created clusterrolebinding.rbac.authorization.k8s.io/dashboard-admin created
[root@master01 dashboard]# kubectl get secrets -n kube-system NAME TYPE DATA AGE dashboard-admin-token-n5dcl kubernetes.io/service-account-token 3 86s default-token-4nhtx kubernetes.io/service-account-token 3 3d11h kubernetes-dashboard-certs Opaque 11 19m kubernetes-dashboard-key-holder Opaque 2 170m kubernetes-dashboard-token-kkpxs kubernetes.io/service-account-token 3 165m
[root@master01 dashboard]# kubectl describe secrets dashboard-admin-token-n5dcl -n kube-system Name: dashboard-admin-token-n5dcl Namespace: kube-system Labels: <none> Annotations: kubernetes.io/service-account.name: dashboard-admin kubernetes.io/service-account.uid: e3600e3f-3a7b-11ec-adb1-000c2959bebe Type: kubernetes.io/service-account-token Data ==== ca.crt: 1359 bytes namespace: 11 bytes token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tbjVkY2wiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZTM2MDBlM2YtM2E3Yi0xMWVjLWFkYjEtMDAwYzI5NTliZWJlIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.kCkjHd8bfgecC_sNrsauvkBy7O09dQY7KAbz-2pNnDwEdKNkAR4y7cwC8zCNLsul7uuHVZs5hCp6iGti1EeUUEMoy8cEBlC4WDOxSQdJQzi9RTCSkCHrReql2nGfGpFHx15JkcyB2CY8BBBRaUvIbe6phX5sOmUlJWf5K4FI0sQHYpefH0vYSr8CWWCeccajlDZPEqgLkUpAUMHT2fjhJNfWgbTZDBMEye6nnyQjS92s8qECF1jBgRbIfTZKWHqPRpKmbz9oFKnKlcH2BBgDonpE3cDSfmSTH6SgYDHKQuGAwV_vjIIP_GMxdXmM_ymGZAdcdw9kd0EBCSdOQH8yIw
将令牌序列号复制填入到浏览器页面中,点击登录
登录成功
[root@node01 opt]# curl 172.17.54.3 this is nginx_dashboard_test web
设定完成后,点击部署
完成部署,通过master节点查看
[root@master01 dashboard]# kubectl get pod NAME READY STATUS RESTARTS AGE nginx-7d498867b6-5mv26 1/1 Running 0 5m15s nginx-7d498867b6-spkgq 1/1 Running 0 5m15s nginx-7d498867b6-w9dvq 1/1 Running 0 5m15s nginx-test-7dc4f9dcc9-bklg4 1/1 Running 0 6h18m [root@master01 dashboard]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE nginx-7d498867b6-5mv26 1/1 Running 0 5m20s 172.17.54.5 node01 <none> nginx-7d498867b6-spkgq 1/1 Running 0 5m20s 172.17.97.4 192.168.122.12 <none> nginx-7d498867b6-w9dvq 1/1 Running 0 5m20s 172.17.97.3 192.168.122.12 <none> nginx-test-7dc4f9dcc9-bklg4 1/1 Running 0 6h18m 172.17.54.3 node01 <none>