2020-11-18

Kubernetes 部署 Metrics Server 获取集群指标数据

系统环境：

Kubernetes 版本：1.19.4
Metrics Server 版本：v4.0.1

示例部署文件地址：

Kubernetes 部署 Metrics Server 部署文件

一、Metrics Server 简介

介绍 Metrics Server 前首先介绍下 Heapster，该工具是用于 Kubernetes 集群监控和性能分析工具，可以收集节点上的指标数据，例如，节点的 CPU、Memory、Network 和 Disk 的 Metric 数据。不过在 Kubernetes V1.11 版本后将被逐渐废弃。而 Metrics Server 正是 Heapster 的代替者。

Metrics Server 是 Kubernetes 集群核心监控数据的聚合器，可以通过 Metrics API 的形式获取 Metrics 数据，不过仅仅是获取指标的最新值，不对旧值进行存储，且不负责将指标转发到第三方目标。Metrics Server 还可以与 Kubectl 工具结合使用，提供 kubectl top 命令来展示集群中的指标数据，接下来我们开始部署 Metrics Server。

二、部署应用权限 RBAC 资源

Kubernetes 部署 Metrics Server 前需要先提前部署 RBAC 相关配置，这样 Metrics Server 才能有足够的权限获取系统组件的信息。

创建 Metrics RBAC 文件

metrics-rbac.yaml

1
## ServiceAccount
2
apiVersion: v1
3
kind: ServiceAccount
4
metadata:
5
  labels:
6
    k8s-app: metrics-server
7
  name: metrics-server
8
  namespace: kube-system
9
---
10
## ClusterRole aggregated-metrics-reader
11
apiVersion: rbac.authorization.k8s.io/v1
12
kind: ClusterRole
13
metadata:
14
  name: system:aggregated-metrics-reader
15
  labels:
16
    k8s-app: metrics-server
17
    rbac.authorization.k8s.io/aggregate-to-view: "true"
18
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
19
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
20
rules:
21
- apiGroups: ["metrics.k8s.io"]
22
  resources: ["pods","nodes"]
23
  verbs: ["get","list","watch"]
24
---
25
## ClusterRole metrics-server
26
apiVersion: rbac.authorization.k8s.io/v1
27
kind: ClusterRole
28
metadata:
29
  name: system:metrics-server
30
  labels:
31
    k8s-app: metrics-server
32
rules:
33
- apiGroups: [""]
34
  resources: ["pods","nodes","nodes/stats","namespaces","configmaps"]
35
  verbs: ["get","list","watch"]
36
---
37
## ClusterRoleBinding auth-delegator
38
apiVersion: rbac.authorization.k8s.io/v1
39
kind: ClusterRoleBinding
40
metadata:
41
  name: metrics-server:system:auth-delegator
42
  labels:
43
    k8s-app: metrics-server
44
roleRef:
45
  apiGroup: rbac.authorization.k8s.io
46
  kind: ClusterRole
47
  name: system:auth-delegator
48
subjects:
49
- kind: ServiceAccount
50
  name: metrics-server
51
  namespace: kube-system
52
---
53
## RoleBinding metrics-server-auth-reader
54
apiVersion: rbac.authorization.k8s.io/v1
55
kind: RoleBinding
56
metadata:
57
  name: metrics-server-auth-reader
58
  namespace: kube-system
59
  labels:
60
    k8s-app: metrics-server
61
roleRef:
62
  apiGroup: rbac.authorization.k8s.io
63
  kind: Role
64
  name: extension-apiserver-authentication-reader
65
subjects:
66
- kind: ServiceAccount
67
  name: metrics-server
68
  namespace: kube-system
69
---
70
## ClusterRoleBinding system:metrics-server
71
apiVersion: rbac.authorization.k8s.io/v1
72
kind: ClusterRoleBinding
73
metadata:
74
  name: system:metrics-server
75
  labels:
76
    k8s-app: metrics-server
77
roleRef:
78
  apiGroup: rbac.authorization.k8s.io
79
  kind: ClusterRole
80
  name: system:metrics-server
81
subjects:
82
- kind: ServiceAccount
83
  name: metrics-server
84
  namespace: kube-system

通过 Kubectl 工具部署 Metrics RBAC

-n：指定部署应用的 Namespace 命名空间

1
$ kubectl apply -f metrics-rbac.yaml -n kube-system

三、部署 APIService 资源

设置扩展 API Service 工作于聚合层，允许使用其 API 扩展 Kubernetes apiserver，而这些 API 并不是核心 Kubernetes API 的一部分。这里部署 APIservice 资源，来提供 Kubernetes Metrics 指标 API 数据。

创建 Metrics APIService 文件

metrics-api-service.yaml

1
## APIService
2
apiVersion: apiregistration.k8s.io/v1
3
kind: APIService
4
metadata:
5
  labels:
6
    k8s-app: metrics-server
7
  name: v1beta1.metrics.k8s.io
8
spec:
9
  group: metrics.k8s.io
10
  service:
11
    name: metrics-server
12
    namespace: kube-system
13
  version: v1beta1
14
  groupPriorityMinimum: 100
15
  insecureSkipTLSVerify: true
16
  versionPriority: 100

通过 Kubectl 工具部署 Metrics APIService

-n：指定部署应用的 Namespace 命名空间

1
$ kubectl apply -f metrics-api-service.yaml -n kube-system

四、部署 Metrics Server 应用

创建 Metrics 部署文件

metrics-server-deploy.yaml

1
## Service
2
apiVersion: v1
3
kind: Service
4
metadata:
5
  labels:
6
    k8s-app: metrics-server
7
  name: metrics-server
8
  namespace: kube-system
9
spec:
10
  ports:
11
  - name: https
12
    port: 443
13
    protocol: TCP
14
    targetPort: https
15
  selector:
16
    k8s-app: metrics-server
17
---
18
## Deployment
19
apiVersion: apps/v1
20
kind: Deployment
21
metadata:
22
  name: metrics-server
23
  namespace: kube-system
24
  labels:
25
    k8s-app: metrics-server
26
spec:
27
  selector:
28
    matchLabels:
29
      k8s-app: metrics-server
30
  strategy:
31
    rollingUpdate:
32
      maxUnavailable: 0
33
  template:
34
    metadata:
35
      name: metrics-server
36
      labels:
37
        k8s-app: metrics-server
38
    spec:
39
      hostNetwork: true
40
      serviceAccountName: metrics-server
41
      containers:
42
      - name: metrics-server
43
        image: bitnami/metrics-server:0.4.1
44
        imagePullPolicy: IfNotPresent
45
        args:
46
          - --cert-dir=/tmp
47
          - --secure-port=4443
48
          - --kubelet-insecure-tls
49
          - --kubelet-use-node-status-port
50
          - --kubelet-preferred-address-types=InternalDNS,InternalIP,ExternalDNS,ExternalIP,Hostname
51
        livenessProbe:
52
          failureThreshold: 3
53
          httpGet:
54
            path: /livez
55
            port: https
56
            scheme: HTTPS
57
          periodSeconds: 10
58
        readinessProbe:
59
          failureThreshold: 3
60
          httpGet:
61
            path: /readyz
62
            port: https
63
            scheme: HTTPS
64
          periodSeconds: 10
65
        ports:
66
        - name: https
67
          containerPort: 4443
68
          protocol: TCP
69
        securityContext:
70
          readOnlyRootFilesystem: true
71
          runAsNonRoot: true
72
          runAsUser: 1000
73
        resources:
74
          limits:
75
            memory: 1Gi
76
            cpu: 1000m
77
          requests:
78
            memory: 1Gi
79
            cpu: 1000m
80
        volumeMounts:
81
        - name: tmp-dir
82
          mountPath: /tmp
83
        - name: localtime
84
          readOnly: true
85
          mountPath: /etc/localtime
86
      volumes:
87
      - name: tmp-dir
88
        emptyDir: {}
89
      - name: localtime
90
        hostPath:
91
          type: File
92
          path: /etc/localtime
93
      nodeSelector:
94
        kubernetes.io/os: linux

通过 Kubectl 工具部署 Metrics 应用

-n：指定部署应用的 Namespace 命名空间

1
$ kubectl apply -f metrics-server-deploy.yaml -n kube-system

五、进行测试

当部署完 Metrics Server 后，可以通过 kubectl 工具进行测试，默认支持下面命令：

kubectl top pod： 获取 Pod 的 CPU、Memory 使用信息。
kubectl top node： 获取 Node 的 CPU、Memory 使用信息。

输入上面命令进行测试，如下：

1
## 获取全部节点指标信息
2
$ kubectl top node
3
NAME            CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
4
k8s-master      228m         5%     1204Mi          44%
5
k8s-node-2-12   131m         1%     1843Mi          23%
6
k8s-node-2-13   73m          0%     576Mi           7%
7

8
## 获取某个 Namespace Pod 的指标信息
9
$ kubectl top pods -n kube-system
10
NAME                                         CPU(cores)   MEMORY(bytes)
11
coredns-9d85f5447-c82w7                      7m           22Mi
12
coredns-9d85f5447-kcmb4                      7m           21Mi
13
dashboard-metrics-scraper-65f454dff8-2pts8   1m           22Mi
14
etcd-k8s-master                              22m          104Mi
15
kube-apiserver-k8s-master                    58m          311Mi
16
kube-controller-manager-k8s-master           27m          46Mi
17

18
## 获取某个 Namespace 下某个 Pod 的指标信息
19
$ kubectl top pods coredns-9d85f5447-c82w7 -n kube-system
20
NAME                      CPU(cores)   MEMORY(bytes)
21
coredns-9d85f5447-c82w7   7m           21Mi
22

23
## 获取全部 Namespace 下的 Pod 的指标信息
24
$ kubectl top pods --all-namespaces
25
NAMESPACE     NAME                                           CPU(cores)   MEMORY(bytes)
26
kube-system   coredns-9d85f5447-c82w7                        6m           22Mi
27
kube-system   coredns-9d85f5447-kcmb4                        6m           21Mi
28
kube-system   dashboard-metrics-scraper-65f454dff8-2pts8     1m           22Mi
29
kube-system   etcd-k8s-master                                21m          106Mi
30
kube-system   kube-apiserver-k8s-master                      62m          311Mi
31
kube-system   kube-controller-manager-k8s-master             26m          46Mi
32
kube-system   kube-proxy-kpt7c                               2m           36Mi
33
kube-system   kube-proxy-zb2l5                               1m           27Mi
34
kube-system   kube-scheduler-k8s-master                      5m           21Mi
35
kube-system   kubernetes-dashboard-7bf47cd79c-nctx2          1m           55Mi
36
kube-system   metrics-server-6d54447849-nnbfk                3m           20Mi

---END---

如果本文对你有帮助，可以关注我的公众号 "小豆丁技术栈" 了解最新动态，顺便也请帮忙 Github 点颗星哦，感谢~