Kube State Metrics

Overview

Kube State Metrics (KSM) is a service that listens to the Kubernetes API server and generates metrics about the state of Kubernetes objects. It is not focused on the health of individual Kubernetes components, but rather on the health of various objects inside, such as deployments, nodes, and pods.

Installation

1. Create Namespace

kubectl create namespace kube-state-metrics

2. Install KSM

kubectl apply -f kube-state-metrics.yaml

kube-state-metrics.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.10.1
  name: kube-state-metrics
  namespace: kube-state-metrics
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.10.1
  name: kube-state-metrics
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - secrets
  - nodes
  - pods
  - services
  - resourcequotas
  - replicationcontrollers
  - limitranges
  - persistentvolumeclaims
  - persistentvolumes
  - namespaces
  - endpoints
  verbs:
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - statefulsets
  - daemonsets
  - deployments
  - replicasets
  verbs:
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - list
  - watch
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - list
  - watch
- apiGroups:
  - authentication.k8s.io
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectaccessreviews
  verbs:
  - create
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - list
  - watch
- apiGroups:
  - certificates.k8s.io
  resources:
  - certificatesigningrequests
  verbs:
  - list
  - watch
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  - volumeattachments
  verbs:
  - list
  - watch
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - mutatingwebhookconfigurations
  - validatingwebhookconfigurations
  verbs:
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - networkpolicies
  - ingresses
  verbs:
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.10.1
  name: kube-state-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
- kind: ServiceAccount
  name: kube-state-metrics
  namespace: kube-state-metrics
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.10.1
  name: kube-state-metrics
  namespace: kube-state-metrics
spec:
  clusterIP: None
  ports:
  - name: http-metrics
    port: 8080
    targetPort: http-metrics
  - name: telemetry
    port: 8081
    targetPort: telemetry
  selector:
    app.kubernetes.io/name: kube-state-metrics
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.10.1
  name: kube-state-metrics
  namespace: kube-state-metrics
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: kube-state-metrics
  template:
    metadata:
      labels:
        app.kubernetes.io/component: exporter
        app.kubernetes.io/name: kube-state-metrics
        app.kubernetes.io/version: 2.10.1
    spec:
      containers:
      - image: registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.10.1
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 5
          timeoutSeconds: 5
        name: kube-state-metrics
        ports:
        - containerPort: 8080
          name: http-metrics
        - containerPort: 8081
          name: telemetry
        readinessProbe:
          httpGet:
            path: /
            port: 8081
          initialDelaySeconds: 5
          timeoutSeconds: 5
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsUser: 65534
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: kube-state-metrics

3. Verify Installation

kubectl get pods -n kube-state-metrics

The output should look like this:

NAME                                 READY   STATUS    RESTARTS   AGE
kube-state-metrics-9d4f9f67-4qnqz   1/1     Running   0          10s

Metrics Collection

After installing KSM, you can configure the OpenTelemetry Collector to collect metrics from KSM. Here's an example configuration:

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: 'kube-state-metrics'
          static_configs:
            - targets: ['kube-state-metrics.kube-state-metrics.svc.cluster.local:8080']

processors:
  batch:

exporters:
  otlp:
    endpoint: "otelcol:4317"
    tls:
      insecure: true

service:
  pipelines:
    metrics:
      receivers: [prometheus]
      processors: [batch]
      exporters: [otlp]

This configuration will:

1. Scrape metrics from the KSM service endpoint
2. Process them in batches
3. Export them to your OpenTelemetry Collector

Available Metrics

KSM provides a wide range of metrics about various Kubernetes objects. Here are some key metric categories:

• Node Metrics: Information about node status, capacity, and allocatable resources
• Pod Metrics: Pod status, resource requests/limits, and container states
• Deployment Metrics: Deployment status, replicas, and update progress
• StatefulSet Metrics: StatefulSet status and replica information
• DaemonSet Metrics: DaemonSet status and update information
• Job Metrics: Job completion status and duration
• CronJob Metrics: CronJob schedule and last schedule time
• Volume Metrics: PersistentVolume and PersistentVolumeClaim status
• Service Metrics: Service type and selector information

For a complete list of available metrics, you can visit the KSM documentation.