Skip to content

Metrics Server📜

Overview📜

Metrics Server is an addon cluster utility that adds functionality to Kubernetes clusters rather than applications. It is used for monitoring pod CPU & memory utilization for use with autoscaling pods horizontally and vertically.

Metrics Server collects resource metrics from Kubelets and exposes them in Kubernetes apiserver through Metrics API for use by Horizontal Pod Autoscaler and Vertical Pod Autoscaler. Metrics API can also be accessed by kubectl top, making it easier to debug autoscaling pipelines.

Metrics Server is not meant for non-autoscaling purposes. For example, don’t use it to forward metrics to monitoring solutions, or as a source of monitoring solution metrics. In such cases please collect metrics from Kubelet /metrics/resource endpoint directly.

Metrics Server offers:

  • A single deployment that works on most clusters (see Requirements)
  • Fast autoscaling, collecting metrics every 15 seconds.
  • Resource efficiency, using 1 mili core of CPU and 2 MB of memory for each node in a cluster.
  • Scalable support up to 5,000 node clusters.

Use cases📜

You can use Metrics Server for:

Don’t use Metrics Server when you need:

  • Non-Kubernetes clusters
  • An accurate source of resource usage metrics
  • Horizontal autoscaling based on other resources than CPU/Memory

For unsupported use cases, check out full monitoring solutions like Prometheus (also deployed by default within the BigBang monitoring package).

Deployment📜

Since Metrics Server is prerequisite for a number of Kubernetes components (HPA, scheduler, kubectl top) it typically will run by default in most Kubernetes clusters. By default within a BigBang deployment, the enabled value is set to auto, which installs only if metrics API endpoint is not present.

Big Bang Touch Points📜

Architecture:📜

Metrics Server collects resource metrics from Kubelets (the primary “node agent” that runs on each node) and exposes them in Kubernetes apiserver through Metrics API for use by Horizontal Pod Autoscaler and Vertical Pod Autoscaler. Metrics API can also be accessed by kubectl top, making it easier to debug autoscaling pipelines.

Storage📜

To store data in memory Metric Server will replace the default storage layer (etcd) by introducing in-memory store which will implement Storage interface.

Only the most recent value of each metric will be remembered.

Users looking to access historical data should look into Prometheus and Grafana packages as part of BigBang’s monitoring stack.

Istio Configuration📜

Istio is disabled in the metric server chart by default and can be enabled by setting the following values in the bigbang chart:

istio:
  enabled: true

These values get passed into the metric server chart here.

High Availability📜

Metrics Server is simply installed in high availability mode by setting the replicas chart value greater than 1. The default configuration within BigBang is a 2 replica deployment.

Additional Metric Server High Availability documentation can be found here.

Requirements📜

Metrics Server has specific requirements for cluster and network configuration. These requirements aren’t the default for all cluster distributions. Please ensure that your cluster distribution supports these requirements before using Metrics Server:

  • The kube-apiserver must enable an aggregation layer.
  • Nodes must have Webhook authentication and authorization enabled.
  • Kubelet certificate needs to be signed by cluster Certificate Authority (or disable certificate validation by passing --kubelet-insecure-tls to Metrics Server)
  • Container runtime must implement a container metrics RPCs (or have cAdvisor support)
  • Network should support following communication:
  • Control plane to Metrics Server. Control plane node needs to reach Metrics Server’s pod IP and port 10250 (or node IP and custom port if hostNetwork is enabled). Read more about control plane to node communication.
  • Metrics Server to Kubelet on all nodes. Metrics server needs to reach node address and Kubelet port. Addresses and ports are configured in Kubelet and published as part of Node object. Addresses in .status.addresses and port in .status.daemonEndpoints.kubeletEndpoint.port field (default 10250). Metrics Server will pick first node address based on the list provided by kubelet-preferred-address-types command line flag (default InternalIP,ExternalIP,Hostname in manifests).

Resource footprint on nodes within cluster are minimal at: 1 mili core of CPU and 2 MB of memory per node.

Single Sign on (SSO)📜

None. This service doesn’t have a web interface.

Other Resources📜

Licensing📜

Metric Server utilizes an Apache 2.0 license.

Last update: 2024-01-31 by Michael Martin