Get the current and the most latest CPU and Memory usage of all the pods

Question

I would like to know how to get current or the last read metric value of CPU and memory usage of all pods.

I tried to call the hawkler endpoint. I went to the browser developer mode by hitting f12 and took this endpoint from list of calls that are made when metrics page of a pod is loaded.

https://metrics.mydev.abccomp.com/hakular/metrics/gauges/myservice%dfjlajdflk-lkejre-12112kljdfkl%2Fcpu%2Fusage_rate/data?bucketDuration=1mn&start=-1mn

However this will give me the cpu usage metrics for the last minute, for that particular pod. I am trying to see if there is a command or way exisits that will give me only the current snapshot of cpu usage and memory stats of all the pods collectively like below:

pod        memory usage     memory max       cpu usage         cpu max
pod1       0.4 mb            2.0 mb            20 m cores       25 m cores
pod2       1.5 mb            2.0 mb            25 m cores       25 m cores

score 1 · Answer 1 · answered Aug 13 '19 at 09:07

To see the pods that use the most cpu and memory you can use the kubectl top command but it doesn't sort yet and is also missing the quota limits and requests per pod. You only see the current usage.

Execute command below:

$ kubectl top pods --all-namespaces --containers=true

Because of these limitations, but also because you want to gather and store this resource usage information on an ongoing basis, a monitoring tool comes in handy. This allows you to analyze resource usage both in real time and historically, and also lets you alert on capacity bottlenecks.

To workaround problem " Error from server (Forbidden): unknown (get services http:heapster"

Make sure that heapster deployment don't forgot to install the Service for heapster, otherwise you will have to do it manually.

E.g.:

$ kubectl create -f /dev/stdin <<SVC
apiVersion: v1
kind: Service
metadata:
  name: heapster
  namespace: kube-system
spec:
  selector:
    whatever-label: is-on-heapster-pods
  ports:
  - name: http
    port: 80
    targetPort: whatever-is-heapster-is-listening-on
SVC

Suresh Vishnoi · Answer 2 · 2019-08-21T14:50:06.957

0

List resource CPU & Memory utilization for all containers (pods)

kubectl top pods --all-namespaces --containers=true

The default admin, edit, view, and cluster-reader cluster roles support cluster role aggregation, where the cluster rules for each role are dynamically updated as new rules are created. This feature is relevant only if you extend the Kubernetes API by creating custom resources. Cluster-role-aggregator

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: top-pods-admin
  labels:
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rbac.authorization.k8s.io/aggregate-to-view: "true"
rules:
- apiGroups:
  - metrics.k8s.io
  resources:
  - pods
  verbs:
  - get
  - list
  - watch

edited Aug 21 '19 at 14:50

answered Aug 10 '19 at 14:17

Suresh Vishnoi

12,639
5
32
44

i dont have kubectl installed. Can this be done like below: ```oc top pods --all-namespaces --containers=true``` – lr-pal Aug 10 '19 at 14:24
yea, `oc adm top pods --all-namespaces --containers=true` – Suresh Vishnoi Aug 10 '19 at 14:26
Btw, when you install oc, it also install kubectl – Suresh Vishnoi Aug 10 '19 at 14:26
i installed kubectl now. and here is what i get ```$ kubectl top pods --all-namespaces --containers=true Error from server (Forbidden): unknown (get services http:heapster:) ``` – lr-pal Aug 10 '19 at 14:45
so, you need to have an RBAC. it seems you have access to project resources only, ask your cluster admin to give you a role (Cluster-reader) to access for these endpoints, – Suresh Vishnoi Aug 10 '19 at 15:09
this might give you some insights https://docs.openshift.com/container-platform/3.11/admin_guide/manage_rbac.html – Suresh Vishnoi Aug 10 '19 at 15:10
Can i use a service account as argument in the command or something? – lr-pal Aug 10 '19 at 15:15
I think ServiceAccounts are namespaced object in the cluster. nevertheless you can try just execute the same command with `--as=TOKEN` flag – Suresh Vishnoi Aug 10 '19 at 15:27
i tried with the service account, it resulted in the same error. I can get the right access before i try – lr-pal Aug 10 '19 at 15:42
@Ir-pal, apply above clusterroles than you will be able to pull the resource usages. – Suresh Vishnoi Aug 22 '19 at 14:40

Get the current and the most latest CPU and Memory usage of all the pods

2 Answers2