Goglides Dev 🌱

Balkrishna Pandey
Balkrishna Pandey

Posted on • Originally published at Medium on

Part 01: How to force quotas limit to Kubernetes resources

How to force quotas limit to Kubernetes resources

Problem Description

By default,

  • Containers run with unbounded compute resources on a Kubernetes cluster.
  • If pod/ containers gone rouge and can monopolize resources, it can kill the entire cluster.

Solutions:

Use k8s features like Resource Quotas, LimitRange, Request/Limits to restrict this behavior. In this trashcan article, I am going through official blog to spike LimitRange feature.

Official page mentioned 4 types of resources,

  • Limiting Container compute resources
  • Limiting Pod compute resources
  • Limiting Storage resources
  • Limits/Requests Ratio

I am going to try all 4.

Limiting container compute resources

  • Create a namespace limitrange-demo using the following kubectl command:
kubectl create namespace limitrange-demo
Enter fullscreen mode Exit fullscreen mode
  • Create a configuration file (limitrange.yaml) for limit range objects
apiVersion: v1
kind: LimitRange
metadata:
  name: limit-mem-cpu-per-container
spec:
  limits:
  - max:
      cpu: "800m"
      memory: "1Gi"
    min:
      cpu: "100m"
      memory: "99Mi"
    default:
      cpu: "700m"
      memory: "900Mi"
    defaultRequest:
      cpu: "110m"
      memory: "111Mi"
    type: Container
Enter fullscreen mode Exit fullscreen mode
  • Now apply this limit range configuration file
kubectl apply -f limitrange.yaml -n limitrange-demo
Output:
limitrange/limit-mem-cpu-per-container created
Enter fullscreen mode Exit fullscreen mode

** You can validate if the object is properly created using the following command

kubectl describe -f limitrange.yaml -n limitrange-demo
Output:
Name: limit-mem-cpu-per-container
Namespace: limitrange-demo
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container memory 99Mi 5Gi 111Mi 900Mi -
Container cpu 100m 8 110m 700m -
# Or using following
kubectl describe limitrange/limit-mem-cpu-per-container -n limitrange-demo
Output:
Name: limit-mem-cpu-per-container
Namespace: limitrange-demo
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container cpu 100m 8 110m 700m -
Container memory 99Mi 5Gi 111Mi 900Mi -
Enter fullscreen mode Exit fullscreen mode
  • Example manifests to test this feature.

Create a manifest file busybox1.yaml with the following content.

apiVersion: v1
kind: Pod
metadata:
  name: busybox1
spec:
  containers:
  - name: busybox-cnt01
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt01; sleep 10;done"]
    resources:
      requests:
        memory: "100Mi"
        cpu: "100m"
      limits:
        memory: "200Mi"
        cpu: "500m"
  - name: busybox-cnt02
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt02; sleep 10;done"]
    resources:
      requests:
        memory: "100Mi"
        cpu: "100m"
  - name: busybox-cnt03
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt03; sleep 10;done"]
    resources:
      limits:
        memory: "200Mi"
        cpu: "500m"
  - name: busybox-cnt04
    image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo hello from cnt04; sleep 10;done"]
Enter fullscreen mode Exit fullscreen mode

Now send this manifest to k8s api.

kubectl apply -f busybox1.yaml -n limitrange-demo
Enter fullscreen mode Exit fullscreen mode

Let’s validate

kubectl describe -f busybox1.yaml -n limitrange-demo
Name: busybox1
Namespace: limitrange-demo
Priority: 0
Node: ip-69-76-24-106.us-west-2.compute.internal/69.76.24.106
Start Time: Tue, 25 Feb 2020 14:38:54 -0700
Labels: <none>
Annotations: cni.projectcalico.org/podIP: 100.96.5.82/32
              kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"busybox1","namespace":"limitrange-demo"},"spec":{"containers":[{"args...
              kubernetes.io/limit-ranger:
                LimitRanger plugin set: cpu, memory limit for container busybox-cnt02; cpu, memory request for container busybox-cnt04; cpu, memory limit ...
Status: Running
IP: 100.96.5.82
IPs: <none>
Containers:
  busybox-cnt01:
    Container ID: docker://afcd9d2ec8e5568fde8d9d9c155266bd1d685a36d8bc79ba6677a119ba6f09b5
    Image: busybox
    Image ID: docker-pullable://busybox@sha256:6915be4043561d64e0ab0f8f098dc2ac48e077fe23f488ac24b665166898115a
    Port: <none>
    Host Port: <none>
    Command:
      /bin/sh
    Args:
      -c
      while true; do echo hello from cnt01; sleep 10;done
    State: Running
      Started: Tue, 25 Feb 2020 14:38:57 -0700
    Ready: True
    Restart Count: 0
    Limits:
      cpu: 500m
      memory: 200Mi
    Requests:
      cpu: 100m
      memory: 100Mi
    Environment: <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-mnfxb (ro)
  busybox-cnt02:
    Container ID: docker://a9bacf09475944c447a021f299ac384087d5af46f139b70ce14ee595098bbcf8
    Image: busybox
    Image ID: docker-pullable://busybox@sha256:6915be4043561d64e0ab0f8f098dc2ac48e077fe23f488ac24b665166898115a
    Port: <none>
    Host Port: <none>
    Command:
      /bin/sh
    Args:
      -c
      while true; do echo hello from cnt02; sleep 10;done
    State: Running
      Started: Tue, 25 Feb 2020 14:38:59 -0700
    Ready: True
    Restart Count: 0
    Limits:
      cpu: 700m
      memory: 900Mi
    Requests:
      cpu: 100m
      memory: 100Mi
    Environment: <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-mnfxb (ro)
  busybox-cnt03:
    Container ID: docker://c8a2520d632e672a5b343f2dc680c178f2612515e70531cc1be0125e9a9c1fd3
    Image: busybox
    Image ID: docker-pullable://busybox@sha256:6915be4043561d64e0ab0f8f098dc2ac48e077fe23f488ac24b665166898115a
    Port: <none>
    Host Port: <none>
    Command:
      /bin/sh
    Args:
      -c
      while true; do echo hello from cnt03; sleep 10;done
    State: Running
      Started: Tue, 25 Feb 2020 14:39:00 -0700
    Ready: True
    Restart Count: 0
    Limits:
      cpu: 500m
      memory: 200Mi
    Requests:
      cpu: 500m
      memory: 200Mi
    Environment: <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-mnfxb (ro)
  busybox-cnt04:
    Container ID: docker://c70901422ba7739e1ef1820d917ebf6471d8efb998de904287e7afdb24e9c909
    Image: busybox
    Image ID: docker-pullable://busybox@sha256:6915be4043561d64e0ab0f8f098dc2ac48e077fe23f488ac24b665166898115a
    Port: <none>
    Host Port: <none>
    Command:
      /bin/sh
    Args:
      -c
      while true; do echo hello from cnt04; sleep 10;done
    State: Running
      Started: Tue, 25 Feb 2020 14:39:02 -0700
    Ready: True
    Restart Count: 0
    Limits:
      cpu: 700m
      memory: 900Mi
    Requests:
      cpu: 110m
      memory: 111Mi
    Environment: <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-mnfxb (ro)
Conditions:
  Type Status
  Initialized True
  Ready True
  ContainersReady True
  PodScheduled True
Volumes:
  default-token-mnfxb:
    Type: Secret (a volume populated by a Secret)
    SecretName: default-token-mnfxb
    Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type Reason Age From Message
  ---- ------ ---- ---- -------
  Normal Pulling 117s kubelet, ip-69-76-24-106.us-west-2.compute.internal pulling image "busybox"
  Normal Pulled 114s kubelet, ip-69-76-24-106.us-west-2.compute.internal Successfully pulled image "busybox"
  Normal Created 114s kubelet, ip-69-76-24-106.us-west-2.compute.internal Created container
  Normal Started 114s kubelet, ip-69-76-24-106.us-west-2.compute.internal Started container
  Normal Pulling 114s kubelet, ip-69-76-24-106.us-west-2.compute.internal pulling image "busybox"
  Normal Pulled 113s kubelet, ip-69-76-24-106.us-west-2.compute.internal Successfully pulled image "busybox"
  Normal Created 113s kubelet, ip-69-76-24-106.us-west-2.compute.internal Created container
  Normal Started 112s kubelet, ip-69-76-24-106.us-west-2.compute.internal Started container
  Normal Pulling 112s kubelet, ip-69-76-24-106.us-west-2.compute.internal pulling image "busybox"
  Normal Pulled 111s kubelet, ip-69-76-24-106.us-west-2.compute.internal Successfully pulled image "busybox"
  Normal Created 111s kubelet, ip-69-76-24-106.us-west-2.compute.internal Created container
  Normal Started 111s kubelet, ip-69-76-24-106.us-west-2.compute.internal Started container
  Normal Pulling 111s kubelet, ip-69-76-24-106.us-west-2.compute.internal pulling image "busybox"
  Normal Pulled 109s kubelet, ip-69-76-24-106.us-west-2.compute.internal Successfully pulled image "busybox"
  Normal Created 109s kubelet, ip-69-76-24-106.us-west-2.compute.internal Created container
  Normal Started 109s kubelet, ip-69-76-24-106.us-west-2.compute.internal Started container
  Normal Scheduled 40s default-scheduler Successfully assigned limitrange-demo/busybox1 to ip-69-76-24-106.us-west-2.compute.internal
Enter fullscreen mode Exit fullscreen mode

If you look at the above output, any containers which are missing limits/request is allocated a default value which is mentioned in LimitRange object. If limits are missing value Kubernetes will allocate value for limits and if requests are missing value it will assign value for requests. If both values are missing it will assign a default value for both. If all values are mentioned in manifest it will do nothing as long as manifest respecting hard max and min limits.

Container 1

kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[0].resources"
Output:
{
  "limits": {
    "cpu": "500m",
    "memory": "200Mi"
  },
  "requests": {
    "cpu": "100m",
    "memory": "100Mi"
  }
}
Enter fullscreen mode Exit fullscreen mode

Container 2

kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[1].resources"
Output:
{
  "limits": {
    "cpu": "700m",
    "memory": "900Mi"
  },
  "requests": {
    "cpu": "100m",
    "memory": "100Mi"
  }
}
Enter fullscreen mode Exit fullscreen mode

Container 3

kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[2].resources"
Output:
{
  "limits": {
    "cpu": "500m",
    "memory": "200Mi"
  },
  "requests": {
    "cpu": "500m",
    "memory": "200Mi"
  }
}
Enter fullscreen mode Exit fullscreen mode

Container 4

kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[3].resources"
Output:
{
  "limits": {
    "cpu": "700m",
    "memory": "900Mi"
  },
  "requests": {
    "cpu": "110m",
    "memory": "111Mi"
  }
}
Enter fullscreen mode Exit fullscreen mode

Top comments (0)