Skip to content

Commit

Permalink
fix: add resource limit to all containers (#220)
Browse files Browse the repository at this point in the history
* fix: add resource limit to all containers

Signed-off-by: Lin Yang <[email protected]>

* fix: a typo

Signed-off-by: Lin Yang <[email protected]>

* fix: nindent

Signed-off-by: Lin Yang <[email protected]>

---------

Signed-off-by: Lin Yang <[email protected]>
  • Loading branch information
reaver-flomesh authored Apr 9, 2024
1 parent 9f3d430 commit 0b4dfa8
Show file tree
Hide file tree
Showing 18 changed files with 136 additions and 10 deletions.
1 change: 1 addition & 0 deletions charts/connector/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ The following table lists the configurable parameters of the fsm chart and their
| fsm.cloudConnector.connectorProvider | string | `""` | |
| fsm.cloudConnector.enable | bool | `false` | |
| fsm.cloudConnector.enablePodDisruptionBudget | bool | `false` | Enable Pod Disruption Budget |
| fsm.cloudConnector.initResources | object | `{"limits":{"cpu":"500m","memory":"512M"},"requests":{"cpu":"200m","memory":"128M"}}` | initContainer resource parameters |
| fsm.cloudConnector.nodeSelector | object | `{}` | |
| fsm.cloudConnector.podLabels | object | `{}` | Sidecar injector's pod labels |
| fsm.cloudConnector.replicaCount | int | `1` | Sidecar injector's replica count (ignored when autoscale.enable is true) |
Expand Down
2 changes: 2 additions & 0 deletions charts/connector/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ spec:
- name: init
image: "{{ include "fsmCurl.image" . }}"
command: ["curl", "http://fsm-bootstrap.{{ include "fsm.namespace" . }}.svc.cluster.local:9091/healthz", "--connect-timeout", "2", "--retry", "50", "--retry-connrefused", "--retry-delay", "5"]
resources:
{{- toYaml .Values.fsm.cloudConnector.initResources | nindent 12 }}
containers:
- name: connector
image: "{{ include "fsmConnector.image" . }}"
Expand Down
6 changes: 5 additions & 1 deletion charts/connector/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,8 @@
"connectorProvider",
"connectorName",
"replicaCount",
"resource"
"resource",
"initResources"
],
"properties": {
"enable": {
Expand Down Expand Up @@ -350,6 +351,9 @@
"resource": {
"$ref": "#/definitions/containerResources"
},
"initResources": {
"$ref": "#/definitions/containerResources"
},
"podLabels": {
"$id": "#/properties/fsm/properties/cloudConnector/properties/podLabels",
"type": "object",
Expand Down
11 changes: 10 additions & 1 deletion charts/connector/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -105,4 +105,13 @@ fsm:

# -- Node tolerations applied to control plane pods.
# The specified tolerations allow pods to schedule onto nodes with matching taints.
tolerations: [ ]
tolerations: [ ]

# -- initContainer resource parameters
initResources:
limits:
cpu: "500m"
memory: "512M"
requests:
cpu: "200m"
memory: "128M"
9 changes: 7 additions & 2 deletions charts/fsm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ The following table lists the configurable parameters of the fsm chart and their
| fsm.cleanup.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[1].values[0] | string | `"amd64"` | |
| fsm.cleanup.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[1].values[1] | string | `"arm64"` | |
| fsm.cleanup.nodeSelector | object | `{}` | |
| fsm.cleanup.resources | object | `{"limits":{"cpu":"500m","memory":"512M"},"requests":{"cpu":"200m","memory":"128M"}}` | FSM cleanup hook's container resource parameters. |
| fsm.cleanup.tolerations | list | `[]` | Node tolerations applied to control plane pods. The specified tolerations allow pods to schedule onto nodes with matching taints. |
| fsm.configResyncInterval | string | `"90s"` | Sets the resync interval for regular proxy broadcast updates, set to 0s to not enforce any resync |
| fsm.controlPlaneTolerations | list | `[]` | Node tolerations applied to control plane pods. The specified tolerations allow pods to schedule onto nodes with matching taints. |
Expand All @@ -95,8 +96,8 @@ The following table lists the configurable parameters of the fsm chart and their
| fsm.egressGateway.podAnnotations | object | `{}` | |
| fsm.egressGateway.podLabels | object | `{}` | |
| fsm.egressGateway.port | int | `1080` | |
| fsm.egressGateway.replicaCount | int | `1` | FSM Operator Manager's replica count (ignored when autoscale.enable is true) |
| fsm.egressGateway.resources | object | `{"limits":{"cpu":"1000m","memory":"512M"},"requests":{"cpu":"300m","memory":"128M"}}` | FSM Operator Manager's container resource parameters. |
| fsm.egressGateway.replicaCount | int | `1` | FSM Egress Gateway's replica count (ignored when autoscale.enable is true) |
| fsm.egressGateway.resources | object | `{"limits":{"cpu":"1000m","memory":"512M"},"requests":{"cpu":"300m","memory":"128M"}}` | FSM Egress Gateway's container resource parameters. |
| fsm.enableDebugServer | bool | `false` | Enable the debug HTTP server on FSM controller |
| fsm.enableEgress | bool | `true` | Enable egress in the mesh |
| fsm.enableFluentbit | bool | `false` | Enable Fluent Bit sidecar deployment on FSM controller's pod |
Expand Down Expand Up @@ -180,6 +181,7 @@ The following table lists the configurable parameters of the fsm chart and their
| fsm.fsmController.autoScale.memory.targetAverageUtilization | int | `80` | Average target memory utilization (%) |
| fsm.fsmController.autoScale.minReplicas | int | `1` | Minimum replicas for autoscale |
| fsm.fsmController.enablePodDisruptionBudget | bool | `false` | Enable Pod Disruption Budget |
| fsm.fsmController.initResources | object | `{"limits":{"cpu":"500m","memory":"512M"},"requests":{"cpu":"200m","memory":"128M"}}` | FSM controller's init-container resource parameters. See https://docs.flomesh.io/docs/guides/ha_scale/scale/ for more details. |
| fsm.fsmController.podLabels | object | `{}` | FSM controller's pod labels |
| fsm.fsmController.replicaCount | int | `1` | FSM controller's replica count (ignored when autoscale.enable is true) |
| fsm.fsmController.resource | object | `{"limits":{"cpu":"2","memory":"2G"},"requests":{"cpu":"0.5","memory":"256M"}}` | FSM controller's container resource parameters. See https://docs.flomesh.io/docs/guides/ha_scale/scale/ for more details. |
Expand Down Expand Up @@ -217,6 +219,7 @@ The following table lists the configurable parameters of the fsm chart and their
| fsm.fsmIngress.http.enabled | bool | `true` | |
| fsm.fsmIngress.http.nodePort | int | `30508` | |
| fsm.fsmIngress.http.port | int | `80` | |
| fsm.fsmIngress.initResources | object | `{"limits":{"cpu":"500m","memory":"512M"},"requests":{"cpu":"200m","memory":"128M"}}` | FSM ingress's init-container resource parameters. |
| fsm.fsmIngress.logLevel | string | `"info"` | |
| fsm.fsmIngress.namespaced | bool | `false` | |
| fsm.fsmIngress.nodeSelector | object | `{}` | Node selector applied to control plane pods. |
Expand Down Expand Up @@ -336,6 +339,7 @@ The following table lists the configurable parameters of the fsm chart and their
| fsm.injector.autoScale.memory.targetAverageUtilization | int | `80` | Average target memory utilization (%) |
| fsm.injector.autoScale.minReplicas | int | `1` | Minimum replicas for autoscale |
| fsm.injector.enablePodDisruptionBudget | bool | `false` | Enable Pod Disruption Budget |
| fsm.injector.initResources | object | `{"limits":{"cpu":"500m","memory":"512M"},"requests":{"cpu":"200m","memory":"128M"}}` | FSM sidecar injector's init-container resource parameters. |
| fsm.injector.nodeSelector | object | `{}` | |
| fsm.injector.podLabels | object | `{}` | Sidecar injector's pod labels |
| fsm.injector.replicaCount | int | `1` | Sidecar injector's replica count (ignored when autoscale.enable is true) |
Expand Down Expand Up @@ -408,6 +412,7 @@ The following table lists the configurable parameters of the fsm chart and their
| fsm.preinstall.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[1].values[0] | string | `"amd64"` | |
| fsm.preinstall.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[1].values[1] | string | `"arm64"` | |
| fsm.preinstall.nodeSelector | object | `{}` | |
| fsm.preinstall.resources | object | `{"limits":{"cpu":"500m","memory":"512M"},"requests":{"cpu":"200m","memory":"128M"}}` | FSM preinstall hook's container resource parameters. |
| fsm.preinstall.tolerations | list | `[]` | Node tolerations applied to control plane pods. The specified tolerations allow pods to schedule onto nodes with matching taints. |
| fsm.prometheus.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].key | string | `"kubernetes.io/os"` | |
| fsm.prometheus.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].operator | string | `"In"` | |
Expand Down
2 changes: 2 additions & 0 deletions charts/fsm/templates/cleanup-hook.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,8 @@ spec:
kubectl delete deploy fsm-egress-gateway -n '{{ include "fsm.namespace" . }}' --ignore-not-found;
kubectl delete svc fsm-egress-gateway -n '{{ include "fsm.namespace" . }}' --ignore-not-found;
kubectl delete cm fsm-egress-gateway-pjs -n '{{ include "fsm.namespace" . }}' --ignore-not-found;
resources:
{{- toYaml .Values.fsm.cleanup.resources | nindent 12 }}
{{- if .Values.fsm.imagePullSecrets }}
imagePullSecrets:
{{ toYaml .Values.fsm.imagePullSecrets | indent 8 }}
Expand Down
4 changes: 4 additions & 0 deletions charts/fsm/templates/fsm-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ spec:
- name: init-fsm-controller
image: "{{ include "fsmCurl.image" . }}"
command: ["curl", "http://fsm-bootstrap.{{ include "fsm.namespace" . }}.svc.cluster.local:9091/healthz", "--connect-timeout", "2", "--retry", "50", "--retry-connrefused", "--retry-delay", "5"]
resources:
{{- toYaml .Values.fsm.fsmController.initResources | nindent 12 }}
- name: init-untar
image: "{{ include "fsmCurl.image" . }}"
imagePullPolicy: IfNotPresent
Expand All @@ -56,6 +58,8 @@ spec:
subPath: scripts.tar.gz
- mountPath: /repo
name: shared-repo
resources:
{{- toYaml .Values.fsm.fsmController.initResources | nindent 12 }}
containers:
{{- if not .Values.fsm.repoServer.standalone }}
- name: fsm-pipy-repo
Expand Down
2 changes: 2 additions & 0 deletions charts/fsm/templates/fsm-ingress-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,8 @@ spec:
- --retry-connrefused
- --retry-delay
- "5"
resources:
{{- toYaml .Values.fsm.fsmIngress.initResources | nindent 10 }}
containers:
- name: ingress
image: {{ include "fsmIngress.image" . }}
Expand Down
2 changes: 2 additions & 0 deletions charts/fsm/templates/fsm-injector-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ spec:
- name: init-fsm-injector
image: "{{ include "fsmCurl.image" . }}"
command: ["curl", "http://fsm-bootstrap.{{ include "fsm.namespace" . }}.svc.cluster.local:9091/healthz", "--connect-timeout", "2", "--retry", "50", "--retry-connrefused", "--retry-delay", "5"]
resources:
{{- toYaml .Values.fsm.injector.initResources | nindent 12 }}
containers:
- name: fsm-injector
image: "{{ include "fsmInjector.image" . }}"
Expand Down
2 changes: 2 additions & 0 deletions charts/fsm/templates/preinstall-hook.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ spec:
- --verbosity={{ .Values.fsm.controllerLogLevel }}
- --enforce-single-mesh={{ .Values.fsm.enforceSingleMesh }}
- --namespace={{ include "fsm.namespace" . }}
resources:
{{- toYaml .Values.fsm.preinstall.resources | nindent 12 }}
{{- if .Values.fsm.imagePullSecrets }}
imagePullSecrets:
{{ toYaml .Values.fsm.imagePullSecrets | indent 8 }}
Expand Down
28 changes: 26 additions & 2 deletions charts/fsm/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,8 @@
"title": "The fsmController schema",
"description": "The details of the fsmController.",
"required": [
"resource"
"resource",
"initResources"
],
"properties": {
"replicaCount": {
Expand All @@ -223,6 +224,9 @@
"resource": {
"$ref": "#/definitions/containerResources"
},
"initResources": {
"$ref": "#/definitions/containerResources"
},
"podLabels": {
"$id": "#/properties/fsm/properties/fsmController/properties/podLabels",
"type": "object",
Expand Down Expand Up @@ -1527,7 +1531,8 @@
"description": "Sidecar injector configurations",
"required": [
"replicaCount",
"resource"
"resource",
"initResources"
],
"properties": {
"replicaCount": {
Expand All @@ -1542,6 +1547,9 @@
"resource": {
"$ref": "#/definitions/containerResources"
},
"initResources": {
"$ref": "#/definitions/containerResources"
},
"podLabels": {
"$id": "#/properties/fsm/properties/injector/properties/podLabels",
"type": "object",
Expand Down Expand Up @@ -1636,6 +1644,7 @@
"replicaCount",
"service",
"resources",
"initResources",
"podLabels",
"podAnnotations",
"podSecurityContext",
Expand Down Expand Up @@ -1782,6 +1791,11 @@
"default": {},
"title": "The resources Schema"
},
"initResources": {
"type": "object",
"default": {},
"title": "The resources Schema"
},
"podLabels": {
"type": "object",
"default": {},
Expand Down Expand Up @@ -2729,6 +2743,11 @@
},
"tolerations": {
"type": "array"
},
"resources": {
"type": "object",
"default": {},
"title": "The resources Schema"
}
},
"additionalProperties": false
Expand All @@ -2748,6 +2767,11 @@
},
"tolerations": {
"type": "array"
},
"resources": {
"type": "object",
"default": {},
"title": "The resources Schema"
}
},
"additionalProperties": false
Expand Down
49 changes: 47 additions & 2 deletions charts/fsm/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,15 @@ fsm:
# The specified tolerations allow pods to schedule onto nodes with matching taints.
tolerations: [ ]

# -- FSM controller's init-container resource parameters. See https://docs.flomesh.io/docs/guides/ha_scale/scale/ for more details.
initResources:
limits:
cpu: "500m"
memory: "512M"
requests:
cpu: "200m"
memory: "128M"

#
# -- FSM interceptor parameters
fsmInterceptor:
Expand Down Expand Up @@ -675,6 +684,15 @@ fsm:
# The specified tolerations allow pods to schedule onto nodes with matching taints.
tolerations: [ ]

# -- FSM sidecar injector's init-container resource parameters.
initResources:
limits:
cpu: "500m"
memory: "512M"
requests:
cpu: "200m"
memory: "128M"

# -- Run init container in privileged mode
enablePrivilegedInitContainer: false

Expand Down Expand Up @@ -873,6 +891,15 @@ fsm:
# -- Node selector applied to control plane pods.
nodeSelector: { }

# -- FSM ingress's init-container resource parameters.
initResources:
limits:
cpu: "500m"
memory: "512M"
requests:
cpu: "200m"
memory: "128M"

#
# -- FSM GatewayAPI parameters
fsmGateway:
Expand Down Expand Up @@ -920,9 +947,9 @@ fsm:
egressGateway:
enabled: false
name: fsm-egress-gateway
# -- FSM Operator Manager's replica count (ignored when autoscale.enable is true)
# -- FSM Egress Gateway's replica count (ignored when autoscale.enable is true)
replicaCount: 1
# -- FSM Operator Manager's container resource parameters.
# -- FSM Egress Gateway's container resource parameters.
resources:
limits:
cpu: 1000m
Expand Down Expand Up @@ -971,6 +998,15 @@ fsm:
# The specified tolerations allow pods to schedule onto nodes with matching taints.
tolerations: [ ]

# -- FSM preinstall hook's container resource parameters.
resources:
limits:
cpu: 500m
memory: 512M
requests:
cpu: 200m
memory: 128M

cleanup:
## Node labels for pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
Expand All @@ -997,6 +1033,15 @@ fsm:
# The specified tolerations allow pods to schedule onto nodes with matching taints.
tolerations: [ ]

# -- FSM cleanup hook's container resource parameters.
resources:
limits:
cpu: 500m
memory: 512M
requests:
cpu: 200m
memory: 128M

#
# -- SMI configuration
smi:
Expand Down
1 change: 1 addition & 0 deletions charts/gateway/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ The following table lists the configurable parameters of the fsm chart and their
| fsm.fsmGateway.enablePodDisruptionBudget | bool | `false` | Enable Pod Disruption Budget |
| fsm.fsmGateway.env[0].name | string | `"GIN_MODE"` | |
| fsm.fsmGateway.env[0].value | string | `"release"` | |
| fsm.fsmGateway.initResources | object | `{"limits":{"cpu":"500m","memory":"512M"},"requests":{"cpu":"200m","memory":"128M"}}` | initContainer resource parameters |
| fsm.fsmGateway.logLevel | string | `"info"` | |
| fsm.fsmGateway.nodeSelector | object | `{}` | Node selector applied to control plane pods. |
| fsm.fsmGateway.podAnnotations | object | `{}` | |
Expand Down
2 changes: 2 additions & 0 deletions charts/gateway/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ spec:
- --retry-connrefused
- --retry-delay
- "5"
resources:
{{- toYaml .Values.fsm.fsmGateway.initResources | nindent 10 }}
{{- with .Values.fsm.fsmGateway.securityContext }}
securityContext:
{{- toYaml . | nindent 10 }}
Expand Down
11 changes: 10 additions & 1 deletion charts/gateway/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -114,4 +114,13 @@ fsm:
tolerations: []

# -- Node selector applied to control plane pods.
nodeSelector: {}
nodeSelector: {}

# -- initContainer resource parameters
initResources:
limits:
cpu: "500m"
memory: "512M"
requests:
cpu: "200m"
memory: "128M"
1 change: 1 addition & 0 deletions charts/namespaced-ingress/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ The following table lists the configurable parameters of the fsm chart and their
| fsm.fsmIngress.http.enabled | bool | `true` | |
| fsm.fsmIngress.http.nodePort | int | `30508` | |
| fsm.fsmIngress.http.port | int | `80` | |
| fsm.fsmIngress.initResources | object | `{"limits":{"cpu":"500m","memory":"512M"},"requests":{"cpu":"200m","memory":"128M"}}` | initContainer resource parameters |
| fsm.fsmIngress.logLevel | string | `"info"` | |
| fsm.fsmIngress.nodeSelector | object | `{}` | Node selector applied to control plane pods. |
| fsm.fsmIngress.podAnnotations | object | `{}` | |
Expand Down
2 changes: 2 additions & 0 deletions charts/namespaced-ingress/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ spec:
- --retry-connrefused
- --retry-delay
- "5"
resources:
{{- toYaml .Values.fsm.fsmIngress.initResources | nindent 10 }}
{{- with .Values.nsig.spec.securityContext }}
securityContext:
{{- toYaml . | nindent 10 }}
Expand Down
Loading

0 comments on commit 0b4dfa8

Please sign in to comment.