-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added Helm charts for deployment. #35
base: developing
Are you sure you want to change the base?
Changes from 4 commits
56c6887
13e1c7a
54d845c
45fde8e
339eb07
3daf4ad
e1fc4e6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
# Patterns to ignore when building packages. | ||
# This supports shell glob matching, relative path matching, and | ||
# negation (prefixed with !). Only one pattern per line. | ||
.DS_Store | ||
# Common VCS dirs | ||
.git/ | ||
.gitignore | ||
.bzr/ | ||
.bzrignore | ||
.hg/ | ||
.hgignore | ||
.svn/ | ||
# Common backup files | ||
*.swp | ||
*.bak | ||
*.tmp | ||
*.orig | ||
*~ | ||
# Various IDEs | ||
.project | ||
.idea/ | ||
*.tmproj | ||
.vscode/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
apiVersion: v2 | ||
name: esdk | ||
description: Huawei Container Storage Interface (CSI) Driver | ||
|
||
# A chart can be either an 'application' or a 'library' chart. | ||
# | ||
# Application charts are a collection of templates that can be packaged into versioned archives | ||
# to be deployed. | ||
# | ||
# Library charts provide useful utilities or functions for the chart developer. They're included as | ||
# a dependency of application charts to inject those utilities and functions into the rendering | ||
# pipeline. Library charts do not define any templates and therefore cannot be deployed. | ||
type: application | ||
|
||
# This is the chart version. This version number should be incremented each time you make changes | ||
# to the chart and its templates, including the app version. | ||
# Versions are expected to follow Semantic Versioning (https://semver.org/) | ||
version: 0.1.0 | ||
|
||
# compatible Kubernetes versions, helm installation fails fail if the cluster runs an unsupported Kubernetes version | ||
kubeVersion: ">= 1.13.0 < 1.19.3" | ||
# This is the version number of the application being deployed. This version number should be | ||
# incremented each time you make changes to the application. Versions are not expected to | ||
# follow Semantic Versioning. They should reflect the version the application is using. | ||
# It is recommended to use it with quotes. | ||
appVersion: "2.2.13" | ||
|
||
home: https://github.com/Huawei/eSDK_K8S_Plugin | ||
sources: | ||
- https://github.com/Huawei/eSDK_K8S_Plugin |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
kind: ConfigMap | ||
apiVersion: v1 | ||
metadata: | ||
name: huawei-csi-configmap | ||
namespace: kube-system | ||
data: | ||
csi.json: | | ||
{{ $length := len .Values.backends }} {{ if gt $length 0 }} { {{ end }} | ||
"backends": {{ .Values.backends | toPrettyJson | nindent 8 }} | ||
{{ $length := len .Values.backends }} {{ if gt $length 0 }} } {{ end }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
--- | ||
kind: ServiceAccount | ||
apiVersion: v1 | ||
metadata: | ||
name: huawei-csi-controller | ||
namespace: kube-system | ||
--- | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. According to Helm best practices each resource definition should be defined in a separate template. |
||
kind: ClusterRole | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
metadata: | ||
name: huawei-csi-provisioner-runner | ||
rules: | ||
- apiGroups: [""] | ||
resources: ["persistentvolumes"] | ||
verbs: ["get", "list", "watch", "create", "delete", "patch"] | ||
- apiGroups: [""] | ||
resources: ["persistentvolumeclaims"] | ||
verbs: ["get", "list", "watch", "update", "patch"] | ||
- apiGroups: ["storage.k8s.io"] | ||
resources: ["storageclasses"] | ||
verbs: ["get", "list", "watch"] | ||
- apiGroups: [""] | ||
resources: ["events"] | ||
verbs: ["list", "watch", "create", "update", "patch"] | ||
- apiGroups: ["snapshot.storage.k8s.io"] | ||
resources: ["volumesnapshots"] | ||
verbs: ["list", "watch", "create", "update", "patch"] | ||
- apiGroups: ["snapshot.storage.k8s.io"] | ||
resources: ["volumesnapshotcontents"] | ||
verbs: ["get", "list", "watch", "create", "delete", "patch"] | ||
- apiGroups: ["snapshot.storage.k8s.io"] | ||
resources: ["volumesnapshots/status"] | ||
verbs: ["update"] | ||
- apiGroups: ["storage.k8s.io"] | ||
resources: ["csinodes"] | ||
verbs: ["get", "list", "watch"] | ||
- apiGroups: [""] | ||
resources: ["nodes"] | ||
verbs: ["get", "list", "watch"] | ||
|
||
--- | ||
kind: Deployment | ||
apiVersion: apps/v1 | ||
metadata: | ||
name: huawei-csi-controller | ||
namespace: kube-system | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hardcoding a namespace is not recommended in Helm charts. We should discuss how to best approach the issue with the |
||
spec: | ||
replicas: 1 | ||
selector: | ||
matchLabels: | ||
app: huawei-csi-controller | ||
template: | ||
metadata: | ||
labels: | ||
app: huawei-csi-controller | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Using a the default template function for labels created by |
||
spec: | ||
serviceAccount: huawei-csi-controller | ||
hostNetwork: true | ||
containers: | ||
- name: csi-provisioner | ||
image: {{ .Values.images.sidecar.provisioner }} | ||
args: | ||
- "--csi-address=$(ADDRESS)" | ||
- "--timeout=6h" | ||
env: | ||
- name: ADDRESS | ||
value: /var/lib/csi/sockets/pluginproxy/csi.sock | ||
imagePullPolicy: {{ .Values.imagePullPolicy }} | ||
volumeMounts: | ||
- name: socket-dir | ||
mountPath: /var/lib/csi/sockets/pluginproxy/ | ||
|
||
- name: csi-attacher | ||
image: {{ .Values.images.sidecar.attacher }} | ||
args: | ||
- "--csi-address=$(ADDRESS)" | ||
env: | ||
- name: ADDRESS | ||
value: /var/lib/csi/sockets/pluginproxy/csi.sock | ||
imagePullPolicy: {{ .Values.imagePullPolicy }} | ||
volumeMounts: | ||
- name: socket-dir | ||
mountPath: /var/lib/csi/sockets/pluginproxy/ | ||
- name: huawei-csi-driver | ||
image: {{ required "Must provide the CSI controller service container image." .Values.images.huaweiCsiControllerService }} | ||
args: | ||
- "--endpoint=$(CSI_ENDPOINT)" | ||
- "--controller" | ||
- "--containerized" | ||
- "--backend-update-interval=120" | ||
- "--driver-name=csi.huawei.com" | ||
env: | ||
- name: CSI_ENDPOINT | ||
value: /var/lib/csi/sockets/pluginproxy/csi.sock | ||
imagePullPolicy: {{ .Values.imagePullPolicy }} | ||
volumeMounts: | ||
- name: socket-dir | ||
mountPath: /var/lib/csi/sockets/pluginproxy/ | ||
- name: log | ||
mountPath: /var/log | ||
- name: config-map | ||
mountPath: /etc/huawei | ||
- name: secret | ||
mountPath: /etc/huawei/secret | ||
volumes: | ||
- name: socket-dir | ||
emptyDir: | ||
- name: log | ||
hostPath: | ||
path: /var/log/ | ||
type: Directory | ||
- name: config-map | ||
configMap: | ||
name: huawei-csi-configmap | ||
- name: secret | ||
secret: | ||
secretName: huawei-csi-secret | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IMO, Add line feed at end of file There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,137 @@ | ||
--- | ||
apiVersion: v1 | ||
kind: ServiceAccount | ||
metadata: | ||
name: huawei-csi-node | ||
namespace: kube-system | ||
|
||
--- | ||
kind: ClusterRole | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. According to Helm best practices each resource definition should be defined in a separate template. |
||
apiVersion: rbac.authorization.k8s.io/v1 | ||
metadata: | ||
name: huawei-csi-driver-registrar-runner | ||
namespace: kube-system | ||
rules: | ||
- apiGroups: [""] | ||
resources: ["events"] | ||
verbs: ["get", "list", "watch", "create", "update", "patch"] | ||
|
||
--- | ||
kind: ClusterRoleBinding | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
metadata: | ||
name: huawei-csi-driver-registrar-role | ||
subjects: | ||
- kind: ServiceAccount | ||
name: huawei-csi-node | ||
namespace: kube-system | ||
roleRef: | ||
kind: ClusterRole | ||
name: huawei-csi-driver-registrar-runner | ||
apiGroup: rbac.authorization.k8s.io | ||
--- | ||
kind: DaemonSet | ||
apiVersion: apps/v1 | ||
metadata: | ||
name: huawei-csi-node | ||
namespace: kube-system | ||
spec: | ||
selector: | ||
matchLabels: | ||
app: huawei-csi-node | ||
template: | ||
metadata: | ||
labels: | ||
app: huawei-csi-node | ||
spec: | ||
serviceAccountName: huawei-csi-node | ||
hostPID: true | ||
hostNetwork: true | ||
containers: | ||
- name: csi-node-driver-registrar | ||
image: {{ .Values.images.sidecar.registrar }} | ||
args: | ||
- "--csi-address=/csi/csi.sock" | ||
- "--kubelet-registration-path=/var/lib/kubelet/plugins/csi.huawei.com/csi.sock" | ||
imagePullPolicy: "IfNotPresent" | ||
volumeMounts: | ||
- name: plugin-dir | ||
mountPath: /csi | ||
- name: registration-dir | ||
mountPath: /registration | ||
- name: huawei-csi-driver | ||
image: {{ required "Must provide the CSI controller service node image." .Values.images.huaweiCsiNodeService }} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IMHO message is for node service. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed this. |
||
args: | ||
- "--endpoint=/csi/csi.sock" | ||
- "--containerized" | ||
- "--driver-name=csi.huawei.com" | ||
- "--volume-use-multipath=true" | ||
securityContext: | ||
privileged: true | ||
capabilities: | ||
add: ["SYS_ADMIN"] | ||
allowPrivilegeEscalation: true | ||
imagePullPolicy: "IfNotPresent" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please add image pull policy template here There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||
lifecycle: | ||
preStop: | ||
exec: | ||
command: ["/bin/sh", "-c", "rm -f /csi/csi.sock"] | ||
env: | ||
- name: CSI_ESDK_NODENAME | ||
valueFrom: | ||
fieldRef: | ||
apiVersion: v1 | ||
fieldPath: spec.nodeName | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IMO, Currently the "CSI_ESDK_NODENAME" is not used. Please remove it There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should I leave the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have removed the env variable There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please remove There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||
volumeMounts: | ||
- name: plugin-dir | ||
mountPath: /csi | ||
- name: pods-dir | ||
mountPath: /var/lib/kubelet | ||
mountPropagation: "Bidirectional" | ||
- name: etc-dir | ||
mountPath: /etc | ||
- name: log-dir | ||
mountPath: /var/log | ||
- name: dev-dir | ||
mountPath: /dev | ||
mountPropagation: "HostToContainer" | ||
- name: iscsi-dir | ||
mountPath: /var/lib/iscsi | ||
- name: config-map | ||
mountPath: /etc/huawei | ||
- name: secret | ||
mountPath: /etc/huawei/secret | ||
volumes: | ||
- name: plugin-dir | ||
hostPath: | ||
path: /var/lib/kubelet/plugins/csi.huawei.com | ||
type: DirectoryOrCreate | ||
- name: registration-dir | ||
hostPath: | ||
path: /var/lib/kubelet/plugins_registry | ||
type: Directory | ||
- name: pods-dir | ||
hostPath: | ||
path: /var/lib/kubelet | ||
type: Directory | ||
- name: etc-dir | ||
hostPath: | ||
path: /etc | ||
type: Directory | ||
- name: dev-dir | ||
hostPath: | ||
path: /dev | ||
type: Directory | ||
- name: iscsi-dir | ||
hostPath: | ||
path: /var/lib/iscsi | ||
- name: log-dir | ||
hostPath: | ||
path: /var/log/ | ||
type: Directory | ||
- name: config-map | ||
configMap: | ||
name: huawei-csi-configmap | ||
- name: secret | ||
secret: | ||
secretName: huawei-csi-secret | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. IMO, Add line feed at end of file There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
backends: | ||
- storage: "fusionstorage-san" | ||
name: "test-esdk-deployment" | ||
urls: | ||
- "https://1.2.3.4:28443" | ||
pools: | ||
- "StoragePool001" | ||
parameters: | ||
protocol: "iscsi" | ||
portals: | ||
- "1.2.3.4" | ||
images: | ||
sidecar: | ||
attacher: quay.io/k8scsi/csi-attacher:v1.2.1 | ||
provisioner: quay.io/k8scsi/csi-provisioner:v1.6.0 | ||
registrar: quay.io/k8scsi/csi-node-driver-registrar:v2.0.1 | ||
huaweiCsiControllerService: huawei-csi:test | ||
huaweiCsiNodeService: huawei-csi:test | ||
imagePullPolicy: "IfNotPresent" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please add comments on each attribute for easy of use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In our chart we decided to just dump the YAML structure from
values.yaml
directly intocsi.json
: https://github.com/adfinis-sygroup/helm-charts/blob/c23bb7d8380df256e454cc143510de60511ae2e2/charts/huawei-csi-plugin/templates/configmap.yaml#L9Might be easier then constructing a JSON and more flexible if you would decide to add more then just the
backends
section in the future.