Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added Tika support #356

Merged
merged 4 commits into from
Jun 28, 2024
Merged

feat: added Tika support #356

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bd77319
feat: added Tika support
fastlorenzo Jun 28, 2024
0e234a5
fix: fixed tika support
fastlorenzo Jun 28, 2024
5628675
chore: deploy tika and oletools in test CI
fastlorenzo Jun 28, 2024
2689d8c
fix: fixed Tika image
fastlorenzo Jun 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions mailu/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -935,6 +935,63 @@ Check that the deployed pods are all running.
| `oletools.extraVolumes` | Optionally specify extra list of additional volumes for the pod(s) | `[]` |
| `oletools.extraContainers` | Add additional containers to the pod | `[]` |

### Tika parameters

| Name | Description | Value |
| -------------------------------------------- | ----------------------------------------------------------------------------------------------- | --------------- |
| `tika.enabled` | Enable OLETools | `true` |
| `tika.logLevel` | Override default log level | `""` |
| `tika.languages` | Array of languages to enable (sets the FULL_TEXT_SEARCH environment variable); "off" to disable | `["en"]` |
| `tika.image.repository` | Pod image repository | `apache/tika` |
| `tika.image.tag` | Pod image tag | `2.9.2.1-full` |
| `tika.image.pullPolicy` | Pod image pull policy | `IfNotPresent` |
| `tika.image.registry` | Pod image registry (specific for tika as it is not part of the mailu organization) | `docker.io` |
| `tika.resources.limits` | The resources limits for the container | `{}` |
| `tika.resources.requests` | The requested resources for the container | `{}` |
| `tika.livenessProbe.enabled` | Enable livenessProbe | `true` |
| `tika.livenessProbe.failureThreshold` | Failure threshold for livenessProbe | `3` |
| `tika.livenessProbe.initialDelaySeconds` | Initial delay seconds for livenessProbe | `10` |
| `tika.livenessProbe.periodSeconds` | Period seconds for livenessProbe | `10` |
| `tika.livenessProbe.successThreshold` | Success threshold for livenessProbe | `1` |
| `tika.livenessProbe.timeoutSeconds` | Timeout seconds for livenessProbe | `5` |
| `tika.readinessProbe.enabled` | Enable readinessProbe | `true` |
| `tika.readinessProbe.initialDelaySeconds` | Initial delay seconds for readinessProbe | `10` |
| `tika.readinessProbe.periodSeconds` | Period seconds for readinessProbe | `10` |
| `tika.readinessProbe.timeoutSeconds` | Timeout seconds for readinessProbe | `5` |
| `tika.readinessProbe.failureThreshold` | Failure threshold for readinessProbe | `3` |
| `tika.readinessProbe.successThreshold` | Success threshold for readinessProbe | `1` |
| `tika.startupProbe.enabled` | Enable startupProbe | `false` |
| `tika.startupProbe.initialDelaySeconds` | Initial delay seconds for startupProbe | `10` |
| `tika.startupProbe.periodSeconds` | Period seconds for startupProbe | `10` |
| `tika.startupProbe.timeoutSeconds` | Timeout seconds for startupProbe | `5` |
| `tika.startupProbe.failureThreshold` | Failure threshold for startupProbe | `3` |
| `tika.startupProbe.successThreshold` | Success threshold for startupProbe | `1` |
| `tika.podLabels` | Add extra labels to pod | `{}` |
| `tika.podAnnotations` | Add extra annotations to the pod | `{}` |
| `tika.nodeSelector` | Node labels selector for pod assignment | `{}` |
| `tika.initContainers` | Add additional init containers to the pod | `[]` |
| `tika.priorityClassName` | Pods' priorityClassName | `""` |
| `tika.podSecurityContext.enabled` | Enabled pods' Security Context | `false` |
| `tika.podSecurityContext.fsGroup` | Set pods' Security Context fsGroup | `1001` |
| `tika.containerSecurityContext.enabled` | Enabled containers' Security Context | `false` |
| `tika.containerSecurityContext.runAsUser` | Set containers' Security Context runAsUser | `1001` |
| `tika.containerSecurityContext.runAsNonRoot` | Set container's Security Context runAsNonRoot | `false` |
| `tika.terminationGracePeriodSeconds` | In seconds, time given to the pod to terminate gracefully | `2` |
| `tika.affinity` | Affinity for tika pod assignment | `{}` |
| `tika.tolerations` | Tolerations for pod assignment | `[]` |
| `tika.revisionHistoryLimit` | Configure the revisionHistoryLimit of the deployment | `3` |
| `tika.hostAliases` | Pod pod host aliases | `[]` |
| `tika.schedulerName` | Name of the k8s scheduler (other than default) | `""` |
| `tika.service.annotations` | tika service annotations | `{}` |
| `tika.topologySpreadConstraints` | Topology Spread Constraints for pod assignment | `[]` |
| `tika.updateStrategy.type` | Can be set to RollingUpdate or OnDelete | `RollingUpdate` |
| `tika.extraEnvVars` | Extra environment variable to pass to the running container | `[]` |
| `tika.extraEnvVarsCM` | Name of existing ConfigMap containing extra environment variables to mount in the pod | `""` |
| `tika.extraEnvVarsSecret` | Name of existing Secret containing extra environment variables to mount in the pod | `""` |
| `tika.extraVolumeMounts` | Optionally specify extra list of additional volumeMounts for the pod | `[]` |
| `tika.extraVolumes` | Optionally specify extra list of additional volumes for the pod(s) | `[]` |
| `tika.extraContainers` | Add additional containers to the pod | `[]` |


## Example values.yaml to get started

Expand Down
30 changes: 30 additions & 0 deletions mailu/templates/_helpers.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,7 @@ Please read the upgrade guide at XXX.
{{- $messages := list -}}
{{- $messages := append $messages (include "mailu.validateValues.deprecated" .) -}}
{{- $messages := append $messages (include "mailu.validateValues.domain" .) -}}
{{- $messages := append $messages (include "mailu.validateValues.tika" .) -}}
{{- $messages := without $messages "" -}}
{{- $message := join "\n" $messages -}}
{{- if $message -}}
Expand All @@ -178,3 +179,32 @@ mailu: domain
You need to set the domain to be used
{{- end -}}
{{- end -}}

{{/* Check if .Values.tika.enabled and .Values.tika.languages is a non-empty array.
If .Values.tika.enabled is false, then mailu.fullTextSearch should be "off".
If .Values.tika.enabled is true, and .Values.tika.languages is an empty array, throw an error.
If .Values.tika.enabled is true, and .Values.tika.languages is a non-empty array, then mailu.fullTextSearch should be all languages joined by a comma.
*/}}
{{- define "mailu.validateValues.tika" -}}
{{- if .Values.tika.enabled -}}
{{/* Check if .Values.tika.languages is an empty array */}}
{{- if not .Values.tika.languages -}}
mailu: tika
Tika is enabled but no languages are set (tika.enabled = true, tika.languages = [])
You need to set at least one language for Tika in tika.languages
{{- end -}}
{{- end -}}
{{- end -}}

{{/* Check if .Values.tika.enabled is false or a comma-separated list of languages in .Values.tika.languages */}}
{{- define "mailu.fullTextSearch" -}}
{{- if .Values.tika.enabled -}}
{{- if not .Values.tika.languages -}}
{{- print "off" -}}
{{- else -}}
{{- join "," .Values.tika.languages -}}
{{- end -}}
{{- else -}}
{{- print "off" -}}
{{- end -}}
{{- end -}}
10 changes: 10 additions & 0 deletions mailu/templates/_services.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Mailu services:
- rspamd
- webdav
- oletools
- tika

Service name can be retrieved with `mailu.SERVICE.serviceName`
Service fqdn (within cluster) can be retrieved with `mailu.SERVICE.serviceFqdn`
Expand Down Expand Up @@ -157,6 +158,15 @@ Service fqdn (within cluster) can be retrieved with `mailu.SERVICE.serviceFqdn`
{{- printf "%s.%s.svc.%s" (include "mailu.oletools.serviceName" . ) (include "common.names.namespace" . ) (include "mailu.clusterDomain" . ) -}}
{{- end -}}

{{/* Returns tika internal service name. */}}
{{- define "mailu.tika.serviceName" -}}
{{- printf "%s-tika" (include "mailu.fullname" .) -}}
{{- end -}}
{{/* Returns tika internal service fqdn. */}}
{{- define "mailu.tika.serviceFqdn" -}}
{{- printf "%s.%s.svc.%s" (include "mailu.tika.serviceName" . ) (include "common.names.namespace" . ) (include "mailu.clusterDomain" . ) -}}
{{- end -}}


{{/* Combine the enabled ports that should be exposed into a comma-separated string */}}
{{- define "mailu.enabledPorts" -}}
Expand Down
8 changes: 8 additions & 0 deletions mailu/templates/envvars-configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,11 @@ data:
{{- else }}
SCAN_MACROS: "false"
{{- end }}

{{- if .Values.tika.enabled }}
FULL_TEXT_SEARCH: {{ include "mailu.fullTextSearch" . }}
FULL_TEXT_SEARCH_ATTACHMENTS: "true"
FTS_ATTACHMENTS_ADDRESS: {{ include "mailu.tika.serviceFqdn" . }}
{{- else }}
FULL_TEXT_SEARCH_ATTACHMENTS: "false"
{{- end}}
128 changes: 128 additions & 0 deletions mailu/templates/tika/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
---
Dismissed Show dismissed Hide dismissed
{{- if .Values.tika.enabled }}
apiVersion: {{ include "common.capabilities.deployment.apiVersion" . }}
kind: Deployment
metadata:
name: {{ printf "%s-tika" (include "mailu.fullname" .) }}
namespace: {{ include "common.names.namespace" . | quote }}
labels: {{- include "common.labels.standard" . | nindent 4 }}
app.kubernetes.io/component: tika
{{- if .Values.commonLabels }}
{{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 4 }}
{{- end }}
{{- if .Values.commonAnnotations }}
annotations: {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
{{- end }}
spec:
replicas: 1
{{- if .Values.tika.updateStrategy }}
strategy: {{- toYaml .Values.tika.updateStrategy | nindent 4 }}
{{- end }}
revisionHistoryLimit: {{ .Values.tika.revisionHistoryLimit }}
selector:
matchLabels: {{- include "common.labels.matchLabels" . | nindent 6 }}
app.kubernetes.io/component: tika
template:
metadata:
labels: {{- include "common.labels.standard" . | nindent 8 }}
app.kubernetes.io/component: tika
{{- if .Values.tika.podLabels }}
{{- include "common.tplvalues.render" (dict "value" .Values.tika.podLabels "context" $) | nindent 8 }}
{{- end }}
{{- if .Values.tika.podAnnotations }}
annotations: {{- include "common.tplvalues.render" ( dict "value" .Values.tika.podAnnotations "context" $) | nindent 8 }}
{{- end }}
spec:
{{- include "common.images.pullSecrets" (dict "images" (list .Values.tika.image) "global" .Values.global) | nindent 6 }}
{{- if .Values.tika.hostAliases }}
hostAliases: {{- include "common.tplvalues.render" (dict "value" .Values.tika.hostAliases "context" $) | nindent 8 }}
{{- end }}
{{- if .Values.tika.nodeSelector }}
nodeSelector: {{- include "common.tplvalues.render" (dict "value" .Values.tika.nodeSelector "context" $) | nindent 8 }}
{{- end }}
{{- if .Values.tika.affinity }}
affinity: {{- include "common.tplvalues.render" (dict "value" .Values.tika.affinity "context" $) | nindent 8 }}
{{- end }}
{{- if .Values.tika.tolerations }}
tolerations: {{- include "common.tplvalues.render" (dict "value" .Values.tika.tolerations "context" $) | nindent 8 }}
{{- end }}
{{- if .Values.tika.priorityClassName }}
priorityClassName: {{ .Values.tika.priorityClassName | quote }}
{{- end }}
{{- if .Values.tika.schedulerName }}
schedulerName: {{ .Values.tika.schedulerName | quote }}
{{- end }}
{{- if .Values.tika.topologySpreadConstraints }}
topologySpreadConstraints: {{- include "common.tplvalues.render" (dict "value" .Values.tika.topologySpreadConstraints "context" $) | nindent 8 }}
{{- end }}
{{- if .Values.tika.terminationGracePeriodSeconds }}
terminationGracePeriodSeconds: {{ .Values.tika.terminationGracePeriodSeconds }}
{{- end }}
{{- if .Values.tika.initContainers }}
initContainers: {{- include "common.tplvalues.render" (dict "value" .Values.tika.initContainers "context" $) | nindent 8 }}
{{- end }}
{{- if .Values.tika.podSecurityContext.enabled }}
securityContext: {{- omit .Values.tika.podSecurityContext "enabled" | toYaml | nindent 8 }}
{{- end }}
containers:
- name: tika
image: {{ .Values.clamav.image.registry }}/{{ .Values.clamav.image.repository }}:{{ .Values.clamav.image.tag }}
imagePullPolicy: {{ .Values.tika.image.pullPolicy }}
{{- if .Values.tika.containerSecurityContext.enabled }}
securityContext: {{- omit .Values.tika.containerSecurityContext "enabled" | toYaml | nindent 12 }}
{{- end }}
{{- if .Values.tika.extraVolumeMounts }}
volumeMounts:
{{- include "common.tplvalues.render" (dict "value" .Values.tika.extraVolumeMounts "context" $) | nindent 12 }}
{{- end }}
env:
- name: LOG_LEVEL
value: {{ default .Values.logLevel .Values.tika.logLevel }}
{{- tpl (include "mailu.envvars.secrets" .) $ | nindent 12 }}
{{- if .Values.tika.extraEnvVars }}
{{- include "common.tplvalues.render" (dict "value" .Values.tika.extraEnvVars "context" $) | nindent 12 }}
{{- end }}
envFrom:
- configMapRef:
name: {{ printf "%s-envvars" (include "mailu.fullname" .) }}
{{- if .Values.tika.extraEnvVarsCM }}
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
Dismissed Show dismissed Hide dismissed
- configMapRef:
name: {{ include "common.tplvalues.render" (dict "value" .Values.tika.extraEnvVarsCM "context" $) }}
{{- end }}
{{- if .Values.tika.extraEnvVarsSecret }}
- secretRef:
name: {{ include "common.tplvalues.render" (dict "value" .Values.tika.extraEnvVarsSecret "context" $) }}
{{- end }}
ports:
- name: http
containerPort: 9998
protocol: TCP
{{- if .Values.tika.resources }}
resources: {{- toYaml .Values.tika.resources | nindent 12 }}
{{- end }}
{{- if .Values.tika.startupProbe.enabled }}
startupProbe: {{- include "common.tplvalues.render" (dict "value" (omit .Values.tika.startupProbe "enabled") "context" $) | nindent 12 }}
httpGet:
path: /tika
port: http
{{- end }}
{{- if .Values.tika.livenessProbe.enabled }}
livenessProbe: {{- include "common.tplvalues.render" (dict "value" (omit .Values.tika.livenessProbe "enabled") "context" $) | nindent 12 }}
httpGet:
path: /tika
port: http
{{- end }}
{{- if .Values.tika.readinessProbe.enabled }}
readinessProbe: {{- include "common.tplvalues.render" (dict "value" (omit .Values.tika.readinessProbe "enabled") "context" $) | nindent 12 }}
httpGet:
path: /tika
port: http
{{- end }}
{{- if .Values.tika.extraContainers }}
{{- toYaml .Values.tika.extraContainers | nindent 8 }}
{{- end }}
{{- if .Values.tika.extraVolumes }}
volumes:
{{- include "common.tplvalues.render" (dict "value" .Values.tika.extraVolumes "context" $) | nindent 8 }}
{{- end }}
{{- end }}
23 changes: 23 additions & 0 deletions mailu/templates/tika/service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
{{- if .Values.tika.enabled }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "mailu.tika.serviceName" . }}
namespace: {{ include "common.names.namespace" . | quote }}
labels: {{- include "common.labels.standard" . | nindent 4 }}
app.kubernetes.io/component: tika
{{- if .Values.commonLabels }}
{{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 4 }}
{{- end }}
{{- if .Values.tika.service.annotations }}
annotations: {{- include "common.tplvalues.render" ( dict "value" .Values.tika.service.annotations "context" $) | nindent 4 }}
{{- end }}
spec:
selector: {{- include "common.labels.matchLabels" . | nindent 4 }}
app.kubernetes.io/component: tika
ports:
- name: http
port: 9998
protocol: TCP
{{- end }}
Loading
Loading