SDP Release 23.7

[adwk67]

Stackable Data Platform (SDP) Release 23.7 is now publicly available!

Highlights

This release introduces the specification of resource quotas and pod overrides and updates the product versions supported by SDP.

Resource Quotas

Explicit resources are now applied to all containers, both operators and products. This allows users to run the Stackable Data Platform on Kubernetes clusters that have set a ResourceQuota or LimitRange. Where these are not specified directly, defaults will be used.

Pod Overrides

It is now possible to add custom settings which specify elements of a pod template (service, stateteful set etc.) on roles or rolegroups, which the operator then merges with the objects it writes before actually applying them. This provides the user with a possibility for specifying any property that can be set on a regular Kubernetes Pod, but which is not directly exposed via the Stackable custom resource definition.

OpenShift certification

OLM bundles - a pre-requisite for the OpenShift certification process - have been created for each operator. All 15 SDP operators in release 23.4.1 are now OpenShift-certified and deployable directly from within an OpenShift cluster.

OpenShift certification means that our components are built using certified images with the latest security updates, and which contain clear license and identification information. They are also regularly monitored for security issues. It is also means that our containers can be deployed consistently and securely across hybrid cloud and on-premise installations.

New Versions

The following new product versions are now supported:

• Apache Airflow: 2.6.1
• Apache Druid: 26.0.0
• Apache Kafka: 3.4.0
• Apache NiFi: 1.20.0, 1.21.0
• Opa: 0.51
• Apache Spark: 3.4.0
• Apache Superset: 1.4.2, 1.5.3, 2.0.1, 2.1.0
• Trino: 414
• Apache ZooKeeper: 3.8.1

New product-specific features

We have also added new product-specific features such as:

• HDFS: support for enabling secure mode with Kerberos
• Spark-k8s: support for using custom certificates when accessing S3 with TLS
• Trino: support arbitrary connectors using the generic connector for e.g. access to PostgreSQL

Learning Stackable

Further details on our release and how to upgrade can be found in our release notes as well as in the change logs of the individual operators.