Skip to content

Commit

Permalink
Adapt to docker image changes (#377)
Browse files Browse the repository at this point in the history 
* use jmx exporter soflink

* remove 2.3.9 support

* adapt to 3.1.3

* adapted changelog

* bump vector aggregator

* bump operator-rs to 0.55.0

* fix time duration

* adapted changelog
  • Loading branch information
@maltesander
maltesander authored Oct 18, 2023
1 parent 0ef7e92 commit 0633cb6
Show file tree
Hide file tree
Showing 11 changed files with 47 additions and 23 deletions.
10 changes: 8 additions & 2 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,19 @@ All notable changes to this project will be documented in this file.

### Changed

- `vector` `0.26.0` -> `0.31.0` ([#361]).
- `operator-rs` `0.44.0` -> `0.52.1` ([#360], [#376]).
- `vector` `0.26.0` -> `0.33.0` ([#361], [#377]).
- `operator-rs` `0.44.0` -> `0.55.0` ([#360], [#376], [#377]).
- jmx-exporter now referenced via soft link without version ([#377]).

### Removed

- Support for 2.3.9 ([#377]).

[#360]: https://github.com/stackabletech/hive-operator/pull/360
[#361]: https://github.com/stackabletech/hive-operator/pull/361
[#365]: https://github.com/stackabletech/hive-operator/pull/365
[#376]: https://github.com/stackabletech/hive-operator/pull/376
[#377]: https://github.com/stackabletech/hive-operator/pull/377

## [23.7.0] - 2023-07-14

Expand Down
8 changes: 4 additions & 4 deletions Cargo.lock
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
serde_yaml = "0.9"
snafu = "0.7"
stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "0.52.1" }
stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "0.55.0" }
strum = { version = "0.25", features = ["derive"] }
tokio = { version = "1.29", features = ["full"] }
tracing = "0.1"
Expand Down
36 changes: 28 additions & 8 deletions deploy/helm/hive-operator/crds/crds.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -728,20 +728,22 @@ spec:
default:
enableVectorAgent: null
containers: {}
description: Logging configuration
properties:
containers:
additionalProperties:
anyOf:
- required:
- custom
- {}
description: Fragment derived from `ContainerLogConfigChoice`
description: Log configuration of the container
properties:
console:
description: Configuration for the console appender
nullable: true
properties:
level:
description: Log levels
description: The log level threshold. Log events with a lower log level are discarded.
enum:
- TRACE
- DEBUG
Expand All @@ -757,14 +759,16 @@ spec:
description: Custom log configuration provided in a ConfigMap
properties:
configMap:
description: ConfigMap containing the log configuration files
nullable: true
type: string
type: object
file:
description: Configuration for the file appender
nullable: true
properties:
level:
description: Log levels
description: The log level threshold. Log events with a lower log level are discarded.
enum:
- TRACE
- DEBUG
Expand All @@ -778,9 +782,10 @@ spec:
type: object
loggers:
additionalProperties:
description: Configuration of a logger
properties:
level:
description: Log levels
description: The log level threshold. Log events with a lower log level are discarded.
enum:
- TRACE
- DEBUG
Expand All @@ -793,10 +798,13 @@ spec:
type: string
type: object
default: {}
description: Configuration per logger
type: object
type: object
description: Log configuration per container
type: object
enableVectorAgent:
description: Wether or not to deploy a container with the Vector log agent
nullable: true
type: boolean
type: object
Expand Down Expand Up @@ -840,6 +848,7 @@ spec:
data:
default:
capacity: null
description: This field is deprecated. It was never used by Hive and will be removed in a future CRD version. The controller will warn if it's set to a non zero value
properties:
capacity:
description: "Quantity is a fixed-point representation of a number. It provides convenient marshaling/unmarshaling in JSON and YAML, in addition to String() and AsInt64() accessors.\n\nThe serialization format is:\n\n``` <quantity> ::= <signedNumber><suffix>\n\n\t(Note that <suffix> may be empty, from the \"\" case in <decimalSI>.)\n\n<digit> ::= 0 | 1 | ... | 9 <digits> ::= <digit> | <digit><digits> <number> ::= <digits> | <digits>.<digits> | <digits>. | .<digits> <sign> ::= \"+\" | \"-\" <signedNumber> ::= <number> | <sign><number> <suffix> ::= <binarySI> | <decimalExponent> | <decimalSI> <binarySI> ::= Ki | Mi | Gi | Ti | Pi | Ei\n\n\t(International System of units; See: http://physics.nist.gov/cuu/Units/binary.html)\n\n<decimalSI> ::= m | \"\" | k | M | G | T | P | E\n\n\t(Note that 1024 = 1Ki but 1000 = 1k; I didn't choose the capitalization.)\n\n<decimalExponent> ::= \"e\" <signedNumber> | \"E\" <signedNumber> ```\n\nNo matter which of the three exponent forms is used, no quantity may represent a number greater than 2^63-1 in magnitude, nor may it have more than 3 decimal places. Numbers larger or more precise will be capped or rounded up. (E.g.: 0.1m will rounded up to 1m.) This may be extended in the future if we require larger or smaller quantities.\n\nWhen a Quantity is parsed from a string, it will remember the type of suffix it had, and will use the same type again when it is serialized.\n\nBefore serializing, Quantity will be put in \"canonical form\". This means that Exponent/suffix will be adjusted up or down (with a corresponding increase or decrease in Mantissa) such that:\n\n- No precision is lost - No fractional digits will be emitted - The exponent (or suffix) is as large as possible.\n\nThe sign will be omitted unless the number is negative.\n\nExamples:\n\n- 1.5 will be serialized as \"1500m\" - 1.5Gi will be serialized as \"1536Mi\"\n\nNote that the quantity will NEVER be internally represented by a floating point number. That is the whole point of this exercise.\n\nNon-canonical values will still parse as long as they are well formed, but will be re-emitted in their canonical form. (So always use canonical form, or don't diff.)\n\nThis format is intended to make it difficult to use these numbers without writing some sort of special handling code in the hopes that that will cause implementors to also use a fixed point implementation."
Expand Down Expand Up @@ -883,6 +892,7 @@ spec:
type: object
type: object
warehouseDir:
description: The location of default database for the Hive warehouse. Maps to the `hive.metastore.warehouse.dir` setting.
nullable: true
type: string
type: object
Expand Down Expand Up @@ -4207,20 +4217,22 @@ spec:
default:
enableVectorAgent: null
containers: {}
description: Logging configuration
properties:
containers:
additionalProperties:
anyOf:
- required:
- custom
- {}
description: Fragment derived from `ContainerLogConfigChoice`
description: Log configuration of the container
properties:
console:
description: Configuration for the console appender
nullable: true
properties:
level:
description: Log levels
description: The log level threshold. Log events with a lower log level are discarded.
enum:
- TRACE
- DEBUG
Expand All @@ -4236,14 +4248,16 @@ spec:
description: Custom log configuration provided in a ConfigMap
properties:
configMap:
description: ConfigMap containing the log configuration files
nullable: true
type: string
type: object
file:
description: Configuration for the file appender
nullable: true
properties:
level:
description: Log levels
description: The log level threshold. Log events with a lower log level are discarded.
enum:
- TRACE
- DEBUG
Expand All @@ -4257,9 +4271,10 @@ spec:
type: object
loggers:
additionalProperties:
description: Configuration of a logger
properties:
level:
description: Log levels
description: The log level threshold. Log events with a lower log level are discarded.
enum:
- TRACE
- DEBUG
Expand All @@ -4272,10 +4287,13 @@ spec:
type: string
type: object
default: {}
description: Configuration per logger
type: object
type: object
description: Log configuration per container
type: object
enableVectorAgent:
description: Wether or not to deploy a container with the Vector log agent
nullable: true
type: boolean
type: object
Expand Down Expand Up @@ -4319,6 +4337,7 @@ spec:
data:
default:
capacity: null
description: This field is deprecated. It was never used by Hive and will be removed in a future CRD version. The controller will warn if it's set to a non zero value
properties:
capacity:
description: "Quantity is a fixed-point representation of a number. It provides convenient marshaling/unmarshaling in JSON and YAML, in addition to String() and AsInt64() accessors.\n\nThe serialization format is:\n\n``` <quantity> ::= <signedNumber><suffix>\n\n\t(Note that <suffix> may be empty, from the \"\" case in <decimalSI>.)\n\n<digit> ::= 0 | 1 | ... | 9 <digits> ::= <digit> | <digit><digits> <number> ::= <digits> | <digits>.<digits> | <digits>. | .<digits> <sign> ::= \"+\" | \"-\" <signedNumber> ::= <number> | <sign><number> <suffix> ::= <binarySI> | <decimalExponent> | <decimalSI> <binarySI> ::= Ki | Mi | Gi | Ti | Pi | Ei\n\n\t(International System of units; See: http://physics.nist.gov/cuu/Units/binary.html)\n\n<decimalSI> ::= m | \"\" | k | M | G | T | P | E\n\n\t(Note that 1024 = 1Ki but 1000 = 1k; I didn't choose the capitalization.)\n\n<decimalExponent> ::= \"e\" <signedNumber> | \"E\" <signedNumber> ```\n\nNo matter which of the three exponent forms is used, no quantity may represent a number greater than 2^63-1 in magnitude, nor may it have more than 3 decimal places. Numbers larger or more precise will be capped or rounded up. (E.g.: 0.1m will rounded up to 1m.) This may be extended in the future if we require larger or smaller quantities.\n\nWhen a Quantity is parsed from a string, it will remember the type of suffix it had, and will use the same type again when it is serialized.\n\nBefore serializing, Quantity will be put in \"canonical form\". This means that Exponent/suffix will be adjusted up or down (with a corresponding increase or decrease in Mantissa) such that:\n\n- No precision is lost - No fractional digits will be emitted - The exponent (or suffix) is as large as possible.\n\nThe sign will be omitted unless the number is negative.\n\nExamples:\n\n- 1.5 will be serialized as \"1500m\" - 1.5Gi will be serialized as \"1536Mi\"\n\nNote that the quantity will NEVER be internally represented by a floating point number. That is the whole point of this exercise.\n\nNon-canonical values will still parse as long as they are well formed, but will be re-emitted in their canonical form. (So always use canonical form, or don't diff.)\n\nThis format is intended to make it difficult to use these numbers without writing some sort of special handling code in the hopes that that will cause implementors to also use a fixed point implementation."
Expand Down Expand Up @@ -4362,6 +4381,7 @@ spec:
type: object
type: object
warehouseDir:
description: The location of default database for the Hive warehouse. Maps to the `hive.metastore.warehouse.dir` setting.
nullable: true
type: string
type: object
Expand Down
1 change: 0 additions & 1 deletion docs/modules/hive/partials/supported-versions.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,3 @@
// Stackable Platform documentation.

- 3.1.3
- 2.3.9
2 changes: 1 addition & 1 deletion examples/simple-hive-cluster.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
name: simple-hive-derby
spec:
image:
productVersion: 2.3.9
productVersion: 3.1.3
stackableVersion: 0.0.0-dev
clusterConfig:
database:
Expand Down
4 changes: 2 additions & 2 deletions rust/crd/src/affinity.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ mod tests {
name: simple-hive
spec:
image:
productVersion: 2.3.9
productVersion: 3.1.3
clusterConfig:
database:
connString: jdbc:derby:;databaseName=/tmp/hive;create=true
Expand Down Expand Up @@ -112,7 +112,7 @@ mod tests {
name: simple-hive
spec:
image:
productVersion: 2.3.9
productVersion: 3.1.3
clusterConfig:
database:
connString: jdbc:derby:;databaseName=/tmp/hive;create=true
Expand Down
2 changes: 1 addition & 1 deletion rust/crd/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -411,7 +411,7 @@ impl Configuration for MetaStoreConfigFragment {
result.insert(
HIVE_METASTORE_HADOOP_OPTS.to_string(),
Some(formatdoc! {"
-javaagent:/stackable/jmx/jmx_prometheus_javaagent-0.16.1.jar={METRICS_PORT}:/stackable/jmx/jmx_hive_config.yaml
-javaagent:/stackable/jmx/jmx_prometheus_javaagent.jar={METRICS_PORT}:/stackable/jmx/jmx_hive_config.yaml
-Djavax.net.ssl.trustStore={STACKABLE_TRUST_STORE}
-Djavax.net.ssl.trustStorePassword={STACKABLE_TRUST_STORE_PASSWORD}
-Djavax.net.ssl.trustStoreType=pkcs12
Expand Down
2 changes: 1 addition & 1 deletion rust/operator-binary/src/controller.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ use stackable_operator::{
rbac::build_rbac_resources,
s3::{S3AccessStyle, S3ConnectionSpec},
},
duration::Duration,
k8s_openapi::{
api::{
apps::v1::{StatefulSet, StatefulSetSpec},
Expand Down Expand Up @@ -61,6 +60,7 @@ use stackable_operator::{
compute_conditions, operations::ClusterOperationsConditionBuilder,
statefulset::StatefulSetConditionBuilder,
},
time::Duration,
};
use std::{
borrow::Cow,
Expand Down
2 changes: 1 addition & 1 deletion tests/templates/kuttl/logging/01-install-hbase-vector-aggregator.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ commands:
- script: >-
helm install hive-vector-aggregator vector
--namespace $NAMESPACE
--version 0.23.0
--version 0.26.0
--repo https://helm.vector.dev
--values hive-vector-aggregator-values.yaml
---
Expand Down
1 change: 0 additions & 1 deletion tests/test-definition.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ dimensions:
- "12.5.6"
- name: hive
values:
- 2.3.9
- 3.1.3
- name: hive-latest
values:
Expand Down

0 comments on commit 0633cb6

Please sign in to comment.