Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Consolidate encryption and auth #366

Closed
wants to merge 21 commits into from
Closed

[Merged by Bors] - Consolidate encryption and auth #366

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
e01e1c1
WIP, it compiles and tests are green :)
sbernauer Jan 3, 2023
dd0ad11
Implement validation
sbernauer Jan 4, 2023
979d60e
fix smoke test
sbernauer Jan 4, 2023
900e435
fix deep storage tests
sbernauer Jan 4, 2023
5d8a5e6
docs: update mentions of tls and auth to new convention
Jan 4, 2023
7d3b7e2
docs: update example to new form
Jan 4, 2023
4057fc7
test: replace tls: null with new form
Jan 4, 2023
2dce5b7
test: update tls secret class format convention
Jan 4, 2023
69edb53
test: adjust rust test yamls to new format
Jan 4, 2023
9602bf6
Only load SIMPLE_CLIENT_SSL_CONTEXT extension when tls is enabled
sbernauer Jan 5, 2023
d4477a5
Update tests
sbernauer Jan 5, 2023
63a93ab
Adress review
sbernauer Jan 9, 2023
97d5583
lowercase error messages
sbernauer Jan 9, 2023
ce37c4c
clippy
sbernauer Jan 9, 2023
6fa17ad
Move common files into dedicated folder
sbernauer Jan 9, 2023
6e2b137
Add FIXME
sbernauer Jan 9, 2023
93154a1
Improve tls tests
sbernauer Jan 10, 2023
5bd2ac8
Allow to disable tls
sbernauer Jan 10, 2023
1bb2b68
Move TLS_DEFAULT_SECRET_CLASS to tls.rs
sbernauer Jan 10, 2023
49869da
Use vec macro rather than pushing
sbernauer Jan 10, 2023
1e63553
Rename internal-tls -> internal-and-server-tls
sbernauer Jan 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

50 changes: 30 additions & 20 deletions deploy/crd/druidcluster.crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ spec:
spec:
properties:
brokers:
description: Configuration of the broker role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -355,22 +356,27 @@ spec:
- roleGroups
type: object
clusterConfig:
description: Common cluster wide configuration that can not differ or be overriden on a role or role group level
description: Common cluster wide configuration that can not differ or be overridden on a role or role group level
properties:
authentication:
description: Authentication class settings for Druid like TLS authentication or LDAP
nullable: true
properties:
tls:
description: TLS based client authentication (mutual TLS)
nullable: true
properties:
authenticationClass:
type: string
required:
- authenticationClass
type: object
type: object
default: []
description: List of Authentication classes using like TLS or LDAP to authenticate users
items:
properties:
authenticationClass:
description: |-
The AuthenticationClass <https://docs.stackable.tech/home/nightly/concepts/authenticationclass.html> to use.

## TLS provider

Only affects client connections. This setting controls: - If clients need to authenticate themselves against Druid via TLS - Which ca.crt to use when validating the provided client certs

Please note that the SecretClass used to authenticate users needs to be the same as the SecretClass used for internal communication.
type: string
required:
- authenticationClass
type: object
type: array
authorization:
description: Authorization settings for Druid like OPA
nullable: true
Expand Down Expand Up @@ -667,15 +673,15 @@ spec:
type: object
tls:
default:
secretClass: tls
description: TLS encryption settings for Druid
serverAndInternalSecretClass: tls
description: TLS encryption settings for Druid. This setting only affects server and internal communication. It does not affect client tls authentication, use `clusterConfig.authentication` instead.
nullable: true
properties:
secretClass:
description: 'Only affects client connections. This setting controls: - If TLS encryption is used at all - Which cert the servers should use to authenticate themselves against the client Important: This will activate encrypted internal druid communication as well!'
serverAndInternalSecretClass:
default: tls
description: 'This setting controls client as well as internal tls usage: - If TLS encryption is used at all - Which cert the servers should use to authenticate themselves against the clients - Which cert the servers should use to authenticate themselves among each other'
nullable: true
type: string
required:
- secretClass
type: object
zookeeperConfigMapName:
description: ZooKeeper discovery ConfigMap
Expand All @@ -686,6 +692,7 @@ spec:
- zookeeperConfigMapName
type: object
coordinators:
description: Configuration of the coordinator role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -1018,6 +1025,7 @@ spec:
- roleGroups
type: object
historicals:
description: Configuration of the historical role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -1507,6 +1515,7 @@ spec:
type: string
type: object
middleManagers:
description: Configuration of the middle managed role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -1839,6 +1848,7 @@ spec:
- roleGroups
type: object
routers:
description: Configuration of the router role
properties:
cliOverrides:
additionalProperties:
Expand Down
50 changes: 30 additions & 20 deletions deploy/helm/druid-operator/crds/crds.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ spec:
spec:
properties:
brokers:
description: Configuration of the broker role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -357,22 +358,27 @@ spec:
- roleGroups
type: object
clusterConfig:
description: Common cluster wide configuration that can not differ or be overriden on a role or role group level
description: Common cluster wide configuration that can not differ or be overridden on a role or role group level
properties:
authentication:
description: Authentication class settings for Druid like TLS authentication or LDAP
nullable: true
properties:
tls:
description: TLS based client authentication (mutual TLS)
nullable: true
properties:
authenticationClass:
type: string
required:
- authenticationClass
type: object
type: object
default: []
description: List of Authentication classes using like TLS or LDAP to authenticate users
items:
properties:
authenticationClass:
description: |-
The AuthenticationClass <https://docs.stackable.tech/home/nightly/concepts/authenticationclass.html> to use.

## TLS provider

Only affects client connections. This setting controls: - If clients need to authenticate themselves against Druid via TLS - Which ca.crt to use when validating the provided client certs

Please note that the SecretClass used to authenticate users needs to be the same as the SecretClass used for internal communication.
type: string
required:
- authenticationClass
type: object
type: array
authorization:
description: Authorization settings for Druid like OPA
nullable: true
Expand Down Expand Up @@ -669,15 +675,15 @@ spec:
type: object
tls:
default:
secretClass: tls
description: TLS encryption settings for Druid
serverAndInternalSecretClass: tls
description: TLS encryption settings for Druid. This setting only affects server and internal communication. It does not affect client tls authentication, use `clusterConfig.authentication` instead.
nullable: true
properties:
secretClass:
description: 'Only affects client connections. This setting controls: - If TLS encryption is used at all - Which cert the servers should use to authenticate themselves against the client Important: This will activate encrypted internal druid communication as well!'
serverAndInternalSecretClass:
default: tls
description: 'This setting controls client as well as internal tls usage: - If TLS encryption is used at all - Which cert the servers should use to authenticate themselves against the clients - Which cert the servers should use to authenticate themselves among each other'
nullable: true
type: string
required:
- secretClass
type: object
zookeeperConfigMapName:
description: ZooKeeper discovery ConfigMap
Expand All @@ -688,6 +694,7 @@ spec:
- zookeeperConfigMapName
type: object
coordinators:
description: Configuration of the coordinator role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -1020,6 +1027,7 @@ spec:
- roleGroups
type: object
historicals:
description: Configuration of the historical role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -1509,6 +1517,7 @@ spec:
type: string
type: object
middleManagers:
description: Configuration of the middle managed role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -1841,6 +1850,7 @@ spec:
- roleGroups
type: object
routers:
description: Configuration of the router role
properties:
cliOverrides:
additionalProperties:
Expand Down
50 changes: 30 additions & 20 deletions deploy/manifests/crds.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ spec:
spec:
properties:
brokers:
description: Configuration of the broker role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -358,22 +359,27 @@ spec:
- roleGroups
type: object
clusterConfig:
description: Common cluster wide configuration that can not differ or be overriden on a role or role group level
description: Common cluster wide configuration that can not differ or be overridden on a role or role group level
properties:
authentication:
description: Authentication class settings for Druid like TLS authentication or LDAP
nullable: true
properties:
tls:
description: TLS based client authentication (mutual TLS)
nullable: true
properties:
authenticationClass:
type: string
required:
- authenticationClass
type: object
type: object
default: []
description: List of Authentication classes using like TLS or LDAP to authenticate users
items:
properties:
authenticationClass:
description: |-
The AuthenticationClass <https://docs.stackable.tech/home/nightly/concepts/authenticationclass.html> to use.

## TLS provider

Only affects client connections. This setting controls: - If clients need to authenticate themselves against Druid via TLS - Which ca.crt to use when validating the provided client certs

Please note that the SecretClass used to authenticate users needs to be the same as the SecretClass used for internal communication.
type: string
required:
- authenticationClass
type: object
type: array
authorization:
description: Authorization settings for Druid like OPA
nullable: true
Expand Down Expand Up @@ -670,15 +676,15 @@ spec:
type: object
tls:
default:
secretClass: tls
description: TLS encryption settings for Druid
serverAndInternalSecretClass: tls
description: TLS encryption settings for Druid. This setting only affects server and internal communication. It does not affect client tls authentication, use `clusterConfig.authentication` instead.
nullable: true
properties:
secretClass:
description: 'Only affects client connections. This setting controls: - If TLS encryption is used at all - Which cert the servers should use to authenticate themselves against the client Important: This will activate encrypted internal druid communication as well!'
serverAndInternalSecretClass:
default: tls
description: 'This setting controls client as well as internal tls usage: - If TLS encryption is used at all - Which cert the servers should use to authenticate themselves against the clients - Which cert the servers should use to authenticate themselves among each other'
nullable: true
type: string
required:
- secretClass
type: object
zookeeperConfigMapName:
description: ZooKeeper discovery ConfigMap
Expand All @@ -689,6 +695,7 @@ spec:
- zookeeperConfigMapName
type: object
coordinators:
description: Configuration of the coordinator role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -1021,6 +1028,7 @@ spec:
- roleGroups
type: object
historicals:
description: Configuration of the historical role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -1510,6 +1518,7 @@ spec:
type: string
type: object
middleManagers:
description: Configuration of the middle managed role
properties:
cliOverrides:
additionalProperties:
Expand Down Expand Up @@ -1842,6 +1851,7 @@ spec:
- roleGroups
type: object
routers:
description: Configuration of the router role
properties:
cliOverrides:
additionalProperties:
Expand Down
13 changes: 5 additions & 8 deletions docs/modules/ROOT/pages/usage.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,11 +265,11 @@ TLS encryption is supported for internal cluster communication (e.g. between Bro
spec:
clusterConfig:
tls:
secretClass: tls # <1>
serverAndInternalSecretClass: tls # <1>
----
<1> Name of the `SecretClass` that is used to encrypt internal and external communication.

IMPORTANT: A Stackable Druid cluster is always encrypted per default. In order to disable this default behavior you can set `spec.clusterConfig.tls: null`.
IMPORTANT: A Stackable Druid cluster is always encrypted per default. In order to disable this default behavior you can set `spec.clusterConfig.tls.serverAndInternalSecretClass: null`.

=== Authentication

Expand All @@ -282,13 +282,10 @@ The access to the Druid cluster can be limited by configuring client authenticat
spec:
clusterConfig:
authentication:
tls:
authenticationClass: druid-tls-auth # <1>
- authenticationClass: druid-tls-auth # <1>
----
<1> Name of the `AuthenticationClass` that is used to encrypt and authenticate communication.

IMPORTANT: The TLS `AuthenticationClass` and its respective `SecretClass` will always take precedence over the TLS encryption `SecretClass` (if provided).

The `AuthenticationClass` may or may not have a `SecretClass` configured:
[source,yaml]
----
Expand All @@ -303,10 +300,10 @@ spec:
tls:
clientCertSecretClass: druid-mtls # <1>
# Option 2
tls: null # <2>
tls: {} # <2>
----
<1> If a client `SecretClass` is provided in the `AuthenticationClass` (here `druid-mtls`), these certificates will be used for encryption and authentication.
<2> If no client `SecretClass` is provided in the `AuthenticationClass`, the `spec.clusterConfig.tls.secretClass` will be used for encryption and authentication. It cannot be explicitly set to null in this case.
<2> If no client `SecretClass` is provided in the `AuthenticationClass`, the `spec.clusterConfig.tls.serverAndInternalSecretClass` will be used for encryption and authentication. It cannot be explicitly set to null in this case.

==== LDAP

Expand Down
5 changes: 2 additions & 3 deletions examples/tls/tls-druid-cluster.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,7 @@ spec:
stackableVersion: 0.3.0
clusterConfig:
authentication:
tls:
authenticationClass: druid-mtls-authentication-class
- authenticationClass: druid-mtls-authentication-class
deepStorage:
hdfs:
configMapName: druid-hdfs
Expand All @@ -95,7 +94,7 @@ spec:
host: localhost
port: 1527
tls:
secretClass: tls
serverAndInternalSecretClass: tls
zookeeperConfigMapName: druid-znode
brokers:
roleGroups:
Expand Down
3 changes: 2 additions & 1 deletion rust/crd/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,6 @@ snafu = "0.7"
lazy_static = "1.4"

[dev-dependencies]
serde_yaml = "0.8"
indoc = "1.0"
rstest = "0.16"
serde_yaml = "0.8"
Loading