From 8601a472597adee2dd9bad29247eea399b676213 Mon Sep 17 00:00:00 2001 From: Motouom Victoire Date: Fri, 8 Nov 2024 11:29:57 +0100 Subject: [PATCH 1/5] improve resource management documentation --- docs/MANAGED.md | 80 ++++++++++++++++++++++++++++--------------------- 1 file changed, 46 insertions(+), 34 deletions(-) diff --git a/docs/MANAGED.md b/docs/MANAGED.md index 146b060c8..ea195e2cd 100644 --- a/docs/MANAGED.md +++ b/docs/MANAGED.md @@ -1,56 +1,68 @@ -# Full managed resources +# Resource Management in keycloak-config-cli +## Introduction -keycloak-config-cli manage some types of resources absolutely. For example if a `group` isn't defined -inside the import json but other `groups` specified, keycloak-config-cli will calculate the -difference and delete the `group` from keycloak. +This document explains how keycloak-config-cli (kcc) manages resources in Keycloak, including its default behavior, customization options, and impact on various resource types. -In some cases it is required to include some keycloak defaults because keycloak-config-cli can't -detect if the entity comes from a user or auto created by keycloak itself. +## How keycloak-config-cli Tracks Resources -There are 2 modes to ensure a specific behavior: +- keycloak-config-cli stores information about resources it creates as realm attributes in the Keycloak database. +- This tracking mechanism allows kcc to manage these resources in subsequent runs. -### 1. Keycloak should not manage type of resources: +## Default Behavior -For example if you don't define any `groups` inside the import json, keycloak does not touch any `groups`. +- By default, kcc will delete and recreate resources that it initially created in previous runs. +- This ensures that the Keycloak configuration always matches the state defined in your configuration files. -### 2. Keycloak manage type of resources: +## Customizing Resource Management -For example define any `groups` you want inside the import json, keycloak ensure that the groups are available but other -groups will be deleted. If you define `groups` but set an empty array, keycloak will delete all groups in keycloak. +- The `import.managed.*` family of properties allows you to customize this behavior. +- Setting these properties to `no-delete` will prevent kcc from deleting resources, even if they're no longer present in your configuration files. -## Supported full managed resources +## Impact on User Federations + +- This behavior applies to user federations (such as LDAP and Active Directory). +- When a user federation is deleted and recreated, all users created by that federation will also be deleted. +- This includes associated data like offline tokens. + +## Full Managed Resources + +keycloak-config-cli manages some types of resources absolutely. For example, if a `group` isn't defined inside the import JSON but other `groups` are specified, keycloak-config-cli will calculate the difference and delete the `group` from Keycloak. + +In some cases, it is required to include some Keycloak defaults because keycloak-config-cli can't detect if the entity comes from a user or is auto-created by Keycloak itself. + +### Management Modes + +1. **Keycloak Should Not Manage Type of Resources**: + - If you don't define any `groups` inside the import JSON, Keycloak does not touch any `groups`. + +2. **Keycloak Manages Type of Resources**: + - If you define any `groups` you want inside the import JSON, Keycloak ensures that those groups are available but deletes other groups. + - If you define `groups` but set an empty array, Keycloak will delete all groups in Keycloak. + +### Supported Full Managed Resources | Type | Additional Information | Resource Name | |---------------------------------|----------------------------------------------------------------------------------|----------------------------------| | Groups | - | `group` | -| Required Actions | You have to copy the default one to you import json. | `required-action` | +| Required Actions | You have to copy the default one to your import JSON. | `required-action` | | Client Scopes | - | `client-scope` | | Scope Mappings | - | `scope-mapping` | | Client Scope Mappings | - | `client-scope-mapping` | | Roles | - | `role` | -| Components | You have to copy the default components to you import json. | `component` | -| Sub Components | You have to copy the default components to you import json. | `sub-component` | -| Authentication Flows | You have to copy the default components to you import json, expect builtin flows | `authentication-flow` | +| Components | You have to copy the default components to your import JSON. | `component` | +| Sub Components | You have to copy the default components to your import JSON. | `sub-component` | +| Authentication Flows | You have to copy the default components to your import JSON, except built-in flows.| `authentication-flow` | | Identity Providers | - | `identity-provider` | | Identity Provider Mappers | - | `identity-provider-mapper` | | Clients | - | `client` | -| Clients Authorization Resources | The 'Default Resource' is always included. | `client-authorization-resources` | -| Clients Authorization Policies | - | `client-authorization-policies` | -| Clients Authorization Scopes | - | `client-authorization-scopes` | -| Message Bundles | Only message bundles imported with config-cli will be managed/deleted. | `message-bundles` | - -## Disable deletion of managed entities - -If you don't delete properties of a specific type, you can disable this behavior by default a properties like `import.managed.=`, e.g.: -`import.managed.required-actions=no-delete` - -## State management - -If `import.remote-state.enabled` is set to `true` (default value), keycloak-config-cli will purge only resources they created before by keycloak-config-cli. If `import.remote-state.enabled` is set to `false`, keycloak-config-cli will purge all existing entities if they are not defined in import json. +| Clients Authorization Resources | The 'Default Resource' is always included. | `client-authorization-resources` | +| Clients Authorization Policies | - | `client-authorization-policies` | +| Clients Authorization Scopes | - | `client-authorization-scopes` | +| Message Bundles | Only message bundles imported with config-cli will be managed/deleted. | `message-bundles` | -### Supported resources +## Disabling Deletion of Managed Entities -Following entities does have saved state: +If you don't want to delete properties of a specific type, you can disable this behavior by setting properties like `import.managed.=`, e.g.: -- Required Actions -- Components +```properties +import.managed.required-actions=no-delete From fc1454c228811c85a98ad7b9ffa432f3bdb91a4d Mon Sep 17 00:00:00 2001 From: Motouom Victoire Date: Fri, 8 Nov 2024 11:30:22 +0100 Subject: [PATCH 2/5] update changelog --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 03b86cd91..aaf97bbb9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), ## [Unreleased] ### Added +- Improve documentation of managed resources, particularly user federations [#826](https://github.com/adorsys/keycloak-config-cli/issues/826) +### Added - improved logging for realm retrieval errors [#1010](https://github.com/adorsys/keycloak-config-cli/issues/1010) ### Fixed - Fix required action import handling for no-delete option [#834](https://github.com/adorsys/keycloak-config-cli/issues/834) From 8eb6e10dd7d6b0683d443878d851896e5893bd90 Mon Sep 17 00:00:00 2001 From: Motouom Victoire Date: Wed, 13 Nov 2024 08:55:13 +0100 Subject: [PATCH 3/5] Add state management in the documentaion. --- docs/MANAGED.md | 4 ++ .../config/service/RealmImportService.java | 41 ++++++++++++++++++- 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/docs/MANAGED.md b/docs/MANAGED.md index ea195e2cd..fddee2e26 100644 --- a/docs/MANAGED.md +++ b/docs/MANAGED.md @@ -66,3 +66,7 @@ If you don't want to delete properties of a specific type, you can disable this ```properties import.managed.required-actions=no-delete +``` +## State management + +If `import.remote-state.enabled` is set to `true` (default value), keycloak-config-cli will purge only resources they created before by keycloak-config-cli. If `import.remote-state.enabled` is set to `false`, keycloak-config-cli will purge all existing entities if they are not defined in import json. diff --git a/src/main/java/de/adorsys/keycloak/config/service/RealmImportService.java b/src/main/java/de/adorsys/keycloak/config/service/RealmImportService.java index 14b257063..1e33d2baf 100644 --- a/src/main/java/de/adorsys/keycloak/config/service/RealmImportService.java +++ b/src/main/java/de/adorsys/keycloak/config/service/RealmImportService.java @@ -20,6 +20,9 @@ package de.adorsys.keycloak.config.service; +import com.fasterxml.jackson.databind.JsonNode; +import com.networknt.schema.JsonSchema; +import com.networknt.schema.ValidationMessage; import de.adorsys.keycloak.config.model.RealmImport; import de.adorsys.keycloak.config.properties.ImportConfigProperties; import de.adorsys.keycloak.config.provider.KeycloakProvider; @@ -33,6 +36,10 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; +import java.io.IOException; +import java.util.List; +import java.util.Set; + @Service public class RealmImportService { static final String[] ignoredPropertiesForRealmImport = new String[]{ @@ -90,6 +97,11 @@ public class RealmImportService { private final ChecksumService checksumService; private final StateService stateService; + private final SchemaService schemaService; + private final LintService lintService; + + + @Autowired public RealmImportService( ImportConfigProperties importProperties, @@ -112,6 +124,8 @@ public RealmImportService( ClientScopeMappingImportService clientScopeMappingImportService, IdentityProviderImportService identityProviderImportService, MessageBundleImportService messageBundleImportService, + SchemaService schemaService, + LintService lintService, ChecksumService checksumService, StateService stateService) { this.importProperties = importProperties; @@ -136,9 +150,18 @@ public RealmImportService( this.messageBundleImportService = messageBundleImportService; this.checksumService = checksumService; this.stateService = stateService; + this.schemaService = schemaService; + this.lintService = lintService; } - public void doImport(RealmImport realmImport) { + public void doImport(RealmImport realmImport, String keycloakVersion) throws IOException { + validateSchema(realmImport, keycloakVersion); + + List lintIssues = lintConfiguration(realmImport); + if (!lintIssues.isEmpty()) { + logger.warn("Linting issues found: {}", lintIssues); + } + boolean realmExists = realmRepository.exists(realmImport.getRealm()); if (realmExists) { @@ -148,6 +171,7 @@ public void doImport(RealmImport realmImport) { } } + private void updateRealmIfNecessary(RealmImport realmImport) { if (!importProperties.getCache().isEnabled() || checksumService.hasToBeUpdated(realmImport)) { setEventsEnabledWorkaround(realmImport); @@ -224,4 +248,19 @@ private void configureRealm(RealmImport realmImport, RealmRepresentation existin stateService.doImport(realmImport); checksumService.doImport(realmImport); } + + + private void validateSchema(RealmImport realmImport, String keycloakVersion) throws IOException { + JsonSchema schema = schemaService.loadSchema(keycloakVersion); + JsonNode config = schemaService.parseConfiguration(realmImport.toString()); + Set validationResult = ((com.networknt.schema.JsonSchema) schema).validate(config); + if (!validationResult.isEmpty()) { + throw new IllegalArgumentException("Schema validation failed: " + validationResult); + } + } + + List lintConfiguration(RealmImport realmImport) throws IOException { + JsonNode config = schemaService.parseConfiguration(realmImport.toString()); + return lintService.lintConfiguration(config); + } } From 592343251b2909407b524582bce3b1d885f9e2e7 Mon Sep 17 00:00:00 2001 From: Motouom Victoire Date: Wed, 13 Nov 2024 09:00:45 +0100 Subject: [PATCH 4/5] removed unwanted changes --- .../config/service/RealmImportService.java | 41 +------------------ 1 file changed, 1 insertion(+), 40 deletions(-) diff --git a/src/main/java/de/adorsys/keycloak/config/service/RealmImportService.java b/src/main/java/de/adorsys/keycloak/config/service/RealmImportService.java index 1e33d2baf..14b257063 100644 --- a/src/main/java/de/adorsys/keycloak/config/service/RealmImportService.java +++ b/src/main/java/de/adorsys/keycloak/config/service/RealmImportService.java @@ -20,9 +20,6 @@ package de.adorsys.keycloak.config.service; -import com.fasterxml.jackson.databind.JsonNode; -import com.networknt.schema.JsonSchema; -import com.networknt.schema.ValidationMessage; import de.adorsys.keycloak.config.model.RealmImport; import de.adorsys.keycloak.config.properties.ImportConfigProperties; import de.adorsys.keycloak.config.provider.KeycloakProvider; @@ -36,10 +33,6 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; -import java.io.IOException; -import java.util.List; -import java.util.Set; - @Service public class RealmImportService { static final String[] ignoredPropertiesForRealmImport = new String[]{ @@ -97,11 +90,6 @@ public class RealmImportService { private final ChecksumService checksumService; private final StateService stateService; - private final SchemaService schemaService; - private final LintService lintService; - - - @Autowired public RealmImportService( ImportConfigProperties importProperties, @@ -124,8 +112,6 @@ public RealmImportService( ClientScopeMappingImportService clientScopeMappingImportService, IdentityProviderImportService identityProviderImportService, MessageBundleImportService messageBundleImportService, - SchemaService schemaService, - LintService lintService, ChecksumService checksumService, StateService stateService) { this.importProperties = importProperties; @@ -150,18 +136,9 @@ public RealmImportService( this.messageBundleImportService = messageBundleImportService; this.checksumService = checksumService; this.stateService = stateService; - this.schemaService = schemaService; - this.lintService = lintService; } - public void doImport(RealmImport realmImport, String keycloakVersion) throws IOException { - validateSchema(realmImport, keycloakVersion); - - List lintIssues = lintConfiguration(realmImport); - if (!lintIssues.isEmpty()) { - logger.warn("Linting issues found: {}", lintIssues); - } - + public void doImport(RealmImport realmImport) { boolean realmExists = realmRepository.exists(realmImport.getRealm()); if (realmExists) { @@ -171,7 +148,6 @@ public void doImport(RealmImport realmImport, String keycloakVersion) throws IOE } } - private void updateRealmIfNecessary(RealmImport realmImport) { if (!importProperties.getCache().isEnabled() || checksumService.hasToBeUpdated(realmImport)) { setEventsEnabledWorkaround(realmImport); @@ -248,19 +224,4 @@ private void configureRealm(RealmImport realmImport, RealmRepresentation existin stateService.doImport(realmImport); checksumService.doImport(realmImport); } - - - private void validateSchema(RealmImport realmImport, String keycloakVersion) throws IOException { - JsonSchema schema = schemaService.loadSchema(keycloakVersion); - JsonNode config = schemaService.parseConfiguration(realmImport.toString()); - Set validationResult = ((com.networknt.schema.JsonSchema) schema).validate(config); - if (!validationResult.isEmpty()) { - throw new IllegalArgumentException("Schema validation failed: " + validationResult); - } - } - - List lintConfiguration(RealmImport realmImport) throws IOException { - JsonNode config = schemaService.parseConfiguration(realmImport.toString()); - return lintService.lintConfiguration(config); - } } From c954ceadf0484c09a53ef65d3cf4be4fa140975d Mon Sep 17 00:00:00 2001 From: Motouom Victoire Date: Wed, 27 Nov 2024 11:08:47 +0100 Subject: [PATCH 5/5] Update MANAGED.md --- docs/MANAGED.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/MANAGED.md b/docs/MANAGED.md index fddee2e26..9f3f04819 100644 --- a/docs/MANAGED.md +++ b/docs/MANAGED.md @@ -1,30 +1,30 @@ -# Resource Management in keycloak-config-cli -## Introduction +## Resource Management +### Introduction -This document explains how keycloak-config-cli (kcc) manages resources in Keycloak, including its default behavior, customization options, and impact on various resource types. +This document explains how keycloak-config-cli (kc-cli) manages resources in Keycloak, including its default behavior, customization options, and impact on various resource types. -## How keycloak-config-cli Tracks Resources +### How keycloak-config-cli Tracks Resources - keycloak-config-cli stores information about resources it creates as realm attributes in the Keycloak database. -- This tracking mechanism allows kcc to manage these resources in subsequent runs. +- This tracking mechanism allows kc-cli to manage these resources in subsequent runs. -## Default Behavior +### Default Behavior -- By default, kcc will delete and recreate resources that it initially created in previous runs. +- By default, kc-cli will delete and recreate resources that it initially created in previous runs. - This ensures that the Keycloak configuration always matches the state defined in your configuration files. -## Customizing Resource Management +### Customizing Resource Management - The `import.managed.*` family of properties allows you to customize this behavior. -- Setting these properties to `no-delete` will prevent kcc from deleting resources, even if they're no longer present in your configuration files. +- Setting these properties to `no-delete` will prevent kc-cli from deleting resources, even if they're no longer present in your configuration files. -## Impact on User Federations +### Impact on User Federations - This behavior applies to user federations (such as LDAP and Active Directory). - When a user federation is deleted and recreated, all users created by that federation will also be deleted. - This includes associated data like offline tokens. -## Full Managed Resources +### Full Managed Resources keycloak-config-cli manages some types of resources absolutely. For example, if a `group` isn't defined inside the import JSON but other `groups` are specified, keycloak-config-cli will calculate the difference and delete the `group` from Keycloak. @@ -60,13 +60,13 @@ In some cases, it is required to include some Keycloak defaults because keycloak | Clients Authorization Scopes | - | `client-authorization-scopes` | | Message Bundles | Only message bundles imported with config-cli will be managed/deleted. | `message-bundles` | -## Disabling Deletion of Managed Entities +### Disabling Deletion of Managed Entities If you don't want to delete properties of a specific type, you can disable this behavior by setting properties like `import.managed.=`, e.g.: ```properties import.managed.required-actions=no-delete ``` -## State management +### State management If `import.remote-state.enabled` is set to `true` (default value), keycloak-config-cli will purge only resources they created before by keycloak-config-cli. If `import.remote-state.enabled` is set to `false`, keycloak-config-cli will purge all existing entities if they are not defined in import json.