Release 3.23.0

Aam-Digital · Aug 21, 2023 · 2566e16 · 2566e16
2 parents 43d51c9 + c43b328
commit 2566e16
Show file tree

Hide file tree

Showing 591 changed files with 9,428 additions and 6,803 deletions.
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -1,4 +1,4 @@
-see issue: #XX
+closes: #XX
 
 ### Visible/Frontend Changes
 - [x] 

diff --git a/README.md b/README.md
@@ -3,8 +3,7 @@
 [![Code Climate](https://codeclimate.com/github/Aam-Digital/ndb-core/badges/gpa.svg)](https://codeclimate.com/github/Aam-Digital/ndb-core)
 [![Test Coverage](https://api.codeclimate.com/v1/badges/4e4a7a6301064019b2c9/test_coverage)](https://codeclimate.com/github/Aam-Digital/ndb-core/test_coverage)
 [![E2E Tests](https://img.shields.io/endpoint?url=https://dashboard.cypress.io/badge/simple/2petka/master&style=flat&logo=cypress)](https://dashboard.cypress.io/projects/2petka/runs)
-[![Guides](https://img.shields.io/badge/Tutorial%20%26%20Guides-22-blue)](https://aam-digital.github.io/ndb-core/documentation/additional-documentation/overview.html)
-[![Doc CoverageDocs](https://aam-digital.github.io/ndb-core/documentation/images/coverage-badge-documentation.svg)](https://aam-digital.github.io/ndb-core/documentation/modules.html)
+[![Guides](https://img.shields.io/badge/Tutorial%20%26%20Guides-available-blue)](https://aam-digital.github.io/ndb-core/documentation/additional-documentation/overview.html)
 [![Known Vulnerabilities](https://snyk.io/test/github/Aam-Digital/ndb-core/badge.svg)](https://snyk.io/test/github/Aam-Digital/ndb-core)
 
 # Aam Digital

diff --git a/build/Dockerfile b/build/Dockerfile
@@ -75,12 +75,13 @@ ENV COUCHDB_URL="http://localhost"
 ENV NOMINATIM_URL="https://nominatim.openstreetmap.org"
 
 # content security policy headers
-ENV CSP_REPORT_URI=""
+# (also see Developer Documentation: https://aam-digital.github.io/ndb-core/documentation/additional-documentation/concepts/security.html)
+ENV CSP_REPORT_URI="https://o167951.ingest.sentry.io/api/1242399/security/"
 # overwrite the Content-Security-Policy rules (report-uri is added automatically)
 ## default includes all required whitelists for production server
 ## to disable any CSP blocking, set to "default-src *  data: blob: filesystem: about: ws: wss: 'unsafe-inline' 'unsafe-eval'"
-ENV CSP="default-src 'self' 'unsafe-eval' data: https://sentry.io https://*.tile.openstreetmap.org/ https://matomo.aam-digital.org  https://*.aam-digital.com https://api.github.com/repos/Aam-Digital/ 'sha256-Sh1PNRUktjifFwuOicavxmlAcFpZMbUqQGiCIoKhDI8='; style-src 'self' 'unsafe-inline'"
-### 'sha256-Sh1PNRUktjifFwuOicavxmlAcFpZMbUqQGiCIoKhDI8=' for index.html writing browser details
+ENV CSP="default-src 'self' 'unsafe-eval' data: https://*.tile.openstreetmap.org/ https://matomo.aam-digital.org  https://*.aam-digital.com https://api.github.com/repos/Aam-Digital/ https://sentry.io $CSP_REPORT_URI 'sha256-gtzIf+c+ujwirISvjI8lnwlaZwnMkh04eA9ZDBCd8TY='; style-src 'self' 'unsafe-inline'"
+### 'sha256-gtzIf+c+ujwirISvjI8lnwlaZwnMkh04eA9ZDBCd8TY=' for index.html writing browser details
 ### 'unsafe-eval' required for pouchdb https://github.com/pouchdb/pouchdb/issues/7853#issuecomment-535020600
 
 # variables are inserted into the nginx config

diff --git a/doc/compodoc_sources/.compodocrc.json b/doc/compodoc_sources/.compodocrc.json
@@ -1,5 +1,5 @@
 {
-  "tsconfig": "tsconfig.json",
+  "tsconfig": "doc/compodoc_sources/tsconfig.json",
   "output": "doc/compodoc",
   "assetsFolder": "doc/images",
   "includes": "doc/compodoc_sources",

diff --git a/doc/compodoc_sources/concepts/entity-schema-system.md b/doc/compodoc_sources/concepts/entity-schema-system.md
@@ -36,11 +36,14 @@ supporting the `EntityMapperService` and transforming the entity object into the
 
 -----
 
-`EntitySchemaService` keeps a registry of "data types",
+`EntitySchemaService` manages a registry of "data types",
 i.e. transformation functions that will be called for a specific schema field's dataType.
 
 Basic data transformations for `string`, `number`, `date` and `month` are supported by default.
-You can register your own transformations by injecting the `EntitySchemaService` and using its `registerSchemaDatatype()` method.
+You can register your own types by implementing services extending `DefaultDatatype` and
+providing these through Angular dependency injection using `multi: true`.
+
+Also see: [How to create a new Datatype](../how-to-guides/create-a-new-datatype.html).
 
 ## Schema options
 

diff --git a/doc/compodoc_sources/concepts/extendability.md b/doc/compodoc_sources/concepts/extendability.md
@@ -0,0 +1,33 @@
+# Extendability and Plugin Approach
+Aam Digital is designed to be an extendable platform.
+We try to define core interfaces that can be implemented in additional feature modules
+to implement further functionality in a modular way.
+
+The following aspects are specifically designed to be extended:
+- **DataTypes**
+  - transformation functions how data is stored in / read from database
+  - `editComponent` how data is displayed and edited in forms
+  - `viewComponent` how data is displayed in tables
+  - `importValueMapping` to support smart import into the data type
+  - *also see [How to create a new Datatype](../how-to-guides/create-a-new-datatype.html)*
+- **Entity Types**
+  - pre-define a data structure with various fields and custom logic that may be interconnected. This mostly is useful if you implement very specialized UI components for a specific data structure.
+  - any entity type can be extended through config for individual clients (e.g. adding further properties at runtime)
+  - demo data generator to automatically provide useful sample records
+  - *also see [How to create a new Entity Type](../how-to-guides/create-a-new-entity-type.html)*
+- **Views**
+  - defining a screen completely, including data loaded, etc. and hook it into the platforms navigation and overall layout
+- **Sub-Views**
+  - defining a screen to display custom details for the entity currently loaded in the active route. The core platform takes care of passing the current entity and config details to the view as inputs.
+- **Dashboard Widgets**
+  - filling the given card template with custom data and visualization 
+- **Filters**
+  - specialized logic and UIs to filter list data
+- **Technical "Backend" Implementations**
+  - less common to change, but possible to implement integrations with different technical systems are
+    - Authentication Services (e.g. switch between native CouchDB users and more advance Keycloak)
+    - Database / Local Storage (e.g. switch between PouchDB using IndexedDB and purely in-memory, discardable data storage - or possibly implement an integration with a different system)
+
+The folder structure of the code base (while containing some intertwined legacy structures) also reflects this architecture:
+- *src/app/core*: generic structures and platform code
+- *src/app/features*: more specialized, modular features that plug into the core code (e.g. a location / map integration type)
diff --git a/doc/compodoc_sources/concepts/security.md b/doc/compodoc_sources/concepts/security.md
@@ -0,0 +1,28 @@
+# Security
+
+We have made both technical and design choices to develop a secure platform.
+The Angular framework itself has some in-built protection against common security issues like cross-site scripting attacks (see [Angular > Security](https://angular.io/guide/security)).
+Beyond this, the following measures are implemented:
+
+- If deployed including our ["replication-backend"](https://github.com/Aam-Digital/replication-backend), that server-side API also ensures authenticated users can only access and sync data their account has permissions for.
+- Password policy enforces users to set a strong password including special characters (either via Keycloak or the platforms user profile form)
+- Content Security Policy (CSP) headers restrict connections to and execution of code from sources that are not whitelisted.
+
+## Content Security Policy (CSP)
+CSP headers are set in the nginx server being built from the code base to serve the Angular app.
+The whitelisted CSP sources can be overwritten and adapted using a docker environment variable `CSP` (the default whitelist is defined in the [Dockerfile](https://github.com/Aam-Digital/ndb-core/blob/master/build/Dockerfile)).
+
+> CSP is currently running in "report-only" mode for testing.
+> Scripts and connections are not yet blocked by default.
+
+### Allowing PouchDB to function under CSP
+The browser-side database system PouchDB uses map-reduce functions for indexing which are defined as strings.
+It is therefore requiring `'unsafe-eval'` in the CSP.
+
+### Whitelisting the index.html
+To whitelist a specific script section (currently only in the index.html) a [CSP hash](https://content-security-policy.com/hash/) can be used.
+Updating the hash should be necessary only rarely, when that script section changes.
+
+The easiest and most reliable way to get the correct hash is to deploy a production build image and check the browser console.
+It states something like `"Refused to execute inline script because it violates the following Content Security Policy directive: "...". Either the 'unsafe-inline' keyword, a hash ('sha256-<RELEVANT HASH>')" or a nonce is required."` from where you can copy the given hash and include/update it in the CSP headers.
+Generating the hash by pasting the script into an online generator does not seem to work, probably because code is minified during the build process.
diff --git a/doc/compodoc_sources/how-to-guides/configure-custom-system.md b/doc/compodoc_sources/how-to-guides/configure-custom-system.md
@@ -0,0 +1,7 @@
+# Configure and Customize a System
+The platform allows very flexible customization of the user interface and data structures to different use cases.
+This is usually possible without changes to the code base, using the configuration system.
+
+> This guide is currently only a "stub".
+> 
+> For the time being, please refer to the documentation about the [Configuration](../concepts/configuration.html).
diff --git a/doc/compodoc_sources/how-to-guides/create-new-datatype.md b/doc/compodoc_sources/how-to-guides/create-new-datatype.md
@@ -0,0 +1,60 @@
+# How to create a new Datatype
+"Datatypes" define how a single field (i.e. entity property) is stored and displayed.
+They are core building blocks for all entities and can enable advanced functionality, like displaying a streetmap for an address.
+
+The Aam Digital core defines most commonly known datatypes already (see `CoreModule`).
+The architecture of datatypes is designed for extension however, so you can easily add further types.
+
+## The base: `DefaultDatatype` and Angular Service
+`DefaultDatatype` is the base class for all implementations of custom datatypes.
+It implements default logic for all the required aspects so that you can override only those parts that are relevant for your new type.
+
+Implementations of datatypes are normal Angular Services.
+This allows you to inject and use any other service that you may need to do sophisticated data transformations.  
+
+## Defining a new Datatype
+- Create a new Angular Service class (according to our file name convention it should follow the pattern `my-custom.datatype.ts`)
+- Use inheritance to extend the `DefaultDatatype` class
+- Define your datatype identifier (which is used in `@DatabaseField` annotations in their `EntitySchemaField` definitions) by setting the `static dataType = "my-custom"` property of your class
+- Override any of the other aspects if you want to customize them
+
+This could result in a Datatype class like this:
+```
+@Injectable()
+export class MyCustomDatatype extends DefaultDatatype<SpecialObject, string> {
+  static dataType = "my-custom";
+  
+  constructor() {
+    super();
+    // use constructor to simply inject other services you need
+  }
+  
+  editComponent = "EditMyCustomFormField";
+  viewComponent = "DisplayText";
+  // make sure to register your new components in the ComponentRegistry
+  
+  transformToDatabaseFormat(value: SpecialObject): string {
+    // storing as string in the database for whatever reason
+    return value.toString();
+  }
+
+  transformToObjectFormat(value: string): SpecialObject {
+    return transformToSpecialObject(value);
+  }
+  
+  importConfigComponent = "SpecialImportValueMapping";
+  
+  importMapFunction(value: any, schemaField: EntitySchemaField, additional?: any) {
+    return SpecialValueParserForDifferentImportFormats(value);
+  }
+}
+```
+
+Please also refer to the extensive JsDoc code comments in the `DefaultDatatype` class.
+
+## Registering the new Datatype
+Provide your datatype service using Angular dependency injection:
+`{ provide: DefaultDatatype, useClass: MyCustomDatatype, multi: true },`
+
+The EntitySchemaService, that handles all data transformations, and other platform modules automatically pick up your datatype
+and use it for any entity properties that use the `dataType` identifier of your implementation.
diff --git a/doc/compodoc_sources/how-to-guides/create-new-entity-type.md b/doc/compodoc_sources/how-to-guides/create-new-entity-type.md
@@ -1,21 +1,40 @@
 # How to create a new Entity Type
 "Entities" are the objects around which our whole system is built.
-There may for example be User, Child or School classes representing the types of objects the users can view interact with. To extend the system with types, you can create a new module and extend the existing base of the Entity system.
+Entity types specify a data structure for records in the case management system,
+defining both the data object as well as some details how the records are displayed in the UI to users.
+There may for example be User, Child or School classes representing the types of objects the users can view in lists and detail forms and interact with.
 
-The [EntityModule](../../modules/EntityModule.html) specifies the base class and generic features, specific entity types can extend upon it.
-For an example, have a look at the [User](../../classes/User.html#source) class that extends `Entity`.
+To adapt the platform to a certain use case, you usually want to adapt entity types or create new ones.
+This defines what fields users can see and edit in forms, among other things.
+
+There are two approaches to define entity types:
+- at runtime in the config file (see [How to Configure and Customize a System](./configure-and-customize-a-system.html))
+- at build time in the code (this guide)
+
+Unless you have special requirements and write further code specifically for your entity type, 
+you usually should rely on the more flexible and easy approach defining them in the config.
 
+_This guide talks about the advance topic of implementing an entity type via code._
 
 
-## The `Entity` class
+---
+## Creating a custom Entity type in code 
+To extend the system with types, you can create a new module and extend the existing base class of the Entity system.
+This is something you would only do if you are also implementing special components or services in the code base to handle this entity type.
+Otherwise, it is easier and more flexible to define an entity type through the config system only, as shown above.
+
+The [EntityModule](../../modules/EntityModule.html) specifies the base class and generic features, specific entity types can extend upon it.
+For an example, have a look at the [User](../../classes/User.html#source) class that extends `Entity`.
+
+### The `Entity` class
 `Entity` and any class extending it are only plain TypeScript classes defining the attributes (and possibly some "business logic" methods).
 
 The classes should never contain code for saving/deleting or other database related code because we are keeping this separate in an EntityMapper class (see [Data Mapper Pattern](https://en.wikipedia.org/wiki/Data_mapper_pattern)).
 
 To keep them easy to understand and extend the Entity (sub-)classes also shouldn't depend on other components or services - Entity classes are not by themselves part of the Angular system and do not have dependency injection!
 
 
-## The EntitySchema
+### The EntitySchema
 The EntitySchema object is defining which attributes of the class will be saved to the database.
 Attributes not explicitly defined in the Entity sub-type's schema are ignored when an instance is saved.
 
@@ -42,15 +61,15 @@ You can also set additional options for DatabaseFields and implement your own da
 For more details see the concept of the [EntitySchema System](../concepts/entity-schema.html).
 
 
-## Other components to be implemented for a new sub-type
+### Other components to be implemented for a new sub-type
 The following components are related to display an Entity sub-type and currently implemented for each new sub-type as Angular components:
 * `EntityDetailsComponent`: displays all attributes of one of the Entity's objects to the user
 * `EntityListComponent`: displays a list of all the Entity's objects
 * `EntityBlockComponent`: displays a small, inline representation of one of the Entity's objects for use in other components
 
 
 
-## Example: Creating a new Entity sub-type
+### Example: Creating a new Entity sub-type
 
 ```
 import { Entity } from '../entity/entity';

diff --git a/doc/compodoc_sources/summary.json b/doc/compodoc_sources/summary.json
@@ -25,10 +25,60 @@
       }
     ]
   },
+  {
+    "title": "Concepts",
+    "file": "concepts/_index.md",
+    "children": [
+      {
+        "title": "Overall Architecture",
+        "file": "concepts/overall-architecture.md"
+      },
+      {
+        "title": "Extendability and Plugin Approach",
+        "file": "concepts/extendability.md"
+      },
+      {
+        "title": "Entity Schema",
+        "file": "concepts/entity-schema-system.md"
+      },
+      {
+        "title": "Configuration",
+        "file": "concepts/configuration.md"
+      },
+      {
+        "title": "Session and Authentication System",
+        "file": "concepts/session-system.md"
+      },
+      {
+        "title": "User Roles and Permissions",
+        "file": "concepts/permissions.md"
+      },
+      {
+        "title": "Security",
+        "file": "concepts/security.md"
+      },
+      {
+        "title": "UX Guidelines",
+        "file": "concepts/ux-guidelines.md"
+      },
+      {
+        "title": "Documentation Structure",
+        "file": "concepts/documentation-structure.md"
+      },
+      {
+        "title": "Infrastructure",
+        "file": "concepts/infrastructure.md"
+      }
+    ]
+  },
   {
     "title": "How-To Guides",
     "file": "how-to-guides/_index.md",
     "children": [
+      {
+        "title": "Configure and Customize a System",
+        "file": "how-to-guides/configure-custom-system.md"
+      },
       {
         "title": "Navigate the Code Structure",
         "file": "how-to-guides/navigate-code-structure.md"
@@ -37,6 +87,18 @@
         "title": "Contribute Code to the Project",
         "file": "how-to-guides/contribute-code.md"
       },
+      {
+        "title": "Create a New Entity Type",
+        "file": "how-to-guides/create-new-entity-type.md"
+      },
+      {
+        "title": "Create a New Datatype",
+        "file": "how-to-guides/create-new-datatype.md"
+      },
+      {
+        "title": "Create a Report",
+        "file": "how-to-guides/reports.md"
+      },
       {
         "title": "Load and Save Data",
         "file": "how-to-guides/load-and-save-data.md"
@@ -69,10 +131,6 @@
         "title": "Use Queries and Indices",
         "file": "how-to-guides/use-queries-and-indices.md"
       },
-      {
-        "title": "Create a New Entity Type",
-        "file": "how-to-guides/create-new-entity-type.md"
-      },
       {
         "title": "Generate Demo Data",
         "file": "how-to-guides/generate-demo-data.md"
@@ -81,10 +139,6 @@
         "title": "Review a Pull Request",
         "file": "how-to-guides/review-pull-request.md"
       },
-      {
-        "title": "Create a Report",
-        "file": "how-to-guides/reports.md"
-      },
       {
         "title": "Format Data Export",
         "file": "how-to-guides/exports.md"
@@ -103,44 +157,6 @@
       }
     ]
   },
-  {
-    "title": "Concepts",
-    "file": "concepts/_index.md",
-    "children": [
-      {
-        "title": "Overall Architecture",
-        "file": "concepts/overall-architecture.md"
-      },
-      {
-        "title": "Entity Schema",
-        "file": "concepts/entity-schema-system.md"
-      },
-      {
-        "title": "Session and Authentication System",
-        "file": "concepts/session-system.md"
-      },
-      {
-        "title": "Configuration",
-        "file": "concepts/configuration.md"
-      },
-      {
-        "title": "User Roles and Permissions",
-        "file": "concepts/permissions.md"
-      },
-      {
-        "title": "UX Guidelines",
-        "file": "concepts/ux-guidelines.md"
-      },
-      {
-        "title": "Documentation Structure",
-        "file": "concepts/documentation-structure.md"
-      },
-      {
-        "title": "Infrastructure",
-        "file": "concepts/infrastructure.md"
-      }
-    ]
-  },
   {
     "title":"Prerequisites",
     "file": "prerequisites.md"