task: rework and cleanup vocabulary initialization logic

intranda · Nov 29, 2024 · 41d56e3 · 41d56e3
1 parent 8313ead
commit 41d56e3
Show file tree

Hide file tree

Showing 14 changed files with 63 additions and 90 deletions.
diff --git a/.github/workflows/develop-build.yml b/.github/workflows/develop-build.yml
@@ -34,7 +34,7 @@ jobs:
       - name: Create ZIP archive for setup scripts
         uses: montudor/action-zip@v1
         with:
-          args: zip -qq -r setup-scripts.zip install
+          args: zip -qq -r vocabulary-init-script.zip install
       - name: Get current date
         id: date
         run: echo "::set-output name=date::$(date +'%Y-%m-%d %H:%M:%S %Z')"
@@ -53,5 +53,5 @@ jobs:
           files: |
             module-*/target/*.jar
             module-core/src/main/resources/application.properties
-            setup-scripts.zip
+            vocabulary-init-script.zip
             migration-tool.zip
diff --git a/.github/workflows/release-build.yml b/.github/workflows/release-build.yml
@@ -34,7 +34,7 @@ jobs:
       - name: Create ZIP archive for setup scripts
         uses: montudor/action-zip@v1
         with:
-          args: zip -qq -r setup-scripts.zip install
+          args: zip -qq -r vocabulary-init-script.zip install
       - name: Release
         id: create_release
         uses: softprops/action-gh-release@v2
@@ -46,5 +46,5 @@ jobs:
           files: |
             module-*/target/*.jar
             module-core/src/main/resources/application.properties
-            setup-scripts.zip
+            vocabulary-init-script.zip
             migration-tool.zip
diff --git a/.gitignore b/.gitignore
@@ -36,4 +36,5 @@ build/
 
 ### Python ###
 /migration/vmenv
+/migration/migration.csv
 __pycache__/
diff --git a/docs/de/setup.md b/docs/de/setup.md
@@ -13,16 +13,21 @@ Diese Dokumentation beschreibt den Prozess der Installation und Ersteinrichtung
 ## Einrichtung von Goobi Workflow zur Kommunikation mit dem Vokabularserver
 - Goobi Workflow verwendet seit Version `24.07` den neuen Vokabularserver.
 - Konfigurieren Sie die Variablen `vocabularyServerHost`, `vocabularyServerPort` und `vocabularyServerToken` in der Datei `goobi_config.properties` entsprechend der Konfiguration Ihres Vokabularservers.
+- Alternativ zu `vocabularyServerHost` und `vocabularyServerPort` kann auch die Variable `vocabularyServerAddress` gesetzt werden (beispielsweise `vocabularyServerAddress=https://external.address.com/vocabulary`). Diese Variable erlaubt auch eine SSL Verbindung.
 - Starten Sie Goobi Workflow neu, damit die Änderungen wirksam werden.
 - Navigieren Sie zu `Administration` > `Vocabulare`, um zu überprüfen, ob alles funktioniert. Sie sollten eine Liste von Vokabularen sehen, wenn alles in Ordnung ist (nicht jetzt, sondern nachdem Sie einige Vokabulare erstellt oder die bestehenden migriert haben). Wenn etwas nicht funktioniert, wird eine rote Fehlermeldung angezeigt.
 
 ## Ersteinrichtung
-- Für den ordnungsgemäßen Betrieb benötigt der Vokabularserver einige Ausgangsdaten.
+- Im Falle einer Datenmigration benötigt der Vokabularserver einige Ausgangsdaten.
 - Diese Daten enthalten Sprachangaben (wenn mehrsprachige Vokabulare verwendet werden) und Feldtypdefinitionen. 
-- Sie können das folgende Skript verwenden, welches einige Beispielsprachen und Feldtypen installiert.
-- Laden Sie das [Initial Data Script](https://github.com/intranda/goobi-vocabulary-server/raw/develop/install/default_setup.sh) herunter.
-- Ändern Sie die Variablen `HOST` und `TOKEN` am Anfang entsprechend der Konfiguration des Vokabularservers, lassen Sie das Suffix `/api/v1` unverändert.
-- Führen Sie das Skript aus.
+- Diese Daten sind in einem Minimaldatensatz hinterlegt und können einfach mithilfe des dafür vorgesehenen Installationsscriptes installiert werden.
+- Laden Sie das [Vokabular-Initialisierungs-Tool](https://github.com/intranda/goobi-vocabulary-server/releases/latest/download/vocabulary-init-script.zip) herunter.
+- Falls nicht bereits geschehen, starten Sie den Vokabularserver.
+- Entpacken Sie das Archiv.
+- Passen Sie die Variablen `HOST`, `PORT` und `TOKEN` an Ihre Konfiguration an und starten Sie das Script wie folgt:
+```bash
+HOST=localhost PORT=8081 TOKEN=secret /path/to/the/script/install.sh minimal
+```
 
 ## Installationsskript
 Der Vokabularserver benötigt Java 17, der Systemd-Service geht davon aus, dass Java 17 der System-Default ist.
@@ -100,8 +105,9 @@ grep ^vocabularyServerToken= /opt/digiverso/goobi/config/goobi_config.properties
 sudo systemctl restart vocabulary.service & sudo journalctl -u vocabulary.service  -f -n 0 | grep -q "Started VocabularyServerApplication in"
 
 # initial set up
-wget https://github.com/intranda/goobi-vocabulary-server/raw/develop/install/default_setup.sh -O /tmp/default_setup.sh
-bash /tmp/default_setup.sh
+wget https://github.com/intranda/goobi-vocabulary-server/releases/latest/download/vocabulary-init-script.zip -O /tmp/vocabulary-init-script.zip
+sudo unzip /tmp/vocabulary-init-script.zip -d "${VOC_PATH}"
+HOST=localhost PORT=${VOC_PORT} TOKEN=${VOC_TOKEN} ${VOC_PATH}/vocabulary-init-script/install.sh minimal
 
 ## test
 curl -s http://localhost:${VOC_PORT}/api/v1/types --header "Authorization: Bearer $VOC_TOKEN" | jq -r '._embedded.fieldTypeList[] .name'
@@ -118,15 +124,8 @@ curl "http://localhost:${VOC_PORT:-8081}/api/v1/types" --header "Authorization:
 ```
 - Das Ergebnis sollte wie folgt aussehen:
 ```bash
+Any Text
 Anything
 Boolean
 Number
-Word
-skos:prefLabel
-skos:altLabel
-skos:definition
-skos:editorialNote
-skos:related
-skos:closeMatch
-skos:exactMatch
 ```
diff --git a/docs/en/setup.md b/docs/en/setup.md
@@ -14,18 +14,22 @@ This documentation describes the process of bootstrapping the vocabulary server.
 
 - Goobi Workflow has been using the new vocabulary server since version `24.07`.
 - Configure the variables `vocabularyServerHost`, `vocabularyServerPort` and `vocabularyServerToken` in the `goobi_config.properties` file according to the configuration of your vocabulary server.
+- As an alternative to `vocabularyServerHost` and `vocabularyServerPort`, the variable `vocabularyServerAddress` can also be set (e. g. `vocabularyServerAddress=https://external.address.com/vocabulary`). This variable also allows an SSL connection.
 - Restart Goobi Workflow for the changes to take effect.
 - Navigate to `Administration` > `Vocabularies` to check that everything is working. You should see a list of vocabularies if everything is OK (not now, but after you have created some vocabularies or migrated the existing ones). If something is not working, you will see a red error message.
 
 
 ## Initial setup
-
-- For proper operation, the vocabulary server requires some initial data.
+- In the case of data migration, the vocabulary server requires some initial data.
 - This data contains language information (if multilingual vocabularies are used) and field type definitions.
-- You can use the following script, which installs some sample languages and field types.
-- Download the [Initial Data Script](https://github.com/intranda/goobi-vocabulary-server/raw/develop/install/default_setup.sh).
-- Change the variables `HOST` and `TOKEN` at the beginning according to the configuration of the vocabulary server, leave the suffix `/api/v1` unchanged.
-- Execute the script.
+- This data is stored in a minimal data set and can be easily installed using the installation script provided.
+- Download the [Vocabulary-Initialization-Tool](https://github.com/intranda/goobi-vocabulary-server/releases/latest/download/vocabulary-init-script.zip).
+- If you have not already done so, start the vocabulary server.
+- Unpack the archive.
+- Adapt the variables `HOST`, `PORT` and `TOKEN` to your configuration and start the script as follows:
+```bash
+HOST=localhost PORT=8081 TOKEN=secret /path/to/the/script/install.sh minimal
+```
 
 ## Installation script
 The vocabulary server requires Java 17, the systemd service assumes that Java 17 is the system default.
@@ -103,8 +107,9 @@ grep ^vocabularyServerToken= /opt/digiverso/goobi/config/goobi_config.properties
 sudo systemctl restart vocabulary.service & sudo journalctl -u vocabulary.service  -f -n 0 | grep -q "Started VocabularyServerApplication in"
 
 # initial set up
-wget https://github.com/intranda/goobi-vocabulary-server/raw/develop/install/default_setup.sh -O /tmp/default_setup.sh
-bash /tmp/default_setup.sh
+wget https://github.com/intranda/goobi-vocabulary-server/releases/latest/download/vocabulary-init-script.zip -O /tmp/vocabulary-init-script.zip
+sudo unzip /tmp/vocabulary-init-script.zip -d "${VOC_PATH}"
+HOST=localhost PORT=${VOC_PORT} TOKEN=${VOC_TOKEN} ${VOC_PATH}/vocabulary-init-script/install.sh minimal
 
 ## test
 curl -s http://localhost:${VOC_PORT}/api/v1/types --header "Authorization: Bearer $VOC_TOKEN" | jq -r '._embedded.fieldTypeList[] .name'
@@ -122,15 +127,8 @@ curl ‘http://localhost:${VOC_PORT:-8081}/api/v1/types’ --header ‘Authorisa
 
 - The result should look like this:
 ```bash
+Any Text
 Anything
 Boolean
 Number
-Word
-skos:prefLabel
-skos:altLabel
-skos:definition
-skos:editorialNote
-skos:related
-skos:closeMatch
-skos:exactMatch
 ```
diff --git a/install/.gitignore b/install/.gitignore
@@ -1 +1 @@
-/cache.txt
+cache.txt
diff --git a/install/default_setup.sh b/install/default_setup.sh
diff --git a/install/samples/minimal/1_languages/Deutsch.json b/install/samples/minimal/1_languages/Deutsch.json
@@ -0,0 +1,4 @@
+{
+    "abbreviation": "ger",
+    "name": "Deutsch"
+}
diff --git a/install/samples/minimal/1_languages/English.json b/install/samples/minimal/1_languages/English.json
@@ -0,0 +1,4 @@
+{
+    "abbreviation": "eng",
+    "name": "English"
+}
diff --git a/install/samples/minimal/1_languages/Français.json b/install/samples/minimal/1_languages/Français.json
@@ -0,0 +1,4 @@
+{
+    "abbreviation": "fre",
+    "name": "Français"
+}
diff --git a/install/samples/minimal/2_types/Any_Text.json b/install/samples/minimal/2_types/Any_Text.json
@@ -0,0 +1,4 @@
+{
+    "name": "Any Text",
+    "large": true
+}
diff --git a/install/samples/minimal/2_types/Anything.json b/install/samples/minimal/2_types/Anything.json
@@ -0,0 +1,3 @@
+{
+    "name": "Anything"
+}
diff --git a/install/samples/minimal/2_types/Boolean.json b/install/samples/minimal/2_types/Boolean.json
@@ -0,0 +1,6 @@
+{
+    "name": "Boolean",
+    "selectableValues": [
+        "true", "false"
+    ]
+}
diff --git a/install/samples/minimal/2_types/Number.json b/install/samples/minimal/2_types/Number.json
@@ -0,0 +1,4 @@
+{
+    "name": "Number",
+    "validation": "\\d+"
+}