From 7e5cc05ca90c4e385426a2461f96c5cfedafac75 Mon Sep 17 00:00:00 2001 From: achtsnits Date: Mon, 15 Jul 2024 20:28:04 +0200 Subject: [PATCH] adding note regarding treatment of object storage as file system --- docs/design/overview.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/design/overview.md b/docs/design/overview.md index 29a04e1..b5eedc5 100644 --- a/docs/design/overview.md +++ b/docs/design/overview.md @@ -4,10 +4,10 @@ As laid out in the [System Architecture document](https://eoepca.readthedocs.io/projects/architecture/) the Workspace building-block comprises: * **Workspace Controller**
- Platform-level API for administration of workspaces with capabilities via an API for **Workspace Provisioning** and **Workspace Utilisation** + Platform-level API for administration of workspaces with capabilities of the Workspace Controller API for **Workspace Provisioning** and **Workspace Utilisation** -* **Storage Controller**
- User-level API for management of storage buckets with capabilities via an API for **Bucket Provisioning**, **Bucket Management** and **Bucket Federation** +* **Storage Controller & Storage Layer**
+ User-level API for management of storage buckets with capabilities of the Storage Controller API for **Bucket Provisioning**, **Bucket Management** and **Bucket Federation** and of the Storage Layer API and UI for **Bucket Content Browsing** and specific **Bucket Content Sharing** via HTTP. * **Workspace Services**
Comprising the **extensible set of Services** dynamically instantiated on a workspace within the scope of a project/user. @@ -40,7 +40,7 @@ An operator can establish a workspace for a project/user imperatively via the Ku Note: The EOEPCA 1.x workspace API is obsolete and not included in EOEPCA+. -A workspace can be accessed via the Kubernetes API and is integrated with GitOps tooling through Flux. Both common and optional components can be installed for a specific project or user. By default, the template GitRepository includes the Storage Controller, responsible for **Bucket Provisioning** and **Bucket Federation**, as well as common EOEPCA Building Blocks (BB) for resource discovery and EO data management. These **Workspace Services** can be enriched with additional tools from the EOEPCA Application Hub BB and EOEPCA MLHub BB. +A workspace can be accessed via the Kubernetes API and is integrated with GitOps tooling through Flux. Both common and optional components can be installed for a specific project or user. By default, the template GitRepository includes the Storage Controller API, responsible for **Bucket Provisioning**, **Bucket Management** and **Bucket Federation**, the Storage Layer API and UI for **Bucket Content Browsing** and specific **Bucket Content Sharing** via HTTP, as well as common EOEPCA Building Blocks (BB) for resource discovery and EO data management. These **Workspace Services** can be enriched with additional tools from the EOEPCA Application Hub BB and EOEPCA MLHub BB. The **Storage Controller** is tasked with reconciling the desired state as specified in the storage manifest (for the new Kubernetes Storage CRD). It uses Terraform internally for the reconciliation process, with documentation providing specifications for common cloud providers like AWS as examples. @@ -57,4 +57,6 @@ and finally exposes: - Access URL, - The current reconciliation state. -TBC +The **Storage Layer** is implemented as a microservice that allows users to search and browse through the content of the storage buckets connected to a given workspace using directory-based file system semantics. For "text files" like markdown, direct preview capabilities are provided. Besides navigation and search, the main use case for the Storage Layer is sharing specific content via pre-signed URLs, enabling direct HTTP access without requiring user authentication. + +Note: The EOEPCA+ acknowledges that object storage solutions like AWS S3 store content as key-value pairs without true directory functionality, which can have performance implications when using a file system abstraction. By supporting only read-only browsing and not allowing modifications like move or rename operations, treating bucket content as files within the storage layer has been identified as the most intuitive approach for users to navigate content hierarchies.