Skip to content

Commit

Permalink
adding note regarding treatment of object storage as file system
Browse files Browse the repository at this point in the history
  • Loading branch information
achtsnits committed Jul 15, 2024
1 parent fa84198 commit 7e5cc05
Showing 1 changed file with 7 additions and 5 deletions.
12 changes: 7 additions & 5 deletions docs/design/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
As laid out in the [System Architecture document](https://eoepca.readthedocs.io/projects/architecture/) the Workspace building-block comprises:

* **Workspace Controller**<br>
Platform-level API for administration of workspaces with capabilities via an API for **Workspace Provisioning** and **Workspace Utilisation**
Platform-level API for administration of workspaces with capabilities of the Workspace Controller API for **Workspace Provisioning** and **Workspace Utilisation**

* **Storage Controller**<br>
User-level API for management of storage buckets with capabilities via an API for **Bucket Provisioning**, **Bucket Management** and **Bucket Federation**
* **Storage Controller & Storage Layer**<br>
User-level API for management of storage buckets with capabilities of the Storage Controller API for **Bucket Provisioning**, **Bucket Management** and **Bucket Federation** and of the Storage Layer API and UI for **Bucket Content Browsing** and specific **Bucket Content Sharing** via HTTP.

* **Workspace Services**<br>
Comprising the **extensible set of Services** dynamically instantiated on a workspace within the scope of a project/user.
Expand Down Expand Up @@ -40,7 +40,7 @@ An operator can establish a workspace for a project/user imperatively via the Ku

Note: The EOEPCA 1.x workspace API is obsolete and not included in EOEPCA+.

A workspace can be accessed via the Kubernetes API and is integrated with GitOps tooling through Flux. Both common and optional components can be installed for a specific project or user. By default, the template GitRepository includes the Storage Controller, responsible for **Bucket Provisioning** and **Bucket Federation**, as well as common EOEPCA Building Blocks (BB) for resource discovery and EO data management. These **Workspace Services** can be enriched with additional tools from the EOEPCA Application Hub BB and EOEPCA MLHub BB.
A workspace can be accessed via the Kubernetes API and is integrated with GitOps tooling through Flux. Both common and optional components can be installed for a specific project or user. By default, the template GitRepository includes the Storage Controller API, responsible for **Bucket Provisioning**, **Bucket Management** and **Bucket Federation**, the Storage Layer API and UI for **Bucket Content Browsing** and specific **Bucket Content Sharing** via HTTP, as well as common EOEPCA Building Blocks (BB) for resource discovery and EO data management. These **Workspace Services** can be enriched with additional tools from the EOEPCA Application Hub BB and EOEPCA MLHub BB.

The **Storage Controller** is tasked with reconciling the desired state as specified in the storage manifest (for the new Kubernetes Storage CRD). It uses Terraform internally for the reconciliation process, with documentation providing specifications for common cloud providers like AWS as examples.

Expand All @@ -57,4 +57,6 @@ and finally exposes:
- Access URL,
- The current reconciliation state.

TBC
The **Storage Layer** is implemented as a microservice that allows users to search and browse through the content of the storage buckets connected to a given workspace using directory-based file system semantics. For "text files" like markdown, direct preview capabilities are provided. Besides navigation and search, the main use case for the Storage Layer is sharing specific content via pre-signed URLs, enabling direct HTTP access without requiring user authentication.

Note: The EOEPCA+ acknowledges that object storage solutions like AWS S3 store content as key-value pairs without true directory functionality, which can have performance implications when using a file system abstraction. By supporting only read-only browsing and not allowing modifications like move or rename operations, treating bucket content as files within the storage layer has been identified as the most intuitive approach for users to navigate content hierarchies.

0 comments on commit 7e5cc05

Please sign in to comment.