From d262cc90562a98ce547335e94e5b15a2b3348e91 Mon Sep 17 00:00:00 2001 From: Simon Gerber Date: Wed, 22 Nov 2023 15:37:15 +0100 Subject: [PATCH 1/4] Add generic APPUiO Managed OpenShift 4 architecture page --- .../ROOT/assets/images/ocp4-architecture.svg | 4 +++ .../images/ocp4-external-dependencies.svg | 4 +++ .../pages/references/architecture/index.adoc | 34 +++++++++++++++++++ docs/modules/ROOT/partials/nav.adoc | 2 +- 4 files changed, 43 insertions(+), 1 deletion(-) create mode 100644 docs/modules/ROOT/assets/images/ocp4-architecture.svg create mode 100644 docs/modules/ROOT/assets/images/ocp4-external-dependencies.svg create mode 100644 docs/modules/ROOT/pages/references/architecture/index.adoc diff --git a/docs/modules/ROOT/assets/images/ocp4-architecture.svg b/docs/modules/ROOT/assets/images/ocp4-architecture.svg new file mode 100644 index 00000000..9b8fb574 --- /dev/null +++ b/docs/modules/ROOT/assets/images/ocp4-architecture.svg @@ -0,0 +1,4 @@ + + + +
Cluster Machine Network (/24, requires DHCP)
Cluster Machine Network (/24, requires DHCP)
OpenShift 4 Control plane nodes
3x 4vCPU/16GiB RAM
OpenShift 4 Control pla...
OpenShift 4 infrastructure nodes
4x 4vCPU/16GiB RAM
OpenShift 4 infrastruct...
OpenShift 4 worker nodes
3x 4vCPU/16GiB RAM
OpenShift 4 worker node...
Pod Network – Cilium VXLAN – 10.128.0.0/14
Pod Network – Cilium VXLAN – 10.128.0.0/14
Kubernetes API
Kubernetes API
Scheduler
Scheduler
Controllers
Controllers
OpenShift API
OpenShift API
etcd
etcd
Ingress Router
Ingress Router
Monitoring
Monitoring
Registry
Registry
Logging
Logging
...
...
Application A
Application A
Application B
Application B
Application C
Application C
6443/tcp
6443/tcp
API VIP
API VIP
80/tcp 443/tcp
80/tcp 443/tcp
Ingress VIP
Ingress VIP
Egress VIP
Egress VIP
SNAT
SNAT
SNAT
SNAT
SNAT
SNAT
Service Network – Cilium eBPF rules – 172.30.0.0/16
Service Network – Cilium eBPF rules – 172.30.0.0/16Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/modules/ROOT/assets/images/ocp4-external-dependencies.svg b/docs/modules/ROOT/assets/images/ocp4-external-dependencies.svg new file mode 100644 index 00000000..e909b89c --- /dev/null +++ b/docs/modules/ROOT/assets/images/ocp4-external-dependencies.svg @@ -0,0 +1,4 @@ + + + +
APPUiO Managed
OpenShift 4 cluster
APPUiO Managed...
VSHN infrastructure
VSHN infrastructure
Secrets store
vault-prod.syn.vshn.net
Secrets store...
Project Syn API
api.syn.vshn.net
Project Syn API...
Central metrics store
metrics-receive.appuio.net
Central metrics store...
Cluster configuration
git.vshn.net
Cluster configuration...
acme-dns
acme-dns-api.vshn.net
acme-dns-ns.vshn.net
acme-dns...
VSHN SSO
id.vshn.net
VSHN SSO...
VSHN LDAP
ldap.vshn.net
VSHN LDAP...
Container image registries
Container image registries
Red Hat registries
registry.redhat.io
registry.access.redhat.com
Red Hat registries...
Google container registry
gcr.io
Google container...
Kubernetes registry
registry.k8s.io
Kubernetes regis...
DockerHub
docker.io
DockerHub...
GitHub container registry
ghcr.io
GitHub container...
Quay container registry
quay.io
Quay container r...
Let's Encrypt API
acme-v02.api.letsencrypt.org
acme-staging-v02.api.letsencrypt.org
Let's Encrypt API...
Red Hat infrastructure
Red Hat infrastructure
NTP servers
1.rhel.pool.ntp.org
2.rhel.pool.ntp.org
3.rhel.pool.ntp.org
NTP servers...
Telemetry
cert-api.access.redhat.com
api.access.redhat.com
infogw.api.openshift.com
console.redhat.com/api/ingress
Telemetry...
OpenShift upgrade service
api.openshift.com
OpenShift upgrade service...
OpsGenie API
api.opsgenie.com
OpsGenie API...
Passbolt API
cloud.passbolt.com/vshn
Passbolt API...Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/modules/ROOT/pages/references/architecture/index.adoc b/docs/modules/ROOT/pages/references/architecture/index.adoc new file mode 100644 index 00000000..44d84bcb --- /dev/null +++ b/docs/modules/ROOT/pages/references/architecture/index.adoc @@ -0,0 +1,34 @@ += APPUiO Managed OpenShift 4 Architecture + +Every APPUiO Managed OpenShift 4 has 3 control plane nodes. +Additionally, APPUiO Managed OpenShift 4 clusters have https://docs.openshift.com/container-platform/4.13/nodes/nodes/nodes-nodes-creating-infrastructure-nodes.html[4 infrastructure nodes]. +Finally, APPUiO Managed OpenShift 4 clusters have 3 worker nodes by default. +The minimum size for each node is 4 vCPUs, 16 GiB of RAM and 100 GiB root disk. + +All APPUiO Managed OpenShift 4 clusters use https://cilium.io[Cilium] as the network plugin. +APPUiO Managed OpenShift 4 uses the default OpenShift IP CIDRs for the pod and service networks: 10.128.0.0/14 for the pod network and 172.30.0.0/24 for the service network. +There's no standard IP CIDR for the cluster machine network (sometimes also "cluster network" or "machine network") for APPUiO Managed OpenShift 4 clusters. +However, a default IP CIDR for the machine network is defined for some infrastructure providers. + +Each cluster is configured with two floating IPs, one for the Kubernetes API (the "API VIP") and one for the ingress router (the "Ingress VIP"). +On most providers, a third floating IP (the "Egress VIP") is configured and all outgoing traffic from the machine network is SNATed to this IP. + +.APPUiO Managed OpenShift 4 base architecture +image::ocp4-architecture.svg[alt=OCP4 Architecture, width=640] + +https://syn.tools[Project Syn] is used to configure APPUiO Managed OpenShift 4 clusters. +Apart from deploying custom configurations for OpenShift components, Project Syn is also used to deploy multiple third party components, such as https://cilium.io[Cilium] and https://cert-manager.io[cert-manager] on each APPUiO Managed OpenShift 4 cluster. + +Each APPUiO Managed OpenShift 4 cluster depends on multiple external systems. +Some of those systems are owned and operated by VSHN while others are operated by Red Hat. +Additionally, APPUiO Managed OpenShift 4 makes use of some SaaS services, such as https://letsencrypt.org/[Let's Encrypt], https://www.passbolt.com/[Passbolt], various container registries, and https://www.atlassian.com/software/opsgenie[OpsGenie]. +The mandatory external systems are shown in the diagram below. + +.APPUiO Managed OpenShift 4 external dependencies +image::ocp4-external-dependencies.svg[alt=OCP4 external dependencies, width=800] + +The architecture documentation for the supported infrastructure providers provides details on how the architecture is adapted to work within the constraints imposed by the provider. + +* cloudscale.ch (coming soon) +* Exoscale (coming soon) +* VMWare vSphere (coming soon) diff --git a/docs/modules/ROOT/partials/nav.adoc b/docs/modules/ROOT/partials/nav.adoc index f7c94d93..13eb22ec 100644 --- a/docs/modules/ROOT/partials/nav.adoc +++ b/docs/modules/ROOT/partials/nav.adoc @@ -4,7 +4,7 @@ // TODO Consider to add a "Getting Started" section -* Architecture +* xxref:oc4:ROOT:references/architecture/index.adoc[Architecture] ** xref:oc4:ROOT:explanations/dns_scheme.adoc[] ** xref:oc4:ROOT:explanations/node_groups.adoc[] ** xref:oc4:ROOT:explanations/etcd_encryption.adoc[] From 330635da0c1c47c1f54455609459ba89dc27cd6b Mon Sep 17 00:00:00 2001 From: Simon Gerber Date: Tue, 28 Nov 2023 16:19:37 +0100 Subject: [PATCH 2/4] Restructure generic architecture * Add better intro * Improve structure * Add links to OCP/K8s architecture docs --- .../pages/references/architecture/index.adoc | 46 ++++++++++++++++--- 1 file changed, 39 insertions(+), 7 deletions(-) diff --git a/docs/modules/ROOT/pages/references/architecture/index.adoc b/docs/modules/ROOT/pages/references/architecture/index.adoc index 44d84bcb..87a59d3c 100644 --- a/docs/modules/ROOT/pages/references/architecture/index.adoc +++ b/docs/modules/ROOT/pages/references/architecture/index.adoc @@ -1,9 +1,31 @@ = APPUiO Managed OpenShift 4 Architecture -Every APPUiO Managed OpenShift 4 has 3 control plane nodes. -Additionally, APPUiO Managed OpenShift 4 clusters have https://docs.openshift.com/container-platform/4.13/nodes/nodes/nodes-nodes-creating-infrastructure-nodes.html[4 infrastructure nodes]. -Finally, APPUiO Managed OpenShift 4 clusters have 3 worker nodes by default. -The minimum size for each node is 4 vCPUs, 16 GiB of RAM and 100 GiB root disk. +APPUiO Managed OpenShift 4 is based on https://docs.openshift.com/container-platform/latest/welcome/index.html[Red Hat OpenShift 4 Container Platform], which is a https://kubernetes.io[Kubernetes] distribution maintained and developed by Red Hat. +This documentation assumes that readers are familiar with Kubernetes concepts. + +For details on the architecture of a Kubernetes cluster, we recommend reading the https://kubernetes.io/docs/concepts/architecture/[cluster architecture section] of the official Kubernetes documentation. + +For details on the architecture of a Red Hat OpenShift 4 cluster, please see the https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html/architecture/index[architecture section] of the OpenShift 4 product documentation or https://docs.openshift.com/container-platform/latest/architecture/index.html[architecture page] in the OpenShift Container Platform technical documentation. + +This page focuses on the default customizations and configurations for a Red Hat OpenShift 4 cluster which make up the APPUiO Managed OpenShift 4 product. + +== Generic APPUiO Managed OpenShift 4 architecture + +.APPUiO Managed OpenShift 4 base architecture +image::ocp4-architecture.svg[alt=OCP4 Architecture, width=640] + +=== Nodes + +At minimum, a Red Hat OpenShift 4 cluster requires https://docs.openshift.com/container-platform/latest/installing/installing_bare_metal/installing-bare-metal.html#installation-minimum-resource-requirements_installing-bare-metal[3 control plane nodes with 4 vCPU and 16 GiB RAM and 2 worker nodes with 2 vCPU and 8 GiB RAM]. + +In contrast to the minimum requirements for a Red Hat OpenShift 4 cluster, the base configuration of an APPUiO Managed OpenShift 4 cluster has https://docs.openshift.com/container-platform/latest/nodes/nodes/nodes-nodes-creating-infrastructure-nodes.html[4 infrastructure nodes with 4 vCPU and 16 GiB RAM] and 3 worker nodes with 4 vCPU and 16 GiB RAM in addition to the required 3 control plane nodes with 4 vCPU and 16 GiB RAM. + +APPUiO Managed OpenShift 4 deploys infrastructure nodes by default because an OpenShift 4 cluster with infrastructure nodes supports better separation of non-control plane infrastructure components (monitoring stack, logging stack, integrated image registry, etc.) from customer applications. +Additionally, infrastructure nodes https://access.redhat.com/solutions/5034771[don't incur Red Hat OpenShift 4 license fees] as long as only approved infrastructure components are scheduled on those nodes. + +All nodes are installed with https://docs.openshift.com/container-platform/latest/architecture/architecture-rhcos.html[Red Hat CoreOS]. + +=== Networking All APPUiO Managed OpenShift 4 clusters use https://cilium.io[Cilium] as the network plugin. APPUiO Managed OpenShift 4 uses the default OpenShift IP CIDRs for the pod and service networks: 10.128.0.0/14 for the pod network and 172.30.0.0/24 for the service network. @@ -13,12 +35,11 @@ However, a default IP CIDR for the machine network is defined for some infrastru Each cluster is configured with two floating IPs, one for the Kubernetes API (the "API VIP") and one for the ingress router (the "Ingress VIP"). On most providers, a third floating IP (the "Egress VIP") is configured and all outgoing traffic from the machine network is SNATed to this IP. -.APPUiO Managed OpenShift 4 base architecture -image::ocp4-architecture.svg[alt=OCP4 Architecture, width=640] - https://syn.tools[Project Syn] is used to configure APPUiO Managed OpenShift 4 clusters. Apart from deploying custom configurations for OpenShift components, Project Syn is also used to deploy multiple third party components, such as https://cilium.io[Cilium] and https://cert-manager.io[cert-manager] on each APPUiO Managed OpenShift 4 cluster. +== External connectivity + Each APPUiO Managed OpenShift 4 cluster depends on multiple external systems. Some of those systems are owned and operated by VSHN while others are operated by Red Hat. Additionally, APPUiO Managed OpenShift 4 makes use of some SaaS services, such as https://letsencrypt.org/[Let's Encrypt], https://www.passbolt.com/[Passbolt], various container registries, and https://www.atlassian.com/software/opsgenie[OpsGenie]. @@ -27,8 +48,19 @@ The mandatory external systems are shown in the diagram below. .APPUiO Managed OpenShift 4 external dependencies image::ocp4-external-dependencies.svg[alt=OCP4 external dependencies, width=800] +IMPORTANT: If either Passbolt or OpsGenie aren't reachable from the environment in which the cluster is deployed, VSHN can't provide ongoing operational support. + +== APPUiO Managed OpenShift 4 architecture for supported infrastructure providers + The architecture documentation for the supported infrastructure providers provides details on how the architecture is adapted to work within the constraints imposed by the provider. * cloudscale.ch (coming soon) * Exoscale (coming soon) * VMWare vSphere (coming soon) + +Additionally, the https://products.vshn.ch/appuio/managed/ocp4.html[APPUiO Managed OpenShift 4 product documentation] provides more details on the minimum required resources, supported Red Hat OpenShift 4 features and components as well as optionally supported features and add-ons for each supported infrastructure provider. + +== References + +* https://access.redhat.com/documentation/en-us/openshift_container_platform/4.14/html/architecture/index[Red Hat OpenShift 4 architecture] +* https://products.vshn.ch/appuio/managed/ocp4.html[APPUiO Managed OpenShift 4 product documentation] From e4995ecd92c1a7e2fb8354fba4fd2707f781f816 Mon Sep 17 00:00:00 2001 From: Simon Gerber Date: Wed, 29 Nov 2023 13:23:44 +0100 Subject: [PATCH 3/4] Add section on user access --- .../modules/ROOT/assets/images/ocp4-architecture.svg | 2 +- .../ROOT/pages/references/architecture/index.adoc | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/docs/modules/ROOT/assets/images/ocp4-architecture.svg b/docs/modules/ROOT/assets/images/ocp4-architecture.svg index 9b8fb574..94464986 100644 --- a/docs/modules/ROOT/assets/images/ocp4-architecture.svg +++ b/docs/modules/ROOT/assets/images/ocp4-architecture.svg @@ -1,4 +1,4 @@ -
Cluster Machine Network (/24, requires DHCP)
Cluster Machine Network (/24, requires DHCP)
OpenShift 4 Control plane nodes
3x 4vCPU/16GiB RAM
OpenShift 4 Control pla...
OpenShift 4 infrastructure nodes
4x 4vCPU/16GiB RAM
OpenShift 4 infrastruct...
OpenShift 4 worker nodes
3x 4vCPU/16GiB RAM
OpenShift 4 worker node...
Pod Network – Cilium VXLAN – 10.128.0.0/14
Pod Network – Cilium VXLAN – 10.128.0.0/14
Kubernetes API
Kubernetes API
Scheduler
Scheduler
Controllers
Controllers
OpenShift API
OpenShift API
etcd
etcd
Ingress Router
Ingress Router
Monitoring
Monitoring
Registry
Registry
Logging
Logging
...
...
Application A
Application A
Application B
Application B
Application C
Application C
6443/tcp
6443/tcp
API VIP
API VIP
80/tcp 443/tcp
80/tcp 443/tcp
Ingress VIP
Ingress VIP
Egress VIP
Egress VIP
SNAT
SNAT
SNAT
SNAT
SNAT
SNAT
Service Network – Cilium eBPF rules – 172.30.0.0/16
Service Network – Cilium eBPF rules – 172.30.0.0/16Text is not SVG - cannot display \ No newline at end of file +
Cluster Machine Network (/24, requires DHCP)
Cluster Machine Network (/24, requires DHCP)
OpenShift 4 Control plane nodes
3x 4vCPU/16GiB RAM
OpenShift 4 Control pla...
OpenShift 4 infrastructure nodes
4x 4vCPU/16GiB RAM
OpenShift 4 infrastruct...
OpenShift 4 worker nodes
3x 4vCPU/16GiB RAM
OpenShift 4 worker node...
Pod Network – Cilium VXLAN – 10.128.0.0/14
Pod Network – Cilium VXLAN – 10.128.0.0/14
Application A
Application A
Application B
Application B
Application C
Application C
6443/tcp
6443/tcp
API VIP
API VIP
80/tcp 443/tcp
80/tcp 443/tcp
Ingress VIP
Ingress VIP
Egress VIP
Egress VIP
SNAT
SNAT
SNAT
SNAT
SNAT
SNAT
Service Network – Cilium eBPF rules – 172.30.0.0/16
Service Network – Cilium eBPF rules – 172.30.0.0/16
API
API
Web
Console
Web...
VSHNeer
VSHNe...
Ingress Router
Ingress Router
Registry
Registry
Logging
Logging
Monitoring
Monitoring
Kubernetes API
Kubernetes API
...
...
OpenShift API
OpenShift API
Controllers
Controllers
Web Console
Web Console
etcd
etcd
Web
Console /
Applications
Web...
API
API
Customer Developer
Custo...
Applications
Applications
Application User
Appli...Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/modules/ROOT/pages/references/architecture/index.adoc b/docs/modules/ROOT/pages/references/architecture/index.adoc index 87a59d3c..7ea51b4b 100644 --- a/docs/modules/ROOT/pages/references/architecture/index.adoc +++ b/docs/modules/ROOT/pages/references/architecture/index.adoc @@ -50,6 +50,18 @@ image::ocp4-external-dependencies.svg[alt=OCP4 external dependencies, width=800] IMPORTANT: If either Passbolt or OpsGenie aren't reachable from the environment in which the cluster is deployed, VSHN can't provide ongoing operational support. +== User access + +There's three different personas accessing an APPUiO Managed OpenShift 4 cluster: + +1. VSHNeers access the cluster in order to operate and maintain it. +VSHNeers access the cluster's API and the OpenShift web console to perform adminstrative tasks. +2. Customer developers access the cluster in order to develop, operate and maintain the customer's applications. +The customer's developers access the cluster's API and the OpenShift web console to operate and maintain the customer's applications. +Additionally, the customer's developers access the customer's applications. +3. Application users access the applications on the cluster. +Application users don't need to know or care about the fact that the applications are running on an APPUiO Managed OpenShift 4 cluster. + == APPUiO Managed OpenShift 4 architecture for supported infrastructure providers The architecture documentation for the supported infrastructure providers provides details on how the architecture is adapted to work within the constraints imposed by the provider. From 4cf2ec18c8c0b1c9cd6646733048653eb98132e2 Mon Sep 17 00:00:00 2001 From: Aline Abler Date: Fri, 1 Dec 2023 14:37:46 +0100 Subject: [PATCH 4/4] Minor language fixes --- docs/modules/ROOT/pages/references/architecture/index.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/modules/ROOT/pages/references/architecture/index.adoc b/docs/modules/ROOT/pages/references/architecture/index.adoc index 7ea51b4b..949cb872 100644 --- a/docs/modules/ROOT/pages/references/architecture/index.adoc +++ b/docs/modules/ROOT/pages/references/architecture/index.adoc @@ -52,7 +52,7 @@ IMPORTANT: If either Passbolt or OpsGenie aren't reachable from the environment == User access -There's three different personas accessing an APPUiO Managed OpenShift 4 cluster: +There are three different personas accessing an APPUiO Managed OpenShift 4 cluster: 1. VSHNeers access the cluster in order to operate and maintain it. VSHNeers access the cluster's API and the OpenShift web console to perform adminstrative tasks. @@ -64,7 +64,7 @@ Application users don't need to know or care about the fact that the application == APPUiO Managed OpenShift 4 architecture for supported infrastructure providers -The architecture documentation for the supported infrastructure providers provides details on how the architecture is adapted to work within the constraints imposed by the provider. +The architecture documentation for the supported infrastructure providers can give more details on how the architecture is adapted to work within the constraints imposed by the respective provider. * cloudscale.ch (coming soon) * Exoscale (coming soon)