Intel AMX accelerated patterns (Multicloud GitOps & Medical Diagnosis) (

#376)

* Intel AMX accelerated patterns (Multicloud GitOps & Medical Diagnosis) documentation.

Co-authored-by: Izabela Irzynska <[email protected]>
Co-authored-by: Paulina Olszewska <[email protected]>
Co-authored-by: Igor Marzynski <[email protected]>
Co-authored-by: Piotr Grabuszynski <[email protected]>

* Using module names instead of full names in _index.adoc - removing target reference from intro, updated attributes for md-amx

* added NFD to hardware list, fixed spelling typo

* updated repo urls

* updated repo urls

---------

Co-authored-by: Izabela Irzynska <[email protected]>
Co-authored-by: Paulina Olszewska <[email protected]>
Co-authored-by: Igor Marzynski <[email protected]>
Co-authored-by: day0hero <[email protected]>

content/patterns/medical-diagnosis-amx/_index.adoc

@@ -0,0 +1,100 @@
+---
+title: Intel AMX accelerated Medical Diagnosis
+date: 2023-10-10
+validated: false
+summary: This pattern is based on a demo implementation of an automated data pipeline for chest X-ray analysis previously developed by Red Hat. The pattern is modified to utilize Intel AMX feature.
+products:
+- Red Hat OpenShift Container Platform
+- Red Hat OpenShift Serverless
+- Red Hat OpenShift Data Foundation
+- Red Hat OpenShift Node Feature Discovery
+- 4th Gen Intel Xeon Scalable processors with Intel Advanced Matrix Extensions (Intel AMX)
+industries:
+- medical
+aliases: /medical-diagnosis-amx/
+pattern_logo: medical-diagnosis.png
+links:
+  install: getting-started
+  arch: https://www.redhat.com/architect/portfolio/architecturedetail?ppid=6
+  help: https://groups.google.com/g/hybrid-cloud-patterns
+  bugs: https://github.com/validatedpatterns-sandbox/amx-accelerated-medical-diagnosis/issues
+---
+
+include::modules/comm-attributes.adoc[]
+
+:toc:
+:imagesdir: /images
+:_content-type: ASSEMBLY
+
+//Module to be included
+//:_content-type: CONCEPT
+//:imagesdir: ../../images
+[id="about-med-diag-pattern"]
+= About the {med-pattern}
+
+Background::
+
+This validated pattern is a modified version of the Medical Diagnosis pattern. It was extended to showcase *{intel-4th-gen-xeon-processors}* capabilities, especially *link:https://www.intel.com/content/www/us/en/products/docs/accelerator-engines/advanced-matrix-extensions/overview.html[{intel-amx}]* that speeds up AI workloads.
+The pattern is based on a demo implementation of an automated data pipeline for chest X-ray analysis that was previously developed by Red Hat. It includes the same functionality as the original demonstration, but uses the GitOps framework to deploy the pattern including Operators, creation of namespaces, and cluster configuration.
+
+Compared to the original Medical Diagnosis pattern, this one was extended by the Node Feature Discovery Operator, whose task is to detect hardware features and expose them as labels for this node.
+Red Hat OpenShift Serverless component is modified to assign AI workload to nodes with available {intel-amx} feature.
+
+Moreover, the Machine Learning model was quantized to int8 precision to improve efficiency thanks to *{intel-amx}*, with only marginal accuracy loss.
+
+This pattern was also adapted to run in on-premise environments.
+
+Workflow::
+
+* Node Feature Discovery operator labels nodes with {intel-amx} capabilities.
+* Ingest chest X-rays from a simulated X-ray machine and puts them into an `objectStore` based on Ceph.
+* The `objectStore` sends a notification to a Kafka topic.
+* A KNative Eventing listener to the topic triggers a KNative Serving function.
+* KNative Serving was modified to schedule pods with AI workload only on nodes with enabled {intel-amx} feature
+* An ML-trained model running in a container makes a risk assessment of Pneumonia for incoming images.
+* A Grafana dashboard displays the pipeline in real time, along with images incoming, processed, anonymized, and full metrics collected from Prometheus.
+
+The simplified pipeline without {intel-amx} is showcased link:https://www.youtube.com/watch?v=zja83FVsm14[in this video].
+
+image::medical-diagnosis-amx/dashboard.png[link="/images/medical-diagnosis-amx/dashboard.png"]
+
+[id="about-solution-med"]
+== About the solution elements
+
+The solution aids the understanding of the following:
+
+* How to use a GitOps approach to keep in control of configuration and operations.
+* How to deploy AI/ML technologies for medical diagnosis using GitOps.
+
+The {med-pattern} uses the following products and technologies:
+
+* {rh-ocp} for container orchestration
+* {rh-gitops}, a GitOps continuous delivery (CD) solution
+* {rh-amq-first}, an event streaming platform based on the Apache Kafka
+* {rh-serverless-first} for event-driven applications
+* {rh-ocp-data-first} for cloud native storage capabilities
+* {grafana-op} to manage and share Grafana dashboards, data sources, and so on
+* {rh-nfd} to label nodes with {intel-amx} capabilities
+
+
+[id="about-architecture-med"]
+== About the architecture
+
+[IMPORTANT]
+====
+Presently, the {amx-med-pattern} does not have an edge component. Edge deployment capabilities are planned as part of the pattern architecture for a future release.
+====
+
+image::medical-diagnosis-amx/edge-medical-diagnosis-marketing-slide.png[link="/images/medical-diagnosis-amx/edge-medical-diagnosis-marketing-slide.png"]
+
+Components are running on OpenShift either at the data center, at the medical facility, or public cloud running OpenShift.
+
+The diagram below shows the components that are deployed with the the data flows and API calls between them.
+
+image::medical-diagnosis-amx/physical-dataflow.png[link="/images/medical-diagnosis-amx/physical-dataflow.png"]
+
+[id="next-steps_med-diag-index"]
+== Next steps
+
+* Getting started link:getting-started[Deploy the Pattern]
+//We have relevant links on the patterns page

content/patterns/medical-diagnosis-amx/cluster-sizing.adoc

@@ -0,0 +1,76 @@
+---
+title: Cluster Sizing
+weight: 20
+aliases: /medical-diagnosis/cluster-sizing/
+---
+
+include::modules/comm-attributes.adoc[]
+
+:toc:
+:imagesdir: /images
+:_content-type: ASSEMBLY
+
+//Module to be included
+//:_content-type: CONCEPT
+//:imagesdir: ../../images
+[id="about-openshift-cluster-sizing-med"]
+= About OpenShift cluster sizing for the {amx-med-pattern}
+
+The minimum requirements for an {ocp} cluster depend on your installation platform, for example:
+
+* For AWS, see link:https://docs.openshift.com/container-platform/4.13/installing/installing_aws/preparing-to-install-on-aws.html#requirements-for-installing-ocp-on-aws[Installing {ocp} on AWS].
+
+* For bare-metal, see link:https://docs.openshift.com/container-platform/4.13/installing/installing_bare_metal/installing-bare-metal.html#installation-minimum-resource-requirements_installing-bare-metal[Installing {ocp} on bare metal].
+
+To understand cluster sizing requirements for the {amx-med-pattern}, consider the following components that the {amx-med-pattern} deploys on the datacenter or the hub OpenShift cluster:
+
+|===
+| Name | Kind | Namespace | Description
+
+| Medical Diagnosis Hub
+| Application
+| medical-diagnosis-hub
+| Hub GitOps management
+
+| {rh-gitops}
+| Operator
+| openshift-operators
+| {rh-gitops-short}
+
+| {rh-ocp-data-first}
+| Operator
+| openshift-storage
+| Cloud Native storage solution
+
+| {rh-amq-streams}
+| Operator
+| openshift-operators
+| AMQ Streams provides Apache Kafka access
+
+| {rh-serverless-first}
+| Operator
+| - knative-serving (knative-eventing)
+| Provides access to Knative Serving and Eventing functions
+
+| {rh-nfd}
+| Operator
+| openshift-nfd
+| Manages the detection and labeling of hardware features and configuration (for example {intel-amx})
+|===
+
+//AI: Removed the following since we have CI status linked on the patterns page
+//[id="tested-platforms-cluster-sizing"]
+//== Tested Platforms
+
+The {amx-med-pattern} also includes the Red Hat Advanced Cluster Management (RHACM) supporting operator that is installed by OpenShift GitOps using Argo CD.
+
+For information about requirements for additional platforms, see link:https://docs.openshift.com/container-platform/4.13/installing/installing-preparing.html[{ocp} documentation].
+
+[id="med-openshift-cluster-size"]
+=== About {amx-med-pattern} OpenShift cluster size
+
+The OpenShift cluster is a standard deployment of 3 control plane nodes and 3 or more worker nodes.
+
+The recommended hardware setup:
+
+include::modules/intel-recommended-cluster-sizing.adoc[]