Merge pull request containers#227 from rhatdan/man

Update README to mention Makefile variables
mhdawson · Apr 10, 2024 · b86c80f · b86c80f
2 parents f4ea876 + 5b90ebb
commit b86c80f
Show file tree

Hide file tree

Showing 7 changed files with 222 additions and 25 deletions.
diff --git a/recipes/audio/audio_to_text/README.md b/recipes/audio/audio_to_text/README.md
@@ -16,7 +16,7 @@
 
 # Build the Application
 
-In order to build this application we will need a model, a Model Service and an AI Application.  
+In order to build this application we will need a model, a Model Service and an AI Application.
 
 * [Download a model](#download-a-model)
 * [Build the Model Service](#build-the-model-service)
@@ -31,17 +31,17 @@ In order to build this application we will need a model, a Model Service and an
 If you are just getting started, we recommend using [ggerganov/whisper.cpp](https://huggingface.co/ggerganov/whisper.cpp).
 This is a well performant model with an MIT license.
 It's simple to download a pre-converted whisper model from [huggingface.co](https://huggingface.co)
-here: https://huggingface.co/ggerganov/whisper.cpp. There are a number of options, but we recommend to start with `ggml-small.bin`. 
+here: https://huggingface.co/ggerganov/whisper.cpp. There are a number of options, but we recommend to start with `ggml-small.bin`.
 
 The recommended model can be downloaded using the code snippet below:
 
 ```bash
 cd models
-wget https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin 
+wget https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin
 cd ../
 ```
 
-_A full list of supported open models is forthcoming._  
+_A full list of supported open models is forthcoming._
 
 
 ### Build the Model Service
@@ -58,12 +58,12 @@ podman build -t whispercppserver .
 The local Model Service relies on a volume mount to the localhost to access the model files. You can start your local Model Service using the following Podman command:
 ```
 podman run --rm -it \
-        -p 8001:8001 \
-        -v Local/path/to/locallm/models:/locallm/models \
-        -e MODEL_PATH=models/<model-filename> \
-        -e HOST=0.0.0.0 \
-        -e PORT=8001 \
-        whispercppserver
+	-p 8001:8001 \
+	-v Local/path/to/locallm/models:/locallm/models \
+	-e MODEL_PATH=models/<model-filename> \
+	-e HOST=0.0.0.0 \
+	-e PORT=8001 \
+	whispercppserver
 ```
 
 ### Build the AI Application
@@ -92,7 +92,7 @@ Once the streamlit application is up and running, you should be able to access i
 From here, you can upload audio files from your local machine and translate the audio files as shown below.
 
 By using this recipe and getting this starting point established,
-users should now have an easier time customizing and building their own LLM enabled applications.   
+users should now have an easier time customizing and building their own LLM enabled applications.
 
 #### Input audio files
 
@@ -102,3 +102,80 @@ To convert your input audio files to 16-bit WAV format you can use `ffmpeg` like
 ```bash
 ffmpeg -i <input.mp3> -ar 16000 -ac 1 -c:a pcm_s16le <output.wav>
 ```
+
+### Deploy the AI Application
+
+Make sure the Model Service is up and running before starting this container image. When starting the AI Application container image we need to direct it to the correct `MODEL_SERVICE_ENDPOINT`. This could be any appropriately hosted Model Service (running locally or in the cloud) using an OpenAI compatible API. In our case the Model Service is running inside the Podman machine so we need to provide it with the appropriate address `10.88.0.1`. The following Podman command can be used to run your AI Application:
+
+```bash
+podman run --rm -it -p 8501:8501 -e MODEL_SERVICE_ENDPOINT=http://10.88.0.1:8001/v1 codegen
+```
+
+### Interact with the AI Application
+
+Everything should now be up an running with the chat application available at [`http://localhost:8501`](http://localhost:8501). By using this recipe and getting this starting point established, users should now have an easier time customizing and building their own LLM enabled code generation applications.
+
+_Note: Future recipes will demonstrate integration between locally hosted LLM's and developer productivity tools like VSCode._
+
+### Embed the AI Application in a Bootable Container Image
+
+To build a bootable container image that includes this sample chatbot workload as a service that starts when a system is booted, cd into this folder
+and run:
+
+
+```
+make BOOTC_IMAGE=quay.io/your/codegen-bootc:latest bootc
+```
+
+Substituting the bootc/Containerfile FROM command is simple using the Makefile FROM option.
+
+```
+make FROM=registry.redhat.io/rhel9-beta/rhel-bootc:9.4 BOOTC_IMAGE=quay.io/your/codegen-bootc:latest bootc
+```
+
+The magic happens when you have a bootc enabled system running. If you do, and you'd like to update the operating system to the OS you just built
+with the codegen application, it's as simple as ssh-ing into the bootc system and running:
+
+```
+bootc switch quay.io/your/codegen-bootc:latest
+```
+
+Upon a reboot, you'll see that the codegen service is running on the system.
+
+Check on the service with
+
+```
+ssh user@bootc-system-ip
+sudo systemctl status codegen
+```
+
+#### What are bootable containers?
+
+What's a [bootable OCI container](https://containers.github.io/bootc/) and what's it got to do with AI?
+
+That's a good question! We think it's a good idea to embed AI workloads (or any workload!) into bootable images at _build time_ rather than
+at _runtime_. This extends the benefits, such as portability and predictability, that containerizing applications provides to the operating system.
+Bootable OCI images bake exactly what you need to run your workloads into the operating system at build time by using your favorite containerization
+tools. Might I suggest [podman](https://podman.io/)?
+
+Once installed, a bootc enabled system can be updated by providing an updated bootable OCI image from any OCI
+image registry with a single `bootc` command. This works especially well for fleets of devices that have fixed workloads - think
+factories or appliances. Who doesn't want to add a little AI to their appliance, am I right?
+
+Bootable images lend toward immutable operating systems, and the more immutable an operating system is, the less that can go wrong at runtime!
+
+##### Creating bootable disk images
+
+You can convert a bootc image to a bootable disk image using the
+[quay.io/centos-bootc/bootc-image-builder](https://github.com/osbuild/bootc-image-builder) container image.
+
+This container image allows you to build and deploy [multiple disk image types](../../common/README_bootc_image_builder.md) from bootc container images.
+
+Default image types can be set via the DISK_TYPE Makefile variable.
+
+`make bootc-image-builder DISK_TYPE=ami`
+
+### Makefile variables
+
+There are several [Makefile variables](../../common/README.md) defined within each `recipe` Makefile which can be
+used to override defaults for a variety of make targets.
diff --git a/recipes/common/README.md b/recipes/common/README.md
@@ -0,0 +1,41 @@
+# Makefile targets
+
+.PHONY: check-model-in-path
+
+| Target               | Description								       |
+|----------------------|-------------------------------------------------------------------------------|
+|bootc                 | Create bootable container image for the application			       |
+|bootc-image-builder   | Create diskimage from your bootc image to be run on a VM or physical hardware |
+|build                 | Build container image to run your app using Containerfile in app directory    |
+|clean                 | Remove contents of the build directory                                        |
+|quadlet               | Modify quadlet files into the build dir                                       |
+|run                   | Run containerizied app as a container                                         |
+| install-chromedriver | Used to testing purposes                                                      |
+| install-chrome       | Used for testing purposes                                                     |
+
+
+# Makefile variables
+
+Makefile variables defined within each `recipe` Makefile which can be
+used to override defaults for a variety of make targets.
+
+| Variable           | Description                                          | Default                                                 |
+|--------------------|------------------------------------------------------|---------------------------------------------------------|
+|REGISTRY            | Container Registry for storing container images      | `quay.io`						      |
+|REGISTRY_ORG        | Containwer Registry organization 	      	    | `ai-lab`						      |
+|IMAGE_NAME          | App image and registry organization            	    | `$(REGISTRY_ORG)/${APP}:latest`			      |
+|APP_IMAGE           | Application image to be built                  	    | `$(REGISTRY)/$(IMAGE_NAME)` 			      |
+|BOOTC_IMAGE         | Bootc image name                               	    | `quay.io/$(REGISTRY_ORG)/${APP}-bootc:latest`	      |
+|BOOTC_IMAGE_BUILDER | Bootc Image Builder container image 	      	    | `quay.io/centos-bootc/bootc-image-builder`	      |
+|CHROMADB_IMAGE      | ChromaDB image to be used for application      	    | `$(REGISTRY)/$(REGISTRY_ORG)/chromadb:latest`	      |
+|DISK_TYPE           | Disk type to be created by BOOTC_IMAGE_BUILDER 	    | `qcow2` (Options: ami, iso, vmdk, raw)		      |
+|MODEL_IMAGE 	     | AI Model to be used by application             	    | `$(REGISTRY)/$(REGISTRY_ORG)/mistral-7b-instruct:latest`|
+|SERVER_IMAGE 	     | AI Model Server Application                    	    | `$(REGISTRY)/$(REGISTRY_ORG)/llamacpp-python:latest`    |
+|SSH_PUBKEY 	     | SSH Public key preloaded in bootc image.             | `$(shell cat ${HOME}/.ssh/id_rsa.pub;)`		      |
+|FROM 		     | Overrides first FROM instruction within Containerfile| `FROM` line defined in the Containerfile		      |
+|ARCH 		     | Use alternate arch for image build                   | Current Arch					      |
+|CONTAINERFILE 	     | Use alternate Containfile for image build            | Containerfile (Containerfile.nocache)		      |
+
+Examples
+
+make bootc FROM=registry.redhat.io/rhel9-beta/rhel-bootc:9.4 APP_IMAGE=quay.io/myorg/chatbot-bootc
diff --git a/recipes/common/README_bootc_image_builder.md b/recipes/common/README_bootc_image_builder.md
@@ -0,0 +1,16 @@
+This tools allows you to build and deploy disk-images from bootc container inputs.
+
+The following image disk types are currently available:
+
+| Image type            | Target environment                                                                    |
+|-----------------------|---------------------------------------------------------------------------------------|
+| `ami`                 | [Amazon Machine Image](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) |
+| `qcow2` **(default)** | [QEMU](https://www.qemu.org/)                                                         |
+| `vmdk`                | [VMDK](https://en.wikipedia.org/wiki/VMDK) usable in vSphere, among others            |
+| `anaconda-iso`        | An unattended Anaconda installer that installs to the first disk found.               |
+| `raw`                 | Unformatted [raw disk](https://en.wikipedia.org/wiki/Rawdisk).                        |
+
+The recipe Makefile can be used to automatically generate a disk image from the bootc image. The following
+command will create an qcow2 image file from the default bootc image in the build subdirectory.
+
+`make bootc-image-builder DISK_TYPE=qcow2`
diff --git a/recipes/natural_language_processing/chatbot/README.md b/recipes/natural_language_processing/chatbot/README.md
@@ -101,15 +101,15 @@ Everything should now be up an running with the chat application available at [`
 
 ### Embed the AI Application in a Bootable Container Image
 
-To build a bootable container image that includes this sample chatbot workload as a service that starts when a system is booted, run: `make -f Makefile bootc`. You can optionally override the default image / tag you want to give the make command by specifiying it as follows: `make -f Makefile BOOTC_IMAGE=<your_bootc_image> bootc`.
+To build a bootable container image that includes this sample chatbot workload as a service that starts when a system is booted, run: `make -f Makefile bootc`. You can optionally override the default image / tag you want to give the make command by specifying it as follows: `make -f Makefile BOOTC_IMAGE=<your_bootc_image> bootc`.
 
 Substituting the bootc/Containerfile FROM command is simple using the Makefile FROM option.
 
 ```bash
 make FROM=registry.redhat.io/rhel9-beta/rhel-bootc:9.4 bootc
 ```
 
-Selecting the ARCH for the bootc/Containerfile is simple using the Makefile ARCH= option.
+Selecting the ARCH for the bootc/Containerfile is simple using the Makefile ARCH= variable.
 
 ```
 make ARCH=x86_64 bootc
@@ -143,3 +143,19 @@ image registry with a single `bootc` command. This works especially well for fle
 factories or appliances. Who doesn't want to add a little AI to their appliance, am I right?
 
 Bootable images lend toward immutable operating systems, and the more immutable an operating system is, the less that can go wrong at runtime!
+
+##### Creating bootable disk images
+
+You can convert a bootc image to a bootable disk image using the
+[quay.io/centos-bootc/bootc-image-builder](https://github.com/osbuild/bootc-image-builder) container image.
+
+This container image allows you to build and deploy [multiple disk image types](../../common/README_bootc_image_builder.md) from bootc container images.
+
+Default image types can be set via the DISK_TYPE Makefile variable.
+
+`make bootc-image-builder DISK_TYPE=ami`
+
+### Makefile variables
+
+There are several [Makefile variables](../../common/README.md) defined within each `recipe` Makefile which can be
+used to override defaults for a variety of make targets.
diff --git a/recipes/natural_language_processing/codegen/README.md b/recipes/natural_language_processing/codegen/README.md
@@ -4,15 +4,15 @@ This demo provides a simple recipe to help developers start building out their o
 
 There are a few options today for local Model Serving, but this recipe will use [`llama-cpp-python`](https://github.com/abetlen/llama-cpp-python) and their OpenAI compatible Model Service. There is a Containerfile provided that can be used to build this Model Service within the repo, [`model_servers/llamacpp_python/base/Containerfile`](/model_servers/llamacpp_python/base/Containerfile).
 
-Our AI Application will connect to our Model Service via it's OpenAI compatible API. In this example we rely on [Langchain's](https://python.langchain.com/docs/get_started/introduction) python package to simplify communication with our Model Service and we use [Streamlit](https://streamlit.io/) for our UI layer. Below please see an example of the code generation application.               
+Our AI Application will connect to our Model Service via it's OpenAI compatible API. In this example we rely on [Langchain's](https://python.langchain.com/docs/get_started/introduction) python package to simplify communication with our Model Service and we use [Streamlit](https://streamlit.io/) for our UI layer. Below please see an example of the code generation application.
 
 
-![](/assets/codegen_ui.png) 
+![](/assets/codegen_ui.png)
 
 
 # Build the Application
 
-In order to build this application we will need a model, a Model Service and an AI Application.  
+In order to build this application we will need a model, a Model Service and an AI Application.
 
 * [Download a model](#download-a-model)
 * [Build the Model Service](#build-the-model-service)
@@ -30,7 +30,7 @@ and quantized into the [GGUF format](https://github.com/ggerganov/ggml/blob/mast
 ways to get a GGUF version of Mistral-7B, but the simplest is to download a pre-converted one from
 [huggingface.co](https://huggingface.co) here: https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF.
 
-There are a number of options for quantization level, but we recommend `Q4_K_M`. 
+There are a number of options for quantization level, but we recommend `Q4_K_M`.
 
 The recommended model can be downloaded using the code snippet below:
 
@@ -40,7 +40,7 @@ wget https://huggingface.co/TheBloke/Mistral-7B-Code-16K-qlora-GGUF/resolve/main
 cd ../
 ```
 
-_A full list of supported open models is forthcoming._  
+_A full list of supported open models is forthcoming._
 
 
 ### Build the Model Service
@@ -62,12 +62,12 @@ The complete instructions for building and deploying the Model Service can be fo
 The local Model Service relies on a volume mount to the localhost to access the model files. You can start your local Model Service using the following Podman command:
 ```
 podman run --rm -it \
-        -p 8001:8001 \
-        -v Local/path/to/locallm/models:/locallm/models \
-        -e MODEL_PATH=models/<model-filename> \
-        -e HOST=0.0.0.0 \
-        -e PORT=8001 \
-        llamacppserver
+	-p 8001:8001 \
+	-v Local/path/to/locallm/models:/locallm/models \
+	-e MODEL_PATH=models/<model-filename> \
+	-e HOST=0.0.0.0 \
+	-e PORT=8001 \
+	llamacppserver
 ```
 
 ### Build the AI Application
@@ -85,7 +85,7 @@ podman build -t codegen app
 Make sure the Model Service is up and running before starting this container image. When starting the AI Application container image we need to direct it to the correct `MODEL_SERVICE_ENDPOINT`. This could be any appropriately hosted Model Service (running locally or in the cloud) using an OpenAI compatible API. In our case the Model Service is running inside the Podman machine so we need to provide it with the appropriate address `10.88.0.1`. The following Podman command can be used to run your AI Application:
 
 ```bash
-podman run --rm -it -p 8501:8501 -e MODEL_SERVICE_ENDPOINT=http://10.88.0.1:8001/v1 codegen   
+podman run --rm -it -p 8501:8501 -e MODEL_SERVICE_ENDPOINT=http://10.88.0.1:8001/v1 codegen
 ```
 
 ### Interact with the AI Application
@@ -140,3 +140,19 @@ image registry with a single `bootc` command. This works especially well for fle
 factories or appliances. Who doesn't want to add a little AI to their appliance, am I right?
 
 Bootable images lend toward immutable operating systems, and the more immutable an operating system is, the less that can go wrong at runtime!
+
+##### Creating bootable disk images
+
+You can convert a bootc image to a bootable disk image using the
+[quay.io/centos-bootc/bootc-image-builder](https://github.com/osbuild/bootc-image-builder) container image.
+
+This container image allows you to build and deploy [multiple disk image types](../../common/README_bootc_image_builder.md) from bootc container images.
+
+Default image types can be set via the DISK_TYPE Makefile variable.
+
+`make bootc-image-builder DISK_TYPE=ami`
+
+### Makefile variables
+
+There are several [Makefile variables](../../common/README.md) defined within each `recipe` Makefile which can be
+used to override defaults for a variety of make targets.
diff --git a/recipes/natural_language_processing/rag/README.md b/recipes/natural_language_processing/rag/README.md
@@ -197,3 +197,19 @@ image registry with a single `bootc` command. This works especially well for fle
 factories or appliances. Who doesn't want to add a little AI to their appliance, am I right?
 
 Bootable images lend toward immutable operating systems, and the more immutable an operating system is, the less that can go wrong at runtime!
+
+##### Creating bootable disk images
+
+You can convert a bootc image to a bootable disk image using the
+[quay.io/centos-bootc/bootc-image-builder](https://github.com/osbuild/bootc-image-builder) container image.
+
+This container image allows you to build and deploy [multiple disk image types](../../common/README_bootc_image_builder.md) from bootc container images.
+
+Default image types can be set via the DISK_TYPE Makefile variable.
+
+`make bootc-image-builder DISK_TYPE=ami`
+
+### Makefile variables
+
+There are several [Makefile variables](../../common/README.md) defined within each `recipe` Makefile which can be
+used to override defaults for a variety of make targets.
diff --git a/recipes/natural_language_processing/summarizer/README.md b/recipes/natural_language_processing/summarizer/README.md
@@ -169,3 +169,18 @@ factories or appliances. Who doesn't want to add a little AI to their appliance,
 
 Bootable images lend toward immutable operating systems, and the more immutable an operating system is, the less that can go wrong at runtime!
 
+##### Creating bootable disk images
+
+You can convert a bootc image to a bootable disk image using the
+[quay.io/centos-bootc/bootc-image-builder](https://github.com/osbuild/bootc-image-builder) container image.
+
+This container image allows you to build and deploy [multiple disk image types](../../common/README_bootc_image_builder.md) from bootc container images.
+
+Default image types can be set via the DISK_TYPE Makefile variable.
+
+`make bootc-image-builder DISK_TYPE=ami`
+
+### Makefile variables
+
+There are several [Makefile variables](../../common/README.md) defined within each `recipe` Makefile which can be
+used to override defaults for a variety of make targets.