Re-factor ArgoCD apps directory, Update README.md

Apps define as relative path

.cluster.dev/minikube-one.yaml

@@ -5,12 +5,16 @@ cluster:
     provider: aws
     region: eu-central-1
     vpc: default
-    domain: shalb.net
+# 'domain' you need to create/have it in your AWS Route53 account
+    domain: cluster.dev
   provisioner:
     type: minikube
     instanceType: m5.large
-# 'apps' are actually directories in '/kubernetes/apps' that contains ArgoCD project and application declarations
+  modules:
+    nginx-ingress: true
+    cert-manager: true
+# 'apps' are list of directories in you repo that contains ArgoCD applications and project manifests
   apps:
-    - helm-all-in-app
-    - helm-dependency
-    - raw-manifest
+    - /kubernetes/apps/samples
+
+

.vscode/settings.json

@@ -32,13 +32,15 @@
       "grafana",
       "gsub",
       "io",
+      "istio",
       "keystore",
       "kibana",
       "kubernetes",
       "kustomize",
       "letsencrypt",
       "linebreak",
       "minikube",
+      "mkdir",
       "modules",
       "nestedkey",
       "noarch",

README.md

@@ -2,59 +2,138 @@
 
 Cluster.dev is an open-source system delivered as GitHub Action or Docker Image for creating and managing Kubernetes clusters with simple manifests by GitOps approach.
 
-Designed for developers that are bored to configure Kubernetes stuff and just need: kubeconfig, dashboard, logging and monitoring out-of-the-box.
+Designed for developers that are bored to configure Kubernetes stuff and just need: kubeconfig, dashboard, logging and monitoring and deployment systems out-of-the-box.
 
-Based on DevOps and SRE best-practices. GitOps cluster management and application delivery. Simple CI/CD integration. Easily extendable by pre-configured applications and modules. Supports different Cloud Providers and Kubernetes versions.
+GitOps infrastructure management with Terraform and continuous deployment with ArgoCD. Easily extendable by pre-configured applications and modules. Quick integration with Jenkins, GitLab or  other CI/CD systems. Supports multiple Cloud Providers and Kubernetes versions.
 
 ----
 
 ## MENU
-
-* [Quick Start](#quick-start-)
-  * [Quick Start on AWS](#quick-start-on-aws-)
-  * [Cleanup](#cleanup-)
 * [Principle diagram](#principle-diagram-)
 * [How it works](#how-it-works-)
+* [Installation](#installation-)
+  * [Quick Start on AWS](#quick-start-on-aws-)
+  * [Cleanup](#cleanup-)
 * [Technical diagram](#technical-diagram-)
 * [Roadmap](#roadmap-)
 * [Contributing](#contributing-)
 
 ----
 
-## Quick Start [`↑`](#menu)
+
+## Principle diagram [`↑`](#menu)
+
+Small and understandable yaml definition:
+
+```yaml
+cluster:
+  installed: true
+  name: develop
+  cloud:
+    provider: aws
+    region: eu-central-1
+    vpc: default
+    domain: cluster.dev
+  provisioner:
+    type: minikube
+    instanceType: m5.large
+  modules:
+    nginx-ingress: true
+    cert-manager: true
+  apps:
+    - /kubernetes/apps/samples
+```
+
+`cluster.dev` turns into powerful ready to use infrastructure in code:
+
+![cluster.dev diagram](docs/images/cluster-dev-diagram.png)
+
+## How it works [`↑`](#menu)
+
+In background:
+
+* Terraform creates a "state bucket" in your Cloud Provider account where all infrastructure objects will be stored. Typically it is defined on Cloud Object Storage like AWS S3.
+* Terraform modules create Minikube/EKS/GKE/etc.. cluster, VPC and DNS zone within your Cloud Provider.
+* ArgoCD Continuous Deployment system is deployed inside Kubernetes cluster. It enables you to deploy your applications from raw manifests, helm charts or kustomize yaml's.
+* GitHub CI runner is deployed into your Kubernetes cluster and used for your apps building CI pipelines with GitHub Actions.
+
+You receive:
+
+* Automatically generated kubeconfig, ssh-access, and ArgoCD UI URLs
+* Configured: Ingress Load Balancers, Kubernetes Dashboard, Logging(ELK), Monitoring(Prometheus/Grafana)
+
+
+## Installation [`↑`](#menu)
 
 ## Quick Start on AWS [`↑`](#menu)
 
-1. Dedicate a separate repository for the infrastructure that will be managed by `cluster.dev`.  
-This repo will host code for your clusters, deployments and other resources managed by GitOps.  
-Next steps should be done in that repo.
+_normally it takes 15 minutes_  
+1. Dedicate a separate repository for the infrastructure code that will be managed by `cluster.dev`.  
+This repo will host code for your clusters, deployments, applications and other resources.  
+Clone the repo locally:
+```
+$ git clone https://github.com/YOUR-USERNAME/YOUR-REPOSITORY
+$ cd YOUR-REPOSITORY
+```
+**Next steps** should be done in that repo.
 
-2. Obtain access credentials for your non-root cloud account.  
-In AWS it is called "Programmatic Access user", and looks like:
+2. Create credentials for your non-root cloud user account.  
+In AWS you need to use the existing or create the new ones ["Programmatic Access user"](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html#id_users_create_console).   
+Required Managed policies: _AmazonEC2FullAccess, AmazonS3FullAccess, AmazonRoute53FullAccess, AmazonDynamoDBFullAccess, IAMFullAccess, AmazonEC2ContainerRegistryReadOnly, ElasticLoadBalancingFullAccess_, or you can add permissions using this [iam-json](https://gist.github.com/dgozalo/bc4b932d51f22ca5d8dad07d9a1fe0f2).
 
+Resulting access pair should look like:
 ```yaml
 aws_access_key_id = ATIAAJSXDBUVOQ4JR
 aws_secret_access_key = SuperAwsSecret
 ```
 
-3. Add credentials to you repo Secrets under GitHub's: "Settings->Secrets":
+3. Add credentials to you repo Secrets under GitHub's repo setting: `Settings → Secrets`:
  ![GitHub Secrets](docs/images/gh-secrets.png)
 
-4. Create a Github Workflow file and cluster.dev manifest with your cluster definition.
+4. In your repo, create a Github Workflow file: [.github/workflows/main.yml](.github/workflows/main.yml) and  
+ cluster.dev example manifest: [.cluster.dev/minikube-one.yaml](.cluster.dev/minikube-one.yaml) with cluster definition.
+
+_Or download example files to your local repo clone using the next commands:_
 
-**Minikube**:
 
 ```bash
-export RELEASE=v0.1.3
+# Sample with Minikube cluster
+export RELEASE=v0.1.4
 mkdir -p .github/workflows/ && wget -O .github/workflows/main.yml "https://raw.githubusercontent.com/shalb/cluster.dev/${RELEASE}/docs/quick-start/aws/github-workflow.yaml"
 mkdir -p .cluster.dev/ && wget -O .cluster.dev/minikube-one.yaml "https://raw.githubusercontent.com/shalb/cluster.dev/${RELEASE}/docs/quick-start/aws/minikube-cluster-definition.yaml"
 ```
 
-5. In cluster definition you should set your own Route53 zone (create it manually, if needed). After implementation of [#2](https://github.com/shalb/cluster.dev/issues/2) this step will not be required.  
-And you can change all other parameters or leave default values.  
-Leave github workflow file as is.
+5. In cluster definition yaml you should set your own Route53 DNS zone. If you don't have any hosted public zone you can create it manually with [instructions from AWS Website](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/CreatingHostedZone.html).  
+
+6. You can change all other parameters or leave default values in cluster.yaml .  
+Leave github workflow file as is.  
 
-6. Commit and Push files to your repo and follow the Github Action execution status. In GitHub action output you'll receive access instructions to your cluster and services.
+7. Copy sample ArgoCD applications from [/kubernetes/apps/samples](https://github.com/shalb/cluster.dev/tree/master/kubernetes/apps/samples) and Helm chart samples from [/kubernetes/charts/wordpress](https://github.com/shalb/cluster.dev/tree/master/kubernetes/charts/wordpress) to the same paths into your repo.
+
+_Or download application samples directly to local repo clone with commands:_
+```bash
+export RELEASE=v0.1.4
+# Create directory and place ArgoCD applications inside 
+mkdir -p kubernetes/apps/samples && wget -O kubernetes/apps/samples/helm-all-in-app.yaml "https://raw.githubusercontent.com/shalb/cluster.dev/${RELEASE}/kubernetes/apps/samples/helm-all-in-app.yaml"
+wget -O kubernetes/apps/samples/helm-dependency.yaml "https://raw.githubusercontent.com/shalb/cluster.dev/${RELEASE}/kubernetes/apps/samples/helm-dependency.yaml"
+wget -O kubernetes/apps/samples/raw-manifest.yaml "https://raw.githubusercontent.com/shalb/cluster.dev/${RELEASE}/kubernetes/apps/samples/raw-manifest.yaml"
+# Download sample chart which with own values.yaml
+mkdir -p kubernetes/charts/wordpress && wget -O kubernetes/charts/wordpress/Chart.yaml "https://raw.githubusercontent.com/shalb/cluster.dev/${RELEASE}/kubernetes/charts/wordpress/Chart.yaml"
+wget -O kubernetes/charts/wordpress/requirements.yaml "https://raw.githubusercontent.com/shalb/cluster.dev/${RELEASE}/kubernetes/charts/wordpress/requirements.yaml"
+wget -O kubernetes/charts/wordpress/values.yaml "https://raw.githubusercontent.com/shalb/cluster.dev/${RELEASE}/kubernetes/charts/wordpress/values.yaml"
+```
+
+Define path to ArgoCD apps in cluster manifest:  
+```
+  apps:
+    - /kubernetes/apps/samples
+```  
+
+8. Commit and Push files to your repo and follow the Github Action execution status.  
+In GitHub action output you'll receive access instructions to your cluster and services:  
+![GHA_GetCredentials](docs/images/gha_get_credentials.png)
+
+9. Voilà! You receive GitOps managed infrastructure in code. So now you can deploy applications, create more clusters, integrate with CI systems, experiment with the new features and everything from Git don't leaving your IDE.
 
 ### Cleanup [`↑`](#menu)
 
@@ -67,24 +146,6 @@ For shutdown cluster and remove all associated resources:
 
 After successful removal, you can safely delete cluster manifest file from `.cluster.dev/` directory.
 
-## Principle diagram [`↑`](#menu)
-
-![cluster.dev diagram](docs/images/cluster-dev-diagram.png)
-
-
-## How it works [`↑`](#menu)
-
-In background:
-
-* Terraform creates a "state bucket" in your Cloud Provider account where all infrastructure objects will be stored. Typically it is defined on Cloud Object Storage like AWS S3.
-* Terraform modules create Minikube/EKS/GKE/etc.. cluster, VPC and DNS zone within your Cloud Provider.
-* ArgoCD Continuous Deployment system is deployed inside Kubernetes cluster. It enables you to deploy your applications from raw manifests, helm charts or kustomize yaml's.
-* GitHub CI runner is deployed into your Kubernetes cluster and used for your apps building CI pipelines with GitHub Actions.
-
-You receive:
-
-* Automatically generated kubeconfig, ssh-access, and ArgoCD UI URLs
-* Configured: Ingress Load Balancers, Kubernetes Dashboard, Logging(ELK), Monitoring(Prometheus/Grafana)
 
 ## Technical diagram [`↑`](#menu)
 

bin/argocd.sh

@@ -21,7 +21,7 @@ function argocd::deploy_apps {
     INFO "Deploy Apps from /kubernetes/apps/<folder> into ArgoCD"
 
     for ARGO_APP_DIR in "${cluster_apps_array[@]}"; do
-        run_cmd "kubectl apply -f $GIT_REPO_ROOT/kubernetes/apps/$ARGO_APP_DIR --recursive" "" "false";
+        run_cmd "kubectl apply -f $GIT_REPO_ROOT$ARGO_APP_DIR --recursive" "" "false";
     done
 
     #TODO: enable deletion from ArgoCD application that are installed but not mentioned in target folders manifests

docs/quick-start/aws/github-workflow.yaml

@@ -17,7 +17,7 @@ jobs:
     # - uses: ./ # Uses an action in the root directory
     - name: Reconcile Clusters
       id: reconcile
-      uses: shalb/[email protected].3 # Cluster.dev release tag
+      uses: shalb/[email protected].4 # Cluster.dev release tag
       with:
         cluster-config: './.cluster.dev/'
         cloud-user: ${{ secrets.aws_access_key_id }}

terraform/aws/minikube/k8s-userdata.tpl.sh

@@ -1,3 +1,6 @@
+# Add additional DNS since AWS could delay on VPC DNS resolving and cause Cert-Manager delays on cert creation
+echo "nameserver 8.8.8.8" >> /etc/resolv.conf
+
 # Install AWS utils
 yum install -y wget unzip python3-pip.noarch
 pip3 install awscli