Installation of kubeflow 1.9 is not complete

[ACodingfreak]

Bug Description

I have 2 nodes in a microk8s cluster and I am trying to install kubeflow charm 1.9 using juju. This deployment is not complete even after 2 hours with current status

$ juju status --color | grep -E "blocked|error|maintenance|waiting|App|Unit"
App                      Version                  Status       Scale  Charm                    Channel          Rev  Address         Exposed  Message
kfp-api                                           waiting          1  kfp-api                  2.2/stable      1611  10.152.183.218  no       installing agent
kfp-db                                            waiting          1  mysql-k8s                8.0/stable       180  10.152.183.202  no       installing agent
kfp-persistence                                   waiting          1  kfp-persistence          2.2/stable      1560  10.152.183.231  no       installing agent
kfp-schedwf                                       maintenance      1  kfp-schedwf              2.2/stable      1571  10.152.183.105  no       Reconciling charm: executing component kubernetes:auth-and-crds
kfp-ui                                            waiting          1  kfp-ui                   2.2/stable      1555  10.152.183.141  no       installing agent
kubeflow-volumes                                  maintenance      1  kubeflow-volumes         1.9/stable       348  10.152.183.146  no       Reconciling charm: executing component kubernetes:auth
pvcviewer-operator                                waiting          1  pvcviewer-operator       1.9/stable       204  10.152.183.217  no       installing agent
Unit                        Workload     Agent  Address       Ports          Message
kfp-api/0*                  maintenance  idle   10.1.121.210                 Creating K8S resources
kfp-persistence/0*          blocked      idle   10.1.121.212                 [relation:kfp-api] Expected data from exactly 1 related applications - got 0.
kfp-schedwf/0*              maintenance  idle   10.1.121.214                 Reconciling charm: executing component kubernetes:auth-and-crds
kfp-ui/0*                   blocked      idle   10.1.69.153                  [relation:kfp-api] Expected data from exactly 1 related applications - got 0.
kubeflow-volumes/0*         maintenance  idle   10.1.121.222                 Reconciling charm: executing component kubernetes:auth
pvcviewer-operator/0*       maintenance  idle   10.1.69.157                  Reconciling charm: executing component pvc-viewer-pebble-service

To Reproduce

I am just running the instructions provided in

https://documentation.ubuntu.com/charmed-mlflow/en/latest/tutorial/mlflow-kubeflow/

Environment

Both the machines are following configuration
16 CPU cores
64GB Ram

Ubuntu 22.04
Microk8s 1.29
Juju 3.4
Kubeflow 1.9

Relevant Log Output

$ juju status --color | grep -E "blocked|error|maintenance|waiting|App|Unit"
App                      Version                  Status       Scale  Charm                    Channel          Rev  Address         Exposed  Message
kfp-api                                           waiting          1  kfp-api                  2.2/stable      1611  10.152.183.218  no       installing agent
kfp-db                                            waiting          1  mysql-k8s                8.0/stable       180  10.152.183.202  no       installing agent
kfp-persistence                                   waiting          1  kfp-persistence          2.2/stable      1560  10.152.183.231  no       installing agent
kfp-schedwf                                       maintenance      1  kfp-schedwf              2.2/stable      1571  10.152.183.105  no       Reconciling charm: executing component kubernetes:auth-and-crds
kfp-ui                                            waiting          1  kfp-ui                   2.2/stable      1555  10.152.183.141  no       installing agent
kubeflow-volumes                                  maintenance      1  kubeflow-volumes         1.9/stable       348  10.152.183.146  no       Reconciling charm: executing component kubernetes:auth
pvcviewer-operator                                waiting          1  pvcviewer-operator       1.9/stable       204  10.152.183.217  no       installing agent
Unit                        Workload     Agent  Address       Ports          Message
kfp-api/0*                  maintenance  idle   10.1.121.210                 Creating K8S resources
kfp-persistence/0*          blocked      idle   10.1.121.212                 [relation:kfp-api] Expected data from exactly 1 related applications - got 0.
kfp-schedwf/0*              maintenance  idle   10.1.121.214                 Reconciling charm: executing component kubernetes:auth-and-crds
kfp-ui/0*                   blocked      idle   10.1.69.153                  [relation:kfp-api] Expected data from exactly 1 related applications - got 0.
kubeflow-volumes/0*         maintenance  idle   10.1.121.222                 Reconciling charm: executing component kubernetes:auth
pvcviewer-operator/0*       maintenance  idle   10.1.69.157                  Reconciling charm: executing component pvc-viewer-pebble-service

Additional Context

No response

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-6632.

This message was autogenerated

[ACodingfreak]

If you need any specific logs for troubleshooting the same I can get it.