Map multiple vineyard objects to one volume while specifying the common prefix in the kubeflow benchmark #1578
Conversation
…mmon prefix. * Update the kubeflow benchmark result and the related tutorial. Signed-off-by: Ye Cao <[email protected]>
| @@ -14,6 +14,7 @@ Prerequisites | |||
| guide `Initialize Kubernetes Cluster`_ to create one. | |||
| - Install the `Vineyardctl`_ by following the official guide. | |||
| - Install the `Argo Workflow CLI`_ by following the official guide. | |||
| - Install the kfp package with version < 2.0.0. | |||
There was a problem hiding this comment.
Why we requires <2.0.0 here?
There was a problem hiding this comment.
The current demo is based on the kfp v1.8.0. In kfp v2, we need to rewrite the demo with the new API.
There was a problem hiding this comment.
As the kfp v2 has some breaking changes compared to v1. Backgound: https://www.kubeflow.org/docs/components/pipelines/v2/migration/
Actually, I think it doesn't matter as they are complied as the argo workflow YAML finally.
| +------------+------------------+---------------+ | ||
| | 15000 Mi | 332s | 286s | | ||
| | 15000 Mi | 252s | 298s | | ||
| +------------+------------------+---------------+ |
There was a problem hiding this comment.
Why there are so huge improvements on "without vineyard"? and "with vineyard" becomes slower than "without vineyard"?
There was a problem hiding this comment.
Sorry, wrong writing here.
| +------------+------------------+---------------+ | ||
| | 15000 Mi | 128.6s | 0.04s | | ||
| | 15000 Mi | 61.7s | 0.04s | | ||
| +------------+------------------+---------------+ |
There was a problem hiding this comment.
Why this PR affects the performances of "without vineyard" case?
There was a problem hiding this comment.
The previous time is wrong as it is mixed with the initial reading time. My bad.
| @@ -1,12 +1,10 @@ | |||
| FROM python:3.10 | |||
|
|
|||
| RUN pip3 install --no-cache-dir pandas requests scikit-learn numpy vineyard | |||
| RUN pip3 install pandas requests scikit-learn numpy vineyard | |||
There was a problem hiding this comment.
Use --no-cache-dir to let pip cleanup the downloaded zips/wheels after installation.
Signed-off-by: Ye Cao <[email protected]>
What do these changes do?
Related issue number
Fixes #1568