mount the dedicated storage for each function #1408
Conversation
|
Based on the current function design, when the Function is provided in the custom image, the code running in the ray nodes is the function code only. The user code doesn't run in the ray nodes so it can not access to the directory. |
charts/qiskit-serverless/charts/gateway/templates/rayclustertemplate.yaml
Show resolved
Hide resolved
| - mountPath: /function_data | ||
| name: user-storage | ||
| subPath: {{`{{ function_data }}`}} |
There was a problem hiding this comment.
Do we need to mount this with any kind of restricted permissions? In theory, and correct me if I'm wrong here, whoever is running the function (user or provider) would be able to access this location via the container...
There was a problem hiding this comment.
My understanding is that with the current custom image function design, the only the code in the custom image is executed in the Ray node. No user code is sent or executed in the Ray node so only the function code read/write to this directory.
There was a problem hiding this comment.
Users could in theory find a way around that restriction (if we misimplement something, CVS / exploits, etc.) so it's a risk, we'd just need to determine if it's an acceptable one
There was a problem hiding this comment.
Could we just add the path when a function is from a provider? That could solve part of the problem. And correct me if I'm wrong here @akihikokuroda but all the providers will write here so they could potentially see files from other providers with this approach. What is the problem to use the name of provider as sub-path instead of function_data?
There was a problem hiding this comment.
My idea was that in that way we could attach the specific provider path only when it's a function from the specific provider (but I don't know if there are limitations in this approach).
There was a problem hiding this comment.
I thought function-data was the provider space and data was for users ? since users would already go to data on their own functions
There was a problem hiding this comment.
For the provider, it may be useful to have both while they develop the function because it is the same environment when the function is executed by the user.
There was a problem hiding this comment.
I thought function-data was the provider space and data was for users ?
yep
since users would already go to data on their own functions
basically that's my comment, for functions that come from the user we are adding /function_data too. It's true that /function_data and /data are going to point to user.username path so not a bid deal but probably we can just avoid add /function_data for user functions.
BTW, another answer that I would understand is: "well, we don't overcomplicate more the template this way" and I would agree. Yep.
There was a problem hiding this comment.
Django template in helm template is very tricky :-)
There was a problem hiding this comment.
Hahaha I agree, I agree. I'm just reviewing the rest of the code, Aki and I will update my review 👍
|
Good from my side. Just need to resolve David's question about mounting |
|
Sorry, I didn't read your answers yesterday. |
|
The Function can write something that it wants to share with the user. I'm planing providing the loggers to the function one for private logs and the other for the shared with the user. The later log is written in the file in |
Summary
Mount the dedicated storage for each function
Details and comments
The dedicated storage area is mounted at
/function_datafor the function provider. When the function doesn't have the associated provider, the storage mounted at/function_dateis the same storage as/data.