You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It should be helpful for data scientists to use command like "arena create data imagenet-full" to create, index and manage different training datasets for different training jobs.
Then when use arena submit training job, the "imagenet-full" can be passed in as parameter "data" directly.
The data actually can be a pointer to specific PVC or hdfs path, etc. It's easy to use 'data' to manage and record which training datasets to load, while needn't care about which storage backend is used for persistent.
The text was updated successfully, but these errors were encountered:
It should be helpful for data scientists to use command like "arena create data imagenet-full" to create, index and manage different training datasets for different training jobs.
Then when use arena submit training job, the "imagenet-full" can be passed in as parameter "data" directly.
The data actually can be a pointer to specific PVC or hdfs path, etc. It's easy to use 'data' to manage and record which training datasets to load, while needn't care about which storage backend is used for persistent.
The text was updated successfully, but these errors were encountered: