When you click the NEW DATASET under DATASET page, you will be asked to add new dataset either from local or external data source. You have three data provider options: VESSL, Amazon Simple Storage Service, and Google Cloud Storage.
When you select a VESSL dataset, you can upload data from the local disk. To create a VESSL dataset:
Enter Dataset Name
Click Upload Files
Click Submit
You can retrieve dataset from S3 by selecting Amazon Simple Storage Service. To create a dataset from S3
Enter Dataset Name
Enter ARN
Enter Bucket Path
Click Create
You also have an option to retrieve dataset from Google Cloud Storage. To create a dataset from GCS:
Enter Dataset Name
Enter Bucket Path
Click Create button
If the dataset exists inside the cluster (NAS, host machine, etc.) and you want to mount it only inside the cluster, you can select Local Storage option. In this case, VESSL only stores the location of the dataset, and mounts the path when an experiment is created.
