Create a Geo Dataset with raster labels instead of GeoJSON

[lcoandrade]

I've just learnt about Rastervision and I'm trying to make a minimal example with it to train a network.
I want to use NAIP and Chesapeake data to create a dataset. NAIP has the images (nail) and Chesapeake is the mask (Chesapeake).

I have this dataset on Kaggle available here: Naip and Chesapeake

I would need to intersect both of them and define that the images would be obtained from NAIP intersection and the masks would be obtained from Chesapeake intersection (raster and not geojson).

Can someone provide a minimal example to achieve that?

Thanks in advance!

[AdeelH]

Hi. You should definitely be able to do that.

Assuming you are using RV as a library and using .from_uris() to construct a SemanticSegmentationSlidingWindowGeoDataset or a SemanticSegmentationRandomWindowGeoDataset

• You can read raster segementation labels by specifying the URI of the label GeoTIFFs in label_raster_uri
• If needed, you can also specify an AOI aoi_uri

I highly recommend checking out the tutorial notebooks.

Let me know if you have any specific questions.

[lcoandrade]

Thanks for the answer!

I'm trying to follow the tutorial with the dataset that I've mentioned on Kaggle.

#Installing rastervision
!pip install rastervision

#Setting up environment variables
import os
from subprocess import check_output

os.environ['GDAL_DATA'] = check_output('pip show rasterio | grep Location | awk \'{print $NF"/rasterio/gdal_data/"}\'', shell=True).decode().strip()
os.environ['AWS_NO_SIGN_REQUEST'] = 'YES'

#Defining class config
from rastervision.core.data import ClassConfig

class_config = ClassConfig(
    names=['water', 'wetlands', 'tree canopy', 'shrubland',
          'low_vegetation', 'barren', 'structures', 'impervious surfaces',
          'impervious roads', 'tree canopy over structures',
           'tree canopy over impervious surfaces', 'tree canopy over impervious roads'],
)

#Defining image and label uris to create a dataset
import os
import albumentations as A
from rastervision.pytorch_learner import (
    SemanticSegmentationRandomWindowGeoDataset,
    SemanticSegmentationSlidingWindowGeoDataset,
    SemanticSegmentationVisualizer)

def make_train_list(root):
    return list(map(lambda x: os.path.join(root, x), os.listdir(root)))

train_image_uris = make_train_list('/kaggle/input/naip-chesapeake-sample/naip')
print(train_image_uris)

train_label_uris = [os.path.join('/kaggle/input/naip-chesapeake-sample/chesapeake', 'DE_STATEWIDE.tif')]
print(train_label_uris)

viz = SemanticSegmentationVisualizer(
    class_names=class_config.names, class_colors=class_config.colors)

#Creating the dataset
data_augmentation_transform = A.Compose([
    A.Flip(),
    A.ShiftScaleRotate(),
    A.OneOf([
        A.HueSaturationValue(hue_shift_limit=10),
        A.RGBShift(),
        A.ToGray(),
        A.ToSepia(),
        A.RandomBrightness(),
        A.RandomGamma(),
    ]),
    A.CoarseDropout(max_height=32, max_width=32, max_holes=5)
])

dataset = SemanticSegmentationRandomWindowGeoDataset.from_uris(
    class_config=class_config,
    image_uri=train_image_uris,
    label_raster_uri=train_label_uris,
    size_lims=(150, 200),
    out_size=256,
    max_windows=400,
    transform=data_augmentation_transform)

len(dataset)

With this, I get the following error:

FileNotFoundError: [Errno 2] No such file or directory: 'gdalbuildvrt'

What can it be?

[AdeelH]

Looks like you might not have GDAL installed. gdalbuildvrt is only needed if you want to mosaic images together, which you are currently doing by passing a list of images to image_uri and label_raster_uri. I would instead recommend creating one GeoDataset per scene/image. You can then combine data multiple datasets together (to pass to a DataLoader) using PyTorch's ConcatDataset.

[lcoandrade]

Good suggestion! Thank you for your attention! I’ll try here.

[lcoandrade]

I was able to create a concatenated dataset as you said. But ,for some reason, the images and labels don't seem to be spatially paired.
What am I missing here?

If it is possible, can you have a look at my Kaggle notebook?

This is how the data is spatially related:

And this is a visualization of a sample:

Thank you in advance.

[lcoandrade]

But, regarding the motive of the discussion, I will mark as answered because I successfully created a dataset with raster labels. Thank you very much!