Need help regarding input data in rastervision.

[NoobOwner]

Can we use image tiles as training and validation images and labels in raster vision? I am working on WMTS xy tiles. i have bunch of satellite images as tiles (training images/labels and validation images/labels). in addition to, i have testing dataset that consists of test images to be used for predictions. Kindly need help.

[AdeelH]

Normally, you feed in large GeoTIFFs as inputs to Raster Vision. These large scenes are broken down into smaller "chips" before being fed into the model for training/prediction. Alternatively, your input can be a "pre-chipped" dataset that comprises a bunch of smaller chips in the form of image files or TIFFs or numpy arrays.

RV does not support directly reading from a tile server. You will have to save your tiles in one of the intermediate forms mentioned above before you can use it.

A couple of other options that might or might not be feasible are:

1. Define a RasterSource subclass that is capable of reading from a tile server.
2. (This one requires the current master branch.) The PR, Add XarraySource to make it easier to consume imagery fetched from a STAC API #1764, just added support for an XarraySource. If there is an easy way to define an Xarray DataArray that reads from a tile server, you might be able to use it to interface with RV.

[AdeelH]

My label images are 8 bit binary images which have black background and white polygons

Yes, but they seem to have 3 bands. You need to make sure that they have only 1 band.

Also when i reduce batch size, it gives me shaoe error.

That's probably from BatchNorm layers, which require batch size > 1.

When i increase batch size, it gives me cudnn error even though torch and tf are using GPU.

That's because 1500x1500 is a very large size and you're running out of GPU memory. You should probably be chipping these large images into smaller ones. See the GeoDataset tutorial.

[NoobOwner]

do i have to convert rgb image tiles in grey scale as well (to make them single channel)?

[NoobOwner]

Your labels should not have 3 channels. They should be 2D rasters with pixel values being integer class IDs. Are your labels RGB images where classes are represented by different colors? If so, RV has utils for converting that to a class ID raster.

my label images are single channel images of shape torch.Size([256, 256]), but my rgb images are of shape torch.Size([3, 256, 256]).

[AdeelH]

do i have to convert rgb image tiles in grey scale as well (to make them single channel)?

No

[AdeelH]

my label images are single channel images of shape torch.Size([256, 256]), but my rgb images are of shape torch.Size([3, 256, 256]).

That's fine. I made the point about labels needing to be single band because earlier you reported that they had 3 bands:

out: y shape for train_ds torch.Size([1500, 1500, 3])
out: y shape for val_ds torch.Size([256, 256, 3])

[NoobOwner]

in
data_cfg = SemanticSegmentationGeoDataConfig(
img_channels = 1,
class_names=class_config.names,
class_colors=class_config.colors,
num_workers=0, # increase to use multi-processing
)
i set img_channels as 1, but still i get "Given groups=1, weight of size [64, 1, 7, 7], expected input[1, 3, 256, 256] to have 1 channels, but got 3 channels instead"

[NoobOwner]

now my data is in same shape as mentioned in rv

a. with torch.Size([3, 256, 256]) torch.Size([256, 256])), i get error " RuntimeError: Given groups=1, weight of size [64, 1, 7, 7], expected input[2, 3, 256, 256] to have 1 channels, but got 3 channels instead " with img_channels = 1, batch size = 1

b. with torch.Size([3, 256, 256]) torch.Size([256, 256])), i get error " RuntimeError: Expected more than 1 value per channel when training, got input size torch.Size([1, 256, 1, 1])" with img_channels = 3, batch size = 1
kindly guide me. as i can only use tiles because evenif i stitch tiles in big tif image to use standard way of RV, i dont have the labels as geojson

[AdeelH]

What is the error in that case?

[NoobOwner]

for img_channels=3 and batch_size=1, i get shape error.
for img_channels=3 and batch_size=2+, i get cuda error. the size of my images is 256*256. for big tif images (according to rv documentation), i dont get anny error of cuda. i am using RTX3060

[AdeelH]

Is it a CUDA out of memory error? If so, try a smaller model. Try using resent50 backbone instead of resnet101. You can also try using this model which is lighter than the default DeepLabV3 model that SemanticSegmentationModelConfig generates.

[NoobOwner]

i used this model
channels=3, batchsize=1 , error: Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: Unable to find a valid cuDNN algorithm to run convolution

channels=3, batchsize>=2 error : Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: Unable to find a valid cuDNN algorithm to run convolution
../aten/src/ATen/native/cuda/NLLLoss2d.cu:103: nll_loss2d_forward_kernel: block: [0,0,0], thread: [896,0,0] Assertion t >= 0 && t < n_classes failed.

[AdeelH]

../aten/src/ATen/native/cuda/NLLLoss2d.cu:103: nll_loss2d_forward_kernel: block: [0,0,0], thread: [896,0,0] Assertion t >= 0 && t < n_classes failed.

This error suggests that your labels have class IDs that are out of range. For example, if the model outputs 2 channels (i.e. scores for 2 classes for each pixel) but your labels have pixels that have a class ID of 2 (only 0 or 1 would be valid if there are 2 classes).

[NoobOwner]

its giving "SemanticSegmentationModelConfig' object has no attribute 'to." in line 110. this code is same as in documentation of rastervision. rv==0.20

[NoobOwner]

if we dont have the test_labels (for predictions), then how to use it? in documentation, the test_label are only used to make make_ss_scene (

)
normally we dont have the test_labels available. for thts, i am getting "missing 1 required positional argument: 'label_dir'" error

[AdeelH]

What part asks for test_labels? Unless I am misunderstanding, you already have the predictions, so you're done. No?

[NoobOwner]

this one. I am using the tiles as test dataset. So i dont have test_labels. But it is asking for label_dir for test dataset as well. Actually i did this for test purposes, i trained the model using training and validation tif and geojson just like the documentation. Then to predict, i am using tile images instead of tif.

[AdeelH]

Ah, now I get it. Yes, the ImageDatasets require labels even though GeoDatasets do not.

Here is what you can do:

• This is the code for SemanticSegmentationImageDataset.
• Copy this and modify it to define you own version of this class in your own code which does not require label_dir. Make sure __getitem__() returns None in place of the labels; so this line should be return x, None.
• Use that class instead of RV's SemanticSegmentationImageDataset.

[NoobOwner]

`from torch.utils.data import Dataset
from pathlib import Path
from rastervision.pytorch_learner.dataset import discover_images
class customSemanticSegmentationDataReader(Dataset):
"""Reads semantic segmentatioin images and labels from files."""

def __init__(self, img_dir: str):#, label_dir: str):
    """Constructor.

    Args:
        img_dir (str): Directory containing images.
        label_dir (str): Directory containing segmentation masks.
    """
    self.img_dir = Path(img_dir)
    #self.label_dir = Path(label_dir)

    # collect image and label paths, match them based on filename
    img_paths = discover_images(img_dir)
    #label_paths = discover_images(label_dir)
    self.img_paths = sorted(img_paths, key=lambda p: p.stem)
    #self.label_paths = sorted(label_paths, key=lambda p: p.stem)
    #self.validate_paths()

"""def validate_paths(self) -> None:
    if len(self.img_paths) != len(self.label_paths):
        raise ImageDatasetError(
            'There should be a label file for every image file. '
            f'Found {len(self.img_paths)} image files and '
            f'{len(self.label_paths)} label files.')
    for img_path, label_path in zip(self.img_paths, self.label_paths):
        if img_path.stem != label_path.stem:
            raise ImageDatasetError(
                f'Name mismatch between image file {img_path.stem} '
                f'and label file {label_path.stem}.')"""

def __getitem__(self, ind: int) -> Tuple[np.ndarray, np.ndarray]:
    img_path = self.img_paths[ind]
    #label_path = self.label_paths[ind]

    x = load_image(img_path)
    #y = None# load_image(label_path).squeeze()

    return x #, y

def __len__(self):
    return len(self.img_paths)

class customSemanticSegmentationImageDataset(ImageDataset):
"""Reads semantic segmentatioin images and labels from files.

Uses :class:`.SemanticSegmentationDataReader` to read the data.
"""

def __init__(self, img_dir: str, *args, **kwargs):
    """Constructor.

    Args:
        img_dir (str): Directory containing images.
        label_dir (str): Directory containing segmentation masks.
        *args: See :meth:`.ImageDataset.__init__`.
        **kwargs: See :meth:`.ImageDataset.__init__`.
    """

    ds = customSemanticSegmentationDataReader(img_dir)#, label_dir)
    super().__init__(
        ds,
        *args,
        **kwargs,
        transform_type=TransformType.semantic_segmentation)

`
this is working. but customSemanticSegmentationImageDataset' object has no attribute 'windows'

the window is given by SemanticSegmentationSlidingWindowGeoDataset.from_uris
https://docs.rastervision.io/en/0.20/usage/tutorials/pred_and_eval_ss.html#:~:text=pred_labels%20%3D%20SemanticSegmentationLabels.from_predictions(%0A%20%20%20%20ds.windows%2C%0A%20%20%20%20predictions%2C%0A%20%20%20%20smooth%3DTrue%2C%0A%20%20%20%20extent%3Dds.scene.extent%2C%0A%20%20%20%20num_classes%3Dlen(class_config))

[AdeelH]

You do not have to create SemanticSegmentationLabels; it doesn't apply to ImageDatasets.

predictions is an iterator over the predictions for each image. You will need to write your own code to save them to file.

[NoobOwner]

#1787 (reply in thread)

This is the main issue that i an encountering. I am using training and validating images and labels to train and test images to predict. But there is an error before training even though i entered correct data. The code is already given above.

[NoobOwner]

For this, kindly guide me. The actual problem is this one. All my data is available in the form of tiles (*.png). It gives me error with shape either way

[NoobOwner]

Is there any possible solution for this issue?

[NoobOwner]

import albumentations as A
import rasterio
import numpy as np
from PIL import Image
from rastervision.core.rv_pipeline import *
from rastervision.core.backend import *
from rastervision.pytorch_backend import *
from rastervision.pytorch_learner import *
from rastervision.pipeline.config import register_config, Config, Field
import albumentations as A
import os
from matplotlib import pyplot as plt
from rastervision.core.data import *
from matplotlib.colors import ListedColormap
from torch.utils.data import Dataset
from pathlib import Path
from rastervision.pytorch_learner.dataset import discover_images
import torch
torch.cuda.empty_cache()

train_image_uri = '/home/nicku/Downloads/EORSSD-dataset-master/train'
train_label_uri = '/home/nicku/Downloads/EORSSD-dataset-master/train_labels'
val_image_uri = '/home/nicku/Downloads/EORSSD-dataset-master/test'
val_label_uri = '/home/nicku/Downloads/EORSSD-dataset-master/test_labels'
test_img_uri = '/home/nicku/PycharmProjects/pythonProject/ma_dataset/pred.tif'

pred = '/home/nicku/PycharmProjects/pythonProject/results'
pred_bw = '/home/nicku/PycharmProjects/pythonProject/results'
pred_overlap = '/home/nicku/PycharmProjects/pythonProject/results'
pred_tif = '/home/nicku/PycharmProjects/pythonProject/results'
learner_dir = '/home/nicku/PycharmProjects/pythonProject/model'
saved_model = '/home/nicku/PycharmProjects/pythonProject/model/model-bundle.zip'

class_config = ClassConfig(
names=['background', 'building'],
colors=['black', 'white'],
null_class='building'
)
class_config.ensure_null_class()

viz = SemanticSegmentationVisualizer(
class_names=class_config.names, class_colors=class_config.colors)

data_augmentation_transform = A.Compose([
A.Flip(),
A.Resize(width=256, height=256),
A.ShiftScaleRotate(),
A.OneOf([
A.HueSaturationValue(hue_shift_limit=10),
A.RGBShift(),
A.ToGray(),
A.ToSepia(),
A.RandomBrightness(),
A.RandomGamma(),
]),
A.CoarseDropout(max_height=32, max_width=32, max_holes=5)
])

train_ds = SemanticSegmentationImageDataset(
img_dir=train_image_uri,
label_dir=train_label_uri,
transform=data_augmentation_transform,

)

val_ds = SemanticSegmentationImageDataset(
img_dir=val_image_uri,
label_dir=val_label_uri,
transform=A.Resize(256, 256)
)

data_cfg = SemanticSegmentationGeoDataConfig(
img_channels=3,
class_names=class_config.names,
class_colors=class_config.colors,
num_workers=0, # increase to use multi-processing
)

solver_cfg = SolverConfig(
batch_sz=2,
lr=6e-5,
class_loss_weights=[1., 10.],
)

model= torch.hub.load(
'AdeelH/pytorch-fpn:0.3',
'make_fpn_resnet',
name='resnet18',
fpn_type='panoptic',
num_classes=len(class_config),
fpn_channels=128,
in_channels=3,
out_size=(256, 256),
pretrained=True)

learner_cfg = SemanticSegmentationLearnerConfig(data=data_cfg, solver=solver_cfg)

learner = SemanticSegmentationLearner(
cfg=learner_cfg,
output_dir=learner_dir,
model = model,
train_ds=train_ds,
valid_ds=val_ds,
)
learner.train(epochs=3)
learner.save_model_bundle()

ds = SemanticSegmentationSlidingWindowGeoDataset.from_uris( #if your test dataset have .tif
class_config=class_config,
image_uri=test_img_uri,
size=256,
stride=256,
transform=A.Resize(256, 256))
predictions = learner.predict_dataset(
ds,
raw_out=True,
numpy_out=True,
predict_kw=dict(out_shape=(256, 256)),
progress_bar=True)

pred_labels = SemanticSegmentationLabels.from_predictions(
ds.windows,
predictions,
smooth=True,
extent=ds.scene.extent,
num_classes=len(class_config))

scores = pred_labels.get_score_arr(pred_labels.extent)
scores_building = scores[0]
scores_background = scores[1]

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(10, 5))
fig.tight_layout(w_pad=-2)
ax1.imshow(scores_building, cmap='plasma')
ax1.axis('off')
ax1.set_title('building')
fig.savefig('/home/nicku/PycharmProjects/pythonProject/results/building.png')
ax2.imshow(scores_background, cmap='plasma')
ax2.axis('off')
ax2.set_title('background')
fig.savefig('/home/nicku/PycharmProjects/pythonProject/results/background.png')
plt.show()

pred_labels.save(
uri=pred_tif,
crs_transformer=ds.scene.raster_source.crs_transformer,
class_config=class_config)

->This is my code. it takes the training and validation images and labels of shape (3,256,256) and (256,256). i am trying to train rv on
these imgs and labels. but somehow it is not working.

Training: 0%| | 0/700 [00:01<?, ?it/s]
Traceback (most recent call last):
File "/home/nicku/PycharmProjects/pythonProject/rv_chips_to_tif.py", line 109, in
learner.train(epochs=3)
File "/home/nicku/miniconda3/envs/rastervision/lib/python3.8/site-packages/rastervision/pytorch_learner/learner.py", line 1269, in train
train_metrics = self.train_epoch(
File "/home/nicku/miniconda3/envs/rastervision/lib/python3.8/site-packages/rastervision/pytorch_learner/learner.py", line 1195, in train_epoch
output['train_loss'].backward()
File "/home/nicku/miniconda3/envs/rastervision/lib/python3.8/site-packages/torch/_tensor.py", line 396, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/nicku/miniconda3/envs/rastervision/lib/python3.8/site-packages/torch/autograd/init.py", line 173, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: Unable to find a valid cuDNN algorithm to run convolution

Process finished with exit code 1

[AdeelH]

Please properly format your code so that it's easier to read.

[NoobOwner]

i have done it. it is the exact code as i am using, and using exactly rv documentation data and procedure