Object detection with augmentors #1706

greenhawktobias · 2023-02-10T09:24:06Z

greenhawktobias
Feb 10, 2023

Hi,
I am trying to increase accuracy with augmentors doing object detection. Code runs without error but are augmentors applied?
I can not validate the use of augmentors in cmd when running. Runnning RV image 0.20 with Docker.

...from os.path import join
import albumentations as A...


...def get_config(runner,
               nochip: bool = True,
               augment: bool = True,...

def make_scene(scene_id, image_uri, label_uri):
        raster_source = RasterioSourceConfig(
            uris=[image_uri], channel_order=channel_order)

        vector_source = GeoJSONVectorSourceConfig(
            uri=label_uri,
            ignore_crs_field=True,
            transformers=[ClassInferenceTransformerConfig(default_class_id=0)])
        label_source = ObjectDetectionLabelSourceConfig(
            vector_source=vector_source)
            

        return SceneConfig(
            id=scene_id,
            raster_source=raster_source,
            label_source=label_source)

    for scene in train_scene_list:
        train_scenes_input.append(make_scene(*scene))
        
    data_augmentation_transform = A.Compose([
        A.Flip(),
        A.ShiftScaleRotate(),
        A.OneOf([
            A.HueSaturationValue(hue_shift_limit=10),
            A.RGBShift(),
            A.ToGray(),
            A.ToSepia(),
            A.RandomBrightness(),
            A.RandomGamma(),
        ]),
        A.CoarseDropout(max_height=32, max_width=32, max_holes=5)
    ])
    class_config = ClassConfig(names=['tree'], colors=['red'])
    scene_dataset = DatasetConfig(
        class_config=class_config,
        train_scenes=train_scenes_input,
        validation_scenes=[make_scene(val_scene_id, val_image_uri, val_label_uri)])
    
    chip_sz = 300
    img_sz = chip_sz
   

    chip_options = ObjectDetectionChipOptions(neg_ratio=1.0, ioa_thresh=0.8)
    if nochip:
        window_opts = ObjectDetectionGeoDataWindowConfig(
            method=GeoDataWindowMethod.random,
            size=chip_sz,
            size_lims=(chip_sz, chip_sz + 1),
            max_windows=500,
            max_sample_attempts=100,
            clip=True,
            neg_ratio=None,
            ioa_thresh=chip_options.ioa_thresh,
            neg_ioa_thresh=0.2)

        data = ObjectDetectionGeoDataConfig(
            scene_dataset=scene_dataset,
            window_opts=window_opts,
            img_sz=img_sz,
            num_workers=4,
            plot_options=PlotOptions(
                channel_display_groups=channel_display_groups))
    else:
        data = ObjectDetectionImageDataConfig(

Answered by AdeelH

Feb 10, 2023

You are defining the variable data_augmentation_transform but not passing it to the DataConfig. You can do it like so:

data = ObjectDetectionGeoDataConfig(
    ...
    aug_transform=A.to_dict(data_augmentation_transform),
    ...
)

Same thing for ObjectDetectionImageDataConfig, if you are using nochip=False.

View full answer

AdeelH · 2023-02-10T10:57:58Z

AdeelH
Feb 10, 2023
Maintainer

You are defining the variable data_augmentation_transform but not passing it to the DataConfig. You can do it like so:

data = ObjectDetectionGeoDataConfig(
    ...
    aug_transform=A.to_dict(data_augmentation_transform),
    ...
)

Same thing for ObjectDetectionImageDataConfig, if you are using nochip=False.

3 replies

greenhawktobias Feb 10, 2023
Author

Seems to work now, but traceback says as below. Image input data is 8 bit integer images.
Maybe skipping some augmentors or try passing a MinMaxTransformer or StatsTransformer to the RasterSource?


2023-02-10 12:27:50:rastervision.pytorch_learner.learner: INFO - Plotting sample training batch.
2023-02-10 12:27:50:rastervision.pytorch_learner.dataset.dataset: WARNING - Many albumentations transforms require uint8 input. Therefore, we recommend passing a MinMaxTransformer or StatsTransformer to the RasterSource so the input will be converted to uint8.
2023-02-10 12:27:50:rastervision.pytorch_learner.dataset.dataset: WARNING - Many albumentations transforms require uint8 input. Therefore, we recommend passing a MinMaxTransformer or StatsTransformer to the RasterSource so the input will be converted to uint8.
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 251, in <module>
    _main()
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 247, in _main
    main()
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 236, in run_command
    _run_command(
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 218, in _run_command
    command_fn()
  File "/opt/src/rastervision_core/rastervision/core/rv_pipeline/rv_pipeline.py", line 154, in train
    backend.train(source_bundle_uri=self.config.source_bundle_uri)
  File "/opt/src/rastervision_pytorch_backend/rastervision/pytorch_backend/pytorch_learner_backend.py", line 120, in train
    learner.main()
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 263, in main
    self.plot_dataloaders(self.cfg.data.preview_batch_limit)
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 895, in plot_dataloaders
    self.plot_dataloader(
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 885, in plot_dataloader
    x, y = next(iter(dl))
  File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 681, in __next__
    data = self._next_data()
  File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1376, in _next_data
    return self._process_data(data)
  File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1402, in _process_data
    data.reraise()
  File "/opt/conda/lib/python3.9/site-packages/torch/_utils.py", line 461, in reraise
    raise exception
NotImplementedError: Caught NotImplementedError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataset.py", line 235, in __getitem__
    return self.datasets[dataset_idx][sample_idx]
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/dataset/dataset.py", line 459, in __getitem__
    return super().__getitem__(window)
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/dataset/dataset.py", line 73, in __getitem__
    raise exc
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/dataset/dataset.py", line 67, in __getitem__
    x, y = self.transform(val)
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/dataset/dataset.py", line 57, in <lambda>
    self.transform = lambda inp: tf_func(inp, transform)
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/dataset/transform.py", line 149, in object_detection_transformer
    out = transform(image=x, bboxes=boxes, category_id=class_ids)
  File "/opt/conda/lib/python3.9/site-packages/albumentations/core/composition.py", line 205, in __call__
    data = t(**data)
  File "/opt/conda/lib/python3.9/site-packages/albumentations/core/composition.py", line 205, in __call__
    data = t(**data)
  File "/opt/conda/lib/python3.9/site-packages/albumentations/core/composition.py", line 205, in __call__
    data = t(**data)
  File "/opt/conda/lib/python3.9/site-packages/albumentations/core/transforms_interface.py", line 118, in __call__
    return self.apply_with_params(params, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/albumentations/core/transforms_interface.py", line 131, in apply_with_params
    res[key] = target_function(arg, **dict(params, **target_dependencies))
  File "/opt/conda/lib/python3.9/site-packages/albumentations/core/transforms_interface.py", line 254, in apply_to_bboxes
    return [self.apply_to_bbox(tuple(bbox[:4]), **params) + tuple(bbox[4:]) for bbox in bboxes]  # type: ignore
  File "/opt/conda/lib/python3.9/site-packages/albumentations/core/transforms_interface.py", line 254, in <listcomp>
    return [self.apply_to_bbox(tuple(bbox[:4]), **params) + tuple(bbox[4:]) for bbox in bboxes]  # type: ignore
  File "/opt/conda/lib/python3.9/site-packages/albumentations/core/transforms_interface.py", line 248, in apply_to_bbox
    raise NotImplementedError("Method apply_to_bbox is not implemented in class " + self.__class__.__name__)
NotImplementedError: Method apply_to_bbox is not implemented in class CoarseDropout

AdeelH Feb 10, 2023
Maintainer

The warning about uint8 is misleading. The error is caused by A.CoarseDropout which is not supported for object detection. So I would just remove that.

After fixing that, I think you will still run into an error about bbox_params. The solution to that is:

...
from rastervision.pytorch_learner import DEFAULT_BBOX_PARAMS
...
data_augmentation_transform = A.Compose([...], bbox_params=DEFAULT_BBOX_PARAMS)
...
data = ObjectDetectionGeoDataConfig(
    ...
    aug_transform=A.to_dict(data_augmentation_transform),
    ...
)

greenhawktobias Feb 10, 2023
Author

It works now! Thanks!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Object detection with augmentors #1706

{{title}}

Replies: 1 comment 3 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Object detection with augmentors #1706

greenhawktobias Feb 10, 2023

Replies: 1 comment · 3 replies

AdeelH Feb 10, 2023 Maintainer

greenhawktobias Feb 10, 2023 Author

AdeelH Feb 10, 2023 Maintainer

greenhawktobias Feb 10, 2023 Author

greenhawktobias
Feb 10, 2023

Replies: 1 comment 3 replies

AdeelH
Feb 10, 2023
Maintainer

greenhawktobias Feb 10, 2023
Author

AdeelH Feb 10, 2023
Maintainer

greenhawktobias Feb 10, 2023
Author