Skip to content

Latest commit

 

History

History
318 lines (283 loc) · 28.1 KB

v0.26.alpha.md

File metadata and controls

318 lines (283 loc) · 28.1 KB
Raw

v0.26.alpha Release Notes

Important!

  • This release has been verified on DeepStream 6.2.
  • DeepStream 5.X is no longer supported. Please use a minimum of 6.0.1 for best performance.
  • There are a number of breaking name changes in this release. Please review the release notes carefully.

Highlights

Having to install OpenCV as a dependency is now optional.

The following "Images Services" - are now optional and are disabled by default.

To enable theses services, you must first install OpenCV or FFmpeg and update the Makefile. Please see the updated Installing Dependencies for new requirements. See Issue #908 for more information.

Source Components

  1. All Source components (currently video only) have been refactored and extended with new base-class services to get/set the following output buffer properties. (See #903 for more information).
    • Format - NV12, I420, P010_10LE, BGRx, RGBA .. although, the NVIDIA Streammux only supports NV12, I420, and RGBA as input.
    • Dimensions - output buffer can now be scaled prior to the Streammux input.
    • Cropping - both src and destination cropping are supported.
    • Orientation - flipping and rotating in various directions.
  2. Two New Video Source components have been added.
    • Application Source - allows the application to insert raw samples or buffers into a DSL Pipeline.
    • Interpipe Source - receives pipeline buffers and events from an Interpipe Sink.
  3. The Multi-Image Source has been completed and extended with new services.
  4. The RTSP Source now supports JPEG encoding.
  5. The RTSP Stream-Connection-Manager has been updated to manage the Source's initial stream connection, and to re-initiate a connection-cycle if the initial connection times outs. Previously, the Stream-Connection-Manager was only started after the first successful connection. A failed first attempt would result in the pipeline failing to play.
  6. The Video Dewarper component has been completed and can be added to any Video Source.
  7. Multiple instances of CSI and USB cameras can now be used.
  8. A new Buffer Timeout Pad Probe Handler can be added to any Source for notification of new-buffer timeout.

Sink Components

  1. Four New Video Sink components have been added.
    • Application Sink - allows the application to receive buffers or samples from a DSL Pipeline.
    • Interpipe Sink - allows pipeline buffers and events to flow to other independent pipelines, each with an Interpipe Source.
    • Multi-Image Sink - encodes and saves video frames to JPEG files at specified dimensions and frame-rate.
    • Frame-Capture Sink - encodes and saves video frames to JPEG files on application/user demand.
  2. Encode Sinks (File, Smart-Recording, RTSP, and WebRTC) now support services to get/set (scale) the buffer dimensions on input to the encoder. See #930

Multi-Object Tracker Updates

  1. The Tracker API has been refactored to be inline with DeepStream 6.X where the individual low-level tracker libs have been combined into one. (DeepStream 5.X is no longer supported). There is now a single DSL Tracker constructor that requires a configuration file for the specific Tracker implementation to use (IOU, NvSORT, NvDeepSORT, NvDCF). Note. NVIDIA's DeepStream low-level library provides "reference implementations" of the four Tracker types.
  2. The low-level library for the Tracker to use - which defaults to NVIDIA's NvMultiObjectTracker lib as defined in the Makefile - can be updated by API as well.

New Non-Maximum Processor (NMP) Pad Probe Handler (PPH)

The NMP PPH implements an inference cluster algorithm providing a more flexible alternative to the default non-maximum suppression (NMS) cluster algorithm performed by the NVIDIA Inference plugin.

Object Detection Event (ODE) Services Updates

  1. New filters have been added to the ODE Triggers - accessible through new services:
    • Minimum and maximum inference confidence (minimum only in previous release)
    • Minimum and maximum tracker confidence (both new)
    • Frame limit - maximum number of frames to trigger on - can be reset on-timer and on-demand.
    • Instance counts - both the number of consecutive instances to trigger ODE occurrence, and the number of following consecutive instances to suppress ODE occurrence can be controlled (Instance Trigger only).
  2. Three new ODE Actions have been added
    • Offset Label Action - offsets the objects label vertically and/or horizontally from its default position relative to the Object's bounding-box. This action solves the issue of labels covering the top portion of small objects.
    • Scale BBox Action - can be used to increase (or decrease) the area of the Object's bounding box. Useful when capturing objects to JPEG images/files to ensure the object is fully show, i.e. not cropped by the bounding box.
    • Remove Object - removes the Object (object's metadata) from the current Frame (frame's metadata).

Issues closed in this release

Bugs closed in this release

  • Resolve "videoCacheSize is deprecated. Use cacheSize instead" - deprecation in DS 6.0 build #704
  • Tiler incorrectly checks for source_id > batch_size on call to show-source #835
  • Fix PadProbeHandler execution order - must be executed in the order they were added. #845
  • "MOTChallenge-format" - should report object_id (tracking id) not class_id #846
  • WebRtc Sink Component fails to create in v0.25.alpha #882
  • The "ode_occurrence_uri_start_record_sink_action.py" has errors #922
  • RTSP smart recording is broken due to changes to the NVIDIA recordbin to support Audio #925
  • Object and Frame capture actions are failing on dGPU - Surface create params using wrong memtype. #942

Enhancements closed in this release

  • Implement new conditional/optional Non Maximum Processor Pad Probe Handler #817
  • Request to provide a service that can change the text offset at display render #820
  • Implement new Inter-Pipe Sink and Source components #828
  • Implement new dsl_tiler_frame_numbering_enabled_get/set services #831
  • Implement new Scale BBox ODE Actions - dsl_action_bbox_scale_new #834
  • Implement new Remove Object ODE Action - dsl_ode_action_object_remove_new #838
  • Implement new dsl_ode_trigger_infer_confidence_max_get/set services. #840
  • Implement new dsl_ode_trigger_tracker_confidence_max_get/set services. #841
  • Implement new dsl_ode_trigger_limit_frame_get/set services #843
  • Implement new dsl_source_usb_device_location_get/set services to support multiple USB cameras #850
  • Implement new dsl_source_csi_sensor_id_get/set services to support multiple CSI cameras. #853
  • Refactor CSI Source constructor to generate a unique/incremental sensor_id for each source. #854
  • Refactor USB Source constructor to generate a unique/incremental device-location for each source. #855
  • Need to complete the Multi-Image source constructor - add caps filter and videorate plugins to control framerate. #860
  • Implement new Buffer Timeout Pad Probe Handler (PPH) #862
  • Implement dsl_source_pph_add/remove services #863
  • Implement new dsl_source_image_multi_loop_enabled_get/set services #865
  • Implement new dsl_source_image_multi_indices_get/set services #866
  • Implement new dsl_ode_trigger_instance_count_settings_get/set services #869
  • Implement new Tracker services dsl_tracker_lib_file_get/set to support custom low level libs. #877
  • Implement new App Sink Component #880
  • Implement new App Source Component #881
  • Implement Source base services dsl_source_do_timestamp_get/set #885
  • Extend the RTSP Source to support JPEG encoding #891
  • Log all Initial property settings for all Bintrs at level INFO (4) #897
  • Refactor all sources - create new Audio and Video base classes - add new video-buffer-out control services #903
  • Implement Dewarper Component and integrate with refactored Base Source #904
  • Make extended image services optional - support none (default), or implemented with OpenCV or FFmpeg #908
  • Implement new Multi-Image Sink to save frames to JPEG files at a specified rate. #912
  • Implement new Frame-Capture Sink to encode/save frame buffers to JPEG files on Application demand. #914
  • Allow client to use the default encoder bitrate with all Encode Sinks #923
  • Extend RTSP stream-manager to manage first connection - retry on timeout #927
  • [request] Implement dsl_sink_encode_dimensions_get/set services (for File, Record, RTSP, WebRTC SInks) #930

Services removed in this release

  • Remove logic & services limiting the number of Source and Sink components that can be created. #892
    • dsl_source_num_in_use_get
    • dsl_source_num_in_use_max_get
    • dsl_source_num_in_use_max_set
    • dsl_sink_num_in_use_get
    • dsl_sink_num_in_use_max_get
    • dsl_sink_num_in_use_max_set

Breaking API name changes - issues closed in this release

  • Refactor Tracker API into a single component - remove IOU, KTL, and DCF specific API's #874 - the following services have been removed/replaced (see below)

    • dsl_tracker_ktl_new
    • dsl_tracker_iou_new
    • dsl_tracker_dcf_new
    • dsl_tracker_dcf_batch_processing_enabled_get
    • dsl_tracker_dcf_batch_processing_enabled_set
    • dsl_tracker_dcf_past_frame_reporting_enabled_get
    • dsl_tracker_dcf_past_frame_reporting_enabled_set

  • Rename/fix the Label and BBox ODE Action names. verb_noun should be noun_verb #823 - the following services have been renamed (see below)

    • dsl_ode_action_format_label_new
    • dsl_ode_action_customize_label_new
    • dsl_ode_action_customize_label_get
    • dsl_ode_action_customize_label_set
    • dsl_ode_action_format_bbox_new

  • Rename dsl_ode_trigger_confidence_min_get/set to dsl_ode_trigger_infer_confidence_min_get/set #839 - the following services have been renamed (see below)

    • dsl_ode_trigger_confidence_min_get
    • dsl_ode_trigger_confidence_min_set

  • Rename dsl_ode_trigger_limit_get/set services to dsl_ode_trigger_limit_event_get/set #842 - the following services have been renamed (see below)

    • dsl_ode_trigger_limit_get
    • dsl_ode_trigger_limit_set

  • Remove "annotate" parameter from dsl_ode_action_capture_frame_new - can no longer support. #909

    • dsl_ode_action_capture_frame_new - has been updated.

New Constants, Typedefs, and Services

New symbolic constants in this release

Constants used by the new Non Maximum Processor (NMP) Pad Probe Handler. Process methods:

#define DSL_NMP_PROCESS_METHOD_SURPRESS                             0
#define DSL_NMP_PROCESS_METHOD_MERGE                                1

Object match determination methods:

#define DSL_NMP_MATCH_METHOD_IOU                                    0
#define DSL_NMP_MATCH_METHOD_IOS                                    1

Valid return values for the dsl_sink_app_new_buffer_handler_cb

#define DSL_FLOW_OK                                                 0
#define DSL_FLOW_EOS                                                1
#define DSL_FLOW_ERROR                                              2

New Services in the release

New Video-Source common services:

New Buffer Timeout PPH and services to add/remove from a Source:

New App Source services:

New Interpipe Source services:

New Camera Source services:

New Multi-Image Source services:

New Video-Source Dewarper services:

New App Sink services:

New Interpipe Sink services:

New Multi-Image Sink services:

New Frame-Capture Sink Services:

New Encode Sink (File, Smart-Record, RTSP, and WebRTC) Services:

New and renamed Tracker services:

New and renamed ODE Trigger services:

New and renamed ODE Action services:

New Tiler services:

Non-Maximum Processor PPH service:

Examples in this release

Python:

C++