deepstream?

[js1972]

This looks amazing - I'm wondering will it work with Deepstream elements as well?

[tomasz-lewicki]

Hi @js1972, thanks for the interest in the project 🙂

Yes, you could possibly use jetmulticam to deploy your own TensorRT model, or a pre-trained model from Nvidia NGC with multiple cameras. Depending on the model, it might require adding some additional code for parsing the output.

What model are you interested in deploying?

Best,
Tomasz

[js1972]

I mean - will it work with nvidia deepstream? Ie 8 cameras at 30fps with inference; re-streaming as rtsp; writing to files and so on.

[aafaqin]

How to use Jetson-inference mobilenet v2 with multiple cameras at once on xavier?

[tomasz-lewicki]

@js1972, by defining the pipeline as follows:

pipeline = CameraPipelineDNN(
        cameras=[0, 1, 2, 3, 4, 5, 6, 7],
        models=[PeopleNet.GPU],
        save_video=True,
        save_video_folder="/home/nx/logs/videos"
    )

on Xavier AGX, you will achieve a similar result to what you mentioned: 30fps inference on 8 cameras + writing to files. The initialized pipeline will use Nvidia Deepstream elements under-the-hood. However, currently no tighter integration with Deepstream is planned.

[js1972]

Thankyou @tomek-l this is awesome. So under the hood you are using the DeepStream hardware-accelerated elements.
Do you think it would be possible to use the smartrecord and analytics elements? If these were possible then a lot of DeepStream pipelines could be moved over to this framework making development far easier...

[tomasz-lewicki]

Interesting question, @aafaqin.

I assume you're interested in running detectnet with mobilenet V2 backbone? Similarly as in jetson-inference repository?
Currently this is not implemented, but feel free to open a new issue as a feature request for that 🙂

[tomasz-lewicki]

@aafaqin in general, you could also implement that functionality yourself by:

• Downloading pre-trained model from TAO
• Adding a new module to jetmulticam.models

[tomasz-lewicki]

@js1972 As far as the two other modules are concerned, I have to admit I never used them. I'd have to take a look into their API in DS docs and get back to you.

[tomasz-lewicki]

@js1972 In the meantime, are you familiar with python bindings for deepstream (pyds)? They make it fairly easy to experiment with DS, and there are sample python apps available.

[js1972]

Hi @tomek-l - No not really familiar with them as I tried it out earlier this year (I think it was after DS 5.0 came out) and it was 4 hours of hair pulling to get it installed and working so I could test the deepstream-test1 python app. Then gave up and went back to what I know - C. ;-)

Looking at your code in this repo though I am pretty impressed and how nice and succinct it is! So its got me excited that maybe I should take the jump and switch to python now.

I have built a smart nvr system. It uses a python controller program as a systemd service which connects to aws iot (so the the system can be controlled from a mobile app). This python controller starts and stops a custom DeepStream C program. The DeepStream program muxes rtsp streams then does person detection with peoplenet and uses the nvdsanalytics element to enable ROI's (privacy/activity zones) on each camera.
Smartrecord is used to start recording when a person is detected and stop recording after a time period (or when there are no detections for a period).

So if I can get the analytics and smart record components working I could move over to this architecture you have here.
I'm not sure about the nvdsanalytics component though as it relies on c++ data structures so don't know how you'd access them in python.