GStreamer libcamerasrc controls

[superware]

Hello,

• Raspberry PI 4B
• Bookworm, 6.6.62+rpt-rpi-v8
• GStreamer 1.22.0-2
• libcamera 0.3.2+rpt20241119-1 arm64
• Camera Module 3 Wide

In a GStreamer pipeline using libcamerasrc, how can I disable automatic gain control (AGC)? I found a way to define the tuning file per execution, but it seems control defaults aren't defined there:

LIBCAMERA_RPI_TUNING_FILE=./imx708_wide.json gst-launch-1.0 libcamerasrc ! ...

Is there a simple way to tweak the camera controls? even setting the defaults will do since I permanently need to disable AGC.

[naushir]

I don't think the gstreamer plugin has AE control yet, @kbingham can you confirm? However, there does seem to be work to get this available in gstreamer, you can see the latest version of the patches here: https://patchwork.libcamera.org/project/libcamera/list/?series=4849

[kbingham]

"Full" gstreamer controls are merged to libcamera in 27cece6 https://patchwork.libcamera.org/project/libcamera/list/?series=4732&state=* but Raspberry Pi fork is not currently based on top of that from what I can see.

[naushir]

Ah, so I was looking at the wrong series. Once we rebase the tree, the controls will become avilable.

[naushir]

I've rebased our next branch with upstream libcamera. This should now have the gstreamer control support that was previously missing. I'm not sure when we are making a new package release, so I suggest compiling and running locally for now.

[kbingham]

There are two blockers for me presently to tag the next libcamera release (which may or may not impact RPi making a new release).

• https://patchwork.libcamera.org/project/libcamera/list/?series=4856
  • Implement wallclock timestamps for frames
• https://patchwork.libcamera.org/project/libcamera/list/?series=4833.
  • Add direction field to ControlId

which both introduce an ABI break so I want those 'in' to make the 0.4.0 release.

[superware]

If I need "full" gstreamer controls now, can I simply clone "next" and run meson setup build --buildtype=release -Dpipelines=rpi/vc4,rpi/pisp -Dipas=rpi/vc4,rpi/pisp -Dv4l2=true -Dgstreamer=enabled -Dtest=false -Dlc-compliance=disabled -Dcam=disabled -Dqcam=disabled -Ddocumentation=disabled -Dpycamera=enabled on my RPI4? Should I first manually uninstall libcamera or others?

Thanks!

[naushir]

You should manually uninstall the system install libcamera packages, then rebuild\install libcamera from the next branch.

[superware]

@naushir Ok great, I hope sudo apt remove libcamera-* and ninja -C build install will do the job.

[superware]

On RPI4 with Raspberry Pi OS Lite, it seems README.rst needs to be updated with python3-pip instead of python-pip and libqt5widgets5 instead of libqt5widgets.

Also, ninja -C build install:

...
Installation failed due to insufficient permissions.
Attempting to use polkit to gain elevated privileges...
Error executing command as another user: Not authorized

This incident has been reported.
FAILED: meson-internal__install
/usr/bin/meson install --no-rebuild
ninja: build stopped: subcommand failed.

sudo and it completes successfully (does it?).

$ gst-launch-1.0 libcamerasrc ae-enable=false ! ...

Fails with:

...
gst-launch-1.0: ../include/libcamera/controls.h:188: T libcamera::ControlValue::get() const [with T = bool; typename std::enable_if<((! libcamera::details::is_span<U>::value) && (! std::is_same<std::__cxx11::basic_string<char>, typename std::remove_cv< <template-parameter-1-1> >::type>::value)), std::nullptr_t>::type <anonymous> = nullptr]: Assertion `type_ == details::control_type<std::remove_cv_t<T>>::value' failed.
Aborted

[naushir]

I wonder if you are still using the system installed libcamerasrc component? You will need to uninstall with sudo apt uninscall gstreamer1.0-libcamera*.

To pick up the local install of the component, I think you might also have to follow the instructions here.

[superware]

@naushir Yes, sudo apt list --installed | grep libcamera returns nothing, yet:

me@me:~/libcamera $ meson devenv -C build
[libcamera] me@me:~/libcamera/build $ gst-launch-1.0 libcamerasrc ae-enable=false ! video/x-raw,format=NV12 ! fakesink
Setting pipeline to PAUSED ...
[0:13:56.525509647] [1101]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/me/libcamera/build/src/ipa' to the IPA search path
[0:13:56.525877297] [1101]  INFO Camera camera_manager.cpp:327 libcamera v0.3.2+218-b7ba311b
[0:13:56.537275678] [1102]  INFO RPI pisp.cpp:720 libpisp version v1.0.7 28196ed6edcf 16-12-2024 (09:34:11)
[0:13:56.623006942] [1102]  INFO IPAProxy ipa_proxy.cpp:134 libcamera is not installed. Loading IPA configuration from '/home/me/libcamera/src/ipa/rpi/pisp/data'
[0:13:56.632311981] [1102]  INFO RPI pisp.cpp:1179 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media3 and ISP device /dev/media0 using PiSP variant BCM2712_C0
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[0:13:56.638410858] [1105]  INFO Camera camera.cpp:1197 configuring streams: (0) 1280x1080-NV12
[0:13:56.638555877] [1102]  INFO RPI pisp.cpp:1484 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a - Selected sensor format: 2304x1296-SBGGR10_1X10 - Selected CFE format: 2304x1296-PC1B
gst-launch-1.0: ../include/libcamera/controls.h:188: T libcamera::ControlValue::get() const [with T = bool; typename std::enable_if<((! libcamera::details::is_span<U>::value) && (! std::is_same<std::__cxx11::basic_string<char>, typename std::remove_cv< <template-parameter-1-1> >::type>::value)), std::nullptr_t>::type <anonymous> = nullptr]: Assertion `type_ == details::control_type<std::remove_cv_t<T>>::value' failed.
Aborted

Maybe gstreamer version mismatch? Seems like something related to the control value parsing.

[naushir]

Maybe gstreamer version mismatch? Seems like something related to the control value parsing.

It definitely feels like a mismatch somewhere. Can you run ldd on the libcamerasrc.so file and see which libcamera build it links to?

[superware]

It definitely feels like a mismatch somewhere. Can you run ldd on the libcamerasrc.so file and see which libcamera build it links to?

So after installing a new RPI OS Lite image and building again:

me@me:~/libcamera $ meson devenv -C build
[libcamera] me@me:~/libcamera $ ldd ./build/src/gstreamer/libgstlibcamera.so
        linux-vdso.so.1 (0x00007fff0652c000)
        libcamera.so.0.3 => /home/me/libcamera/build/src/libcamera/libcamera.so.0.3 (0x00007fff06300000)
        libcamera-base.so.0.3 => /home/me/libcamera/build/src/libcamera/base/libcamera-base.so.0.3 (0x00007fff062b0000)
        ...

With no controls gst-launch-1.0 libcamerasrc ! video/x-raw,format=NV12,width=1280,height=720 ! fakesink works, but:

[libcamera] me@me:~/libcamera $ gst-launch-1.0 libcamerasrc ae-enable=false ! video/x-raw,format=NV12,width=1280,height=720 ! fakesink
Setting pipeline to PAUSED ...
[0:18:12.538571438] [3599]  INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/me/libcamera/build/src/ipa' to the IPA search path
[0:18:12.539115352] [3599]  INFO Camera camera_manager.cpp:327 libcamera v0.3.2+218-b7ba311b
[0:18:12.550718803] [3600]  INFO RPI pisp.cpp:720 libpisp version v1.0.7 28196ed6edcf 20-12-2024 (15:41:06)
[0:18:12.635042660] [3600]  INFO IPAProxy ipa_proxy.cpp:134 libcamera is not installed. Loading IPA configuration from '/home/me/libcamera/src/ipa/rpi/pisp/data'
[0:18:12.644661698] [3600]  INFO RPI pisp.cpp:1179 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media2 and ISP device /dev/media0 using PiSP variant BCM2712_C0
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[0:18:12.647133709] [3603]  INFO Camera camera.cpp:1197 configuring streams: (0) 1280x720-NV12
[0:18:12.647329859] [3600]  INFO RPI pisp.cpp:1484 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B
gst-launch-1.0: ../include/libcamera/controls.h:188: T libcamera::ControlValue::get() const [with T = bool; typename std::enable_if<((! libcamera::details::is_span<U>::value) && (! std::is_same<std::__cxx11::basic_string<char>, typename std::remove_cv< <template-parameter-1-1> >::type>::value)), std::nullptr_t>::type <anonymous> = nullptr]: Assertion `type_ == details::control_type<std::remove_cv_t<T>>::value' failed.
Aborted

Also, specifying ae-constraint-mode=2 works, so it seems like a Boolean property parsing issue. Any idea?

[naushir]

I would suggest posting this issue to the libcamera developer mailing list.