FastCV Extension code for Opencv APIs #3811

sssanjee-quic · 2024-10-16T12:41:20Z

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

Change-Id: I721b810181dd9127221b6e2182d7aad5ac679b68

Added FastCV Extension code for Opencv APIs

savuor

The rest is OK, waiting for other wrapped functions to be added.

The tests can be merged as-is, without fixing all my comments. But I still have to leave these comments so that they can be fixed in the future.

savuor · 2024-10-22T23:47:09Z

modules/fastcv/CMakeLists.txt

+
+  # here you can change the library to a specific one
+  set(FASTCV_LIB "/libs/Android/lib64/libfastcvopt.so")
+


savuor · 2024-10-22T23:53:27Z

modules/fastcv/README.md

+
+This module provides wrappers for several FastCV functions not covered by the corresponding HAL in OpenCV.
+Please note that:
+1. This module supports ARM architecture only. This means that CMake script aborts configuration under x86 platform even if you don't want to build binaries for your machine and just want to build docs or enable code analysis in your IDE. In that case you should fix CMakeLists.txt file as told inside it.


This behavior was changed, please change this part of readme.
Now CMake will just emit a warning if the platform is not ARM but won't stop configuration. Any attempts to link under any platform but ARM would lead to a failure. The reason is to let various IDEs run their code analysis tools that require the code to be compiled at least.

savuor · 2024-10-22T23:54:10Z

modules/fastcv/include/opencv2/arithm.hpp

+ * @param src2 Second source matrix of type CV_8S
+ * @param dst Resulting matrix of type CV_32S
+ */
+CV_EXPORTS_W void matmuls8s32(InputArray src1, _InputArray src2, OutputArray dst);


_InputArray src2 -> InputArray src2

savuor · 2024-10-22T23:57:46Z

modules/fastcv/include/opencv2/bilateralFilter.hpp

+
+/**
+ * @brief Applies Bilateral filter to an image considering d-pixel diameter of each pixel's neighborhood.
+          This filter does not work inplace.


This should be checked in the code with something like CV_Assert(src.data != dst.data);

savuor · 2024-10-23T00:01:23Z

modules/fastcv/include/opencv2/moments.hpp

+ * @brief Calculates all of the moments up to the third order of the image pixels' intensities
+          The results are returned in the structure cv::Moments.
+ * @param _src Input image with type CV_8UC1, CV_32SC1, CV_32FC1
+ * @param binary If 1, binary image (0x00-black, oxff-white); if 0, grayscale image


-> If true, assumes the image to be binary (0x00 for black, 0xff for white), otherwise assumes the image to be grayscale

savuor · 2024-10-23T01:15:55Z

modules/fastcv/test/test_remap.cpp

+    cv::minMaxLoc(diffImage, nullptr, &maxVal);
+
+    // Assert if the difference is acceptable (max difference should be less than 10)
+    CV_Assert(maxVal < 10 && "Difference between images is too high!");


The same about L_inf norm

savuor · 2024-10-23T01:16:22Z

modules/fastcv/test/test_scale.cpp

+    cv::minMaxLoc(diffImage, nullptr, &maxVal);
+
+    // Assert if the difference is acceptable (max difference should be less than 10)
+    CV_Assert(maxVal < 10 && "Difference between images is too high!");


The same about L_inf norm

savuor · 2024-10-23T01:17:13Z

modules/fastcv/test/test_scale.cpp

+    cv::minMaxLoc(diffImage, nullptr, &maxVal);
+
+    // Assert if the difference is acceptable (max difference should be less than 10)
+    CV_Assert(maxVal < 10 && "Difference between images is too high!");


The same about L_inf norm

savuor · 2024-10-23T01:18:36Z

modules/fastcv/test/test_scale.cpp

+    cv::fastcv::resizeDownBy2(inputImage, resized_image);
+
+    // Check if the output size is correct
+    EXPECT_EQ(resized_image.size().width, size.width * 0.5);


These checks can be moved to the accuracy test

savuor · 2024-10-23T01:18:41Z

modules/fastcv/test/test_scale.cpp

+    cv::fastcv::resizeDownBy4(inputImage, resized_image);
+
+    // Check if the output size is correct
+    EXPECT_EQ(resized_image.size().width, size.width * 0.25);


These checks can be moved to the accuracy test

cleanup and added make rules to download the fastcv bins from the 3rdparty repo Change-Id: I9a9db68a310263658ec93ea299ea37860d07c507

Fastcv ext changes for Opencv Acceleration

asmorkalov · 2024-10-29T13:43:07Z

modules/fastcv/CMakeLists.txt

+if((ARM OR AARCH64) AND (ANDROID OR (UNIX AND NOT APPLE AND NOT IOS AND NOT XROS)))
+  set(the_description       "FastCV extension module")
+  set(FCV_MODULE_HEADER_DIR "${CMAKE_CURRENT_BINARY_DIR}/inc")
+  set(FCV_MODULE_LIB_DIR    "${CMAKE_CURRENT_BINARY_DIR}/libs")
+
+  # here you can change the library to a specific one
+  set(FASTCV_LIB "${FCV_MODULE_LIB_DIR}/libfastcvopt.so")


There is the same code in main repo. I propose to remove duplicates.

we download the headers and libs to different folders in case someone build the opencv_contrib without enabling WITH_FASTCV flag

asmorkalov · 2024-10-29T13:43:41Z

modules/fastcv/CMakeLists.txt

+  if((NOT EXISTS ${FCV_MODULE_HEADER_DIR}) OR (NOT EXISTS ${FCV_MODULE_LIB_DIR}))
+    include("${CMAKE_CURRENT_SOURCE_DIR}/fastcv.cmake")
+    download_fastcv("${CMAKE_CURRENT_BINARY_DIR}")
+  endif()


Let it be called from the root cmake in main repo.

we have this cmake/OpenCVFindFastCV.cmake for the HAL in the main repo, this part is for extension, they are downloading to different folders

asmorkalov · 2024-10-29T13:44:23Z

modules/fastcv/README.md

+FastCV extension for OpenCV
+===========================
+
+This module provides wrappers for several FastCV functions not covered by the corresponding HAL in OpenCV.


or have implementation incompatible with OpenCV.

asmorkalov · 2024-10-29T13:45:25Z

modules/fastcv/fastcv.cmake

@@ -0,0 +1,35 @@
+function(download_fastcv root_dir)


It should be in main CMake.

there are already on in the main repo, will try to reuse that one

asmorkalov · 2024-10-29T13:47:11Z

modules/fastcv/include/opencv2/moments.hpp

+ * @param _src Input image with type CV_8UC1, CV_32SC1, CV_32FC1
+ * @param binary If 1, binary image (0x00-black, oxff-white); if 0, grayscale image
+ */
+CV_EXPORTS_W cv::Moments moments(InputArray _src, bool binary);


Moments are part of HAL and already wrapped, right?

asmorkalov · 2024-10-29T14:01:40Z

modules/fastcv/test/test_cluster_euclidean.cpp

+        double normL2  = cv::norm(diffs, NORM_L2) / nClusters;
+
+        EXPECT_LT(normL2, 0.392);


Average distance hides corner cases, e.g. all points, besides one are the same, but the last one is significant outlier. The check should be reworked.

asmorkalov · 2024-10-29T14:01:58Z

modules/fastcv/test/test_cluster_euclidean.cpp

+        double normH = cv::norm(bindings8u, trueBindings8u, NORM_HAMMING) / nPts;
+        EXPECT_LT(normH, 0.658);


asmorkalov · 2024-10-29T14:02:17Z

modules/fastcv/test/test_fast10.cpp

+    int border      = std::get<2>(p);
+    bool nmsEnabled = std::get<3>(p);
+
+    cv::Mat src = imread(cvtest::findDataFile("cv/shared/lena.png"), cv::IMREAD_GRAYSCALE);


asmorkalov · 2024-10-29T14:07:28Z

modules/fastcv/test/test_thresh.cpp

+
+    RNG& rng = cv::theRNG();
+    Mat src(size, CV_8UC1);
+    cvtest::randUni(rng, src, Scalar::all(0), Scalar::all(256));


256 - overflow.

asmorkalov · 2024-10-29T14:08:35Z

modules/fastcv/src/utils.cpp

+namespace cv {
+namespace fastcv {
+
+bool isInitialized = false;


The same initialization issue as in main repo.

will add singleton similar to main

sssanjee-quic added 2 commits October 16, 2024 17:55

Added FastCV Extension code for Opencv APIs

ca11f8f

Change-Id: I721b810181dd9127221b6e2182d7aad5ac679b68

Merge pull request #1 from CodeLinaro/FastcvHAL_1stPost

b8b5757

Added FastCV Extension code for Opencv APIs

savuor reviewed Oct 23, 2024

View reviewed changes

sssanjee-quic added 2 commits October 25, 2024 17:52

Fastcv ext changes for Opencv Acceleration

ddaff0f

cleanup and added make rules to download the fastcv bins from the 3rdparty repo Change-Id: I9a9db68a310263658ec93ea299ea37860d07c507

Merge pull request #2 from CodeLinaro/FastcvHAL_1stPost

25a572f

Fastcv ext changes for Opencv Acceleration

asmorkalov reviewed Oct 29, 2024

View reviewed changes

asmorkalov mentioned this pull request Nov 7, 2024

FastCV-based HAL for OpenCV acceleration opencv/opencv#26316

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FastCV Extension code for Opencv APIs #3811

FastCV Extension code for Opencv APIs #3811

sssanjee-quic commented Oct 16, 2024

savuor left a comment

savuor Oct 22, 2024

savuor Oct 22, 2024

savuor Oct 22, 2024

savuor Oct 22, 2024

savuor Oct 23, 2024

savuor Oct 23, 2024

savuor Oct 23, 2024

savuor Oct 23, 2024

savuor Oct 23, 2024

savuor Oct 23, 2024

asmorkalov Oct 29, 2024

ghost Oct 30, 2024

asmorkalov Oct 29, 2024

ghost Oct 30, 2024

asmorkalov Oct 29, 2024

ghost Oct 30, 2024

asmorkalov Oct 29, 2024

ghost Oct 30, 2024

asmorkalov Oct 29, 2024

asmorkalov Oct 29, 2024

asmorkalov Oct 29, 2024

asmorkalov Oct 29, 2024

asmorkalov Oct 29, 2024

asmorkalov Oct 29, 2024

quic-apreetam Oct 30, 2024


		# here you can change the library to a specific one
		set(FASTCV_LIB "/libs/Android/lib64/libfastcvopt.so")

		double normL2 = cv::norm(diffs, NORM_L2) / nClusters;

		EXPECT_LT(normL2, 0.392);

		double normH = cv::norm(bindings8u, trueBindings8u, NORM_HAMMING) / nPts;
		EXPECT_LT(normH, 0.658);

FastCV Extension code for Opencv APIs #3811

Are you sure you want to change the base?

FastCV Extension code for Opencv APIs #3811

Conversation

sssanjee-quic commented Oct 16, 2024

Pull Request Readiness Checklist

savuor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment