diff --git a/docs/PsychCV-April3DSettings.md b/docs/PsychCV-April3DSettings.md new file mode 100644 index 00000000..e584b210 --- /dev/null +++ b/docs/PsychCV-April3DSettings.md @@ -0,0 +1,31 @@ +# [PsychCV('April3DSettings')](PsychCV-April3DSettings) +##### [Psychtoolbox](Psychtoolbox)>[PsychCV](PsychCV).{mex*} subfunction + +[glProjectionMatrix, camCalib, tagSize, minD, maxD] = PsychCV('April3DSettings' [, camCalib][, tagSize][, minD][, maxD]); + +Return current 3D marker pose reconstruction and 3D rendering parameters, +optionally change them. +For 6-[DoF](DoF) 3D marker tag pose computation, and for return of [OpenGL](OpenGL) compliant +rendering matrices, information about the physical size of april tag markers, +[OpenGL](OpenGL) near and far clipping planes, andthe intrinsic optical parameters of the +camera that captures the marker images are needed. +These parameters can be changed anytime with this subfunction. Following +settings are available: +'camCalib' Intrinsic camera parameters vector: [cx, cy, fx, fy]. (cx,cy) is +sensor center in pixels, (fx,fy) is focal length in pixels. +'tagSize' Size of an April tag in meters, ie. length of one side of the square +tag. +'minD' Near clipping distance for [OpenGL](OpenGL) frustum computation - Affects +'glProjectionMatrix' matrix only. +'maxD' Far clipping distance for [OpenGL](OpenGL) frustum computation - Affects +'glProjectionMatrix' matrix only. + +The optionally returned 'glProjectionMatrix', based on these parameters, can be +used as [OpenGL](OpenGL) GL\_PROJECTION\_MATRIX to render 3D objects superimposed and +aligned with the markers in the video input image from the camera, for +debugging, diagnostics, or AR / mixed reality applications. + + + +###See also: +[AprilDetectMarkers](PsychCV-AprilDetectMarkers) diff --git a/docs/PsychCV-AprilDetectMarkers.md b/docs/PsychCV-AprilDetectMarkers.md new file mode 100644 index 00000000..c8fb373f --- /dev/null +++ b/docs/PsychCV-AprilDetectMarkers.md @@ -0,0 +1,78 @@ +# [PsychCV('AprilDetectMarkers')](PsychCV-AprilDetectMarkers) +##### [Psychtoolbox](Psychtoolbox)>[PsychCV](PsychCV).{mex*} subfunction + +[detectedMarkers] = PsychCV('AprilDetectMarkers'[, markerSubset=all][, infoType=all]); + +Detect apriltags in the current video image, return information about them. + +Analyzes the current video image, stored in the internal input image buffer, and +tries to detect the apriltag markers in them. For all detected tags, their 2D +center, their 2D corners, and their 3D position and orientation is computed, +using the provided camera calibration. The return argument 'detectedMarkers' is +an array of structs, with one struct for each successfully detected tag. The +struct contains info about the identity of the tag, confidence values for the +reliability of detection, and the estimated 6-[DoF](DoF) 3D position and 3D pose of the +marker tag in 3D space, relative to the cameras reference frame and origin. +Please note that, in general, 3D pose estimates are not as reliable and accurate +as the 2D detection of markers. Furthermore, the 3D orientation of the marker is +way less well defined and accurate than the 3D position of the markers center. +Often the estimated 3D orientation may be outright rubbish! +You can use [PsychCV](PsychCV)('AprilSettings'); to tune various parameters related to the +2D marker detection, including the use of multiple processing threads for higher +performance. +For 6-[DoF](DoF) 3D estimation, you need to provide camera intrinsic parameters and the +size of the tags via [PsychCV](PsychCV)('April3DSettings'). +If you don't want to detect all tags, but only a subset, then pass a list of +candidate tag id's via the list 'markerSubset', to reduce your codes complexity +and computation time. +To further reduce computation time, you can ask for only a subset of information +by providing 'infoType'. By default all information is returned at highest +quality and robustness with longest computation time: +- 2D marker detection data is always returned. +- A value of +1 will return 3D pose. +Omitting the value +1 from 'infoType' will avoid 3D pose estimation. + +The returned structs contain the following fields: +'Id' The decoded apriltag id. Hamming code error correction is used for +decoding. +'MatchQuality' A measure of the quality of the binary decoding process. This is +what the apriltag library calls decision\_margin. Higher numbers roughly indicate +better decodes. It is a reasonable measure of detection quality only for small +tags, mostly meaningless for bigger tags. +'HammingErrorBits' Number of error bits corrected. Smaller numbers are better. +'Corners2D' A 2-by-4 matrix with the 2D pixel coordinates of the detected +corners of the tag in the input image, each column representing one corner [x ; +y]. These always wrap counter-clock wise around the tag. +'Center2D' A vector with the 2D pixel coordinates of the estimated center of the +tag in the input image. +'PoseError' If 3D pose estimation was used, the object space error of the +returned pose. +'T' A 3-component [x ; y; z] translation vector which encodes the estimated 3D +location of the center of the tag in space, in units of meters, relative to the +origin of the camera. +'R' A 3x3 rotation matrix, encoding the estimated pose of the tag, relative to +the cameras reference frame. Convention is that the tag itself lies in the x-y +plane of its local reference frame, and the positive z-axis sticks out of the +tags surface like a surface normal vector. +'TransformMatrix' A 4x4 transformation matrix representing position and +orientation all in one, for convenience. Simply the product [TransformMatrix](TransformMatrix) = T +\* R, extended to a 4x4 format. This represents pose relative to the cameras +origin, x-axis to the right, y-axis down, z-axis along the looking direction aka +optical axis. +'ModelViewMatrix' A 4x4 RHS transformation matrix, directly usable for 3D [OpenGL](OpenGL) +rendering of objects in the tags local reference frame. It can be used directly +as GL\_MODELVIEW\_MATRIX for rendering 3D content on top of the tag in the video +image, or right-multiplied to the active GL\_MODELVIEW\_MATRIX to represent the +tags 6 [DoF](DoF) pose relative to the 3D [OpenGL](OpenGL) cameras origin. You need to use the +GL\_PROJECTION\_MATRIX returned by matrix = [PsychCV](PsychCV)('April3DSettings'); for +rendering superimposed to images from the camera that captured the april tags. +This matrix is a rotated version of 'TransformMatrix', rotated 180 degrees +around the x-axis for [OpenGL](OpenGL) compatibility, as apriltag has x-axis to the right, +y-axis down, z-axis along optical looking direction axis, whereas [OpenGL](OpenGL) has its +x-axis to the right, y-axis up, and the negative z-axis along optical looking +direction axis / viewing direction, ie. 180 degrees rotated. + + + +###See also: +[AprilInitialize](PsychCV-AprilInitialize) [AprilSettings](PsychCV-AprilSettings) [April3DSettings](PsychCV-April3DSettings) diff --git a/docs/PsychCV-AprilInitialize.md b/docs/PsychCV-AprilInitialize.md new file mode 100644 index 00000000..e529bdd1 --- /dev/null +++ b/docs/PsychCV-AprilInitialize.md @@ -0,0 +1,52 @@ +# [PsychCV('AprilInitialize')](PsychCV-AprilInitialize) +##### [Psychtoolbox](Psychtoolbox)>[PsychCV](PsychCV).{mex*} subfunction + +[inputImageMemBuffer] = PsychCV('AprilInitialize', tagFamilyName, imgWidth, imgHeight, imgChannels [, imgFormat][, maxNrTags]); + +Initialize apriltag prior to first use. + +Apriltag markers are loaded for the given apriltag family 'tagFamilyName'. +The following tag families are currently supported: + tag36h11 + tag25h9 + tag16h5 + tagCircle21h7 + tagCircle49h12 - Use maxNrTags to restrict size! + tagCustom48h12 - Use maxNrTags to restrict size! + tagStandard41h12 + tagStandard52h13 - Use maxNrTags to restrict size! + +Internal video image memory buffers are set up for input images of size +'imgWidth' x 'imgHeight' pixels, with 'imgChannels' mono or color channels (1 +MONO, 3 RGB, or 4 RGBA are valid settings, but 1 for MONO is the most efficient +choice for minimal computation time). For 3 or 4 channels, the input color +format 'imgFormat' (or a default setting if 'imgFormat' is omitted) defines +color channel ordering for the input pixel bytes. 'imgFormat' can be one of: +RGB = 1, BGR = 2, BGRA = 4 (default for 4 channels), ARGB = 7, MONO = 6. Other +pixel formats are not supported for input images, but these are the ones +provided by Psychtoolbox's various video capture engines for different video +sources and settings. For 1 channel mono/grayscale or 3 channel RGB color +content, you don't need to specify 'imgFormat' as the chosen default will always +work, but for 4 channel content with alpha channel, you may have to specify +'imgFormat' if your machine does not return BGRA ordered pixels, but ARGB +ordered pixels, or marker detection on 4 channel content may fail or perform +poorly. +When using a tag family with many potential tags, you can limit the number of +tags to use to the first 'maxNrTags' tags if you specify 'maxNrTags'. This is +important for certain large tag families, as using the tag family at its full +capacity may consume a lot of memory and take a very long time to initialize! +Then apriltags is initialized, and a memory buffer handle 'inputImageMemBuffer' +to the internal video memory input buffer is returned. + +You should pass this handle to Psychtoolbox functions for videocapture to +acquire video images from the scene containing the tags, and to store that input +image inside PsychCV's video buffer. + +After this step, you can commence the actual tracking operations by calls to +[Screen](Screen)()'s video capture engine and the [PsychCV](PsychCV)('AprilDetectMarkers', ...); +subfunction. + + + +###See also: +[AprilShutdown](PsychCV-AprilShutdown) [AprilDetectMarkers](PsychCV-AprilDetectMarkers) [AprilSettings](PsychCV-AprilSettings) [April3DSettings](PsychCV-April3DSettings) diff --git a/docs/PsychCV-AprilSettings.md b/docs/PsychCV-AprilSettings.md new file mode 100644 index 00000000..6245da24 --- /dev/null +++ b/docs/PsychCV-AprilSettings.md @@ -0,0 +1,37 @@ +# [PsychCV('AprilSettings')](PsychCV-AprilSettings) +##### [Psychtoolbox](Psychtoolbox)>[PsychCV](PsychCV).{mex*} subfunction + +[nrThreads, imageDecimation, quadSigma, refineEdges, decodeSharpening, criticalRadAngle, deglitch, maxLineFitMse, minWhiteBlackDiff, minClusterPixels, maxNMaxima] = PsychCV('AprilSettings' [, nrThreads][, imageDecimation][, quadSigma][, refineEdges][, decodeSharpening][, criticalRadAngle][, deglitch][, maxLineFitMse][, minWhiteBlackDiff][, minClusterPixels][, maxNMaxima]); + +Return current tracker parameters, optionally change tracker parameters. +These settings are set to reasonable defaults at startup, but can be changed +anytime with this subfunction. Most of these settings define tradeoffs between +computation time aka tracking speed and quality/robustness of tracking. +Following settings are available: +'nrThreads' Number of processing threads to use for speeding up operation. +Default is 1 for single-threaded operation. +'imageDecimation' 1 = Process full image. \> 1 = Only work on resolution +decimated image for higher speed at lower precision. Default is 2. +'quadSigma' How much blurring (values \> 0) or sharpening (values < 0) to apply +to input images to reduce noise. Default is 0 for none. +'refineEdges' 1 = Perform edge refinement on detected edges (cheap, and the +default), 0 = Use simpler strategy. +'decodeSharpening' How much sharpening should be done to decoded images? Can +help small tags. Default is 0.25. +'criticalRadAngle' How close pairs of edges can be to straight before rejection. +0 = Don't reject. Default is 10 degrees. +'deglitch' Should the thresholded image be deglitched (1) or not (0)? Only +useful for very noisy images. Default is 0 for false. +'maxLineFitMse' When fitting lines to contours, what is the maximum mean squared +error allowed? For rejecting contours far from quad shape. Default is 10.0. +'minWhiteBlackDiff' How much brighter (in pixel values 0 - 255) must white +pixels be than black pixels? Default is 5. +'minClusterPixels' Reject quads containing less than this number of pixels. +Default is 5. +'maxNMaxima' How many corner candidates to consider when segmenting a group of +pixels into a quad. Default is 10. + + + +###See also: +[AprilDetectMarkers](PsychCV-AprilDetectMarkers) diff --git a/docs/PsychCV-AprilShutdown.md b/docs/PsychCV-AprilShutdown.md new file mode 100644 index 00000000..4762bd9d --- /dev/null +++ b/docs/PsychCV-AprilShutdown.md @@ -0,0 +1,14 @@ +# [PsychCV('AprilShutdown')](PsychCV-AprilShutdown) +##### [Psychtoolbox](Psychtoolbox)>[PsychCV](PsychCV).{mex*} subfunction + +PsychCV('AprilShutdown'); + +Shut down apriltag after use, release all resources. +The memory buffer handle 'inputImageMemBuffer' returned by a prior call to +inputImageMemBuffer = [PsychCV](PsychCV)('AprilInitialize', ...); will be invalid after +this shutdown call and must not be used anymore, or Psychtoolbox will crash! + + + +###See also: +[AprilInitialize](PsychCV-AprilInitialize) diff --git a/docs/PsychCV-CopyMatrixToMemBuffer.md b/docs/PsychCV-CopyMatrixToMemBuffer.md new file mode 100644 index 00000000..4bc662aa --- /dev/null +++ b/docs/PsychCV-CopyMatrixToMemBuffer.md @@ -0,0 +1,14 @@ +# [PsychCV('CopyMatrixToMemBuffer')](PsychCV-CopyMatrixToMemBuffer) +##### [Psychtoolbox](Psychtoolbox)>[PsychCV](PsychCV).{mex*} subfunction + +PsychCV('CopyMatrixToMemBuffer', matrix, memBufferPtr); + +Copies a Matlab/Octave uint8 or double matrix 'matrix' into a C memory buffer, +whose memory (void\*) pointer is encoded as a double scalar value 'memBufferPtr'. +The target buffer must be of sufficient size, no checking is performed! This is +essentially a C memcpy() operation, so use with caution, or Matlab/Octave will +crash! + + +###See also: + diff --git a/docs/PsychCV-DescribeModuleFunctionsHelper.md b/docs/PsychCV-DescribeModuleFunctionsHelper.md new file mode 100644 index 00000000..094341a9 --- /dev/null +++ b/docs/PsychCV-DescribeModuleFunctionsHelper.md @@ -0,0 +1,15 @@ +# [PsychCV('DescribeModuleFunctionsHelper')](PsychCV-DescribeModuleFunctionsHelper) +##### [Psychtoolbox](Psychtoolbox)>[PsychCV](PsychCV).{mex*} subfunction + +subfunctionNames = Modulename('DescribeModuleFunctionsHelper' [, mode] [, subfunctionName]); + +Return a cell array of strings naming all subfunctions supported by this module +if the optional 'subfunctionName' argument is omitted. If 'subfunctionName' is a +string with a valid subfunction name for the module, and 'mode' is 1, return a 3 +element cell array of strings which describe the detailed syntax, help and see +also strings for that function - the text you'd get for +Modulename('subfunctionName?'); + + +###See also: + diff --git a/docs/PsychCV-Verbosity.md b/docs/PsychCV-Verbosity.md new file mode 100644 index 00000000..39bcca47 --- /dev/null +++ b/docs/PsychCV-Verbosity.md @@ -0,0 +1,14 @@ +# [PsychCV('Verbosity')](PsychCV-Verbosity) +##### [Psychtoolbox](Psychtoolbox)>[PsychCV](PsychCV).{mex*} subfunction + +oldlevel = PsychCV('Verbosity' [,level]); + +Set level of verbosity for error/warning/status messages. 'level' optional, new +level of verbosity. 'oldlevel' is the old level of verbosity. The following +levels are supported: 0 = Shut up. 1 = Print errors, 2 = Print also warnings, 3 += Print also some info, 4 = Print more useful info (default), \>5 = Be very +verbose (mostly for debugging the driver itself). + + +###See also: + diff --git a/docs/PsychCV-Version.md b/docs/PsychCV-Version.md new file mode 100644 index 00000000..d368863b --- /dev/null +++ b/docs/PsychCV-Version.md @@ -0,0 +1,10 @@ +# [PsychCV('Version')](PsychCV-Version) +##### [Psychtoolbox](Psychtoolbox)>[PsychCV](PsychCV).{mex*} subfunction + +struct=PsychCV('Version') + +return the version of [PsychCV](PsychCV) in a struct + + +###See also: + diff --git a/docs/PsychCV.md b/docs/PsychCV.md index f69e2f1e..2c8f2dd6 100644 --- a/docs/PsychCV.md +++ b/docs/PsychCV.md @@ -1,25 +1,43 @@ # [PsychCV](PsychCV) -##### >[Psychtoolbox](Psychtoolbox)>[PsychBasic](PsychBasic) +##### [Psychtoolbox](Psychtoolbox)>[PsychCV](PsychCV) -[PsychCV](PsychCV) is a MEX file for computer-vision applications. [PsychCV](PsychCV) has -many functions; type "[PsychCV](PsychCV)" for a list: - [PsychCV](PsychCV) +PsychCV - Helper module for miscellaneous functionality related to image processing and/or computer vision: -Please note that [PsychCV](PsychCV) is only supported on recent platforms, e.g., -Matlab versions 7.4 (R2007a) and later or Octave-3 on Windows, and only -on Intel based Macintosh computers under OS/X but not on [PowerPC](PowerPC) -machines. +Copyright 2008 - 2024 Mario Kleiner. Licensed under MIT license. For potential statically included libraries, +see licenses below. This module employs various different 3rd party software, so here are the credits for those parts: + +The 'Aprilxxx' subfunctions for April tag tracking are implemented by use of the apriltag library from: +https://april.eecs.umich.edu/software/apriltag and https://github.com/AprilRobotics/apriltag . +The apriltag library is licensed under BSD-2 clause license. See Psychtoolbox main License.txt file for details. + + +General information and settings: + +version = PsychCV('[Version](PsychCV-Version)'); +oldlevel = PsychCV('[Verbosity](PsychCV-Verbosity)' [,level]); + +Helper functions for memory buffer copies: + +PsychCV('[CopyMatrixToMemBuffer](PsychCV-CopyMatrixToMemBuffer)', matrix, memBufferPtr); + +Support for the apriltag 2D/3D april tag marker tracking library: + +[inputImageMemBuffer] = PsychCV('[AprilInitialize](PsychCV-AprilInitialize)', tagFamilyName, imgWidth, imgHeight, imgChannels [, imgFormat][, maxNrTags]); +[detectedMarkers] = PsychCV('[AprilDetectMarkers](PsychCV-AprilDetectMarkers)'[, markerSubset=all][, infoType=all]); +PsychCV('[AprilShutdown](PsychCV-AprilShutdown)'); +[nrThreads, imageDecimation, quadSigma, refineEdges, decodeSharpening, criticalRadAngle, deglitch, maxLineFitMse, minWhiteBlackDiff, minClusterPixels, maxNMaxima] = PsychCV('[AprilSettings](PsychCV-AprilSettings)' [, nrThreads][, imageDecimation][, quadSigma][, refineEdges][, decodeSharpening][, criticalRadAngle][, deglitch][, maxLineFitMse][, minWhiteBlackDiff][, minClusterPixels][, maxNMaxima]); +[glProjectionMatrix, camCalib, tagSize, minD, maxD] = PsychCV('[April3DSettings](PsychCV-April3DSettings)' [, camCalib][, tagSize][, minD][, maxD]); + PsychCV is a MEX file for computer-vision applications. PsychCV has + many functions; type "PsychCV" for a list: + PsychCV + + Please note that PsychCV is only supported on recent platforms, e.g., + Matlab versions 7.4 (R2007a) and later or Octave-3 on Windows, and only + on Intel based Macintosh computers under OS/X but not on PowerPC + machines. + + - -
Psychtoolbox/PsychBasic/PsychCV.m
-