Skip to content
This repository has been archived by the owner on Jul 16, 2024. It is now read-only.

Commit

Permalink
First pass at multi-tag docs
Browse files Browse the repository at this point in the history
  • Loading branch information
mcm001 committed Jan 20, 2024
1 parent 66cfd77 commit f571416
Show file tree
Hide file tree
Showing 3 changed files with 50 additions and 1 deletion.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
51 changes: 50 additions & 1 deletion source/docs/apriltag-pipelines/multitag.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,53 @@
MultiTag Localization
=====================

Coming soon!
PhotonVision can combine AprilTag detections from multiple simultaniously observed apriltags from a particular camera wih information about where tags are expected to be located on the field to produce a better estimate of where the camera (and therefore robot) is located on the field. PhotonVision can calculate this multi-target result on your coprocessor, reducing CPU usage on your RoboRio. This result is sent over NetworkTables along with other detected targets as part of the ``PhotonPipelineResult`` provided by PhotonLib.

.. warning:: MultiTag requires an accurate field layout JSON be uploaded! Differences between this layout and tag's physical location will drive error in the estimated pose output.

Enabling MultiTag
^^^^^^^^^^^^^^^^^

Ensure that 3D mode is enabled (and do so if not). Navigate to the Output tab and enable "Do Multi-Target Estimation". This enables MultiTag using the uploaded field layout JSON to calculate your camera's pose in the field. This 3D transform will be shown as an additional table in the "targets" tab, along with the IDs of AprilTags used to compute this transform.

.. image:: images/multitag-ui.png
:width: 600
:alt: Multitarget enabled and running in the PhotonVision UI

.. note:: By default, enabling multi-target will disable calculating camera-to-target transforms for each observed AprilTag target to increase performance; the X/Y/angle numbers shown in the target table of the UI are instead calculated using the tag's expected location (per the field layout JSON) and the field-to-camera transform calculated using MultiTag. If you additionally want the individual camera-to-target transform calculated using SolvePNP for each target, enable "Always Do Single-Target Estimation".

This multi-target pose estimate can be accessed using PhotonLib. We suggest using the PhotonPoseEstimator class with the ``MULTI_TAG_PNP_ON_COPROCESSOR`` strategy to simplify code, but the transform can be directly accessed using ``getMultiTagResult``/``MultiTagResult()`` (Java/C++).



.. tab-set-code::

.. code-block:: java
var result = camera.getLatestResult();
if (result.getMultiTagResult().estimatedPose.isPresent) {
Transform3d fieldToCamera = result.getMultiTagResult().estimatedPose.best;
}
.. code-block:: C++

auto result = camera.GetLatestResult();
if (result.MultiTagResult().result.isPresent) {
frc::Transform3d fieldToCamera = result.MultiTagResult().result.best;
}

.. note:: The returned field to camera transform is a transform from the fixed field origin to the camera's coordinate system. This does not change based on alliance color, and by convention is on the BLUE ALLIANCE wall.

Updating the Field Layout
^^^^^^^^^^^^^^^^^^^^^^^^^

PhotonVision ships by default with the `2024 field layout JSON <https://github.com/wpilibsuite/allwpilib/blob/main/apriltag/src/main/native/resources/edu/wpi/first/apriltag/2024-crescendo.json>`_. The layout can be inspected by navigating to the settings tab and scrolling down to the "AprilTag Field Layout" card, as shown below.

.. image:: images/field-layout.png
:width: 600
:alt: The currently saved field layout in the Photon UI

An updated field layout can be uploaded by navigating to the "Device Control" card of the Settings tab and clicking "Import Settings". In the pop-up dialog, select the "Apriltag Layout" type and choose a updated layout JSON (in the same format as the WPILib field layout JSON linked above) using the paperclip icon, and select "Import Settings". The apriltag layout in the "AprilTag Field Layout" card below should update to reflect the new layout.

.. note:: Currently, there is no way to update this layout using PhotonLib, although this feature is under consideration.

0 comments on commit f571416

Please sign in to comment.