Support UIA LabeledBy property

[michaelweghorn]

Link to issue number:

Fixes #17442

Summary of the issue:

UIA has a UIA_LabeledByPropertyId property, see https://learn.microsoft.com/en-us/windows/win32/winauto/uiauto-automation-element-propids :

Identifies the LabeledBy property, which is an automation element that contains the text label for this element.
This property can be used to retrieve, for example, the static text label for a combo box.
Variant type: VT_UNKNOWN
Default value: NULL

However, that one was so far not taken into account by NVDA's "labeledBy" property for UIA elements.

Description of user facing changes

None, unless they're using the Python console or addons making use of the property.

Description of development approach

Override NVDAObject._get_labeledBy in UIA to retrieve and return the UIAElement reported by the UIA_LabeledByPropertyId property.

Testing strategy:

1. start NVDA

2. build Qt's qtbase module including an additional change that implements support for UIA_LabeledByPropertyId in Qt ( https://codereview.qt-project.org/c/qt/qtbase/+/606710 )

3. run Qt's "calendarwidgets" sample app (qtbase/examples/widgets/widgets/calendarwidget/calendarwidget.exe)

4. move focus to the "Locale" combobox

5. open NVDA's Python console (Ctrl+Insert+Z)

6. print the object that the currently focused edit is labelled by and some of its properties:

   focus.labeledBy
   <NVDAObjects.UIA.UIA object at 0x0BE47ED0>
   focus.labeledBy.name
   'Locale'
   focus.labeledBy.role
   <Role.STATICTEXT: 7>

Known issues with pull request:

None

Code Review Checklist:

• Documentation:
  • Change log entry
  • User Documentation
  • Developer / Technical Documentation
  • Context sensitive help for GUI changes
• Testing:
  • Unit tests
  • System (end to end) tests
  • Manual testing
• UX of all users considered:
  • Speech
  • Braille
  • Low Vision
  • Different web browsers
  • Localization in other languages / culture than English
• API is compatible with existing add-ons.
• Security precautions taken.

@coderabbitai summary

[LeonarddeR]

@michaelweghorn Could you elaborate on use cases for making labeledBy available like this? I.e. are there cases where NVDA doesn't read a label properly in which case using labeledBy can be a potential improvement?

[michaelweghorn]

@michaelweghorn Could you elaborate on use cases for making labeledBy available like this? I.e. are there cases where NVDA doesn't read a label properly in which case using labeledBy can be a potential improvement?

@LeonarddeR At least on Linux, toolkits indeed don't necessarily report an accessible name for a control that already has a labelled-by relation set to a label that describes what the control is for.
For the Qt example used in this PR's description, there would be a label "Locale:" and then the combobox to select the locale. The combobox doesn't have an accessible name set, but the Orca screen reader would follow the labelled-by relation and read out the label instead when the combobox gets focus. If it wouldn't do that, it would be unclear to the user what the combobox is there for.

Now I'm not 100% sure of whether not setting an accessible name is valid with UIA on Windows. The UIA_NamePropertyId doc doesn't seem to mention that explicitly. (From what I've seen at least Qt would fall back to using the text of the labelled-by widget for the accessible name of the combobox on Windows - other than on Linux.)

For the web, the "Accessible Name and Description Computation" specification says that the "aria-labelledby" relation takes preference over the "aria-label" ARIA attribute. However, IIUC, that's not something that NVDA would have to do, but it's the responsibility of the user agent/browser to evaluate that when calculating an accessible name, so NVDA can presumably just use the accessible name as is.

Right now, I don't have a specific case at hand that's broken without this change in place. The primary reason why I looked into it was that I noticed the property being available in NVDA's Python console, but then not actually reporting what I was expecting while analyzing an issue in an app where the labelled-by property might have been relevant. In any case, I think it can be useful for analysis.