WPF combobox with datatemplate - class name read instead of value

[nvaccessAuto]

Reported by manish on 2013-10-11 06:37
in WPF, if I use a combo box in an application, NVDA assigns the UIA class ComboBoxWithoutValuePattern to this control.
If the control contains list items directly, then this works correctly and the list item name is spoken.
However, if I use a data template to display a value in the combo box, then instead of the text displayed in the list item, the name of the class of the list item is spoken. I have to use object navigation to go to the child of each list item to read the actual text displayed.

The attached application has an example of a working and a non working combo box. Also attached is a zip of the source of this app.

In this app, the second combo box shows values like "apple", "orange", "cherry" while NVDA reads "TestComboBox.Fruit" for all the items , which is actually the name of the underlying class bound to the combo box.

This problem occurs in many applications built using WPF, including VS 2012, ms office 2013 etc.

[nvaccessAuto]

Attachment TestComboBox.exe added by manish on 2013-10-11 06:41
Description:
sample app.
Update:
File added from Trac
TestComboBox.exe.zip

[nvaccessAuto]

Attachment TestComboBox-source.zip added by manish on 2013-10-11 06:41
Description:
source of sample app.
Update:
File added from Trac
TestComboBox-source.zip

[nvaccessAuto]

Comment 1 by jteh on 2013-10-11 07:30
Yet another bug in WPF's UIA implementation. WPF is actually exposing that name as the name of the combo box items. The question is how to detect this and work around it (use the child of the item) without breaking other (correct) implementations.

[nvaccessAuto]

Comment 2 by manish on 2013-10-31 06:01
This problem occurs because the combo box or the list item has no knowledge of how the item has been rendered on the screen. The list item may itself consist of multiple smaller widgets - as is in the case of windows explorer for example.
The list item simply calls the "ToString" method of the item that is its data source. If the underlying class implements the "ToString()" method, then that is returned as the name. Otherwise, the base class' System.Object.ToString() gets called and it returns the fully qualified class name, which is what we see in the attached example.

If I implement the ToString() method in the Fruit class in this example, then we'll get a valid name read out.

In the .Net world, however, we hardly ever implement the ToString() method on a class unless if we explicitly need it for some reason. There is almost never a reason to do this.
So, short of going back to the app developers and asking them to implement ToString methods on all classes that they bind to list boxes and combo boxes, is there a way we can have NVDA try to build this information itself?

A potential huristic way of getting around this problem is for NVDA to check if the ListItem has any children (which will mean that a dataTemplate has been used) and then to check if the first "n" elements in the list all return the same "name" (which will mean that "ToString()" has not been implemented on the underlying object.
If the above 2 conditions are true, then iterate over all the children of the listItem and concatenate their name properties and return as the name property of the ListItem.

This has obvious flaws in the sense that all first "n" may actually be the same (e.g. the first 10 mails in the mailbox may all be from the same sender and subject etc.)
or the children may not expose a useful "name" property as well (e.g. the child may just be an image of a fruit in this example).

However, in either case, we aren't really any worse off that the current situation and the upside is much better usability of WPF combo boxes and list boxes.
Thoughts?

[nvaccessAuto]

Comment 3 by jteh on 2013-10-31 08:15
This kind of hack is always pretty ugly and is always likely to result in some unexpected use case being broken. For example, what if there just happen to be multiple children with the same text, but that text is actually intended?

That aside, my main concern is speed, which is why I'd prefer to restrict this to only certain combo boxes. Fetching/iterating over children is not cheap as far as performance is concerned. Maybe we can just do it for all WPF combo boxes based either on the window class name or UIA provider description/class name.

[LeonarddeR]

Sounds a bit like a needs an external fix. Note that wpf has more brokenness in it, for example regarding data grids.

[Adriani90]

@manish, @LeonarddeR, @jcsteh if this is still an issue, are there any improvements made by UIA in this regard?

[Adriani90]

I have tested with NVDA 2019.1.1, this is still an issue.
cc: @ABuffEr, @Robert-J-H

[A-jitHub]

Hello,
Were you able to get any work around for this issue? I am also facing this issue with JAWS. Please let me know if anybody has any fixes for this.

[Adriani90]

This is still an issue in NVDA 2023.3.

[Hades948]

It's kind of hidden in the post from Oct 31, 2023, so I think some people are missing it. But the workaround is to just implement ToString in the model class. That worked for me.

[Adriani90]

cc: @siagupta0202, @anjali-wpf, @arpitmathur, @dipeshmsft, @himgoyalmicro, @harshit7962 could someone please look into this? It seems this needs a fix from WPF side, so combo boxes are handled properly. NVDA users need to use a quite uncomfortalbe workaround with object navigation in order to work with such combo boxes, and this type of navigation is not really intuitive especially for new screen reader users.

[Adriani90]

@michaelweghorn you implemented #17443. Would it help in this particular case? Does the labelled by property support make data items to be announced instead of the class name on the parent object level?
cc: @LeonarddeR maybe you also have suggestions here, we also see this problem very often e.g. in Microsoft access.

[michaelweghorn]

@michaelweghorn you implemented #17443. Would it help in this particular case? Does the labelled by property support make data items to be announced instead of the class name on the parent object level?

I haven't tried to run the attached example, but this sounds unrelated to me at first.
I would rather expect the labelled-by relation for a combobox to be set e.g. to point to a label next to the combobox that says what the user can set in the combobox. I currently don't see a connection how that would be related to the announcement of the actually selected value. But in any case, with #17443 in place, you can try whether focus.labeledBy prints anything for the object you're interested in to see whether that relation is set on the UIA level.