Deprecate recording client width and client height

[srallen]

Back in 2019, we thought about deprecating recording client width and client height in the classification metadata. The reasoning:

1. Researchers may not understand the difference between natural width, height and client width, height. One such confusion example:

Basically, I use it to normalize coordinates in an image (annotation coords -> real value in the range [0,1]) so that I can aggregate them together. I think that natural is probably what I actually want and so far it hasn't been important because all my trial images have been the same size. I just grabbed the client versions because they were the first apparently relevant quantities I saw in the classification extract.

Response

I think what you want is natural for this use. Client sizes will vary for each classification and user because it’s how it rendered in the browser, so if a user has low resolution monitor or another is using an iPad, it’ll change.

I think this is a good example for us to not record client size because it might be confusing in the analysis about which one to use.

2. Analysis of classification quality with regard to viewport size can be derived using the the classification metadata viewport recorded separately (see:

   front-end-monorepo/packages/lib-classifier/src/store/Classification/ClassificationMetadata.js

   Lines 28 to 31 in 9a62abf

   	viewport: types.frozen({
   	height: types.optional(types.integer, window.innerHeight),
   	width: types.optional(types.integer, window.innerWidth)
   	}),

   )

3. The viewport metadata can be used as a proxy to determine separate non-logged in users who share IP and user-agent in post-classification analysis.

4. Client width and height becomes meaningless in complex composite subject viewers that we're building now. See this issue regarding the SubjectGroupViewer: Subject Group Viewer: re-examine how viewer records subject/grid width & height? #1594. With the VariableStarViewer, client width and height of the container element doesn't really mean anything when the viewer is a composite of two SVG scatter plots, two SVG bar plots, and a single image. On deck, we also have a scatter plot and image viewer as well aka DataImageViewer. Demand for complex composites and/or SVG based plots are going to continue where client height and width of the container of the modules won't have useful meaning.

5. Client width and height doesn't really tell us much about what the subject looked like when the volunteer classified since we allow pan and zoom by default now. Drawing marks are normalized using natural width and height.

6. Deprecating recording client width and height will help simplify a lot of our subject viewer code.

Is there a use case for client width and height that we're missing?

[srallen]

Confirmed with @CKrawczyk that aggregation does not use it:

My aggregation code does not use image dimensions in any way

[eatyourgreens]

ALICE looks up image dimensions on load, in the browser, so I don't think it would be affected either.
https://github.com/zooniverse/alice/blob/dae27c93cc397049ce10afa89d3cb7abddca76f5/src/components/SubjectViewer/components/SVGView/SVGViewContainer.js#L39-L50

I'm struggling to think how it would use clientWidth and clientHeight either. I'm guessing that code was copied straight from the classifier.

The original motivation for adding it to classifications seems to be 'it might be useful.' zooniverse/Panoptes-Front-End#986

If it isn't being used in any of the data-digging scripts, then I'm inclined to say get rid of it too.

[srallen]

The PH TESS annotation task records the actual center data point on the x-axis and the width of the column annotation box. Since it is SVG, it is vectored and does not go through a similar normalization process like images would against natural width and height. SVG plots wouldn't have a natural width and height, only client width and height, but again, I am unsure if it is used or has any purpose. I am checking with the PH TESS team if it does or not.

[eatyourgreens]

I think Cliff, maybe, once looked at whether subject size affects classification accuracy. As you say, viewport dimensions could be used to get a handle on that.

[srallen]

I've asked @snblickhan and @lcjohnso to review and respond to this. Thanks!

[trouille]

Does mobile use it? For example, in how the rectangle task is recorded?

[srallen]

The mobile app does record something akin to it in the same classification metadata property: https://github.com/zooniverse/mobile/search?q=clientWidth

It appears to be derived differently since a web browser is not involved, however, if we confirm that it is not used in post-classification analysis, we could also recommend that the mobile app deprecate it?

[srallen]

As an aside, whether or not the mobile app uses it I don't think should impact our decision. PFE would continue to record it, too, so this would be a deprecation for the new classifier and classifications which come from the new classifier can be determined by metadata value as well.

[lcjohnso]

@srallen

If you confirm that viewport is essentially the browser window size, I find this measurement much less useful than the old style clientWidth and clientHeight (which in my mind maps to the subject-area / frame-annotator / subject-image-frame size for PFE workflows and other simple cases) for understanding what the interface looked like to the volunteer. I acknowledge that zoom tools blur the notion of knowing exactly what the volunteer saw as they annotated, but the rendered size of the subject viewer provides a useful baseline.

I understand that a choice must be made regarding multi-component viewers (as discussed in #1594), but I advocate there is a straightforward option of recording a summary clientWidth and clientHeight that was representative of the overall viewer interface -- see examples for PH TESS and Stellar Sleuths below:

[srallen]

@lcjohnso I think it's even more blurry with the SVG plots what the meaning of the client size would be even outside of the already mentioned issues. As you point out, we could record the client size of the container HTML element, but I'm still unclear about what that is telling us. We have a set of fractional columns to size the area using CSS grid, but within it, at least as you can see with the VariableStarViewer, the inner components may render differently depending on the current size of the window. In your screenshot, it has switched to its stacked layout. The plot itself is vectored, so there's no issue with resolution like an image has. I'd posit that metadata about pan/zoom views would be more relevant to determine affects on classification quality for SVG plots and that's something we've never recorded, but possibly could consider depending on the SVG plot type.

More generally though, were you using the client size to see if it affected classification quality? What was the result?

[snblickhan]

On my end I'm generally fine with this, but wonder whether we have a way of recording whether classifications were made on projects with the max height settings turned off? I can see teams (at least those who I work with) potentially wanting that for comparative work on data quality and image size for transcription projects. But I'll note for the record that I'm not aware of any teams I'm working with using this component in the data export.

[srallen]

We don't have a max height for the default layout for the new classifier. That was a quirk of PFE's CSS. We also have several unimplemented layouts for the future that project builders would be able to choose from for their workflow settings.

[chrislintott]

I tend to agree with Cliff that these are useful values to save. I acknowledge that viewport tells us about the device, but recording (a guess at) the size of the display itself could be useful in a situation where we were trying to diagnose why a subset of classifiers are behaving differently. Galaxy Zoo, Space Warps and (IIRC) Andromeda Project have all had to dig deep for these things to explain odd features of their dataset in the past, and I think we do well to future proof by recording what we can.

Though it's less useful in a world where people can zoom, and for complex projects like PH and certainly Stellar Sleuths, most projects which the new classifier will support are image based, and most classifiers do not zoom for most subjects, so this is still useful information.

Unless the cost of keeping it is large in dev time or in storage cost, I vote we continue to record both client width and height.

[srallen]

I've updated the ScatterPlotViewer and the two complex composite viewers we have, VariableStarViewer and DataImageViewer, to store client sizes of the wrapping container div in the classification metadata in #2383, #2387, and #2388 since we're close to having some project beta test with them. It'd be great to have some follow up to find out if this information actually gets used in any analysis later.