Improve the performance of Enketo repeated sections

[kennsippell]

What feature do you want to improve?
There are some good examples of signifiant performance issues in the CHT form-filling experience for projects using repeat sections in XForms: https://github.com/medic/config-moh-mali/issues/56 and #6136. These have lead to an initiative to "quantify the upper bounds of what the cht-core can handle" medic/cht-docs#308.

End-users are affected because their forms load slowly.

This is also impacting the CHT app building experience. This performance limitation is below what some developers would intuitively consider devices to be capable of (even legacy devices). This irregularity results in increased educational burdens for onboarding new TPs. The guidelines for avoiding these performance traps are not always totally straightforward as seen with #6136. The end result is a poor app development experience for complex forms, as it is easy for an inexperienced TP to design an app based around a complex form that later needs to broken up. That team may not learn about these performance limits until late in app development, resulting in a need to re-architect the flows. The end result is a disrupted app-building experience and an increased time to deliver the app.

cc @mrjones-plip

Describe the improvement you'd like
We should investigate this problem at its source. Could we meaningfully improve the performance of our form experiences through contributions to Enketo? Is this experience in Enketo slow because it is actually computationally hard, or can Enketo be improved?

Describe alternatives you've considered
No idea. Maybe we can use a different library than Enketo? Doesn't seem like a great alternative.

[mrjones-plip]

@kennsippell - thanks for filing this ticket! It's good to know App Services pain points (though I'm getting to know them first hand through CMoHM ;)

I think the main alternative, aside from using a different library than Enketo, that we have to consider in the very short term is designing the forms we use to work with in the confines of Enketo as we have it implemented today. That is, if a form is using repeats, but could not use them by having CHWs submit multiple forms, do this instead. Multiple forms likely will slow down data input or make it more confusing, but may be better than UI locking up as it is in config-moh-mali#56. Agreed that knowing ahead of time how to write forms to not do this, via cht-docs#308 will be very helpful.

Maybe part of cht-docs#308 might to check the feasibility reproducing config-moh-mali#56's issue, but in an Enketo only environment. This would allows to see if upgrading Enketo will actually improve anything. Being new to Medic, I'm not sure if this is a small, large or even possible task.

Otherwise, the plan to upgrade Enketo is currently scheduled for 3.12, with the work being done in #6345 and #4386. Per above, knowing if this will help or not is key.

My notes above are largely just the state of things though, I'll defer to @garethbowen, @abbyad & @MaxDiz about the more technical nitty gritty and when this might get scheduled to be worked on.

[garethbowen]

I agree this is worth investigating and will help with the documentation by finding what the limits are. We should hold off doing deep investigation until the Enketo upgrade is complete otherwise any improvements or recommendations we find may be obsolete.

[garethbowen]

Another alternative approach is to investigate whether we can use multiple pages for complex repeats, for example if registering all the people in a household, and each person has a number of fields, it may make more sense to add n pages so that the UI only has to render one at a time. I'm not sure if this is even possible in Enketo, and even if it is it may not improve performance, but it's something else to look in to.

[jkuester]

Regarding multi-page repeats, I can confirm that this is possible in Enketo by using the repeating as long as a condition is met pattern. Unfortunately, as I have documented on #7237, the functionality required for that pattern is not supported in the CHT at this time.

One additional observation to note is that if you have no fixed repeat_count (and instead are using the ➕ button to add more entries to the repeat) then everything in the repeat is required to be on the same field-list (on the same page). This seems to be the case for Enketo in general and not just the CHT. However, in the CHT, added entries to that repeat group all show up one-after-another on the same page. On the other hand, the Enketo web form preview functionality deployed at https://getodk.org/xlsform/ will put each entry in the repeat on a different page (so each time you tap the ➕ button you are taken to a new page where you can enter the information for that new repeat entry). That behavior pattern does seem superior from a performance perspective....