Update to latest enketo-core #6345

garethbowen · 2020-04-07T04:28:16Z

The next time we bump our dependency on enketo-core we will have to switch api to depend on enketo-transformer instead of the archived enketo-xslt. The transformer depends on the version of libxslt which works on node 12 so we could consider swapping to use that instead of the native xsltproc process. Additionally we may be able to use the transformer for more of the work that we do ourselves in the generate-xform service in API.

Summary of AT Issues:

1. Default config. Form validation error needs to be specific when a phone number has been taken/used
- Existing issue, not caused by Enketo uplift (comment)
- Mentioned in a comment on Allow for phone number validation but also allow for duplicate phone numbers #6390
2. Default config. Client-side phone validation does not 'complain' where some characters are added to a 'valid' phone number
- Existing behavior, not caused by Enketo uplift (comment)
- Seems like this might actually be the desired behavior
3. Default config. Custom place name label not showing on create/edit place (this is also the same on master)
- Existing issue, not caused by Enketo uplift
- Logged Labeling Issues with Default and Standard Config #7478 and this should be fixed in Fix labeling Issues with Default and Standard Config #7479
4. Default config. When adding/editing a person's DOB, if you check Date of birth unknown and then do not enter a value for Months then the value stored for the person's date_of_birth is an "Invalid Date"
- Behavior change triggered by upgrading our version of openrosa-xpath-evaluator (comment)
- We inadvertently fixed Empty integer values in Enketo forms evaluate to 0 #7222
- ~~Proposal is to use a patch to revert behavior change for now (and wait for a major version bump)~~
- Decided to keep behavior. Have updated the configs (including the person add/edit forms) to properly handle the behavior change. (comment)
5. Default config. Date picker pop up does not close when clicking outside of it. Only when pressing tab key.
- Behavior changed. Now clicking anywhere within the date question's div will open the widget (comment). Need to decide if this is acceptable behavior...
6. Default config. Also, date picker is already open when entering the Pregnancy registration form, when selecting Last menstrual period (LMP) or Expected date of delivery (EDD)
- More generally, questions that are not part of a field-list group are not being automatically selected when the page is opened
- Caused by a code change in enketo-core to deliberately focus on the first element on the page (comment)
- I have updated the android-widget to prevent this from triggering text input (and on-screen keyboard) when a date question is focused on (the widget, itself, is not automatically opened by the focus).
7. Default config. In the Pregnancy registration form, ANC visits at Health Facility (past) input, if you type a big number, the page completely freeze)
- Existing behavior, not caused by Enketo uplift
- The big number triggers a bunch of entries in the repeat so technically this is expected behavior that we probably do not need to worry about (comment)
8. Default config. In the Gestational Age section and and ANC Visit at Health Facility (Scheduled) section, from the Pregnancy Registration, all the information presented is in bold.
- Layout changed due to a change in the css. The css has been updated so that notes are not bold. (comment)
9. Default config. When completing a delivery, the child registered during birth is not created.
- Caused by behavior chance in Enketo form validation that caused inputs to be cleared. Fixed by updating our logic for making inputs always relevant. (comment)
10. Default config. Date label not showing on ANC Visits at Health Facility (Scheduled) / Pregnancy registration
- Existing behavior, not caused by Enketo uplift
- This "works as expected" since the date field on the form in marked with NO_LABEL (comment)
11. Default config. When a Pregnancy registration is submitted, and there are more than 4 ANC Visits registered, the report is is only showing the correct labels for the first four visits
- Existing behavior, not caused by Enketo uplift
- This "works as expected" since there are only translations for the first 4 past ANC visits (comment)
12. Default config. On enketo branch with default config pregnancy_home_visit form throwing an error when trying to open
- Initial error loading the form caused by config code in the form that is no longer supported. I updated the form to properly pass one parameter to the decimal-date-time function.
- Additionally there were issues loading the external contact-summary data into the form which required code changes (comment)
13. Default config. When a Health facility ANC reminder is created, the report is showing an Invalid Date value for the Pregnancy visit task date label.
- This appears to be fixed now as I cannot recreate it. I expect the issue was caused by the problems loading the contact-summary that were fixed in #12 (comment)
14. Default config. When completing a delivery, the child death registered during birth is not created.(related to #9)
- This appears to be fixed now as I cannot recreate it. I expect the issue was fixed by changes related to #9.
15. Standard config. After completing the child_health_registration form, on the Area detail view, the report generated is displayed showing an id instead of the child's name.
- Caused by an issue in the code for making the inputs group relevant. Fixed now. (comment)
16. Standard config. Adding to #15, the report generated for the Child Health Registration is not showing the correct information .
- Caused by same issue with inputs group as #15. Fixed by same changes.
17. Standard config. After creating an Immunization Visit the report say that Baby did not attend their immunizations visit, and the Vaccines that I chosen (Hepatitis A and B) are marked as No.
- Caused by change in Enketo behavior for non-relevant fields with default behaviors. Refactored form logic to fix behavior. (comment)
18. Standard config. Nutrition Treatment Exit report is showing long labels for the collected information. For example, report.nutrition_exit.patient_name instead of Name. This is also the same on master.
- Existing behavior not caused by Enketo uplift. Similar to #11, the "long labels" are shown for this form because there is no provided translation.
19 Standard config. Child Nutrition Screening is presenting #15, #16, and #18. The difference is that this is happening also in master.
- Existing issue, not caused by Enketo uplift. (comment)
- I have included this in Labeling Issues with Default and Standard Config #7478 and fixed it in Fix labeling Issues with Default and Standard Config #7479
20 Standard config. Pregnancy Form displays a user icon and warning icon.
- Caused by a change somewhere in the processing of embedded HTML where invalid whitespace is no longer supported. (comment)
- I have updated the logic for parsing the embedded HTML to clear out any invalid whitespace
21 Standard config. Pregnancy Form doesn't display summary information on submit step.
- Caused by same issue as #21. Fixed by same solution.
22 Standard config. The report generated by the pregnancy form do not match the correct person.
- This seems to be working now since. I expect it was caused by the same issue as #9 and/or #15.
23 Standard config. Adding to #22. The report generated by the delivery form do not match the correct person.
- This seems to be working now since. I expect it was caused by the same issue as #9 and/or #15.
24 Standard config. The analytics related to the delivery are not updated.
- This seems to be working now since. I expect it was caused by the same issue as #15.
25 Standard config. After submitting nutrition_followup form, several things break.
- Caused by same issue as #17 where invalid data was being saved for forms. Fixed by updating the nutrition_followup form. (contact)
26 Standard config. Postnatal Care Visit and Pregnancy Visit reports are not showing Danger Signs.
- Caused by same issue as #17 where invalid data was being saved for forms. Fixed by updating the forms.
27 Standard config. First report in the list has to be deleted twice to actually be deleted. This is also the same on master.
- Not a real issue, it was a sync problem
28 Muso config. Unable to install config-muso on an instance running 6345-enketo-uplift.
- I am not able to recreate this issue. It may have been fixed by one of the previous changes, or we may need to see if there is some difference in our setup.
29 Brac config. Creating a CHP Area or a Family, the error message for mobile number format is not displayed. This is present in master and 6345-enketo-uplift.
- Existing issue, not caused by Enketo uplift. Some form configuration is missing and I have logged a separate issue on the config-brac repo to address it. (comment)
30 Brac config. Unable to edit a person's information.
- Caused by the custom phone-widget being too broadly applied now. Fixed by making the selector for that widget more precise. (comment)
31 Brac config. Unable to load pregnancy follow up form from task tab.
- Caused by behavior change in the new Enketo/Openrosa xPath logic to be more stringent with regards to mal-formed expressions.
- The pregnancy_visit form will need to be updated. I have logged: https://github.com/medic/config-brac/issues/436
32 Brac config. Target tab is not showing any information (one pregnancy was registered).
- Caused (mostly?) by a change to the format-date function behavior. When the pregnancy form submits with valid data, this problem does not present. (comment)
- ~~We still need to land on a long-term fix for this. Waiting on 'Invalid Date' returned by format-date function when given an empty value enketo/enketo#864~~
  - In the meantime I have added a custom format-date implementation to the branch that reverts to the old behavior. Going forwards we can consider keeping these changes or reverting them in favor of something else.
- Our change to openrosa was accepted! I have pulled in the latest version of that library so that we should have the original behavior for format-date
33 Brac config. After submitting the pregnancy_referral_follow_up the task was not deleted and it throws an error.
- I believe this was caused by invalid data created as a result of #32. Once I addressed those issues, this problem no longer presents.
34 Brac config. Household Equity survey task is not being generated when a new family is created.
- This problem is no longer presenting for me. I am honestly not sure which changes may have fixed it (maybe related to #15?), but it seems to be working fine now.
35 Brac config. After creating a FP Visit, in master, the total number of visits was increment to 1, but in enketo branch the total number remained in 0.
- As with #34,this problem is no longer presenting, but I am not sure exactly why or what was causing it in the first place (also maybe #15?).

The text was updated successfully, but these errors were encountered:

garethbowen · 2020-05-26T00:13:10Z

Moved to 3.12.0 to free up engineers to work on 3.11.0.

garethbowen · 2020-07-23T20:45:25Z

This is blocked on updating to 5.9.2 first: #4386

iesmail-znz · 2020-09-30T09:05:35Z

@garethbowen, I wanted to know if CHT has the capability to track a CHV's visit length in XLSForm. I checked at the documentation and I found that the the user_edit_time in this link
mentions Enketo, and the XLSForm spec has astart/endproperty (http://xlsform.org/en/#metadata), but due to an Enketo or CHT bug/issue that was not usable in CHT XLSForms.

Would like to know when it will be fixed.

abbyad · 2020-10-05T15:13:37Z

Hi @iesmail-znz, the start and end meta fields should be available since 3.7. What problems are you currently seeing trying to use them? If they aren't working as expected it would be worth opening a bug report with details as a new issue.

On a related note, I see documentation on those fields in our old (and now archived) docs site, but can't find it in the new CHT docs site. I will port that over now.

MaxDiz · 2020-10-15T13:26:01Z

bumping to release following completion of #4386

abbyad · 2021-01-06T20:39:21Z

Adding to the App Builders backlog too since this is needed to use supported ODK XPath functions in CHT apps.

Edit: added link to Forum post.

MaxDiz · 2021-01-21T20:59:21Z

@garethbowen a couple of follow-up questions from the roadmap planning meeting:

Is there a QA plan to test forms?
Based on the QA plan, would it make more sense to upgrade to enketo latest in a single release (ie combine with improvements scheduled in 3.12 - Update enketo-core to 5.9.2 #4386 - the original plan, I believe)?

garethbowen · 2021-01-24T22:56:02Z

Many forms are tested as part of the standard release test plan. It would be very useful to test a wider range of real world forms as well and that plan should be developed when this issue is being QAed. If time permits this test plan should be automated because we usually update enketo-core every release and we need to catch regressions early.

The reason there are two very similar looking issues is we've made good progress towards the other one, so it should be relatively easy from a development perspective to finish that off ready for AT. This issue would then be a separate body of work which may be trivial or highly difficult depending on what breaking changes exist in the later versions of enketo-core. Obviously the two issues can only be developed one at a time but they could be ATed together to save time. In short I'm happy for this issue to be scheduled in 3.12 to reduce testing effort, but we should keep the issues separate so we can ship the other one, and postpone this one if it turns out to be difficult.

MaxDiz · 2021-01-25T13:40:35Z

ok, moving to v3.12 to align with #4386

tatilepizs · 2022-06-23T16:38:46Z

Brac config - fp_follow_up_long_term

There is a section in the fp_follow_up_long_term form with the question: Did you receive the long term method?* (Q1) and depending on the answer (Yes/No) a different question will be displayed, and this new question (Q2) will also display messages or new questions depending on the answer. The problem is that if I select No from Q1 and then select several options from Q2 different messages will be displayed, and if after that I changed my mind and select Yes from Q1 the messages that were displayed will remain in there, and if I change my mind again, and select No again from Q1, all the options that I selected the first time will be marked, all the options that I selected before will be there, the answers are not clean.
This is not happening in master, no matter how many times I change the answer from Q1 the options that were selected and the messages will be clean.

This description might be a little confusing, I think that a video can help to clarify.

Video - 6345-enketo-uplift

enketoUplift.mp4

Video - master

master.mp4

_______________________________________________________

Muso config - pregnancy_family_planning

When a pregnancy is registered using the pregnancy_family_planning form, the delivery information is not shown.

tatilepizs · 2022-06-29T16:57:42Z

Muso Config - Several forms

In the first page of the form, there is a required input for the patient's name, and no matter what value I select it is throwing an error saying: This field is required, I am not able to continue with the next page.

NOTE: This is only happening if I access the report through the "History" tab.

Another important thing is that, in master, if you go back to select another patient, it starts to show the same behavior that is in the new enketo (6345-enketo-uplift). At least the new enketo is more consistent, because it is failing every time, maybe it is doing a better validation from the beginning 🤷‍♀️

supervision_calendar

6345-enketo-uplift

6345-enketo-uplift.mp4

master

master.mp4

supervision_with_chw_iccm

6345-enketo-uplift

6345-enketo-uplift.mov

master

master.mov

supervision_with_chw_proccm

6345-enketo-uplift

6345-enketo-uplift.mov

master

master.mov

tatilepizs · 2022-07-01T17:03:45Z

Muso config - Some forms are not able to load.

Details of the user used to login:

...
"type": "person",
"is_active": "true",
"name": "Pumba",
...
"parent": {
  "_id": "ea2134e6-fe48-4aa1-be4a-b2421cb8fb90", -> "contact_type": "c40_chw_area"
  "parent": {
    "_id": "062a63ed-bc82-4fd7-9b28-495aeb09e05d", -> "contact_type": "c30_supervisor_area"
    "parent": {
      "_id": "2bec5f7a-5549-493a-8102-a2fd7fb2035c", -> "contact_type": "c20_health_area"
      "parent": {
        "_id": "3fa4a4fc-e492-470d-a98c-8f6281ffb834" -> "contact_type": "c10_site"
      }
    }
  }
},

Failed forms:

tb_followup_daily

tb_followup_weekly

tb_test_result_fp

treatment_followup_over_5

treatment_followup

edit contact

jkuester · 2022-07-11T20:10:25Z

Regarding the issues with the Brac assessment_follow_up form, there are a could of things going on here. First of all, the issue with the constraint violation error message still being displayed (as seen in the video) is fixed by the new changes I recently made to the db-object widget.

Then, the problem with not being able to advance to the next page after selecting the answer to How are you following up? is caused because there is a hidden question on the page that is required. That question's value is set by input data that should be provided when by the task. Since, in this case, the form is being manually created, no data is provided from the task and the form cannot advance. This behavior is the same as in master and I think is acceptable (given how this form is supposed to be used...)

jkuester · 2022-07-13T17:59:12Z

Regarding the Muso pregnancy_family_planning form display issues, this is actually caused by a relatively serious OpenRosa issue with the decimal-date-time function. I have logged enketo/enketo#862 and submitted enketo/openrosa-xpath-evaluator#162 to address this.

jkuester · 2022-07-13T20:12:45Z

Regarding the issue with the Muso supervision forms where you could never successfully choose two persons, this turned out to be a bug in the db-object-widget where it could not properly handle multiple db-objects in nested groups (at least in some cases). That bug should be fixed by these changes.

jkuester · 2022-07-14T16:28:52Z

Regarding the issues in the Muso notification_to_chw form trying to select a "Health Center", I cannot recreate the issue. Perhaps it was caused by not having any health_center contacts that were available to the user? Also, it may have been resolved by one of the recent changes to the db-object-widget.

jkuester · 2022-07-19T21:06:22Z

Okay, I think I have finally figured out what was going on with the Muso contact edit form issue.

The db-object-widget was manually changing its input element to be a select element. However, Enketo had already cached the reference to the input element and this manual change broke that reference. So, when you have a form like Muso's edit contact form, that sets the value of a db-object input using a calculation, it causes an error because the Enekto framework does not have a working reference when it tries to set the calculation value.

This particular situation was made more complicated because, in certain circumstances the Muso edit contact form would load without error. @tatilepizs noticed that the form worked when the contact being edited was created by the user. It turns out this behavior was caused because the db-object value in the form could be set in a couple of different ways (due to some weird circular configuration in the form). Its value would initially come from the current user's id. Then it would later be set to the person_id on the contact being edited. Now, if the contact was one that was created by the user, then the person_id would be the user's id and Enketo would not try and set the value a second time (since it was already set to the user's id). However, if the contact was not created by the user, the person_id would be different and Enketo would try to set the db-object column a second time. This time, though, the db-object-widget will already have been initialized and the reference broken. So an error will result.

Regardless, I have been able to clearly recreate the error in the interaction between the db-object and the calculation with a very simple app form, so I don't think too much of the complexity from the Muso form is actually related to this issue. The following form will trigger an error in the Enekto branch, but not in master:

type	name	label::en	appearance	bind:jr:constraintMsg	calculation
calculate	my_calc				string('bd3fdfa2-caf9-4eac-88b2-09fa739f03ac')
db:person	_id	What is patients name	db-object		${my_calc}

I have had to do a pretty significant refactor of the db-object-widget to fix this situation. But in the end, this should be resolved.

tatilepizs · 2022-07-21T01:04:55Z

🎉 Really good news 🎉

All the issues that were raised were retested and all of them are solved, some completely fixed, some with an external ticket created, and one (an unusual scenario) was marked as a known issue. All the details are in this comment

QA team is done with the testing and this uplift is good to go from QA perspective 🎉

Huge thank you to @jkuester for all the help, support and patience during the process!! 🎉 🎉

jkuester · 2022-08-01T17:17:52Z

Following up again on the Muso pregancy_familiy_planning form display issues, the OpenRosa PR was not accepted (for spec-compliance reasons). Instead, this case will require a change to the form (unless we want to add a custom implementation of the function to the CHT).

To give some more details here, the form is currently calculating a value based off the result of a decimal-date-time call where the parameter passed to the function is an explicitly formatted date string (using format-date-time), but the format is not ISO 8601. One way to fix the issue is to pass the formatted date string to the date function before passing to decimal-date-time. Another possibility is that the form logic might be able to be simplified to not use the decimal-date-time function at all by leveraging the new XPath function we are adding in #7694

garethbowen added Type: Technical issue Improve something that users won't notice Priority: 2 - Medium Normal priority labels Apr 7, 2020

mrjones-plip mentioned this issue Jul 23, 2020

When creating new Health Facility, date picker shows over search box #6486

Closed

garethbowen added the Status: Blocked Unable to progress this. label Jul 23, 2020

mrjones-plip mentioned this issue Sep 19, 2020

Improve the performance of Enketo repeated sections #6647

Open

garethbowen mentioned this issue Feb 3, 2021

Video and Audio continues playing even after navigating to next screen #6589

Closed

jkuester mentioned this issue Jul 12, 2022

Answers to non-relevant questions in forms are not immediately cleared with new Enekto #7674

Closed

jkuester mentioned this issue Jul 25, 2022

Values of non-relevant fields should be maintained until submission #4132

Closed

garethbowen added this to Product Team Activities Jul 31, 2022

garethbowen moved this to Ready to Merge in Product Team Activities Jul 31, 2022

jkuester mentioned this issue Aug 16, 2022

Update today XPath function to not return current time #7731

Closed

jkuester closed this as completed in #7256 Aug 16, 2022

Repository owner moved this from Ready to Merge to Done in Product Team Activities Aug 16, 2022

jkuester moved this to Done in Enketo Uplift AT Issues Aug 16, 2022

garethbowen added the Breaking change Has the potential to break a deployment when upgrading label Oct 31, 2022

jkuester mentioned this issue Dec 23, 2022

Bullets not displaying properly in form labels #8002

Closed

jkuester mentioned this issue Mar 2, 2022

Don’t use non-relevant values in calculations (I.e., like ODK Collect) enketo/enketo#82

Closed

jkuester mentioned this issue Mar 27, 2024

Form end meta field not correctly populated when the form ends #8974

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to latest enketo-core #6345

Update to latest enketo-core #6345

garethbowen commented Apr 7, 2020 •

edited by tatilepizs

Loading

garethbowen commented May 26, 2020

garethbowen commented Jul 23, 2020

iesmail-znz commented Sep 30, 2020

abbyad commented Oct 5, 2020

MaxDiz commented Oct 15, 2020

abbyad commented Jan 6, 2021 •

edited

Loading

MaxDiz commented Jan 21, 2021

garethbowen commented Jan 24, 2021

MaxDiz commented Jan 25, 2021

tatilepizs commented Jun 23, 2022 •

edited

Loading

jkuester commented Jun 24, 2022 •

edited

Loading

jkuester commented Jun 27, 2022 •

edited

Loading

tatilepizs commented Jun 27, 2022

garethbowen commented Jun 27, 2022

tatilepizs commented Jun 28, 2022

tatilepizs commented Jun 29, 2022 •

edited

Loading

tatilepizs commented Jul 1, 2022 •

edited

Loading

jkuester commented Jul 11, 2022

jkuester commented Jul 13, 2022

jkuester commented Jul 13, 2022 •

edited

Loading

jkuester commented Jul 14, 2022

jkuester commented Jul 19, 2022

tatilepizs commented Jul 21, 2022 •

edited

Loading

jkuester commented Aug 1, 2022

Update to latest enketo-core #6345

Update to latest enketo-core #6345

Comments

garethbowen commented Apr 7, 2020 • edited by tatilepizs Loading

garethbowen commented May 26, 2020

garethbowen commented Jul 23, 2020

iesmail-znz commented Sep 30, 2020

abbyad commented Oct 5, 2020

MaxDiz commented Oct 15, 2020

abbyad commented Jan 6, 2021 • edited Loading

MaxDiz commented Jan 21, 2021

garethbowen commented Jan 24, 2021

MaxDiz commented Jan 25, 2021

tatilepizs commented Jun 23, 2022 • edited Loading

Brac config - fp_follow_up_long_term

Other forms having the same problem in one of its questions:

jkuester commented Jun 24, 2022 • edited Loading

jkuester commented Jun 27, 2022 • edited Loading

tatilepizs commented Jun 27, 2022

garethbowen commented Jun 27, 2022

tatilepizs commented Jun 28, 2022

Muso config - pregnancy_family_planning

tatilepizs commented Jun 29, 2022 • edited Loading

Muso Config - Several forms

tatilepizs commented Jul 1, 2022 • edited Loading

Muso config - Some forms are not able to load.

jkuester commented Jul 11, 2022

jkuester commented Jul 13, 2022

jkuester commented Jul 13, 2022 • edited Loading

jkuester commented Jul 14, 2022

jkuester commented Jul 19, 2022

tatilepizs commented Jul 21, 2022 • edited Loading

🎉 Really good news 🎉

jkuester commented Aug 1, 2022

garethbowen commented Apr 7, 2020 •

edited by tatilepizs

Loading

abbyad commented Jan 6, 2021 •

edited

Loading

tatilepizs commented Jun 23, 2022 •

edited

Loading

jkuester commented Jun 24, 2022 •

edited

Loading

jkuester commented Jun 27, 2022 •

edited

Loading

tatilepizs commented Jun 29, 2022 •

edited

Loading

tatilepizs commented Jul 1, 2022 •

edited

Loading

jkuester commented Jul 13, 2022 •

edited

Loading

tatilepizs commented Jul 21, 2022 •

edited

Loading