Staff Dashboard - External Form Upload: Display import summary / errors

[jmilljr24]

Staff can upload a csv file with form submission data. This data is matched to people in the DB. Currently staff can upload the file but there is no indication in the UI for success or any errors.

In the ExternalFormUploadController, use the data returned from the import service to display a separate card below the upload form that gives a summary of the import. This should be implemented using turbo_frames.

This is just a rough example. Feel free to play around with what looks best. It may even make sense to use two cards. Maybe display one for Success information and conditionally display a second card if errors are present?

Use green for the Success title and Red for the Errors title. It may be worth added a flash notice/alert as well, as the summary card may not be visible on a smaller screen until you scroll down.

The import service returns a Data object Data.define(:success?, :count, :no_match, :errors). For this PR the following should be used:

• import.success? will return true or false
• import.count returns the number of form submissions successfully added to the DB
• import.errors returns a nested array with each error having the row number and the error

This file should result in a successful upload. A simple way to create an error is to change one of the timestamps to have month "13" which is out of range "2024-13-03 08:09:10 -0400"
test_google_upload.csv

[khiga8]

Hi! Can I pick this one up?

[jmilljr24]

@khiga8 Just checking in to see how this is going. No pressure.

[khiga8]

Hey there! Planning to dig into this in the next week! Hope that's okay :)

[jmilljr24]

Hey there! Planning to dig into this in the next week! Hope that's okay :)

Absolutely!! Thank you

[khiga8]

Hey @jmilljr24!

I noticed there are many potential outcomes when a CSV is uploaded.

I feel like import.success? implies an all or nothing outcome, but that isn't really the case. We need to look at .count, .no_match, and .errors attributes and craft feedback based on each of these individual attributes.

I took some notes on the different scenarios, and potential messages we could display. I wanted to check in with you to make sure these scenarios align with your understanding!

1. the uploaded file format is invalid.
   • This is now handled by Add CSV Import File Validations #1211, so nothing to do as part of this issue.
2. there's new data in the CSV, resulting in new form submissions being added.
   • potential feedback: "5 form submissions were added."
3. there's no new responses in the CSV, so nothing happens. (surprisingly, import.success? still returns true.)
   • potential feedback: "No new form submissions were added because no new response was detected in the CSV."
4. there are new responses in the CSV, but they're skipped over because the e-mails don't match with any known Person.
   • potential feedback: "Form submissions for were not added because they do not match ABC. Please do XYZ"
5. there's malformed data. (import.success? will always return false in this case, but some form submissions may still be added successfully).
   • potential feedback: "5 form submissions were added. However, some submissions due to the errors in the CSV. Please review the following ."

If anything looks off here, let me know! I also welcome any suggestions you have on the messages we provide! I crafted these based on best guess, but I may lack some context. For example, I'm not sure sure about what action (if any) we want the user to take in scenario 4 (where form submissions aren't added due to the e-mail not being recognized).

[jmilljr24]

@khiga8 I built the csv import service with a lot of assumptions. We do not have actual user data yet so changes will likely happen when we get feedback from our stakeholders in the near future. Your summary matches my train of thought of what information the service returns. Please let me know if I miss anything or need to explain better!!

I do think for the time being we can keep what is displayed a bit simpler due to the good chance there will be changes happening. I viewed success as all or nothing as you state (specifically for errors). In the most basic sense if a user uploads a file and there are no errors they can move on in their work. For MVP we really don't even need to know "what" was imported, just that there were no issues with their file. Staff can go to a different location in the dashboard and see what adopters now have form questions/answers. If there are errors, Staff will need to know the error and potentially edit their csv to make it work.

You can expand on this but I'd like to keep it flexible for future changes. Here are a couple of examples of what I would do below the upload form:

A.
"File successfully scanned: 5 new form submissions were added"

B.
"File successfully scanned: 0 new form submissions were added"

C.
"File upload resulted in error(s)"
File type must be csv

D.
"File upload resulted in error(s)"
5 new form submissions were added
Errors:
Line 3 - error.message
Line 4 - error.message

Regarding #4, it is something we should probably have but that list could be excessively long for a new Organization first getting set up with this website. I say for now we can skip that unless you have a nice solution you would like to implement.

Side note: We can certainly change what's being returned to the controller from the service if it makes things easier, just let me know.