Skip to content

Commit

Permalink
GitHub Research Docs Fixes (#2868)
Browse files Browse the repository at this point in the history
* Move over changes from Camilla's branch

* Update 2020, Fall - Understanding STT Roles, Source of Truth, and Metadata.md

Referenced file that no longer exists on the repo (now deep in archives in Figma & implemented in-app)

* Update 2020, Spring - Understanding the local experience.md

replaced old link pointing to project wiki

* Update 2023, Spring - Testing CSV & Excel-based error reports.md

Final jump to link fix

* Update 2023, Winter - TDP 3.0 Pilot Program.md

last research synthesis naming convention link fix

* Update docs/User-Experience/Research-Syntheses/2020, Fall - Flatfile Metadata Guide.md

Co-authored-by: Lauren Frohlich <[email protected]>

* Update docs/User-Experience/Research-Syntheses/2021, Spring - Evil User Journeys & TANF Data Errors.md

Co-authored-by: Lauren Frohlich <[email protected]>

* Update docs/User-Experience/Research-Syntheses/2022, Fall - TDP 2.2 Pilot Expansion.md

Co-authored-by: Lauren Frohlich <[email protected]>

* Update 2023, Winter - TDP 3.0 Pilot Program.md

* Update 2023, Winter - TDP 3.0 Pilot Program.md

---------

Co-authored-by: Lauren Frohlich <[email protected]>
  • Loading branch information
reitermb and lfrohlich authored Mar 18, 2024
1 parent abd5099 commit f38f6df
Show file tree
Hide file tree
Showing 11 changed files with 32 additions and 33 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ For tribes, ### represents their 3 digit Tribe Code



🔍 **Research**: There are open questions and we’d like to base a recommendation on insights from Phase III Research ([October STT Research](https://github.com/HHS/TANF-app/blob/main/docs/User-Research/2020%2C%20Fall%20-%20Understanding%20STT%20Roles%2C%20Source%20of%20Truth%2C%20and%20Metadata.md)) and continued solutions ideation (Both design and technical).
🔍 **Research**: There are open questions and we’d like to base a recommendation on insights from Phase III Research ([October STT Research](https://github.com/HHS/TANF-app/blob/main/docs/User-Experience/Research-Syntheses/2020%2C%20Fall%20-%20Understanding%20STT%20Roles%2C%20Source%20of%20Truth%2C%20and%20Metadata.md)) and continued solutions ideation (Both design and technical).

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ We created this guide to provide context to the [STT Attributes Spreadsheet](<ht

We collaborated with OFA analysts to gather reports that contained information related to data submission errors for all of the grantees from Quarter 3 2018 to Quarter 2 2020. We are starting our analysis with two data sets focused on data quality and common errors given the critical role of data quality in the success of TDRS:

- [Error count across submissions and resubmissions](https://hackmd.io/_PV7z3UAQmOPKbuFQ2BuOQ#Number-of-errors-discovered-from-TANF-reports-for-all-STTs)
- [Error types across submissions and resubmissions](https://hackmd.io/_PV7z3UAQmOPKbuFQ2BuOQ#Most-frequent-errors-from-TANF-reports-for-all-STTs)
- [Error count across submissions and resubmissions](#error-count-across-submissions-and-resubmissions)
- [Error count across submissions and resubmissions](#error-count-across-submissions-and-resubmissions)

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,6 @@ Concept testing was focused around a flow depicting the process of report creati

[Figma Prototype 2](<https://www.figma.com/proto/y15co5xc7MIZXBnWBOF6hJ/Conceptual-Mockups-for-STT-Feedback?node-id=1%3A1070&viewport=530%2C320%2C0.1293068528175354&scaling=min-zoom>)

[PDF Prototype 1 & 2](<https://github.com/HHS/TANF-app/tree/main/docs/User-Research/2020%2C%20Fall%20-%20Concept%20Prototypes>)

[TANF Uploading Tool_Mockup #1](./TANF%20Uploading%20Tool_Mockup%20%231.pdf)
[TANF Uploading Tool_Mockup #2](./TANF%20Uploading%20Tool_Mockup%20%232.pdf)
Expand Down Expand Up @@ -155,7 +154,7 @@ Raw notes from interviews were synthesized in Mural with the OFA data division.

[Synthesis Workshop Mural](https://app.mural.co/t/officeoffamilyassistance2744/m/officeoffamilyassistance2744/1605279076254/8ba4f36f5bc1b2d724fd8c61daf6dc18da096e97) 🔒


PDF [Prototype 1](https://github.com/HHS/TANF-app/blob/main/docs/User-Experience/Research-Syntheses/TANF%20Uploading%20Tool_Mockup%20%231.pdf) and [Prototype 2](https://github.com/HHS/TANF-app/blob/main/docs/User-Experience/Research-Syntheses/TANF%20Uploading%20Tool_Mockup%20%232.pdf)



Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@ We wanted to talk to those who rely most on sunsetting TDRS tools so that we cou

We interviewed 7 people, with two pairs working in the same office and owning different parts of the data submission process.

By their role and location
By their role and location:

- 1 OFA regional program specialist
- 1 grantee from tribal government
- 5 grantees from 3 state governments

By the tools they use
By the tools they use:

- 4 folks currently using fTANF
- 4 folks familiar with the TDRS extranet
Expand Down Expand Up @@ -61,28 +61,28 @@ Raw notes from interviews were synthesized in Mural with the OFA data division.

**About the data submission process**

- [There isn’t shared language between OFA and STTs on TANF data concepts and tools](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#there-isnt-shared-language-between-ofa-and-stts-on-tanf-data-concepts-and-tools)
- [Data submitters often have years of TANF reporting experience and manage other mandated reports and data requests](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#data-submitters-often-have-years-of-tanf-reporting-experience-and-manage-other-mandated-reports-and-data-requests)
- [Creating flat files in fTANF is a time-consuming, “moody” process, but there’s also anxiety about changing something established and well-known.](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#creating-flat-files-in-ftanf-is-a-time-consuming-moody-process-but-theres-also-anxiety-about-changing-something-established-and-well-known)
- [It can take multiple data sources to create a complete data submission.](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#it-can-take-multiple-data-sources-to-create-a-complete-data-submission)
- [Getting data to OFA can be a collaborative effort between many people](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#getting-data-to-ofa-can-be-a-collaborative-effort-between-many-people)
- [Creating “a perfect file” is really hard; Data submitters have to resolve downstream data errors, meet TDRS formatting requirements, and wait a long time for feedback.](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#creating-a-perfect-file-is-really-hard-data-submitters-have-to-resolve-downstream-data-errors-meet-tdrs-formatting-requirements-and-wait-a-long-time-for-feedback)
- [It’s hard to confirm when and what data has been transmitted, and data submitters need that information.](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#its-hard-to-confirm-when-and-what-data-has-been-transmitted-and-data-submitters-need-that-information)
- [Covid-19 means adapting to new processes and distractions.](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#covid-19-means-adapting-to-new-processes-and-distractions)
- [Data submitters do a mix of importing and manual entry to build flat files, depending on the section](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#data-submitters-do-a-mix-of-importing-and-manual-entry-to-build-flat-files-depending-on-the-section)
- [The TDRS extranet is really missed, especially how it provided a way to submit data and view past data to confirm status](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#the-tdrs-extranet-is-really-missed-especially-how-it-provided-a-way-to-submit-data-and-view-past-data-to-confirm-status)
- [There isn’t shared language between OFA and STTs on TANF data concepts and tools](#there-isnt-shared-language-between-ofa-and-stts-on-tanf-data-concepts-and-tools)
- [Data submitters often have years of TANF reporting experience and manage other mandated reports and data requests](#data-submitters-often-have-years-of-tanf-reporting-experience-and-manage-other-mandated-reports-and-data-requests)
- [Creating flat files in fTANF is a time-consuming, “moody” process, but there’s also anxiety about changing something established and well-known.](#creating-flat-files-in-ftanf-is-a-time-consuming-moody-process-but-theres-also-anxiety-about-changing-something-established-and-well-known)
- [It can take multiple data sources to create a complete data submission.](#it-can-take-multiple-data-sources-to-create-a-complete-data-submission)
- [Getting data to OFA can be a collaborative effort between many people](#getting-data-to-ofa-can-be-a-collaborative-effort-between-many-people)
- [Creating “a perfect file” is really hard; Data submitters have to resolve downstream data errors, meet TDRS formatting requirements, and wait a long time for feedback.](#creating-a-perfect-file-is-really-hard-data-submitters-have-to-resolve-downstream-data-errors-meet-tdrs-formatting-requirements-and-wait-a-long-time-for-feedback)
- [It’s hard to confirm when and what data has been transmitted, and data submitters need that information.](#its-hard-to-confirm-when-and-what-data-has-been-transmitted-and-data-submitters-need-that-information)
- [Covid-19 means adapting to new processes and distractions.](#covid-19-means-adapting-to-new-processes-and-distractions)
- [Data submitters do a mix of importing and manual entry to build flat files, depending on the section](#data-submitters-do-a-mix-of-importing-and-manual-entry-to-build-flat-files-depending-on-the-section)
- [The TDRS extranet is really missed, especially how it provided a way to submit data and view past data to confirm status](#the-tdrs-extranet-is-really-missed-especially-how-it-provided-a-way-to-submit-data-and-view-past-data-to-confirm-status)

**About the prototypes**

- [The XLS template could be a stop-gap for Sections 3 and 4, but submitters are skeptical about Sections 1 and 2.](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#the-xls-template-could-be-a-stop-gap-for-sections-3-and-4-but-submitters-are-skeptical-about-sections-1-and-2)
- [There were unfamiliar fields and layouts in the XLS template](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#there-were-unfamiliar-fields-and-layouts-in-the-xls-template)
- [Seeing why errors happen and getting training on data rules is appreciated, but more detail is needed on where errors are happening](https://github.com/HHS/TANF-app/wiki/2020,-Spring:-Understanding-the-local-experience#seeing-why-errors-happen-and-getting-training-on-data-rules-is-appreciated-but-more-detail-is-needed-on-where-errors-are-happening)
- [The XLS template could be a stop-gap for Sections 3 and 4, but submitters are skeptical about Sections 1 and 2.](#the-xls-template-could-be-a-stop-gap-for-sections-3-and-4-but-submitters-are-skeptical-about-sections-1-and-2)
- [There were unfamiliar fields and layouts in the XLS template](#there-were-unfamiliar-fields-and-layouts-in-the-xls-template)
- [Seeing why errors happen and getting training on data rules is appreciated, but more detail is needed on where errors are happening](#seeing-why-errors-happen-and-getting-training-on-data-rules-is-appreciated-but-more-detail-is-needed-on-where-errors-are-happening)

**Resulting artifacts**

- We updated [our stakeholder map](https://github.com/HHS/TANF-app/wiki/Product-Strategy#project-stakeholders) and [user personas](https://github.com/HHS/TANF-app/wiki/User-types).
- We [created an updated process map](https://github.com/HHS/TANF-app/wiki/current-tdrs-system) to show how STTs submit data to OFA.
- Research findings contributed to [our roadmap and initial backlog](https://github.com/HHS/TANF-app/wiki/Roadmap-and-backlog).
- We updated [our stakeholder map](https://github.com/HHS/TANF-app/blob/main/docs/Product-Strategy/Vision-and-Stakeholders.md ) and [user personas](https://github.com/HHS/TANF-app/blob/main/docs/User-Experience/Research-Syntheses/2020%2C%20Summer%20-%20Understanding%20Stakeholders%20and%20creating%20personas.md ).
- We [created an updated process map](https://app.mural.co/t/hhsacf3483/m/officeoffamilyassistance2744/1608314993566/5724cd3c3e5e964ab9c85743e5b8e1d89c767788?sender=mreiter1745) to show how STTs submit data to OFA.
- Research findings contributed to [our roadmap and initial backlog](https://github.com/HHS/TANF-app/blob/main/docs/Product-Strategy/README.md).

## About the data submission process

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ We expect that there will be a couple hundred users who interact with the TDP sy
**Supporting Documentation**

- [Personas mural](https://app.mural.co/t/officeoffamilyassistance2744/m/gsa6/1592254280716/2ae8293a3233a95941d548cda4f373faab96b40b) :lock:
- [Vision and Stakeholders Doc](https://github.com/raft-tech/TANF-app/blob/restore-stakeholder-and-personas/docs/Product-Strategy/Vision-and-Stakeholders.md)
- [Vision and Stakeholders Doc](https://github.com/HHS/TANF-app/blob/main/docs/Product-Strategy/Vision-and-Stakeholders.md)

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ Our Error Data workshops also identified linking element errors (which are relat
**Project Impact(s)**

- We prioritized Dan and Malcolm for use in short term follow-on workshops; these could potentially include stakeholders from outside the project team so as to better identify risks and collaborate on controls.
- The [evil user journey maps]() :lock: synthesized from both workshops can be used to generate acceptance criteria for continued work on TDP.
- The [evil user journey maps](https://github.com/HHS/TANF-app/blob/main/docs/User-Experience/Research-Syntheses/2021%2C%20Spring%20-%20Evil%20User%20Journeys%20%26%20TANF%20Data%20Errors.md) synthesized from both workshops can be used to generate acceptance criteria for continued work on TDP.

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ Some Tribes have been invested in [particular case management systems] for most
>
> "Our system tells us if we have errors before we transmit [and] errors are corrected prior to pulling the data from [system] to report."
Just as we heard from some participants during [Fall 2020 Research](https://github.com/HHS/TANF-app/blob/main/docs/User-Research/2020%2C%20Fall%20-%20Understanding%20STT%20Roles%2C%20Source%20of%20Truth%2C%20and%20Metadata.md), some tribes have been managing their human services programs in one software solution for the better part of twenty years—often in software structured modularly like other integrated eligibility systems we've seen from participants representing States and Territories. At least two of the tools in use by Tribes still rely on fTANF to produce the final report files.
Just as we heard from some participants during [Fall 2020 Research](https://github.com/HHS/TANF-app/blob/main/docs/User-Experience/Research-Syntheses/2020%2C%20Fall%20-%20Understanding%20STT%20Roles%2C%20Source%20of%20Truth%2C%20and%20Metadata.md), some tribes have been managing their human services programs in one software solution for the better part of twenty years—often in software structured modularly like other integrated eligibility systems we've seen from participants representing States and Territories. At least two of the tools in use by Tribes still rely on fTANF to produce the final report files.

> "We have had [system] for 19 years"
>
Expand Down Expand Up @@ -396,7 +396,7 @@ Communication in this category is different from prior steps in so much as it's

**Immediate Next Steps**

- [Issue #804](https://github.com/raft-tech/TANF-app/issues/804) delivered updates to the [Stakeholders and Personas document](https://github.com/HHS/TANF-app/blob/main/docs/User-Research/Stakeholders-and-Personas.md), including recommendations on needed personas and follow-on tickets [#840](https://github.com/raft-tech/TANF-app/issues/840) and [#841](https://github.com/raft-tech/TANF-app/issues/841) which respectively relate to research focused around grantee program directors and regional specialists.
- [Issue #804](https://github.com/raft-tech/TANF-app/issues/804) delivered updates to the [Stakeholders and Personas document](https://github.com/HHS/TANF-app/blob/main/docs/Product-Strategy/Vision-and-Stakeholders.md), including recommendations on needed personas and follow-on tickets [#840](https://github.com/raft-tech/TANF-app/issues/840) and [#841](https://github.com/raft-tech/TANF-app/issues/841) which respectively relate to research focused around grantee program directors and regional specialists.

**Top Future Validations**

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ We matched our errors were matched with grantee participants accordingly to whic
> | Adult Record | T2 |
> | Child Record | T3 |
>
> For more details, refer to the Section 1 (Active Cases) [Transmission Layout]()
> For more details, refer to the Section 1 (Active Cases) [Transmission Layout](https://www.acf.hhs.gov/ofa/policy-guidance/final-tanf-ssp-moe-data-reporting-system-transmission-files-layouts-and-edits)
### Error Two

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Based on the task scenarios defined for v2.2 the pilot expansion program was a s

### Retest observed gaps from August 2022 pilot program

In addition to reviewing new functionality, we wanted to revisit and retest the areas noted from the August 2022 pilot program. Please reference [August 2022](https://tdp-project-updates.app.cloud.gov/august-2022-update.html) for full details
In addition to reviewing new functionality, we wanted to revisit and retest the areas noted from the August 2022 pilot program. Please reference [August 2022](https://github.com/HHS/TANF-app/blob/main/docs/User-Experience/Research-Syntheses/2022%2C%20Summer%20-%20TDP%202.0%20Pilot%20Program.md) for full details
- Tribal programs listed in the STT selection combobox may pose a discoverability issue
- To aide with locating the correct STT for tribal programs, we added content within the Knowledge Center as a short term resolution. We did *still* observe this occurrence during our moderated session with one of two tribal STTs.
- OFA staff held a tribal division staff meeting and together propose changing the request access form such that users must select the STT type (tribe, state, territory), first. Based on this selection, the relevant STT combo-box options would appear.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ ___

Jump to:
* [Collapsed columns in the Error Report created extra steps for participants when first opening the CSV-based prototype](#Collapsed-columns-in-the-error-report-created-extra-steps-for-participants-when-first-opening-the-csv-based-prototype)
* [We identified which columns should be included—and in what order—to maximize the actionable : extraneous data ratio of reports](#We-identified-which-columns-should-be-included-and-in-what-order-to-maximize-the-actionable-extraneous-data-ratio-of-reports)
* [We identified which columns should be included—and in what order—to maximize the actionable : extraneous data ratio of reports](#we-identified-which-columns-should-be-includedand-in-what-orderto-maximize-the-actionable--extraneous-data-ratio-of-reports)
* [Participants use the coding instructions in conjunction with the Error Report for error definition, but it can be challenging to know whether all information is available](#Participants-use-the-coding-instructions-in-conjunction-with-the-error-report-for-error-definition-but-it-can-be-challenging-to-know-whether-all-information-is-available)
* [Item number and field name represent an area where the language used in TANF documentation has shifted over time](#Item-number-and-field-name-represent-an-area-where-the-language-used-in-TANF-documentation-has-shifted-over-time)
* [We further validated the utility of submission history and notification emails as ways for TANF programs to show proof of transmission for audit purposes](#We-further-validated-the-utility-of-submission-history-and-notification-emails-as-ways-for-tanf-programs-to-show-proof-of-transmission-for-audit-purposes)
Expand Down Expand Up @@ -206,4 +206,4 @@ A common subsequent question is "Are my files without errors or do they need cor

### What's next
- We plan to capture final prototype encompassing all changes informed by this research in [#2527](https://github.com/raft-tech/TANF-app/issues/2527).
- Further errors research with participants from this round who submitted files with errors for the May 15th deadline to repeat many elements of this test but directly in the context of their real data (rather than example data).
- Further errors research with participants from this round who submitted files with errors for the May 15th deadline to repeat many elements of this test but directly in the context of their real data (rather than example data).
Loading

0 comments on commit f38f6df

Please sign in to comment.