diff --git a/docs/Sprint-Review/sprint-104-summary.md b/docs/Sprint-Review/sprint-104-summary.md new file mode 100644 index 000000000..93beb6ef8 --- /dev/null +++ b/docs/Sprint-Review/sprint-104-summary.md @@ -0,0 +1,82 @@ +# sprint-104-summary + +7/17/2024 - 7/30/2024 + +### Sprint Goal + +**Dev:** + +_**Plain Language Error Messaging and Application Health Monitoring work, improved dev tooling, and fixing bugs**_ + +* \#2792 — \[Error Audit] Category 3 error messages clean-up +* \#3059 — Bug: file stuck in pending state when DOB or SSN field is space-filled +* \#2965 — As tech lead, I want a database seed implemented for testing +* \#2175 — \[Bug] Data Files “Download” button(s) disappear when clicked +* \#3055 — Service timeout blocks parsing completion +* \#3061 — \[a11y fix] Django multi-select filter +* \#2960 — As a engineer I need to replace bash script with task file for local dev + +**DevOps:** + +_**Successful deployments across environments and pipeline stability investments**_ + +* \#2458 — Integrate Nexus into CircleCI +* \#3043 — Sentry: Local environment for Debugging +* \#2526 — "nightly" owasp scan after qasp deployment +* \#1623 — As tech lead, I want CircleCI pipelines to catch migration and/or deployment failures + +**Design:** + +_**Support reviews, Finalize Django Admin Experience epic research, Draft research synthesis**_ + +* \#2910 — Django Admin Experience Improvements Research Session (Part 2) +* \#3078 — DIGIT Admin Experience Synthesis + +## Tickets + +### Completed/Merged + +* [#1620 \[SPIKE\] As tech lead, I need to know the real-time branches deployed in Cloud.gov spaces](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/1620) +* [#2910 \[Research Facilitation\] Admin Experience Improvements](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2910) +* [#2960 As a engineer I need to replace bash script with task file for local dev](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2960) +* [#3004 Implement (small) data lifecycle (backup/archive ES)](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3004) +* [#3016 Spike - Cat2 Validator Improvement](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3016) +* [#3049 as an STT user, I need the error message related to the header update indicator clarified](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3049) +* [#3059 Bug: file stuck in pending state when DOB or SSN field is space-filled](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3059) + +### Submitted (QASP Review, OCIO Review) + +* [#3055 Service timeout blocks parsing completion](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3055) +* [#3061 \[a11y fix\] Django multi-select filter ](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3061) +* [#1621 As a TDP user, I'd like to see a descriptive error message page if authentication source is unavailable.](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/1621) +* [#2996 Add dynamic field name to cat4 error messages](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2996) +* [#3058 \[Design Deliverable\] Release notes email template](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3058) +* [#3057 \[Design Deliverable\] Spec for light-lift fiscal quarter / calendar quarter explainer in TDP](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3057) +* [#2985 \[Design Deliverable\] Email spec for Admin Notification for stuck files](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2985) +* [#2883 Pre-Made Reporting Dashboards on Kibana](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2883) +* [#2954 Extend SESSION\_COOKIE\_AGE](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2954) +* [#2993 Kibana Dashboard MVP](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2993) + +### Ready to Merge + +* + +### Closed (Not Merged) + +* [#2526 "nightly" owasp scan after qasp deployment](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2526) +* [#1623 As tech lead, I want CircleCI pipelines to catch migration and/or deployment failures](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/1623) + +### Moved to Next Sprint + +**In Progress** + +* [#2792 \[Error Audit\] Category 3 error messages clean-up](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2792) + +#### Blocked + +* + +**Raft Review** + +* [#3043 Sentry: Local environment for Debugging](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3043) +* [\[Research Synthesis\] DIGIT Admin Experience Improvements#3078](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3078) diff --git a/docs/Sprint-Review/sprint-105-summary.md b/docs/Sprint-Review/sprint-105-summary.md new file mode 100644 index 000000000..b9b57f9ef --- /dev/null +++ b/docs/Sprint-Review/sprint-105-summary.md @@ -0,0 +1,92 @@ +# sprint-105-summary + +7/31/2024 - 8/14/2024 + +### Priority Setting + +* Reparsing + * Tickets: + * \#3064 — Re-parse Meta Model + * \#3113 — As tech lead, I need the validation on the header update indicator revised to unblock parsing + * \#3073 — \[bug] TDP is raising cat 4 error on TANF/SSP closed case files that is not present +* System Monitoring +* DIGIT Work + +### Sprint Goal + +**Dev:** + +_**Plain Language Error Messaging and Application Health Monitoring work, improved dev tooling, and fixing bugs**_ + +* \#2792 — \[Error Audit] Category 3 error messages clean-up +* \#2965 — As tech lead, I want a database seed implemented for testing +* \#3064 — Re-parse Meta Model +* \#3113 — As tech lead, I need the validation on the header update indicator revised to unblock parsing +* \#3073 — \[bug] TDP is raising cat 4 error on TANF/SSP closed case files that is not present +* \#3062 — bug: ES docker image for non-dev spaces stored in personal dockerhub +* \#1646 — \[A11y Fix] Correct TDP home : aria label mismatch + +**DevOps:** + +_**Successful deployments across environments and pipeline stability investments**_ + +* \#2458 — Integrate Nexus into CircleCI + +**Design:** + +_**Support reviews, Complete Research Synthesis, Continue Error Audit (Cat 4)**_ + +* \#3078 — DIGIT Admin Experience Synthesis +* \#3114 — \[Design Spike] In-app banner for submission history pages +* \#2968 — \[Design Deliverable] Update Error Audit for Cat 4 / QA + +## Tickets + +### Completed/Merged + +* [#1621 As a TDP user, I'd like to see a descriptive error message page if authentication source is unavailable.](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/1621) +* [#1646 \[A11y Fix\] Correct TDP home : aria label mismatch](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/1646) +* [#3033 As tech lead, I need the sections 3 and 4 calendar quarter logic updated](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3033) +* [#3055 Service timeout blocks parsing completion](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3055) +* [#3057 \[Design Deliverable\] Spec for light-lift fiscal quarter / calendar quarter explainer in TDP](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3057) +* [#3113 As tech lead, I need the validation on the header update indicator revised to unblock parsing ](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3113) + +### Submitted (QASP Review, OCIO Review) + +* [#2954 Extend SESSION\_COOKIE\_AGE](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2954) +* [#3061 \[a11y fix\] Django multi-select filter ](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3061) +* [#3079 DB Backup Script Fix](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3079) +* [#2883 Pre-Made Reporting Dashboards on Kibana](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2883) +* [#2985 \[Design Deliverable\] Email spec for Admin Notification for stuck files](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2985) +* [#2996 Add dynamic field name to cat4 error messages](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2996) +* [#2993 Kibana Dashboard MVP](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2993) + +### Ready to Merge + +* [#3058 \[Design Deliverable\] Release notes email template](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3058) +* [#3062 bug: ES docker image for non-dev spaces stored in personal dockerhub](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3062) +* [#3073 \[bug\] TDP is raising cat 4 error on TANF/SSP closed case files that is not present](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3073) +* [#3107 \[Re-parse command\] Retain original submission date when command runs](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3107) + +### Closed (Not Merged) + +* [#1355 Research questions around DIGIT teams query usage for parsed data](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/1355) + +### Moved to Next Sprint + +**In Progress** + +* [#2965 As tech lead, I want a database seed implemented for testing](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2965) + +#### Blocked + +* [#2458 Integrate Nexus into CircleCI](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2458) + +**Raft Review** + +* [#3043 Sentry: Local environment for Debugging](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3043) +* [#3064 Re-parse Meta Model](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3064) +* [#3065 Spike - Guarantee Sequential Execution of Re-parse Command](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3065) +* [#3078 \[Research Synthesis\] DIGIT Admin Experience Improvements](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3078) +* [#3087 Admin By Newest Filter Enhancements for Data Files Page](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/3087) +* [#2792 \[Error Audit\] Category 3 error messages clean-up](https://app.zenhub.com/workspaces/sprint-board-5f18ab06dfd91c000f7e682e/issues/gh/raft-tech/tanf-app/2792) diff --git a/docs/Technical-Documentation/diagrams/tdp-environments.drawio b/docs/Technical-Documentation/diagrams/tdp-environments.drawio index 255c6061e..4449dff1a 100644 --- a/docs/Technical-Documentation/diagrams/tdp-environments.drawio +++ b/docs/Technical-Documentation/diagrams/tdp-environments.drawio @@ -1,6 +1,6 @@ - + - + @@ -14,7 +14,7 @@ - + @@ -212,11 +212,6 @@ - - - - - @@ -300,7 +295,7 @@ - + @@ -324,8 +319,16 @@ + + + + + + + + - + @@ -477,18 +480,16 @@ - + - + - + + - - - @@ -498,38 +499,39 @@ - + - + - - + + + + + + + - + + - + + - + - - - - - - @@ -625,6 +627,9 @@ + + + @@ -634,9 +639,6 @@ - - - @@ -669,10 +671,13 @@ - + - + + + + @@ -729,13 +734,47 @@ - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + + + + + + + diff --git a/docs/Technical-Documentation/diagrams/tdp-environments.drawio.png b/docs/Technical-Documentation/diagrams/tdp-environments.drawio.png index a56b1c516..757d7ec10 100644 Binary files a/docs/Technical-Documentation/diagrams/tdp-environments.drawio.png and b/docs/Technical-Documentation/diagrams/tdp-environments.drawio.png differ diff --git a/docs/User-Experience/Research-Syntheses/2024, Summer - OFA Admin Experience.md b/docs/User-Experience/Research-Syntheses/2024, Summer - OFA Admin Experience.md new file mode 100644 index 000000000..4f769a7ba --- /dev/null +++ b/docs/User-Experience/Research-Syntheses/2024, Summer - OFA Admin Experience.md @@ -0,0 +1,62 @@ +# 2024, Summer - OFA Admin Experience + +**Jump To:** + +* [Background](#background) +* [What we did & who we talked to](#what-we-did--who-we-talked-to) +* [What we learned](#what-we-learned) +* [Next steps](#next-steps) + +*** + +## Background + +The Django Admin Console (DAC) is an internal tool tailored for our OFA System Admin and DIGIT Team users. It allows for access to the Postgres database & system logs, managing user permissions, and has been increasingly adopted to provide the DIGIT team with quick insights into data file errors and STT submissions. + +*** + +## What we did & who we talked to + +We ran two workshops with the DIGIT team; which overlaps with our two OFA System Admins. While those teams overlap, note that the permissions for each user group differ. OFA System Admins have privileged access to DAC while DIGIT team users have non-privileged access. Our product manager and two developers were also in attendance for alignment and work estimation purposes. These workshops facilitated conversation around DAC enhancement requests & pain points provided by OFA in tickets [#2930](https://github.com/raft-tech/TANF-app/issues/2930), [#1662](https://github.com/raft-tech/TANF-app/issues/1662), [#960](https://github.com/raft-tech/TANF-app/issues/960), and [#2910](https://github.com/raft-tech/TANF-app/issues/2910) in order to achieve: + +* A clear understanding of current enhancement requests and described pain points +* Initial estimation of the work required to deliver each change to the DAC +* Alignment on the scope of potential work to support prioritizing all issues within our product roadmap and upcoming sprints + +*** + +## What we learned + +We identified and refined our understanding of Django Admin Console enhancements in the following categories: + +### Filtering & readability + +
EnhancementDescriptionDAC PageTicketRecommended Priority
Filter data files by relative dateAdds filter to the DAC Data Files page that filters by submission date and includes options for submissions yesterday, today, the past 7 days, the current month, and the current year. #3077 captures a higher lift enhancement to this use case.Data Files#30764.0 / P3
Default filter on DAC Data Files page to show only the most recent submissions per STT, fiscal period, and section. Currently DAC Search Indexes pages default to filtering results to "Newest". This ticket updates the language of that filter to "Most recent version" and adds that behavior to the Data Files page.Search Indexes, Data Files#30874.0 / P4
Add multiselect control to Search Indexes Fiscal Period FilterCurrently the filter control on DAC Search Indexes pages is a single option dropdown. This ticket replaces it with the multiselect control that we use when filtering by STT. This replicates the SQL union queries used in the legacy system.Search Indexes#31024.0 / P4
Add filters from DAC Data Files page to Data File Summaries pagesCurrently Data File Summaries pages lack filtering capability. This ticket delivers filter options matching those on the Data Files page.Data File Summaries#30934.0 / P2
[Spike] Investigate adding Change Message typesThis spike investigates whether we can supplement the current Change Message column on log entries with a change type to allow for filtering capability. Log Entries#30924.0 / P2
[Spike] Investigates how we can provide a tabular view of data file summariesCurrently data file summaries are served up in a raw JSON format. To make them easier to read we should investigate how we might map these data to a table view.Data File Summaries (Specific Summary view)#30954.0 / P2
Rearrange Data Files filters and implement multiselect fiscal period filterAdds the multiselect control for fiscal period filtering (as seen on current Search Index pages on the Data Files page and rearanges filters into a more intuitive order.Data Files#30974.0 / P2
[Spike] Investigate YYYYMMDD value filtering for data filesHigher lift ideal solution following on from #3076. Investigates how we might add the ability to filter the DAC Data Files page to those submitted on a specific date or within a specific date range. Data Files#3077Beyond 4.0 / P2
List of Cat 1,4 rejected case numbersProvides a method of filtering current django outputs to produce a list of case numbers and months which are associated with category 1 and 4 errors. This provides DIGIT with a new analogue to the transmission reports of the legacy system.Data File Summaries (Specific Summary view)#3096Beyond 4.0 / P2
+ +### DAC actions & behavior + +
EnhancementDescriptionDAC PageTicketRecommended Priority
Read-only data file summaries Modifies the DAC Data File Summaries view to make it read-only to better correspond to how it's used by the DIGIT team.Data File Summaries (Specific summary view)#30944.0 / P2
User deletionCurrently user deletion is not supported via the DAC. This ticket delivers that capability while retaining all objects associated to deleted users.Users#3089Beyond 4.0 / P2
Mass actions on Users tableCurrently System Admins cannot select and deactivate multiple users at a time. This ticket also delivers a filter to restrict the Users table to only those who have been inactive for 180+ days.Users#3090Beyond 4.0 / P2
Add mailto: functionality to user email addresses in metadataCurrently in views of date file metadata, clicking on the user's email address links to that users entry in the DAC Users page. This ticket delivers an update that changes it to a mailto: that will open in the device's default email application.Users (Specific user view)#3120Beyond 4.0 / P2
+ +### Bugs & system performance + +
EnhancementDescriptionDAC PageTicketRecommended Priority
[Bug] Misleading file status column on DAC data files pageThe file status column can return incorrect status values when viewed from the data files table rather than an individual data file's metadata page.Data Files#30684.0 / P2
[Spike] Investigate handling of custom filters During implementation of the first DAC multiselect filter we discovered problems with the handling of query strings which will pose scalability problems as we introduce new filters. N/A#31104.0 / P3
[Spike] Investigate latency when clicking into the parsing errors column on DAC data files pageCurrently when clicking into "Parser Errors" for a given row of the DAC Data Files page there is significant latency before the system returns results. Data Files#30754.0 / P3
+ +### Parsing + +
EnhancementDescriptionTicketRecommended Priority
Update Section 3, 4 validation to screen for ≥ 1 families rather than ≥ 0 Sections 3 and 4 of TANF data concern aggregate values that are highly unlikely to be 0. This ticket delivers a parsing logic fix to reflect that.#30884.0 / P3
+ +### User permissions + +
EnhancementDescriptionTicketRecommended Priority
TDP Data Files page permissions for DIGIT & Sys Admin user groupsCurrently users assigned to the DIGIT or System Admin user groups cannot reach and browse TDP's Data Files page. This ticket adds those permissions for both groups.#30744.0 / P4
+ +### Security Controls + +
EnhancementDescriptionTicketRecommended Priority
Auto-deactivation of usersUser deactivation is currently a manual process for system admins. This ticket delivers automation that will automatically deactivate users who have been inactive for 180 days.#25614.0 / P3
System owner notification upon assigned admin permissionsSince very few people should be granted System Admin permissions in production, the system owner should be notified whenever the role is assigned/unassigned.#13374.0 / P2
+ +*** + +## Next Steps + +Following this research, the design team will fully refine all the tickets referenced above and coordinate time with development and the DIGIT team to determine which enhancements will be tackled in release 4.0 and which will be deprioritized for a subsequent release. + +Additionally, the design team will prioritize [#3121](https://github.com/raft-tech/TANF-app/issues/3121) which delivers the email template that will be implemented by development in [#1337.](https://github.com/raft-tech/TANF-app/issues/1337) diff --git a/docs/User-Experience/Research-Syntheses/README.md b/docs/User-Experience/Research-Syntheses/README.md index 43496f75a..1602253be 100644 --- a/docs/User-Experience/Research-Syntheses/README.md +++ b/docs/User-Experience/Research-Syntheses/README.md @@ -5,6 +5,14 @@ With a few exceptions, we've tended to publish markdown research syntheses to su The syntheses included herein are organized reverse-chronologically from newest to oldest: +### [2024, Summer - OFA Admin Experience](https://github.com/raft-tech/TANF-app/blob/develop/docs/User-Experience/Research-Syntheses/2024,%20Summer%20-%20OFA%20Admin%20Experience.md) +- Ran two workshops with the OFA DIGIT team focused on enhancement requests for the Django Admin Console (DAC) to achieve: + - A clear understanding of current enhancement requests and described pain points + - Initial estimation of the work required to deliver each change to the DAC + - Alignment on the scope of potential work to support prioritizing all issues within our product roadmap and upcoming sprints + + + ### [2023, Sprint - TDP 3.0 Pilot Program](https://github.com/raft-tech/TANF-app/blob/develop/docs/User-Experience/Research-Syntheses/2023%2C%20Spring%20-%20Testing%20CSV%20%26%20Excel-based%20error%20reports.md#spring-2023---testing-csv--excel-based-error-reports) - Research sessions conducted with 5 states and 4 Tribes with a focus on programs that had errors on their Section 1 Data Files.