Skip to content

Commit

Permalink
Merge branch '3046-plg-cloud' into feat/3171-nexus-integration
Browse files Browse the repository at this point in the history
  • Loading branch information
@andrew-jameson
andrew-jameson committed Oct 14, 2024
2 parents 617faff + bb44fb9 commit e217849
Show file tree
Hide file tree
Showing 46 changed files with 5,621 additions and 2,482 deletions.
1 change: 1 addition & 0 deletions .gitconfig
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@
allowed = .git/config:.*
allowed = .gitconfig:.*
allowed = .*DJANGO_SECRET_KEY=.*
allowed = ./tdrs-backend/plg/loki/manifest.yml:*
8 changes: 7 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -108,4 +108,10 @@ tfapply
cypress.env.json

# Patches
*.patch
*.patch

# Logs
*.log

# DB seeds
tdrs-backend/*.pg
17 changes: 17 additions & 0 deletions Taskfile.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,23 @@ tasks:
cmds:
- docker-compose -f docker-compose.yml exec web sh -c "python ./manage.py shell"

backend-exec:
desc: Execute a command in the backend container
dir: tdrs-backend
vars:
CMD: '{{.CMD}}'
cmds:
- docker-compose -f docker-compose.yml exec web sh -c "python manage.py {{.CMD}}"

backend-exec-seed-db:
desc: Execute seed_db command in the backend container
dir: tdrs-backend
vars:
CMD: '{{.CMD}}'
cmds:
- docker-compose -f docker-compose.yml up -d
- docker-compose -f docker-compose.yml exec web sh -c "python manage.py populate_stts; python ./manage.py seed_db"

backend-pytest:
desc: 'Run pytest in the backend container E.g: task backend-pytest PYTEST_ARGS="tdpservice/test/ -s -vv"'
dir: tdrs-backend
Expand Down
5 changes: 4 additions & 1 deletion codecov.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,7 @@ flags:
carryforward: true

ignore:
- "tdrs-backend/tdpservice/scheduling/db_backup.py"
- "tdrs-backend/tdpservice/scheduling/db_backup.py"
- "tdrs-backend/tdpservice/search_indexes/admin/mulitselect_filter.py"
- "tdrs-backend/tdpservice/email/helpers/account_access_requests.py"
- "tdrs-backend/tdpservice/search_indexes/admin/filters.py"
Binary file modified product-updates/img/error-reports/error-report.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
93 changes: 93 additions & 0 deletions product-updates/knowledge-center/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,99 @@ <h2 class="usa-card__heading" id="existing">Use an existing Login.gov Account</h

<h2>What's new in TDP</h2>


<h3 style="margin-top: 32px;">October 10th 2024 (v 3.6.4)</h3>
<h4 style="margin-top: 16px; color: #595959;">Added:</h4>
<ul class="usa-icon-list">
<li class="usa-icon-list__item" style="margin-top: 16px;">
<div class="usa-icon-list__icon text-ink">
<img src="../img/added.svg" aria-hidden="true" alt="" style="background-color: #d3ffcc; border-radius: 5px; width: 18px; height: 18px; position: relative; top: 2px; margin-right: 5px; margin-left: 5px;"/>
</div>
<div class="usa-icon-list__content">
<h5 class="usa-icon-list__title">
New Error Type Column Added to Error Report
</h5>
<p>A new column called "Error Type" has been added to error reports. This feature will help users quickly locate high-priority errors affecting file submission.</p>
<p>The new column categorizes four types of errors by their priority and impact on file status. Detailed guidance to filter high-priority errors along with further information on this update is available in the <a href="viewing-error-reports.html#interpreting-error-types">Knowledge Center.</a></p>

</div>
</li>

</ul>
<table class="usa-table">
<thead>
<tr>
<th scope="col">Error Type</th>
<th scope="col">Priority</th>
<th scope="col">Impact</th>
<th scope="col">File Status</th>
</tr>
</thead>
<tbody>
<tr>
<th scope="row">File Pre-Check</th>
<td>
High
</td>
<td>
Some or all of the record(s) in the file are rejected
</td>
<td>
Rejected or Partially Accepted with errors
</td>

</tr>
<tr>
<th scope="row">Case Consistency</th>
<td>
High
</td>
<td>
Record(s) rejected
</td>
<td>
Partially accepted with errors
</td>
</tr>
<tr>
<th scope="row">Record Value Invalid</th>
<td>
Low
</td>
<td>
Record(s) accepted
</td>
<td>
Accepted with errors
</td>

<tr>
<th scope="row">Record Value Inconsistency</th>
<td>
Low
</td>
<td>
Record(s) accepted
</td>
<td>
Accepted with errors
</td>

</tr>

</tbody>
</table>
</ul>



<hr style="border: none;
height: 1px;
background: #000;
margin-top: 32px;
background: repeating-linear-gradient(90deg,#000,#000 6px,transparent 6px,transparent 12px);">


<h3 style="margin-top: 32px;">September 10th 2024 (v 3.6.0)</h3>
<h4 style="margin-top: 16px; color: #595959;">In Development:</h4>
<ul class="usa-icon-list">
Expand Down
145 changes: 108 additions & 37 deletions product-updates/knowledge-center/viewing-error-reports.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -346,8 +346,6 @@ <h2 id="download-and-view">Download and View Error Reports</h2>
<h2 id="data-file-structure">Data File Structure</h2>
<div style="min-height: 22px;"></div>



<p>The files that you submit on a quarterly basis for your TANF or SSP program contain large amounts of data in a <a href="https://www.acf.hhs.gov/ofa/policy-guidance/final-tanf-ssp-moe-data-reporting-system-transmission-files-layouts-and-edits">specific format</a> that enables TDP to process and provide feedback when uploaded and submitted. Please note that these files are designed for system processing and, as such, are not as human-readable. For example, below is an illustration of a short TANF, Section 1 (Active Case Data) file:</p>

<a href="../img/error-reports/flatfile.png" data-lity data-lity-desc="TANF data file opened in a text editor beginning with a Header record and ending with a Trailer record">
Expand Down Expand Up @@ -558,20 +556,109 @@ <h2 id="overview-of-the-error-report">Overview of the Error Report</h2>
<li>Item Number and Item Name — which can be used alongside the error message to cross-reference with the coding instructions.</li>
<li>Internal Variable Name — a value specific to TDP's database for use in support sessions with the TANF data team.</li>
<li>Row Number — identifying which row of the submitted file contains the record associated with a given error.</li>
<li>Error Type — indicates the category of error and is related to both the cause of the error, and its effect on whether records are accepted into the database.</li>
</ul>
<div style="min-height: 22px;"></div>


<a href="../img/error-reports/error-report.png" data-lity data-lity-desc="TDP error report containing some of the error examples below opened in Excel">
<img src="../img/error-reports/error-report.png" alt="TDP error report containing some of the error examples below opened in Excel">
</a>

<div style="min-height: 22px;"></div> <div style="min-height: 22px;"></div>
<h3 id="interpreting-error-types">Interpreting Error Types</h3>
<div style="min-height: 22px;"></div>
<p>There are four error types which affect data submission differently. Unlike low priority errors, high priority errors will prevent some/all records in the file from being accepted into the TDP system's database. You may filter the report to display high-priority errors first. Low priority errors will not affect whether a record makes it into the data base, but we encourage you to refer to the coding instructions to make corrections for the most accurate data. <a href="https://support.microsoft.com/en-us/office/filter-data-in-a-range-or-table-01832226-31b5-4568-8806-38c37dcc180e" target="_blank">Learn how to filter data here.</a></p>
<br>
<p>For users familiar with the legacy system, errors are equivalent to edits. File Pre-Check and Case Consistency errors are similar to fatal edits and Record Value Invalid and Record Value Inconsistency are similar to warnings.</p>

<table class="usa-table">
<thead>
<tr>
<th scope="col">Error Type</th>
<th scope="col">Description</th>
<th scope="col">Priority</th>
<th scope="col">Impact</th>
<th scope="col">File Status</th>
</tr>
</thead>
<tbody>
<tr>
<th scope="row">File Pre-Check</th>
<td>
Relate to the format of the records within a file (e.g. incorrect headers, incorrect record length).
</td>
<td>
High
</td>
<td>
Some or all of the record(s) in the file are rejected </td>
<td>
Rejected or Partially Accepted with errors

</td>

</tr>
<tr>
<th scope="row">Case Consistency</th>
<td>
Relates to inconsistency across records (e.g. inconsistency across related person- and family-level records).
These errors are triggered by a series of checks that confirm that all the records for a family in a given month are related in a meaningful way. </td>
<td>
High
</td>
<td>
Record(s) rejected
</td>
<td>
Partially accepted with errors
</td>

</tr>
<tr>
<th scope="row">Record Value Invalid</th>
<td>
These errors involve <i>unexpected</i> values in the record (e.g. missing values or out-of-range values).
</td>
<td>
Low
</td>
<td>
Record(s) accepted
</td>
<td>
Accepted with errors
</td>

</tr>
<tr>
<th scope="row">Record Value Inconsistency</th>
<td>
These errors stem from inconsistencies in the record (e.g. conflicting data between items in the same record).
</td>
<td>
Low
</td>
<td>
Record(s) accepted
</td>
<td>
Accepted with errors
</td>

</tr>

</tbody>
</table>

<div style="min-height: 22px;"></div> <div style="min-height: 22px;"></div>



<h3>Examples of Common Errors</h3>
<div style="min-height: 22px;"></div>
<p>Below are examples of error messages associated with common issues that may be listed in an error report.</p>



<div class="usa-alert usa-alert--info usa-alert--no-icon">
<div class="usa-alert__body">
<p class="usa-alert__text">
Expand All @@ -581,41 +668,42 @@ <h3>Examples of Common Errors</h3>

<div style="min-height: 22px;"></div>
<div style="min-height: 22px;"></div>
<h4>Errors related to header or trailer records:</h4>
<h4>File Pre-Check: Errors related to header or trailer records:</h4>
<div style="min-height: 22px;"></div>
<p>Header-related errors are often the result of submitting files for a fiscal period that is not consistent with the time period in the header record (e.g. trying to submit 2022 data for a 2024 submission). Other header or trailer errors may be related to how the file was generated (e.g. the file produced is missing a header or trailer record). Some examples of how these types of error may appear in your error report are included below:</p>


<div style="min-height: 22px;"></div>
<code style="border-left: solid 5px; border-color: #005ea2; padding: 8px; background-color: #f8f8f8; display: inline-block;">Submitted reporting year: 2024, quarter: 1 doesn't match file reporting year 2022, quarter: 4.
</code>
<div style="min-height: 22px;"></div>
<code style="border-left: solid 5px; border-color: #005ea2; padding: 8px; background-color: #f8f8f8; display: inline-block;">Your file does not begin with a HEADER record.
</code>
<div style="min-height: 22px;"></div>


<p>Please refer to the <a href="https://www.acf.hhs.gov/sites/default/files/documents/ofa/transmission_file_header_trailer_record.pdf">Transmission File Header Record definitions</a> to compare your file's header or trailer to the expected layout.


<div style="min-height: 22px;"></div><div style="min-height: 22px;"></div> <div style="min-height: 22px;"></div>
<h4>Errors related to record length:</h4>
<h4>File Pre-Check: Errors related to record length:</h4>
<div style="min-height: 22px;"></div>
<p>Record length-related errors will be raised if the specified record is not aligned with the record layout requirements. For example, this kind of error may appear as follows in the report:</p>
<div style="min-height: 22px;"></div>




<code style="border-left: solid 5px; border-color: #005ea2; padding: 8px; background-color: #f8f8f8; display: inline-block;">T6 record length is 409 characters but must be 379.
</code>
<div style="min-height: 22px;"></div>

<p>Please refer to the <a href="https://www.acf.hhs.gov/ofa/policy-guidance/final-tanf-ssp-moe-data-reporting-system-transmission-files-layouts-and-edits">Transmission File Layout documents</a> to compare your records against their expected layouts.</p>

<div style="min-height: 22px;"></div>
<p>Please refer to the <a href="https://www.acf.hhs.gov/ofa/policy-guidance/final-tanf-ssp-moe-data-reporting-system-transmission-files-layouts-and-edits">Transmission File Layout documents</a> to compare your records against their expected layouts.</p>
<div style="min-height: 22px;"></div><div style="min-height: 22px;"></div> <div style="min-height: 22px;"></div>
<h4 id="related-records">Case Consistency: Errors related to inconsistent values across related records:</h4>
<div style="min-height: 22px;"></div>
<p>Errors with inconsistent values across related records may require review of the coding instructions to determine the proper correction. In the example below, the error is communicating that a T1 (family) record was found in the file that did not have a corresponding T2 (adult) or T3 (child) record, which effectively means that person-level records associated with this family are missing from the file. </p>
<div style="min-height: 22px;"></div>
<code style="border-left: solid 5px; border-color: #005ea2; padding: 8px; background-color: #f8f8f8; display: inline-block;">Every T1 record should have at least one corresponding T2 or T3 record with the same Report Month & Year and Case Number.

</code>

<div style="min-height: 22px;"></div><div style="min-height: 22px;"></div> <div style="min-height: 22px;"></div>
<h4>Errors related to invalid values for a specific item/data element:</h4>
<h4>Record Value Invalid: Errors related to invalid values for a specific item/data element:</h4>
<div style="min-height: 22px;"></div>
<p>Invalid value errors can come up when a specific item/data element has an unexpected value (e.g. a letter or a symbol was reported for the zip code field, such as: "462$1"): </p>

Expand All @@ -630,32 +718,15 @@ <h4>Errors related to invalid values for a specific item/data element:</h4>
<li><a href="https://www.acf.hhs.gov/ofa/policy-guidance/acf-ofa-pi-23-04">TANF (ACF-199) and SSP-MOE (ACF-209) Coding Instructions</a></li>
</ul>


<div style="min-height: 22px;"></div><div style="min-height: 22px;"></div> <div style="min-height: 22px;"></div>
<h4 id="inconsistent-values">Errors related to inconsistent values for related items/data elements in the same record:</h4>
<h4 id="inconsistent-values">Record Value Inconsistency: Errors related to inconsistent values for related items/data elements in the same record:</h4>
<div style="min-height: 22px;"></div>
<p>Some errors may require review of the coding instructions for multiple items (and their respective values) to determine the proper correction. In the example below, the error is communicating that the value reported for Item 49 is in a conflict with the value for Item 30 in the same record. This message suggests a problem with either the value of Item 49 or the value of Item 30. Refer to the coding instructions and your own data to determine which value needs to be corrected.</p>



<div style="min-height: 22px;"></div>
<code style="border-left: solid 5px; border-color: #005ea2; padding: 8px; background-color: #f8f8f8; display: inline-block;">If Item 30 (Family Affiliation) is 1 then Item 49 (Work Participation Status) must be in set of values [01, 02, 05, 07, 09, 15, 17, 18, 19, 99].
</code>
<div style="min-height: 22px;"></div>

<div style="min-height: 22px;"></div><div style="min-height: 22px;"></div> <div style="min-height: 22px;"></div>
<h4 id="related-records">Errors related to inconsistent values across related records:</h4>
<div style="min-height: 22px;"></div>
<p>Errors with inconsistent values across related records may require review of the coding instructions to determine the proper correction. In the example below, the error is communicating that a T1 (family) record was found in the file that did not have a corresponding T2 (adult) or T3 (child) record, which effectively means that person-level records associated with this family are missing from the file. </p>



<div style="min-height: 22px;"></div>
<code style="border-left: solid 5px; border-color: #005ea2; padding: 8px; background-color: #f8f8f8; display: inline-block;">Every T1 record should have at least one corresponding T2 or T3 record with the same Report Month & Year and Case Number.
</code>
<code style="border-left: solid 5px; border-color: #005ea2; padding: 8px; background-color: #f8f8f8; display: inline-block;">If Item 30 (Family Affiliation) is 1 then Item 49 (Work Participation Status) must be in set of values [01, 02, 05, 07, 09, 15, 17, 18, 19, 99].
</code>
<div style="min-height: 22px;"></div>
<div style="min-height: 22px;"></div>

<div style="min-height: 22px;"></div> <div style="min-height: 22px;"></div>

<hr style="border: none;
height: 1px;
Expand Down
Loading

0 comments on commit e217849

Please sign in to comment.