Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat/1620 merge pg upgrade #3072

Merged
merged 12 commits into from
Jul 11, 2024
Merged

Feat/1620 merge pg upgrade #3072

merged 12 commits into from
Jul 11, 2024

Conversation

andrew-jameson
Copy link
Collaborator

Summary of Changes

Provide a brief summary of changes
Pull request closes #_

How to Test

List the steps to test the PR
These steps are generic, please adjust as necessary.

cd tdrs-frontend && docker-compose -f docker-compose.yml -f docker-compose.local.yml up -d
cd tdrs-backend && docker-compose -f docker-compose.yml -f docker-compose.local.yml up -d
  1. Open http://localhost:3000/ and sign in.
  2. Proceed with functional tests as described herein.
  3. Test steps should be captured in the demo GIF(s) and/or screenshots below.

Demo GIF(s) and screenshots for testing procedure

Deliverables

More details on how deliverables herein are assessed included here.

Deliverable 1: Accepted Features

Checklist of ACs:

  • [insert ACs here]
  • lfrohlich and/or adpennington confirmed that ACs are met.

Deliverable 2: Tested Code

  • Are all areas of code introduced in this PR meaningfully tested?
    • If this PR introduces backend code changes, are they meaningfully tested?
    • If this PR introduces frontend code changes, are they meaningfully tested?
  • Are code coverage minimums met?
    • Frontend coverage: [insert coverage %] (see CodeCov Report comment in PR)
    • Backend coverage: [insert coverage %] (see CodeCov Report comment in PR)

Deliverable 3: Properly Styled Code

  • Are backend code style checks passing on CircleCI?
  • Are frontend code style checks passing on CircleCI?
  • Are code maintainability principles being followed?

Deliverable 4: Accessible

  • Does this PR complete the epic?
  • Are links included to any other gov-approved PRs associated with epic?
  • Does PR include documentation for Raft's a11y review?
  • Did automated and manual testing with iamjolly and ttran-hub using Accessibility Insights reveal any errors introduced in this PR?

Deliverable 5: Deployed

  • Was the code successfully deployed via automated CircleCI process to development on Cloud.gov?

Deliverable 6: Documented

  • Does this PR provide background for why coding decisions were made?
  • If this PR introduces backend code, is that code easy to understand and sufficiently documented, both inline and overall?
  • If this PR introduces frontend code, is that code easy to understand and sufficiently documented, both inline and overall?
  • If this PR introduces dependencies, are their licenses documented?
  • Can reviewer explain and take ownership of these elements presented in this code review?

Deliverable 7: Secure

  • Does the OWASP Scan pass on CircleCI?
  • Do manual code review and manual testing detect any new security issues?
  • If new issues detected, is investigation and/or remediation plan documented?

Deliverable 8: User Research

Research product(s) clearly articulate(s):

  • the purpose of the research
  • methods used to conduct the research
  • who participated in the research
  • what was tested and how
  • impact of research on TDP
  • (if applicable) final design mockups produced for TDP development

@andrew-jameson andrew-jameson self-assigned this Jul 11, 2024
@andrew-jameson andrew-jameson added the Deploy with CircleCI-raft Deploy to https://tdp-frontend-raft.app.cloud.gov through CircleCI label Jul 11, 2024
@andrew-jameson andrew-jameson changed the base branch from develop to feat/1620-code-signature July 11, 2024 13:55
@andrew-jameson andrew-jameson merged commit 54a8ec1 into feat/1620-code-signature Jul 11, 2024
13 checks passed
@andrew-jameson andrew-jameson deleted the feat/1620-merge-pg-upgrade branch July 11, 2024 14:19
andrew-jameson added a commit that referenced this pull request Jul 29, 2024
@andrew-jameson @elipe17
@jtimpe @ADPennington @robgendron @lfrohlich @reitermb @raftmsohani
Feat/1620 code signature (#3070)
590aa02 
* Adding action during deploy

* Bad branch name, quick merge. (#3069)

* - Initial commit for duplicate record manager for TANF section 1 records

* - more efficient error generation

* - Added a way to track precedence in the error messages to avoid generating less meaningful messages
- Updated error generation

* -  Fix lint
- Add docstrings

* - Added support to delete deplicate records

* - add support for bulk/raw delete

* - Update deletes on rollbacks to raw delete since it wont hurt anything

* - Updated elastic and django to support raw/bulk deleting

* - Updating rollback test since it is very relevant now that we are doing raw deletes

* - raise log level

* - Update test

* - remove import

* - Update error generation to use record and schema
- rename var

* - Update error precedence logic

* - fix lint

* - use correct method

* - remove extra call to bulk create errors

* - move duplicate manager into case consistency validator
- Starting to remove SortedRecordSchemaPairs from case consistency validator
- updating to support in memory record removal if they havent been serialized

* - update add_record logic to include rpt_month_year

* - Move SortedRecordSchema pairs to util
- Update case consistency validator to not use OG SortedRecordSchemaPairs
- Update dup logic to not consider records on the same line

* - remove unused import

* - Update class to store records based on hash pending the section type

* - Update parse to leverage new data structure

* - clearing after bulk create

* - Rename to be more accurate
- Add method to generate bulk create dictionary
- Stub function for removing cases

* - Updated class name
- basing off of doc instead of schema

* - Adding infrastructure to remove records from memory or the DB pending their state

* - fix logic issue

* - fix logic error

* - added back list of cases

* - fixed test_add_record

* - Fix all case consistency tests

* - Fix most lint errors

* - remove print

* - update rollback logic
- fix most parse tests

* - Update duplicate error messages
- Added logging
- Updated tests

* - functionized dup manager a bit
- updated validator to query errors from dup manager
- added test for s1 duplicate detection

* - Add test for S1 partial duplicate detection

* - fix lint

* - Add section 2 tests

* - Fix reference error

* - order by id

* - parametrizing batch size
- initial integration tests

* - move creation logic in parse.py
- add container to monitor what cases have serialized records
- add test for in memory and DB based record removal with dupes detected

* - added section 2 duplicate tests

* - Test for partial duplicates
- test doesnt behave as I would expect

* - fix lint

* - Fixed logic error where last case in file wouldnt be removed if it has cat4 errors

* - Update SortedRecords to leverage an unsorted container to make bulk  createing faster. Dont have to iterate/generate the required structure

* - naming and doc string updates

* - renaming functions
- removing unnecessary members

* - update doc strings

* - Add test for family affiliation negating partial duplicity

* - Updated to support duplicate checking on section 3/4

* - fix lint

* - Update docstring

* - Update to allow all record types to have duplicate detection

* - add duplicate detection unit tests for all program types and record types

* - fix lint

* - Add cases for ssp

* - Move parser fixtures to their own conftest.py

* - Move conftest.py to the test folder

* - removing whitespace

* - Remove 'test' from fixture names

* - Fix failing tests due to merge

* - Update message for failing test

* - Remove Vars from compose file
- Remove unnecessary checks from cat4 validator
- Updated error counts after removing check

* - fixed test

* - linting

* - move partial hash checking logic to schema/fields

* - Move fixtures to conftest.py

* - Fix failing tests

* - fixed tests

* Revert "- move partial hash checking logic to schema/fields"

This reverts commit ab379ce.

* - Better way to do partial hash checking
- Still in schema

* - Moved hash generation into schema
- updated tests and factories to now support agnostic hash generation

* - Updated error message
- fixed incorrect indentation on error counting

* - move fixtures to conftest

* add radio button for jurisdiction type

* - fix lint

* - fixed tests from merge conflict

* add tests

* - Update error creation logging to track all errors
- Updated queries to most correct syntax for aggregates function
- Updated tests

* - lint

* - move fixture

* - Updated aggregates query to take into account cat4 errors

* - lint

* - import as Query

* - moved fixture to correct location

* - Updated cat4 duplicate errors to include fields that are duplicated

* - fix lint

* - update tests

* - update from merge conflict

* - Add additional exclusion criteria for necessary tests
- add missing fixture decorators

* - renamed parameters and classes to make more sense with their duties.
- Added much more detailed descriptions where information surrounding duplicate detection can be confused.

* - implement TODOs that came out of OH

* - remove todo
- add warning labels to shared object

* - fix lint

* - adding some generic parsing flow docs

* update error string

* fix tests

* - Added useful debug logging to partial hash functions

* - Updated duplicate detection to allow "partial duplicates" of s3 and s4 files
- Added tests for partial dup s3/s4

* - updated file status to be consistent with expectations

* - Updated stt filter to display stt name instead of code to make it easier to filter

* - Remove custom filter in favor of dependency
- Update STTFilter to leverage dependency filter

* - update doc string

* - fix lint

* - Updated case_consistency validator to track which case has been validated to identify which case might need to be removed
- decoupled dup manager and case consistency val error counts
- Added more descriptive logging

* - Update tests

* - Add edge case cat4 test
- unblock util function that was causing errors

* - Updated manager to correctly delete serialized records
- Updated DFS to correctly count total records and total records created
- Updated cat4 test case to be more extensive

* - Updated test

* - fix lint

* - Add elastic specific exception handling

* - add extra case to file for duplicate detection

* - Added creation of LogEntry objects in exception blocks

* - Updated fiscal year filter to not list `All` twice
- Updated STT filter lookup options

* - fix lint

* - Add line number to duplicate error ParserError objects

* - Remove dup test class
- Updated to use correct error generator

* - fix lint

* - Updated based on review feedback

* - Update tests

* - fix lint

* don't show jurisdiction type for acf users

* Create sprint-99-summary.md (#3038)

* Create sprint-99-summary.md

* Update docs/Sprint-Review/sprint-99-summary.md

Co-authored-by: Lauren Frohlich <[email protected]>

---------

Co-authored-by: Lauren Frohlich <[email protected]>

* Create sprint-100-summary.md (#3039)

* Create sprint-100-summary.md

* Update docs/Sprint-Review/sprint-100-summary.md

Co-authored-by: Lauren Frohlich <[email protected]>

* Update docs/Sprint-Review/sprint-100-summary.md

Co-authored-by: Lauren Frohlich <[email protected]>

* Update docs/Sprint-Review/sprint-100-summary.md

Co-authored-by: Lauren Frohlich <[email protected]>

---------

Co-authored-by: Lauren Frohlich <[email protected]>
Co-authored-by: Andrew <[email protected]>

* #3021 Adds FAQ update & release notes (#3040)

* Adds FAQ update & release notes

* Update product-updates/knowledge-center/index.html

Co-authored-by: Alex P.  <[email protected]>

* Update index.html

* Adds request access updates

* Update product-updates/knowledge-center/index.html

Co-authored-by: Alex P.  <[email protected]>

* Whitespace change to re-trigger full pipeline

Whitespace change to re-trigger full pipeline

---------

Co-authored-by: Alex P. <[email protected]>
Co-authored-by: Andrew <[email protected]>

* 2693 cat2 messaging cleanup (#2963)

* clean option string util

* add friendly name and item num to cat 2

* fix linting

* fix a couple tests

* add parens to friendly names

* lint

* fix tests

* fix more tests

* fix merge duplication

* fix tests

* lint

* fix tets

* udpate required validator

* Merge branch 'develop' into 2693-cat2-messaging-cleanup

* fix tests

* fix test

* fix tests

* fix lint

---------

Co-authored-by: Alex P <[email protected]>
Co-authored-by: Eric Lipe <[email protected]>

* 2687 Added email text for sendgrid (#3050)

* 2687 Added email text for sendgrid

* 2687 linting

---------

Co-authored-by: Alex P <[email protected]>
Co-authored-by: Andrew <[email protected]>

* - Added missing territories

* adding placeholder for action to table

* Frontpage link

* Update TDP-environments-README.md

---------

Co-authored-by: Eric Lipe <[email protected]>
Co-authored-by: Jan Timpe <[email protected]>
Co-authored-by: jtimpe <[email protected]>
Co-authored-by: Eric Lipe <[email protected]>
Co-authored-by: Alex P <[email protected]>
Co-authored-by: robgendron <[email protected]>
Co-authored-by: Lauren Frohlich <[email protected]>
Co-authored-by: Miles Reiter <[email protected]>
Co-authored-by: raftmsohani <[email protected]>

* swapping lines, readying for deploy

* YAML spacing.

* bypassing PR checks for faster turnaround

* Pointing to blank revision, fixed to generic/latest

* Dynamic filenames for testing differing environments.

* moving badges out of table for sizing, hopefully

* Newlines

* appending git info to frontend base page

* Feat/1620 merge pg upgrade (#3072)

* - Upgraded local to postrgres 15.7

* - updating version on terraform

* - disabling tests for now

* - adding timeouts to TF for safety

* - Deleting DB from TF state

* - re-add DB

* - remove deployment until DB is ready

* - updated apt.yml to have pg15 client
- updated deploy to only run apt commands

* - re-enable deply dev

* - UPdate client version
- re-enable migrations

* - really disable tests

---------

Co-authored-by: Eric Lipe <[email protected]>
Co-authored-by: andrew-jameson <[email protected]>

* removing comment

* undoing mass line comments

* Update .circleci/build-and-test/workflows.yml

* Apply suggestions from code review

Co-authored-by: Alex P.  <[email protected]>

---------

Co-authored-by: andrew-jameson <[email protected]>
Co-authored-by: Eric Lipe <[email protected]>
Co-authored-by: Jan Timpe <[email protected]>
Co-authored-by: jtimpe <[email protected]>
Co-authored-by: Eric Lipe <[email protected]>
Co-authored-by: Alex P <[email protected]>
Co-authored-by: robgendron <[email protected]>
Co-authored-by: Lauren Frohlich <[email protected]>
Co-authored-by: Miles Reiter <[email protected]>
Co-authored-by: raftmsohani <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@andrew-jameson andrew-jameson

Labels
Deploy with CircleCI-raft Deploy to https://tdp-frontend-raft.app.cloud.gov through CircleCI
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@andrew-jameson @elipe17