Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PROD] Coverage, data consistency, devops, screen shots #2500

Merged
merged 268 commits into from
Nov 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
268 commits
Select commit Hold shift + click to select a range
81f7b68
fix for content being printed to terminal
GarrettEHill Aug 6, 2024
7241b52
extra char on extension
GarrettEHill Aug 6, 2024
ab07803
Update db_restore.sh
GarrettEHill Aug 6, 2024
b5d7df7
Update cf_lambda.sh
GarrettEHill Aug 7, 2024
19216d3
Update db_restore.sh
GarrettEHill Aug 7, 2024
57f5bb5
Update cf_lambda.sh
GarrettEHill Aug 7, 2024
5ca09ec
add configurable timeout and extend the time for the restore
GarrettEHill Aug 7, 2024
4cbd741
Update db_restore.sh
GarrettEHill Aug 7, 2024
b64f0a8
Update config.yml
GarrettEHill Aug 7, 2024
8ff92b8
Update cf_lambda.sh
GarrettEHill Aug 7, 2024
02fe773
Update cf_lambda.sh
GarrettEHill Aug 7, 2024
c559431
Update db_restore.sh
GarrettEHill Aug 7, 2024
0d78ca3
Update cf_lambda.sh
GarrettEHill Aug 7, 2024
caf8a38
Update manifest-restore.yml
GarrettEHill Aug 7, 2024
2ac2164
reduce resources as more are not needed
GarrettEHill Aug 7, 2024
10cc8f3
Update config.yml
GarrettEHill Aug 7, 2024
7222079
Update config.yml
GarrettEHill Aug 7, 2024
44da07b
debug
GarrettEHill Aug 9, 2024
0fe788e
more debugging
GarrettEHill Aug 9, 2024
4798895
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Aug 9, 2024
434add5
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Aug 23, 2024
c09a7e6
refactor
GarrettEHill Aug 23, 2024
269f4fe
Update yarn-audit-known-issues
GarrettEHill Aug 23, 2024
a48ed6d
Update yarn-audit-known-issues
GarrettEHill Aug 23, 2024
15caf4f
Update yarn-audit-known-issues
GarrettEHill Aug 23, 2024
7c4f329
fix formating
GarrettEHill Aug 23, 2024
f7c8a1c
try to get the manifest file to work
GarrettEHill Aug 23, 2024
027629c
Update cf_lambda.sh
GarrettEHill Aug 23, 2024
9eec2f8
try a different method for BOUND_SERVICES
GarrettEHill Aug 23, 2024
48c15fe
switch to var_files model
GarrettEHill Aug 26, 2024
ca352a7
Update cf_lambda.sh
GarrettEHill Aug 26, 2024
4d09a3a
missing values
GarrettEHill Aug 26, 2024
8a82bb3
Update config.yml
GarrettEHill Aug 26, 2024
457ab01
old package no longer available
GarrettEHill Aug 26, 2024
ad8aa57
Update config.yml
GarrettEHill Aug 26, 2024
da46894
make a method to run commands within the lifecycle shell as a task
GarrettEHill Aug 26, 2024
7f27f6a
another test
GarrettEHill Aug 27, 2024
2c8f781
Update config.yml
GarrettEHill Aug 27, 2024
e19a38a
Update run.sh
GarrettEHill Aug 27, 2024
793373d
Update run.sh
GarrettEHill Aug 27, 2024
ec8df6f
Update config.yml
GarrettEHill Aug 27, 2024
d86366f
another test
GarrettEHill Aug 27, 2024
fecc958
Update run.sh
GarrettEHill Aug 27, 2024
af989ec
Update run.sh
GarrettEHill Aug 27, 2024
8f8dbba
Update process.yml
GarrettEHill Aug 27, 2024
615fac7
Update process.yml
GarrettEHill Aug 27, 2024
3f778cd
change needed to not require redis if no redis service is in env
GarrettEHill Aug 27, 2024
37f3c24
change to not require s3 if there is no s3 in the env
GarrettEHill Aug 27, 2024
2e98cb2
remove unneeded services
GarrettEHill Aug 27, 2024
fe112c6
process reports in batches to reduce the memory load
GarrettEHill Aug 27, 2024
9a34d05
better check for s3 existing
GarrettEHill Aug 27, 2024
c15f0fe
second attempt to remove the requirement for redis when using the cod…
GarrettEHill Aug 27, 2024
972dfb9
Update run.sh
GarrettEHill Aug 27, 2024
5026424
give it more memory and make the node limit dynamic
GarrettEHill Aug 27, 2024
4f8d0ab
fix memory format check, add force garbage collection
GarrettEHill Aug 27, 2024
c63becd
Update run.sh
GarrettEHill Aug 28, 2024
6f2f4a7
Update run.sh
GarrettEHill Aug 28, 2024
10803d7
Update run.sh
GarrettEHill Aug 28, 2024
adaf7db
Update process.yml
GarrettEHill Aug 28, 2024
2672172
env var is not correct
GarrettEHill Aug 28, 2024
b671c02
bc not available in buildpack
GarrettEHill Aug 28, 2024
6ad94c0
Update process.yml
GarrettEHill Aug 28, 2024
45d7cc9
more ram
GarrettEHill Aug 28, 2024
912d05b
try different manifest structure
GarrettEHill Aug 28, 2024
ef51f18
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Aug 28, 2024
df93fd3
Update yarn-audit-known-issues
GarrettEHill Aug 28, 2024
06c47b5
Update s3.test.js
GarrettEHill Aug 28, 2024
3cd6fd6
Update s3.test.js
GarrettEHill Aug 28, 2024
538109f
Update s3.test.js
GarrettEHill Aug 28, 2024
9dd4219
updates to support running without s3
GarrettEHill Aug 28, 2024
0547f12
test changes to support running without s3
GarrettEHill Aug 28, 2024
c44f86c
try to get the right configuration to get enough memory
GarrettEHill Aug 28, 2024
932051f
memory needs to be passed into run-task for it to not use the default
GarrettEHill Aug 28, 2024
676b902
yq not available
GarrettEHill Aug 28, 2024
a44ccff
Update cf_lambda.sh
GarrettEHill Aug 29, 2024
3cf44fa
Update cf_lambda.sh
GarrettEHill Aug 29, 2024
3cc3921
try to clean up the app more for each use
GarrettEHill Aug 29, 2024
f6ae629
move where the memory is parsed out
GarrettEHill Aug 29, 2024
c2573ef
refresh log monitoring if it gets disconnected
GarrettEHill Aug 29, 2024
6089249
correct the path
GarrettEHill Aug 29, 2024
ed3806b
revert that last change
GarrettEHill Aug 29, 2024
4b336df
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Aug 29, 2024
2ed3b65
adjust ram
GarrettEHill Aug 29, 2024
6183180
increase memory
GarrettEHill Aug 29, 2024
b5bdaa3
Refactor Process data script to perform the operation more on the pos…
GarrettEHill Aug 30, 2024
e8ecdc6
corrections for refactor
GarrettEHill Aug 30, 2024
972fc60
add comments
GarrettEHill Aug 30, 2024
d8952b2
name change required
GarrettEHill Aug 30, 2024
699dbda
Update latest_backup.sh
GarrettEHill Aug 30, 2024
cc20c78
Update latest_backup.sh
GarrettEHill Aug 30, 2024
1cccdc1
set up daily job
GarrettEHill Aug 30, 2024
dac4772
lint
GarrettEHill Aug 31, 2024
ef20090
lint
GarrettEHill Aug 31, 2024
22b08a4
clean up
GarrettEHill Sep 3, 2024
64dc065
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Sep 3, 2024
751c026
Update processData.test.js
GarrettEHill Sep 3, 2024
1044171
Update s3.test.js
GarrettEHill Sep 3, 2024
2fbb9f9
Update processData.js
GarrettEHill Sep 3, 2024
a91c0b3
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Oct 2, 2024
a28e242
refactor tests
GarrettEHill Oct 3, 2024
54d5e05
add obfuscation to training report data
GarrettEHill Oct 8, 2024
d2782e3
process all og the granteenames on activity reports
GarrettEHill Oct 8, 2024
80e0449
lint
GarrettEHill Oct 8, 2024
42364c9
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Oct 8, 2024
8213711
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Oct 9, 2024
419c127
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Oct 10, 2024
969eb01
refactor to have less branches
GarrettEHill Oct 11, 2024
e333c39
Fix one test, cleanup
thewatermethod Oct 11, 2024
012ce71
Merge remote-tracking branch 'origin/TTAHUB-3061/process-data' into T…
thewatermethod Oct 11, 2024
4e2ceb7
Update s3.test.js
GarrettEHill Oct 11, 2024
3389c6c
Update s3.test.js
GarrettEHill Oct 11, 2024
e4277f0
Update processData.test.js
GarrettEHill Oct 11, 2024
e963e04
only allow mock for tests
GarrettEHill Oct 11, 2024
12cfde6
Update s3.test.js
GarrettEHill Oct 11, 2024
c9b7385
Update s3.test.js
GarrettEHill Oct 12, 2024
8269b49
Update s3.test.js
GarrettEHill Oct 12, 2024
30b1109
try this
GarrettEHill Oct 14, 2024
32ec01e
Update s3.js
GarrettEHill Oct 14, 2024
f5598e3
Update s3.test.js
GarrettEHill Oct 14, 2024
586cdca
Update s3.test.js
GarrettEHill Oct 14, 2024
ca911b0
Update s3.test.js
GarrettEHill Oct 14, 2024
f61ce34
Update s3.test.js
GarrettEHill Oct 14, 2024
44c724a
Update s3.test.js
GarrettEHill Oct 14, 2024
af0ba2d
Update s3.js
GarrettEHill Oct 15, 2024
73ec01b
Update cf_lambda.sh
GarrettEHill Oct 15, 2024
e4f604f
e2e
GarrettEHill Oct 15, 2024
0190c3d
Update config.yml
GarrettEHill Oct 15, 2024
09979ba
Update process.yml
GarrettEHill Oct 15, 2024
695de26
Update dynamic-manifest.yml
GarrettEHill Oct 16, 2024
cebfade
Update cf_lambda.sh
GarrettEHill Oct 16, 2024
9f113d3
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Oct 28, 2024
1aecd6e
remove bad match
GarrettEHill Oct 29, 2024
de0565e
refactor
GarrettEHill Oct 29, 2024
dfe66e9
fix typo
GarrettEHill Oct 29, 2024
051ec80
Update cf_lambda.sh
GarrettEHill Oct 29, 2024
89613a2
longer timeout
GarrettEHill Oct 29, 2024
c8f87a2
v2.1.7
thewatermethod Nov 10, 2024
4e8a234
Refactor line graph
thewatermethod Nov 10, 2024
3c6853f
Improve code coverage for currentUser
kryswisnaskas Nov 11, 2024
3c940ec
Improve code coverage for currentUser
kryswisnaskas Nov 11, 2024
60bd596
Add more tests
kryswisnaskas Nov 11, 2024
b3ccf93
Fix FE tests
thewatermethod Nov 12, 2024
6e57b06
Fix API validation test
thewatermethod Nov 12, 2024
7d7a75b
change Programs date cols to datetype
Nov 12, 2024
2412efa
pacify linter
Nov 12, 2024
6789b38
Fix TTA hours test
thewatermethod Nov 12, 2024
9fa2187
Fix RegionalGoalDashboard test
thewatermethod Nov 12, 2024
4d4e610
update model
Nov 12, 2024
1a038de
update tests
Nov 12, 2024
0b57623
update tests
Nov 12, 2024
b6c0ad5
update tests
Nov 12, 2024
2f9ea06
typo
Nov 12, 2024
81be5df
typos
Nov 12, 2024
eaa4bce
update tests
Nov 12, 2024
4c35b1a
Use consistent and correct screenshot names
thewatermethod Nov 12, 2024
6fcd2e4
Remove weird character
thewatermethod Nov 12, 2024
0196db7
add support for docker:clear and docker:native
GarrettEHill Nov 13, 2024
697f678
make yarn docker:stop work for all cases
GarrettEHill Nov 13, 2024
e252eee
update model
Nov 13, 2024
4d2812b
update model
Nov 13, 2024
b553854
update to support Adam
GarrettEHill Nov 13, 2024
4c44268
Refactor for DRYness
thewatermethod Nov 13, 2024
d1144dc
Merge remote-tracking branch 'origin/main' into mb/TTAHUB-3506/fix-sc…
thewatermethod Nov 13, 2024
c9a546f
fix race condition
nvms Nov 13, 2024
5708b1b
migration to remove duplicates
nvms Nov 13, 2024
86cedc9
Add more tests to cover conditional statements.
kryswisnaskas Nov 13, 2024
952a3a6
more clean up
GarrettEHill Nov 13, 2024
1dfe240
Merge branch 'main' into TTAHUB-3516/native-and-clear
GarrettEHill Nov 13, 2024
afe6325
Fix tests
kryswisnaskas Nov 14, 2024
ed88228
Merge branch 'main' into ttahub-3460/make-program-dates-datetypes
Nov 14, 2024
b9a87e8
initial checkin
GarrettEHill Nov 14, 2024
e49e717
typo
Nov 14, 2024
fe2e1d9
freshen migration prefix
Nov 14, 2024
36c7251
Merge branch 'main' into kw-ttahub-3539
kryswisnaskas Nov 14, 2024
fdf9f04
use docker for docker users
GarrettEHill Nov 14, 2024
50beeef
typo
GarrettEHill Nov 14, 2024
858017a
macOS does not have -P and a grep arg, but both Linux and macOS have -E
GarrettEHill Nov 14, 2024
e95f4b5
Update src/migrations/20241113202427-remove-grant-replacement-type-du…
nvms Nov 14, 2024
254140d
lint
GarrettEHill Nov 14, 2024
863c5b3
move simple-git from dev to normal dependencies
GarrettEHill Nov 14, 2024
04ef082
mv migration
nvms Nov 14, 2024
fb2240c
lint
GarrettEHill Nov 14, 2024
3581a25
Update BuildInfo.js
GarrettEHill Nov 14, 2024
34cb65a
Fix angle bracket alignment
thewatermethod Nov 14, 2024
8b697ff
lint
GarrettEHill Nov 14, 2024
a9bd509
update puml
Nov 14, 2024
b3856fa
roll back version
GarrettEHill Nov 14, 2024
0de4ba8
test fixes
GarrettEHill Nov 15, 2024
f50e8d5
more cleanup
GarrettEHill Nov 15, 2024
d05dc36
front end tests
GarrettEHill Nov 15, 2024
588e48b
more fixes
GarrettEHill Nov 15, 2024
ff37e09
lint
GarrettEHill Nov 15, 2024
07e9000
Merge branch 'main' into TTAHUB-3613/admin-build-info
GarrettEHill Nov 15, 2024
d828380
fix tests
GarrettEHill Nov 15, 2024
43c1e8b
fix unrelated bug
GarrettEHill Nov 15, 2024
4c8349e
Update BuildInfo.js
GarrettEHill Nov 15, 2024
bc71592
Bump cross-spawn from 7.0.3 to 7.0.6 in /frontend
dependabot[bot] Nov 18, 2024
ec5b5d0
Merge pull request #2492 from HHS/dependabot/npm_and_yarn/frontend/cr…
GarrettEHill Nov 18, 2024
0da4190
remove unused default value branches
nvms Nov 20, 2024
c435782
coverage
nvms Nov 20, 2024
0d13cab
status coverage
nvms Nov 20, 2024
bc2ede8
ttaType coverage
nvms Nov 20, 2024
2ff9ff7
add src/scopes/index.test.js and coverage for mergeIncludes
nvms Nov 20, 2024
1b1c112
fix test
nvms Nov 20, 2024
acbd8b8
Merge pull request #2493 from HHS/jp/scope-default-value-branches
nvms Nov 20, 2024
f791605
Merge pull request #2496 from HHS/jp/3547/coverage
nvms Nov 20, 2024
49cd775
Merge pull request #2495 from HHS/jp/3549/status-ttatype-coverage
nvms Nov 20, 2024
75b30e0
Merge pull request #2494 from HHS/jp/3548/commlogdate-coverage
nvms Nov 20, 2024
475a3d0
Merge pull request #2477 from HHS/jp/3614/grant-replacement-race-cond…
nvms Nov 20, 2024
643aee2
Merge branch 'main' into TTAHUB-3613/admin-build-info
GarrettEHill Nov 20, 2024
3d305f1
use materialized view for AR goal retrieval
nvms Nov 20, 2024
5bec940
update test
nvms Nov 20, 2024
58601ea
Merge pull request #2485 from HHS/kw-ttahub-3539
kryswisnaskas Nov 20, 2024
ed8e52d
need to figure out why this broke
nvms Nov 20, 2024
6831ec2
this works? maybe?
nvms Nov 20, 2024
367b7a5
Merge pull request #2471 from HHS/TTAHUB-3516/native-and-clear
GarrettEHill Nov 20, 2024
1cf39af
Merge pull request #2483 from HHS/TTAHUB-3508/pre-commit-lint
GarrettEHill Nov 20, 2024
f6093b8
restore '$grant.id$' portion of clause
nvms Nov 22, 2024
f2d483d
revert this e2e change
nvms Nov 22, 2024
f51ab18
update test
nvms Nov 22, 2024
3b4c84e
Update postgrescli_install.sh
GarrettEHill Nov 25, 2024
3fd28db
Merge pull request #2499 from HHS/TTAHUB-3661/update-postgres-client
GarrettEHill Nov 25, 2024
2c0ae16
update tests
GarrettEHill Nov 25, 2024
60a166a
Update buildInfo.test.js
GarrettEHill Nov 25, 2024
1eb9aa1
Update buildInfo.test.js
GarrettEHill Nov 25, 2024
a3998ef
Update buildInfo.test.js
GarrettEHill Nov 25, 2024
20ceaa4
Merge branch 'main' into TTAHUB-3613/admin-build-info
GarrettEHill Nov 25, 2024
9dd8abd
Update buildInfo.test.js
GarrettEHill Nov 25, 2024
da89fff
Merge pull request #2466 from HHS/mb/TTAHUB-3609/line-graph-refactors
thewatermethod Nov 26, 2024
637e4eb
Merge pull request #2472 from HHS/mb/TTAHUB-3506/fix-screenshot-names…
thewatermethod Nov 26, 2024
4965c69
Merge branch 'main' into TTAHUB-3061/process-data
GarrettEHill Nov 26, 2024
8986d3b
Merge pull request #2481 from HHS/TTAHUB-3613/admin-build-info
GarrettEHill Nov 26, 2024
c47f133
Merge pull request #2497 from HHS/jp/3614/grant-materialized-view-ar
nvms Nov 26, 2024
3126934
Merge pull request #2468 from HHS/ttahub-3460/make-program-dates-date…
hardwarehuman Nov 26, 2024
c17edd9
Update README.md
GarrettEHill Nov 26, 2024
bac8e5e
Merge pull request #2299 from HHS/TTAHUB-3061/process-data
GarrettEHill Nov 26, 2024
f936840
Update config.yml
GarrettEHill Nov 27, 2024
cf44d7a
Update config.yml
GarrettEHill Nov 27, 2024
3f8b3e6
Update config.yml
GarrettEHill Nov 27, 2024
82418b0
Update config.yml
GarrettEHill Nov 27, 2024
07be610
Update config.yml
GarrettEHill Nov 27, 2024
807715d
Update config.yml
GarrettEHill Nov 27, 2024
7ac5ec8
Update config.yml
GarrettEHill Nov 27, 2024
6e41fe1
Update config.yml
GarrettEHill Nov 27, 2024
8b51dd7
Update config.yml
GarrettEHill Nov 27, 2024
a962390
Update config.yml
GarrettEHill Nov 27, 2024
9ca3a55
Update manifest.yml
GarrettEHill Nov 27, 2024
bbe661a
disable trainingReportTaskDueNotifications by requiring env var
Nov 27, 2024
f52ae0c
Merge pull request #2501 from HHS/TTAHUB-3613/admin-build-info2
GarrettEHill Nov 27, 2024
8f5b3cb
Merge pull request #2502 from HHS/ttahub-3600/allow_disabling_email_jobs
hardwarehuman Nov 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
288 changes: 248 additions & 40 deletions .circleci/config.yml

Large diffs are not rendered by default.

43 changes: 33 additions & 10 deletions .githooks/pre-commit
Original file line number Diff line number Diff line change
@@ -1,6 +1,21 @@
#!/usr/bin/env bash
set -e

# Set USE_DOCKER based on whether both backend and frontend containers are running
USE_DOCKER=false
if [ "$(docker ps | grep -E "head-start-ttadp-backend|head-start-ttadp-frontend" | wc -l)" -eq 2 ]; then
USE_DOCKER=true
fi

# Ensure node_modules is populated for frontend or backend (only for non-Docker users)
check_and_run_yarn() {
local dir=$1
if [ ! -d "$dir/node_modules" ] || [ -z "$(ls -A "$dir/node_modules")" ]; then
echo "Installing dependencies in $dir"
(cd "$dir" && yarn)
fi
}

common_changed=false
common_package_json_changed=false
frontend_yarn_lock_changed=false
Expand All @@ -9,7 +24,6 @@ files=$(git diff --cached --name-only)

for f in $files
do

# check if packages/common/src/index.js was changed
if [ -e "$f" ] && [[ $f == packages/common/src/index.js ]]; then
common_changed=true
Expand Down Expand Up @@ -37,19 +51,28 @@ do
git add "$f"
fi

# Autolint changed .js files
if [ -e "$f" ] && [[ $f == *.js ]]; then
yarn lint:fix:single "$f"
git add "$f"
fi

# Autolint changed .ts files
if [ -e "$f" ] && [[ $f == *.ts ]]; then
yarn lint:fix:single "$f"
# Autolint changed .js and .ts files
if [ -e "$f" ] && ([[ $f == *.js ]] || [[ $f == *.ts ]]); then
if [ "$USE_DOCKER" = true ]; then
if [[ $f == frontend/* ]]; then
yarn docker:yarn:fe lint:fix:single "$f"
else
yarn docker:yarn:be lint:fix:single "$f"
fi
else
if [[ $f == frontend/* ]]; then
check_and_run_yarn frontend
yarn lint:fix:single "$f"
else
check_and_run_yarn "."
yarn lint:fix:single "$f"
fi
fi
git add "$f"
fi
done

# Versioning and lock file checks
if [ $common_changed = true ]; then
if [ $common_package_json_changed = false ]; then
echo "ERROR: common/src/index.js was changed, but common/package.json was not updated. Please make sure to publish a new version."
Expand Down
Loading