Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Prod] Modify transaction wrapper; font change; remove ElasticSearch; historical goal status change #2266

Merged
merged 42 commits into from
Jul 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
88f0c0c
initial pass
nvms Jun 11, 2024
7ab4c77
Merge branch 'main' into jp/2795/GSC-migration
nvms Jun 11, 2024
1402353
update ldm, update model, change mig date
nvms Jun 11, 2024
0be561a
replace tabs with spaces
nvms Jun 28, 2024
0803f09
Garrett's change to prevent re-insertion of existing records.
nvms Jun 28, 2024
669649a
Merge branch 'main' into jp/2795/GSC-migration
nvms Jun 28, 2024
5015cc6
mv migration
nvms Jun 28, 2024
b5f0304
userRoles is nullable now
nvms Jul 8, 2024
7e4705b
mv migration
nvms Jul 8, 2024
1601fe5
make reason nullable
nvms Jul 8, 2024
ed2c42d
nullable columns, nullif user id, left join on Goals
nvms Jul 8, 2024
288bba8
fix datatype, update ldm
nvms Jul 8, 2024
87abe33
rm src/lib/awsElasticSearch
nvms Jul 9, 2024
65d87ea
rm constants and hooks
nvms Jul 9, 2024
2d0bda2
rm export
nvms Jul 9, 2024
7ba6007
bump @ttahub/common
nvms Jul 9, 2024
9120e87
rm src/routes/search
nvms Jul 9, 2024
1e232dc
rm worker job
nvms Jul 9, 2024
9d3e507
finish removing src/routes/search
nvms Jul 9, 2024
186f47f
remove packages
nvms Jul 9, 2024
91aec02
remove terraform stuff
nvms Jul 9, 2024
db4ffa8
remove openapi stuff
nvms Jul 9, 2024
188dee0
remove containers
nvms Jul 9, 2024
6a57ff9
remove CI executor
nvms Jul 9, 2024
c3c1c75
remove from cf manifest
nvms Jul 9, 2024
71c1979
update readme
nvms Jul 9, 2024
863d5e4
update yarn-audit-known-issues
nvms Jul 9, 2024
b14d499
deploy to sandbox
nvms Jul 9, 2024
d643209
rm more refs
nvms Jul 9, 2024
bc2a51d
font updates per Kelly
AdamAdHocTeam Jul 10, 2024
04b4f5e
fix transaction wrapper, lower pool size
nvms Jul 10, 2024
64950f6
don't commit this
nvms Jul 10, 2024
aeae0ec
stringify fix
nvms Jul 11, 2024
67d081c
Per Kelly we should hide the profile button in the header on the erro…
AdamAdHocTeam Jul 11, 2024
6dcc25b
remove css
AdamAdHocTeam Jul 11, 2024
60dd1cd
also remove this
AdamAdHocTeam Jul 11, 2024
d11235f
Merge pull request #2262 from HHS/al-adjust-paragraph-font
AdamAdHocTeam Jul 11, 2024
5466d3b
safe accessor
nvms Jul 11, 2024
b11531c
Merge pull request #2261 from HHS/jp/3144/restart-on-seq-err
nvms Jul 11, 2024
f4f5cd4
Merge branch 'main' into jp/3112/rm-elasticsearch
nvms Jul 11, 2024
8ca3bf4
Merge pull request #2253 from HHS/jp/3112/rm-elasticsearch
nvms Jul 11, 2024
355e692
Merge pull request #2201 from HHS/jp/2795/GSC-migration
nvms Jul 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 5 additions & 41 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,27 +19,6 @@ executors:
docker-python-executor:
docker:
- image: cimg/python:3.9.19
docker-postgres-elasticsearch-executor:
docker:
- image: cimg/node:18.20.3-browsers
environment:
DATABASE_URL: postgresql://postgres@localhost/ttasmarthub
- image: cimg/postgres:15.6
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: secretpass
POSTGRES_DB: ttasmarthub
- image: opensearchproject/opensearch:1.3.2
name: opensearch
environment:
transport.host: 0.0.0.0
network.host: 0.0.0.0
http.port: 9200
cluster.name: opensearch-cluster
node.name: opensearch-node1
discovery.type: single-node
DISABLE_SECURITY_PLUGIN: true
ES_JAVA_OPTS: "-Xms256m -Xmx256m"
machine-executor:
machine:
image: ubuntu-2204:current
Expand Down Expand Up @@ -427,7 +406,7 @@ parameters:
default: "al-ttahub-2939-add-fei-root-cause-to-goal-card"
type: string
sandbox_git_branch: # change to feature branch to test deployment
default: "jp/3005/ipd-courses-widget"
default: "jp/3112/rm-elasticsearch"
type: string
prod_new_relic_app_id:
default: "877570491"
Expand Down Expand Up @@ -536,11 +515,8 @@ jobs:
- similarity_api
resource_class: large
test_backend:
executor: docker-postgres-elasticsearch-executor
executor: docker-postgres-executor
environment:
AWS_ELASTICSEARCH_ENDPOINT: http://opensearch:9200
AWS_ELASTICSEARCH_ACCESS_KEY: admin
AWS_ELASTICSEARCH_SECRET_KEY: admin
SFTP_EXPOSED_PORT: 2222
steps:
- attach_workspace:
Expand Down Expand Up @@ -636,11 +612,7 @@ jobs:
path: frontend/coverage/
resource_class: large
test_e2e:
executor: docker-postgres-elasticsearch-executor
environment:
AWS_ELASTICSEARCH_ENDPOINT: http://opensearch:9200
AWS_ELASTICSEARCH_ACCESS_KEY: admin
AWS_ELASTICSEARCH_SECRET_KEY: admin
executor: docker-postgres-executor
steps:
- attach_workspace:
at: .
Expand Down Expand Up @@ -677,11 +649,7 @@ jobs:
path: playwright/e2e
resource_class: large
test_api:
executor: docker-postgres-elasticsearch-executor
environment:
AWS_ELASTICSEARCH_ENDPOINT: http://opensearch:9200
AWS_ELASTICSEARCH_ACCESS_KEY: admin
AWS_ELASTICSEARCH_SECRET_KEY: admin
executor: docker-postgres-executor
steps:
- attach_workspace:
at: .
Expand Down Expand Up @@ -718,11 +686,7 @@ jobs:
path: playwright/api
resource_class: large
test_utils:
executor: docker-postgres-elasticsearch-executor
environment:
AWS_ELASTICSEARCH_ENDPOINT: http://opensearch:9200
AWS_ELASTICSEARCH_ACCESS_KEY: admin
AWS_ELASTICSEARCH_SECRET_KEY: admin
executor: docker-postgres-executor
steps:
- attach_workspace:
at: .
Expand Down
5 changes: 0 additions & 5 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,6 @@ CLAMAV_ENDPOINT=https://clamav-rest:9443
# CLAMAV_ENDPOINT=http://localhost:8081
REDIS_PASS=SUPERSECUREPASSWORD

# Vars for AWS OpenSearch (ES) local docker image
AWS_ELASTICSEARCH_ENDPOINT=http://localhost:9200
AWS_ELASTICSEARCH_ACCESS_KEY=admin
AWS_ELASTICSEARCH_SECRET_KEY=admin

# Email Address that notifications should come from
[email protected]

Expand Down
15 changes: 2 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,6 @@ The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-r

Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the `docs/openapi` folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files.

We use an AWS OpenSearch docker image (Elasticsearch fork) and require that the following variables get added to the env file.
* `AWS_ELASTICSEARCH_ENDPOINT=http://opensearch-node1:9200`
* `AWS_ELASTICSEARCH_ACCESS_KEY=admin`
* `AWS_ELASTICSEARCH_SECRET_KEY=admin`

#### Apple Silicon & Chromium
On a Mac with Apple Silicon, puppeteer install fails with the message:
```"The chromium binary is not available for arm64"```
Expand All @@ -73,11 +68,6 @@ You can also run build commands directly on your host (without docker). Make sur

You must also install and run minio locally to use the file upload functionality. Please comment out `S3_ENDPOINT=http://minio:9000` and uncomment `S3_ENDPOINT=http://localhost:9000` in your .env file.

We use an AWS OpensSearch docker image (Elasticsearch fork) and require that the following variables get added to the env file.
* `AWS_ELASTICSEARCH_ENDPOINT=http://localhost:9200`
* `AWS_ELASTICSEARCH_ACCESS_KEY=admin`
* `AWS_ELASTICSEARCH_SECRET_KEY=admin`

#### Precommit hooks

Our CI will fail if code is committed that doesn't pass our linter (eslint). This repository contains a pre-commit hook that runs eslint's built in "fix" command on all staged javascript files so that any autofixable errors will be fixed. The precommit hook, in .gihooks/pre-commit, also contains code to auto-format our terraform files, which you can read more about [here](https://github.com/adhocteam/Head-Start-TTADP/tree/main/terraform#set-up).
Expand All @@ -97,12 +87,12 @@ If you are already using git hooks, add the .githooks/pre-commit contents to you
### Building Tests

#### Helpful notes on writing (backend) tests
It's important that our tests fully clean up after themselves if they interact with the database. This way, tests do not conflict when run on the CI and remain as deterministic as possible.The best way to do this is to run them locally in an isolated environment and confirm that they are sanitary.
It's important that our tests fully clean up after themselves if they interact with the database. This way, tests do not conflict when run on the CI and remain as deterministic as possible.The best way to do this is to run them locally in an isolated environment and confirm that they are sanitary.

With that in mind, there a few "gotchas" to remember to help write sanitary tests.
- ```Grant.destroy``` needs to run with ```individualHooks: true``` or the related GrantNumberLink model prevents delete.
- When you call ```Model.destroy``` you should be adding ```individualHooks: true``` to the Sequelize options. Often this is required for proper cleanup. There may be times when this is undesirable; this should be indicated with a comment.
- Be aware of paranoid models. For those models: force: true gets around the soft delete. If they are already soft-deleted though, you need to remove the default scopes paranoid: true does it, as well as Model.unscoped()
- Be aware of paranoid models. For those models: force: true gets around the soft delete. If they are already soft-deleted though, you need to remove the default scopes paranoid: true does it, as well as Model.unscoped()
- There are excellent helpers for creating and destroying common Model mocks in ```testUtils.js```. Be aware that they take a scorched earth approach to cleanup. For example, when debugging a flaky test, it was discovered that ```destroyReport``` was removing a commonly used region.
- The next section details additional tools, found in `src/lib/programmaticTransaction.ts`, which make maintaining a clean database state when writing tests a breeze.

Expand Down Expand Up @@ -627,4 +617,3 @@ ex:
[backend]:http://localhost:8080
[API documentation]:http://localhost:5003
[minio]:http://localhost:3000
[elasticsearch]:http://localhost:9200
15 changes: 0 additions & 15 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,20 +59,6 @@ services:
ports:
- "1025:1025"
- "1080:1080"
opensearch-node1:
image: opensearchproject/opensearch:1.3.2
env_file: .env
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- "DISABLE_SECURITY_PLUGIN=true" # disables security plugin entirely in OpenSearch by setting plugins.security.disabled: true in opensearch.yml
- "discovery.type=single-node" # disables bootstrap checks that are enabled when network.host is set to a non-loopback address
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
testingonly:
build:
context: .
Expand All @@ -97,4 +83,3 @@ services:
volumes:
dbdata: {}
minio-data: {}
opensearch-data1: {}
1 change: 0 additions & 1 deletion docs/boundary_diagram.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ Boundary(aws, "AWS GovCloud") {
ContainerDb(www_db, "PostgreSQL Database", "AWS RDS", "Contains content and configuration for the TTA Hub")
ContainerDb(www_s3, "AWS S3 bucket", "AWS S3", "Stores static file assets")
ContainerDb(www_redis, "Redis Database", "AWS Elasticache", "Queue of background jobs to work on")
Container(Elasticache, "Elasticache", "AWS Elasticache", "Elasticache for search results")
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion docs/logical_data_model.encoded

Large diffs are not rendered by default.

10 changes: 5 additions & 5 deletions docs/logical_data_model.puml
Original file line number Diff line number Diff line change
Expand Up @@ -351,16 +351,16 @@ class GoalSimilarityGroups{

class GoalStatusChanges{
* id : integer : <generated>
userId : integer : REFERENCES "Users".id
* goalId : integer : REFERENCES "Goals".id
* userId : integer : REFERENCES "Users".id
* createdAt : timestamp with time zone : now()
* newStatus : varchar(255)
* reason : text
* updatedAt : timestamp with time zone : now()
* userName : varchar(255)
* userRoles : varchar(255)[]
context : text
newStatus : varchar(255)
oldStatus : varchar(255)
reason : text
userName : varchar(255)
userRoles : varchar(255)[]
}

class GoalTemplateFieldPrompts{
Expand Down
2 changes: 0 additions & 2 deletions docs/openapi/paths/index.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,6 @@
$ref: './widgets/reasonList.yaml'
'/widgets/topicAndFrequencyGraph':
$ref: './widgets/topicAndFrequencyGraph.yaml'
'/search':
$ref: './search/search.yaml'
'/settings':
$ref: './settings/index.yaml'
'/settings/email':
Expand Down
43 changes: 0 additions & 43 deletions docs/openapi/paths/search/search.yaml

This file was deleted.

4 changes: 2 additions & 2 deletions frontend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
"@fortawesome/free-solid-svg-icons": "^6.4.2",
"@fortawesome/react-fontawesome": "^0.2.0",
"@hookform/error-message": "^0.0.5",
"@silevis/reactgrid": "3.1",
"@react-hook/resize-observer": "^1.2.6",
"@silevis/reactgrid": "3.1",
"@trussworks/react-uswds": "4.1.1",
"@ttahub/common": "^2.1.5",
"@use-it/interval": "^1.0.0",
Expand Down Expand Up @@ -213,4 +213,4 @@
}
}
}
}
}
6 changes: 5 additions & 1 deletion frontend/src/components/Header.js
Original file line number Diff line number Diff line change
@@ -1,18 +1,20 @@
import React from 'react';
import React, { useContext } from 'react';
import PropTypes from 'prop-types';
import HeaderUserMenu from './HeaderUserMenu';

import logo1x from '../images/eclkc-blocks-logo-43x56.png';
import logo2x from '../images/eclkc-blocks-logo-86x111.png';
import ReadOnlyEditor from './ReadOnlyEditor';
import SiteAlert from './SiteAlert';
import SomethingWentWrongContext from '../SomethingWentWrongContext';

function Header({
authenticated,
alert,
areThereUnreadNotifications,
setAreThereUnreadNotifications,
}) {
const { errorResponseCode, showingNotFound } = useContext(SomethingWentWrongContext);
const headerClassNames = [
'smart-hub-header',
'pin-top',
Expand Down Expand Up @@ -49,12 +51,14 @@ function Header({
<p className="smart-hub-title font-family-sans text-bold margin-y-1">Office of Head Start TTA Hub</p>
</div>
</div>
{ !errorResponseCode && !showingNotFound && (
<div className="flex-column flex-align-self-center">
<HeaderUserMenu
areThereUnreadNotifications={areThereUnreadNotifications}
setAreThereUnreadNotifications={setAreThereUnreadNotifications}
/>
</div>
)}
</div>
</header>
);
Expand Down
6 changes: 3 additions & 3 deletions frontend/src/components/SomethingWentWrong.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ function SomethingWentWrong({ passedErrorResponseCode }) {
message: '403 error - forbidden',
title: 'Restricted access.',
body: (
<p>
<p className="usa-prose">
Sorry, but it looks like you&apos;re trying to access a restricted area. Here&apos;s what you can do:
<ul>
<li>
Expand Down Expand Up @@ -76,7 +76,7 @@ function SomethingWentWrong({ passedErrorResponseCode }) {
message: '404 error',
title: 'Page not found.',
body: (
<p>
<p className="usa-prose">
Well, this is awkward. It seems like the page you&apos;re looking for has taken a detour into the unknown. Here&apos;s what you can do:
<ul>
<li>
Expand All @@ -101,7 +101,7 @@ function SomethingWentWrong({ passedErrorResponseCode }) {
message: null,
title: 'Something went wrong.',
body: (
<p>
<p className="usa-prose">
Well, this is awkward. It seems like the page you&apos;re looking for has taken a detour into the unknown. Here&apos;s what you can do:
<ul>
<li>
Expand Down
11 changes: 2 additions & 9 deletions frontend/src/components/SomethingWentWrong.scss
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,6 @@
max-width: 700px;
}

.smart-hub--something-went-wrong p,
.smart-hub--something-went-wrong ul li button {
color: $base-darkest;
font-size: 1.25rem;
font-weight: 400;
}

.smart-hub--something-went-wrong li {
margin-bottom: .5rem;
.smart-hub--something-went-wrong ul:last-child {
margin-bottom: 1rem;
}
Loading