Skip to content

Commit

Permalink
Merge pull request #2266 from HHS/main
Browse files Browse the repository at this point in the history
[Prod] Modify transaction wrapper; font change; remove ElasticSearch; historical goal status change
  • Loading branch information
Jones-QuarteyDana authored Jul 12, 2024
2 parents 0236187 + 355e692 commit 689b01f
Show file tree
Hide file tree
Showing 49 changed files with 305 additions and 1,959 deletions.
46 changes: 5 additions & 41 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,27 +19,6 @@ executors:
docker-python-executor:
docker:
- image: cimg/python:3.9.19
docker-postgres-elasticsearch-executor:
docker:
- image: cimg/node:18.20.3-browsers
environment:
DATABASE_URL: postgresql://postgres@localhost/ttasmarthub
- image: cimg/postgres:15.6
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: secretpass
POSTGRES_DB: ttasmarthub
- image: opensearchproject/opensearch:1.3.2
name: opensearch
environment:
transport.host: 0.0.0.0
network.host: 0.0.0.0
http.port: 9200
cluster.name: opensearch-cluster
node.name: opensearch-node1
discovery.type: single-node
DISABLE_SECURITY_PLUGIN: true
ES_JAVA_OPTS: "-Xms256m -Xmx256m"
machine-executor:
machine:
image: ubuntu-2204:current
Expand Down Expand Up @@ -427,7 +406,7 @@ parameters:
default: "al-ttahub-2939-add-fei-root-cause-to-goal-card"
type: string
sandbox_git_branch: # change to feature branch to test deployment
default: "jp/3005/ipd-courses-widget"
default: "jp/3112/rm-elasticsearch"
type: string
prod_new_relic_app_id:
default: "877570491"
Expand Down Expand Up @@ -536,11 +515,8 @@ jobs:
- similarity_api
resource_class: large
test_backend:
executor: docker-postgres-elasticsearch-executor
executor: docker-postgres-executor
environment:
AWS_ELASTICSEARCH_ENDPOINT: http://opensearch:9200
AWS_ELASTICSEARCH_ACCESS_KEY: admin
AWS_ELASTICSEARCH_SECRET_KEY: admin
SFTP_EXPOSED_PORT: 2222
steps:
- attach_workspace:
Expand Down Expand Up @@ -636,11 +612,7 @@ jobs:
path: frontend/coverage/
resource_class: large
test_e2e:
executor: docker-postgres-elasticsearch-executor
environment:
AWS_ELASTICSEARCH_ENDPOINT: http://opensearch:9200
AWS_ELASTICSEARCH_ACCESS_KEY: admin
AWS_ELASTICSEARCH_SECRET_KEY: admin
executor: docker-postgres-executor
steps:
- attach_workspace:
at: .
Expand Down Expand Up @@ -677,11 +649,7 @@ jobs:
path: playwright/e2e
resource_class: large
test_api:
executor: docker-postgres-elasticsearch-executor
environment:
AWS_ELASTICSEARCH_ENDPOINT: http://opensearch:9200
AWS_ELASTICSEARCH_ACCESS_KEY: admin
AWS_ELASTICSEARCH_SECRET_KEY: admin
executor: docker-postgres-executor
steps:
- attach_workspace:
at: .
Expand Down Expand Up @@ -718,11 +686,7 @@ jobs:
path: playwright/api
resource_class: large
test_utils:
executor: docker-postgres-elasticsearch-executor
environment:
AWS_ELASTICSEARCH_ENDPOINT: http://opensearch:9200
AWS_ELASTICSEARCH_ACCESS_KEY: admin
AWS_ELASTICSEARCH_SECRET_KEY: admin
executor: docker-postgres-executor
steps:
- attach_workspace:
at: .
Expand Down
5 changes: 0 additions & 5 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,6 @@ CLAMAV_ENDPOINT=https://clamav-rest:9443
# CLAMAV_ENDPOINT=http://localhost:8081
REDIS_PASS=SUPERSECUREPASSWORD

# Vars for AWS OpenSearch (ES) local docker image
AWS_ELASTICSEARCH_ENDPOINT=http://localhost:9200
AWS_ELASTICSEARCH_ACCESS_KEY=admin
AWS_ELASTICSEARCH_SECRET_KEY=admin

# Email Address that notifications should come from
FROM_EMAIL_ADDRESS=[email protected]

Expand Down
15 changes: 2 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,6 @@ The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-r

Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the `docs/openapi` folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files.

We use an AWS OpenSearch docker image (Elasticsearch fork) and require that the following variables get added to the env file.
* `AWS_ELASTICSEARCH_ENDPOINT=http://opensearch-node1:9200`
* `AWS_ELASTICSEARCH_ACCESS_KEY=admin`
* `AWS_ELASTICSEARCH_SECRET_KEY=admin`

#### Apple Silicon & Chromium
On a Mac with Apple Silicon, puppeteer install fails with the message:
```"The chromium binary is not available for arm64"```
Expand All @@ -73,11 +68,6 @@ You can also run build commands directly on your host (without docker). Make sur

You must also install and run minio locally to use the file upload functionality. Please comment out `S3_ENDPOINT=http://minio:9000` and uncomment `S3_ENDPOINT=http://localhost:9000` in your .env file.

We use an AWS OpensSearch docker image (Elasticsearch fork) and require that the following variables get added to the env file.
* `AWS_ELASTICSEARCH_ENDPOINT=http://localhost:9200`
* `AWS_ELASTICSEARCH_ACCESS_KEY=admin`
* `AWS_ELASTICSEARCH_SECRET_KEY=admin`

#### Precommit hooks

Our CI will fail if code is committed that doesn't pass our linter (eslint). This repository contains a pre-commit hook that runs eslint's built in "fix" command on all staged javascript files so that any autofixable errors will be fixed. The precommit hook, in .gihooks/pre-commit, also contains code to auto-format our terraform files, which you can read more about [here](https://github.com/adhocteam/Head-Start-TTADP/tree/main/terraform#set-up).
Expand All @@ -97,12 +87,12 @@ If you are already using git hooks, add the .githooks/pre-commit contents to you
### Building Tests

#### Helpful notes on writing (backend) tests
It's important that our tests fully clean up after themselves if they interact with the database. This way, tests do not conflict when run on the CI and remain as deterministic as possible.The best way to do this is to run them locally in an isolated environment and confirm that they are sanitary.
It's important that our tests fully clean up after themselves if they interact with the database. This way, tests do not conflict when run on the CI and remain as deterministic as possible.The best way to do this is to run them locally in an isolated environment and confirm that they are sanitary.

With that in mind, there a few "gotchas" to remember to help write sanitary tests.
- ```Grant.destroy``` needs to run with ```individualHooks: true``` or the related GrantNumberLink model prevents delete.
- When you call ```Model.destroy``` you should be adding ```individualHooks: true``` to the Sequelize options. Often this is required for proper cleanup. There may be times when this is undesirable; this should be indicated with a comment.
- Be aware of paranoid models. For those models: force: true gets around the soft delete. If they are already soft-deleted though, you need to remove the default scopes paranoid: true does it, as well as Model.unscoped()
- Be aware of paranoid models. For those models: force: true gets around the soft delete. If they are already soft-deleted though, you need to remove the default scopes paranoid: true does it, as well as Model.unscoped()
- There are excellent helpers for creating and destroying common Model mocks in ```testUtils.js```. Be aware that they take a scorched earth approach to cleanup. For example, when debugging a flaky test, it was discovered that ```destroyReport``` was removing a commonly used region.
- The next section details additional tools, found in `src/lib/programmaticTransaction.ts`, which make maintaining a clean database state when writing tests a breeze.

Expand Down Expand Up @@ -627,4 +617,3 @@ ex:
[backend]:http://localhost:8080
[API documentation]:http://localhost:5003
[minio]:http://localhost:3000
[elasticsearch]:http://localhost:9200
15 changes: 0 additions & 15 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,20 +59,6 @@ services:
ports:
- "1025:1025"
- "1080:1080"
opensearch-node1:
image: opensearchproject/opensearch:1.3.2
env_file: .env
container_name: opensearch-node1
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1
- "DISABLE_SECURITY_PLUGIN=true" # disables security plugin entirely in OpenSearch by setting plugins.security.disabled: true in opensearch.yml
- "discovery.type=single-node" # disables bootstrap checks that are enabled when network.host is set to a non-loopback address
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
testingonly:
build:
context: .
Expand All @@ -97,4 +83,3 @@ services:
volumes:
dbdata: {}
minio-data: {}
opensearch-data1: {}
1 change: 0 additions & 1 deletion docs/boundary_diagram.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ Boundary(aws, "AWS GovCloud") {
ContainerDb(www_db, "PostgreSQL Database", "AWS RDS", "Contains content and configuration for the TTA Hub")
ContainerDb(www_s3, "AWS S3 bucket", "AWS S3", "Stores static file assets")
ContainerDb(www_redis, "Redis Database", "AWS Elasticache", "Queue of background jobs to work on")
Container(Elasticache, "Elasticache", "AWS Elasticache", "Elasticache for search results")
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion docs/logical_data_model.encoded

Large diffs are not rendered by default.

10 changes: 5 additions & 5 deletions docs/logical_data_model.puml
Original file line number Diff line number Diff line change
Expand Up @@ -351,16 +351,16 @@ class GoalSimilarityGroups{

class GoalStatusChanges{
* id : integer : <generated>
userId : integer : REFERENCES "Users".id
* goalId : integer : REFERENCES "Goals".id
* userId : integer : REFERENCES "Users".id
* createdAt : timestamp with time zone : now()
* newStatus : varchar(255)
* reason : text
* updatedAt : timestamp with time zone : now()
* userName : varchar(255)
* userRoles : varchar(255)[]
context : text
newStatus : varchar(255)
oldStatus : varchar(255)
reason : text
userName : varchar(255)
userRoles : varchar(255)[]
}

class GoalTemplateFieldPrompts{
Expand Down
2 changes: 0 additions & 2 deletions docs/openapi/paths/index.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,6 @@
$ref: './widgets/reasonList.yaml'
'/widgets/topicAndFrequencyGraph':
$ref: './widgets/topicAndFrequencyGraph.yaml'
'/search':
$ref: './search/search.yaml'
'/settings':
$ref: './settings/index.yaml'
'/settings/email':
Expand Down
43 changes: 0 additions & 43 deletions docs/openapi/paths/search/search.yaml

This file was deleted.

4 changes: 2 additions & 2 deletions frontend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
"@fortawesome/free-solid-svg-icons": "^6.4.2",
"@fortawesome/react-fontawesome": "^0.2.0",
"@hookform/error-message": "^0.0.5",
"@silevis/reactgrid": "3.1",
"@react-hook/resize-observer": "^1.2.6",
"@silevis/reactgrid": "3.1",
"@trussworks/react-uswds": "4.1.1",
"@ttahub/common": "^2.1.5",
"@use-it/interval": "^1.0.0",
Expand Down Expand Up @@ -213,4 +213,4 @@
}
}
}
}
}
6 changes: 5 additions & 1 deletion frontend/src/components/Header.js
Original file line number Diff line number Diff line change
@@ -1,18 +1,20 @@
import React from 'react';
import React, { useContext } from 'react';
import PropTypes from 'prop-types';
import HeaderUserMenu from './HeaderUserMenu';

import logo1x from '../images/eclkc-blocks-logo-43x56.png';
import logo2x from '../images/eclkc-blocks-logo-86x111.png';
import ReadOnlyEditor from './ReadOnlyEditor';
import SiteAlert from './SiteAlert';
import SomethingWentWrongContext from '../SomethingWentWrongContext';

function Header({
authenticated,
alert,
areThereUnreadNotifications,
setAreThereUnreadNotifications,
}) {
const { errorResponseCode, showingNotFound } = useContext(SomethingWentWrongContext);
const headerClassNames = [
'smart-hub-header',
'pin-top',
Expand Down Expand Up @@ -49,12 +51,14 @@ function Header({
<p className="smart-hub-title font-family-sans text-bold margin-y-1">Office of Head Start TTA Hub</p>
</div>
</div>
{ !errorResponseCode && !showingNotFound && (
<div className="flex-column flex-align-self-center">
<HeaderUserMenu
areThereUnreadNotifications={areThereUnreadNotifications}
setAreThereUnreadNotifications={setAreThereUnreadNotifications}
/>
</div>
)}
</div>
</header>
);
Expand Down
6 changes: 3 additions & 3 deletions frontend/src/components/SomethingWentWrong.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ function SomethingWentWrong({ passedErrorResponseCode }) {
message: '403 error - forbidden',
title: 'Restricted access.',
body: (
<p>
<p className="usa-prose">
Sorry, but it looks like you&apos;re trying to access a restricted area. Here&apos;s what you can do:
<ul>
<li>
Expand Down Expand Up @@ -76,7 +76,7 @@ function SomethingWentWrong({ passedErrorResponseCode }) {
message: '404 error',
title: 'Page not found.',
body: (
<p>
<p className="usa-prose">
Well, this is awkward. It seems like the page you&apos;re looking for has taken a detour into the unknown. Here&apos;s what you can do:
<ul>
<li>
Expand All @@ -101,7 +101,7 @@ function SomethingWentWrong({ passedErrorResponseCode }) {
message: null,
title: 'Something went wrong.',
body: (
<p>
<p className="usa-prose">
Well, this is awkward. It seems like the page you&apos;re looking for has taken a detour into the unknown. Here&apos;s what you can do:
<ul>
<li>
Expand Down
11 changes: 2 additions & 9 deletions frontend/src/components/SomethingWentWrong.scss
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,6 @@
max-width: 700px;
}

.smart-hub--something-went-wrong p,
.smart-hub--something-went-wrong ul li button {
color: $base-darkest;
font-size: 1.25rem;
font-weight: 400;
}

.smart-hub--something-went-wrong li {
margin-bottom: .5rem;
.smart-hub--something-went-wrong ul:last-child {
margin-bottom: 1rem;
}
Loading

0 comments on commit 689b01f

Please sign in to comment.