-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat!: Add JS-based configuration capabilities to Learner Record #275
Conversation
* webpack.dev.config.js will be able to use the port number provided in the env.config.js * webpack.prod.config.js will be able to build Learner Record with a JS config passed in during build time * Jest is now set up to retrieve environment variables from a JS config * some tests needed to be fixed
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #275 +/- ##
=======================================
Coverage 70.22% 70.22%
=======================================
Files 27 27
Lines 403 403
Branches 85 85
=======================================
Hits 283 283
Misses 119 119
Partials 1 1 ☔ View full report in Codecov by Sentry. |
@@ -22,7 +22,7 @@ describe('program-certificates-list', () => { | |||
it('renders the Program Certificates List', () => { | |||
render(<ProgramCertificatesList />); | |||
|
|||
expect(screen.getByText('Verifiable Credentials')).toBeTruthy(); | |||
expect(screen.queryAllByText('Verifiable Credentials')).toBeTruthy(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
these tests were failing locally because there were 2 instances in the DOM where 'Verifiable Credentials' existed, same with 'My Learner Records' in the following test file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what's different about this branch that made those tests pass before and not here? Or had they always been failing locally?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To add onto the above question, how is the duplicate text rendered? Might make sense to verify with a screen.debug()
to ensure it's not showing as duplicate headings/text to users.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TLDR;
A child component of these tested components was checking ENABLE_VERIFIABLE_CREDENTIALS in the ConfigDocument, which returned true
due to the env.config.js
existing in my branch. This led to the screen
seeing My Learner Records
and Verifiable Credentials
twice, as noted in a screenshot below.
Even though the variables are an exact copy of .env.test
, because process.env
doesn't get initialized in these tests, the child component's check for ENABLE_VERIFIABLE_CREDENTIALS would be undefined
.
Solution:
Mocking the child component (NavigationBar
) in the tests, as I think it originally should have been.
Longer Read:
I found it! Alright, so in the .env.development
and .env.test
files, the variable ENABLE_VERIFIABLE_CREDENTIALS is set to 'true'.
When set to true, it will show something like the screenshot below:
But because the components are tested separately in Jest and the components -- and their children -- don't reference process.env.ENABLE_VERIFIABLE_CREDENTIALS
, they can safely ignore the .env.development
and .env.test
boolean for ENABLE_VERIFIABLE_CREDENTIALS and will instead expect something like the following screenshot:
However, I did see that there's a child component within these tests that calls getConfig().ENABLE_VERIFIABLE_CREDENTIALS
. This will return undefined
because, again, the .env.*
variables weren't initialized into the ConfigDocument as the component being tested doesn't go through the initialize
process that occurs in the parent App.
Now, the reason why these tests had to be changed was because I have been using a env.config.js
file in my repo, which is copy/paste of the values in env.development
. The problem with that is the child component in these tests that calls getConfig().ENABLE_VERIFIABLE_CREDENTIALS
will now pick up on the fact that ENABLE_VERIFIABLE_CREDENTIAL is true
because the env.config.js
has been merged into the config via the setupTest.js
file that I modified.
import messages from './i18n'; | ||
|
||
jest.mock('@edx/frontend-platform/react/hooks', () => ({ | ||
...jest.requireActual('@edx/frontend-platform/react/hooks'), | ||
useTrackColorSchemeChoice: jest.fn(), | ||
})); | ||
|
||
mergeConfig(envConfig); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Jest doesn't rely on webpack to merge the JS-based config into ConfigDocument, so we have to implement it here. I might create a separate file that does this merge in the Jest directory if it makes more sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is reasonable to have in setupTest.js
. I guess one question/suggestion might be whether it could be in the base setupTest.js
upstream in @edx/frontend-build
so that it'll be applicable to all consuming MFEs. I imagine other consuming MFEs of env.config
would otherwise need to follow this same approach? Is there an opportunity to make it part of your patch and upstream contribution?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From my understanding, frontend-platform
is able to consume frontend-build
, but frontend-build
wouldn't be able to use frontend-platform
's mergeConfig
out of the box because Build isn't set up to import
modules. Not to say it isn't a good idea, but I believe we'd need to convert everything in Build to use import
instead of require
so we can bring frontend-platform
's getConfig
in, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, good call out. I agree we can't do much here with setupTest
upstream after all. Perhaps the action item then is possibly about documentation in your upstream @edx/frontend-build
contribution and/or in @edx/frontend-platform
to ensure consumers know to include this in their setupTest.js
files if the intent is to have those settings come through in tests.
@@ -22,7 +22,7 @@ describe('program-certificates-list', () => { | |||
it('renders the Program Certificates List', () => { | |||
render(<ProgramCertificatesList />); | |||
|
|||
expect(screen.getByText('Verifiable Credentials')).toBeTruthy(); | |||
expect(screen.queryAllByText('Verifiable Credentials')).toBeTruthy(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what's different about this branch that made those tests pass before and not here? Or had they always been failing locally?
jest/fallback.env.config.js
Outdated
@@ -0,0 +1,12 @@ | |||
module.exports = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[clarification] Similar to the other comment, I have a suspicion the jest/fallback.env.config.js
used in the base jest.config.js
provided by @(open)edx/frontend-build
might be sufficient without having to duplicate into consuming projects?
// This file is used as a fallback to prevent build errors if an env.config.js file has not been
// defined in a consuming application.
I read this as though @(open)edx/frontend-build
handles the fallback behind-the-scenes for consuming repositories such that the changes could be released without a breaking change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As noted in your comment in the jest.config.js
file, I'll very likely remove this for the PR because I agree that frontend-build
should already provide this fallback.
@@ -22,7 +22,7 @@ describe('program-certificates-list', () => { | |||
it('renders the Program Certificates List', () => { | |||
render(<ProgramCertificatesList />); | |||
|
|||
expect(screen.getByText('Verifiable Credentials')).toBeTruthy(); | |||
expect(screen.queryAllByText('Verifiable Credentials')).toBeTruthy(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To add onto the above question, how is the duplicate text rendered? Might make sense to verify with a screen.debug()
to ensure it's not showing as duplicate headings/text to users.
webpack.prod.config.js
Outdated
config.resolve.modules = [ | ||
path.resolve(__dirname, './src'), | ||
'node_modules', | ||
]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[curious/clarification] Do you mind expanding a bit on the intent for this config change?
webpack.prod.config.js
Outdated
if (process.env.JS_CONFIG_FILEPATH) { | ||
fs.copyFile(process.env.JS_CONFIG_FILEPATH, 'env.config.js', (err) => { | ||
if (err) { throw err; } | ||
}); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[curious, suggestion] I'm wondering if this condition could make sense upstream as part of the default base webpack.prod.config.js
file returned by createConfig('webpack-prod')
. For example, is the JS_CONFIG_FILEPATH
environment variable approach something we could bake into the usage of env.config.js
more formally for others in the Open edX community looking to migrate to env.config.js
?
That said, perhaps that might be worth bringing as a discussion to Frontend Working Group. Nothing to act on in this PR directly :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great suggestion! I think if we end up going with this implementation, then we should definitely consider moving it upstream. For the sake of stating the obvious and making sure I know the why, similarly with the webpack.config.js
issue I already made in frontend-build, this would mean that teams wouldn't have to create a custom webpack config with this logic or modify their already-existing configs per repo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright, I removed the custom configurations in webpack prod/dev and tested them through the various permutations of keeping/removing .env.*
and env.config.js
, as well as with the local production build using Tubular. All worked as before, and the only things kept in those configs now are related to handling env.config.js
Thanks for bringing these up!
webpack.dev.config.js
Outdated
/** Uncomment the lines below if you wish to use an env.config.js in development */ | ||
// const envConfig = require('./env.config'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[curious, suggestion] If you uncomment this without an env.config.js
file defined in the repository, does it fail? Assuming not due to frontend-build's moduleNameMapper
configuration. I believe if it's not defined, envConfig
would fallback as an empty object {}
.
If that is true, in the commented out line below, e.g., could you have it such that its not necessary to have commented out code here? For example, if you do want to use env.config.js
as is and I uncomment this line, I wouldn't feel I can commit it upstream since not everyone might have an env.config.js
, which adds some burden to local development to ensure I don't commit these changes.
Would config.devServer.port = envConfig?.PORT || process.env.PORT || 8080;
work while keeping the import uncommented?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It'll fail on this line because it'll first try to find this file before it gets to line 6 (createConfig('webpack-dev')
. Unless I add some condition before to check that the file exists, then I imagine it would need to remain commented like this (but I totally agree on the local dev burden to not commit these changes) or we escalate the frontend-build ticket to handle this PORT assignment for us. Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, yes I see what you mean...
I do imagine we could get a ✅ fairly quickly if we opened a PR in frontend-build upstream and request a review in FWG, as the ideal path forward, IMHO.
That said, an alternative shorter term idea might be to (temporarily) utilize patch-package
(docs) to make a patch to your installed copy of @edx/frontend-build
that is committed and applied anytime npm install
is run (including in CI/GoCD).
FWIW, we also had to utilize patch-package
in an enterprise MFE semi-recently at one point, given we were blocked on upgrading to latest version of Paragon but needed a fix in one of its components for the currently installed version we were stuck on (see this ADR describing the rationale).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on your suggestions, I moved the webpack logic for both prod and dev to a patch file, and I also started working on a branch in frontend-build
to include these upgrades.
You'll also notice in the patch file that I modified the webpack and jest configs to be able to handle a JS or JSX extension of env.config
, which I know we'll want to be able to have for upcoming work.
…rogramCertificatesList
* add JS/JSX compatability to jest and webpack configs
+const envConfigPath = fs.existsSync(appEnvConfigPathJs) ? appEnvConfigPathJs | ||
+ : fs.existsSync(appEnvConfigPathJsx) ? appEnvConfigPathJsx | ||
+ : path.resolve(__dirname, './jest/fallback.env.config.js'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[sanity check] Will this __dirname
will refer to the appropriate path when used in a .patch
? Presumably the patch is brought into the node_modules
during the postinstall
so it should be OK; just wanted to see whether that's your understanding as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had the exact same understanding! However, since I applied your ternary suggestion, this __dirname, './jest/fallback.env.config.js'
logic was removed from the patch and performs just as it did when first implemented in frontend-build :D
-if (fs.existsSync(appEnvConfigPath)) { | ||
- envConfigPath = appEnvConfigPath; | ||
-} | ||
+const envConfigPath = fs.existsSync(appEnvConfigPathJs) ? appEnvConfigPathJs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering if the breaking up the nested ternary conditionals would improve readability a bit?
let envConfigPath = path.resolve(__dirname, './jest/fallback.env.config.js');
const appEnvConfigPathJs = path.resolve(process.cwd(), './env.config.js');
const appEnvConfigPathJsx = path.resolve(process.cwd(), './env.config.jsx');
if (fs.existsSync(appEnvConfigPath)) {
envConfigPath = appEnvConfigPath;
} else if (fs.existsSync(appEnvConfigPathJsx)) {
envConfigPath = appEnvConfigPathJsx;
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Totally agree!
|
||
+const envConfigPathJs = path.resolve(process.cwd(),'./env.config.js'); | ||
+const envConfigPathJsx = path.resolve(process.cwd(), './env.config.jsx'); | ||
+const envConfig = fs.existsSync(envConfigPathJs) ? require(envConfigPathJs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similar comment/suggestion as above regarding nested ternary.
import messages from './i18n'; | ||
|
||
jest.mock('@edx/frontend-platform/react/hooks', () => ({ | ||
...jest.requireActual('@edx/frontend-platform/react/hooks'), | ||
useTrackColorSchemeChoice: jest.fn(), | ||
})); | ||
|
||
mergeConfig(envConfig); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is reasonable to have in setupTest.js
. I guess one question/suggestion might be whether it could be in the base setupTest.js
upstream in @edx/frontend-build
so that it'll be applicable to all consuming MFEs. I imagine other consuming MFEs of env.config
would otherwise need to follow this same approach? Is there an opportunity to make it part of your patch and upstream contribution?
"@edx/frontend-build": { | ||
"sharp": "$sharp" | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[inform] It looks like the upstream revert PR to get sharp
back to 0.32.6
was merged upstream in latest version of @openedx/frontend-build
. Likely still need to keep this override here, unless it's possible to migrate to the @openedx/frontend-build
scope.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw! Glad it already got addressed upstream. I think for this PR we'll want to keep the override, and depending on how urgently we want to handle the @edx
--> @openedx
migration we'd be able to remove this override and the patches. Arguably, it'd be a good reason to do a follow-up ticket to migrate Learner Record and any other MFEs we want to experiment JS prod builds with.
….jsx to .gitignore
@adamstankiewicz I am now realizing that the patch-package probably won't work for our production build because |
Hmm, so typically I think
We should still see the above output in GoCD (e.g. in the stage build) given it runs (or should be running) |
…ork in development * since frontend-build is a devdependency, post-package will not include its patches in prod
@adamstankiewicz ah, my problem was that I forgot to add the |
.gitignore
Outdated
env.config.js | ||
env.config.jsx |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: might be able to tackle both of these in 1 line?
env.config.js | |
env.config.jsx | |
env.config.js* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm learning something new every day haha. Thanks for the suggestion!
|
||
+console.log('If you are expecting to use an env.config.jsx config, make sure to run `npm run postinstall` first!') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[clarification] I'm assuming this console.log wouldn't be in the upstream contribution to @edx/frontend-build
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes to it not being added upstream although... I'm now realizing that the postinstall
script should be able to handle this when anyone runs npm install
. I'll remove it!
import messages from './i18n'; | ||
|
||
jest.mock('@edx/frontend-platform/react/hooks', () => ({ | ||
...jest.requireActual('@edx/frontend-platform/react/hooks'), | ||
useTrackColorSchemeChoice: jest.fn(), | ||
})); | ||
|
||
mergeConfig(envConfig); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, good call out. I agree we can't do much here with setupTest
upstream after all. Perhaps the action item then is possibly about documentation in your upstream @edx/frontend-build
contribution and/or in @edx/frontend-platform
to ensure consumers know to include this in their setupTest.js
files if the intent is to have those settings come through in tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
APER-2786
Description
This PR is to add JS-based configuration to Learner Record, which includes:
example.env.config.js
that includes details around how to set up a JS-based configpatch
in node_modules to allow for:env.config.js
) to declare the port number during local development, thus removing reliance onenv.development
env.test
sharp
as@edx/frontend-build
is using a version that fails innpm install
. Until we are able to switch to@openedx/frontend-build
-- which includes the older version ofsharp
that doesn't break during install, we will need this override.Steps to run locally
module.exports
object fromexample.env.config.js
into a new fileenv.config.js
at the root directorywebpack.dev.config.js
to enable fetching the port number fromenv.config.js
.env.*
files. These can exist alongside the JS-config without causing an error as long as we fetch from the ConfigDocument (done withgetConfig
) and not theprocess.env
object.npm install
npm run start
localhost:1990
Steps to run build locally
env.config.js
exists in root directory, but changeNODE_ENV
toproduction
sowebpack.prod.config.js
is used for the build.npm run build
npm run serve
localhost:1990
More on running production webpack locally
Steps to run with
tubular
andedx-internal
tubular
intoedx-repos
directory: https://www.github.com/openedx/tubularedx-internal
intoedx-repos
directory: https://www.github.com/edx/edx-internaljwesson/add-js-config-to-learner-record
branchtubular
directory:python scripts/frontend_build.py --common-config-file ../edx-internal/frontends/common/stage_config.yml --env-config-file ../edx-internal/frontends/frontend-app-learner-record/test_config.yml --version-file ../frontend-app-learner-record/dist/version.json --app-name ../frontend-app-learner-record
env.config.js
,package.json
, and the addition of adist
npm run serve
in Learner Recordlocalhost:1990
Historical Context
Frontend Build ADR - JavaScript-based environment configuration
Frontend Platform ADR - Promote JavaScript file configuration and deprecate environment variable configuration