Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shared config updates #452

Merged
merged 7 commits into from
Aug 17, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .changeset/eleven-paws-invite.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
'@jpmorganchase/mosaic-core': patch
'@jpmorganchase/mosaic-plugins': patch
'@jpmorganchase/mosaic-site': patch
'@jpmorganchase/mosaic-types': patch
---

`SharedConfigPlugin` can now apply a shared config to a source that doesn't have one but shares a namespace with 1 that does.
6 changes: 4 additions & 2 deletions packages/core/src/Source.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,8 @@ export default class Source {
pageExtensions: this.#pageExtensions,
ignorePages: this.#ignorePages,
serialiser: this.serialiser,
config: this.config.asReadOnly()
config: this.config.asReadOnly(),
namespace: this.namespace
}
);
const timeTaken = new Date().getTime() - initTime;
Expand Down Expand Up @@ -149,7 +150,8 @@ export default class Source {
pageExtensions: this.#pageExtensions,
ignorePages: this.#ignorePages,
serialiser: this.serialiser,
config: this.config.asReadOnly()
config: this.config.asReadOnly(),
namespace: this.namespace
});
const timeTaken = new Date().getTime() - initTime;
if (timeTaken > 800) {
Expand Down
6 changes: 4 additions & 2 deletions packages/core/src/worker/Source.worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@ if (isMainThread) {
config,
serialiser,
ignorePages: workerData.ignorePages,
pageExtensions: workerData.pageExtensions
pageExtensions: workerData.pageExtensions,
namespace: workerData.namespace
})
),
switchMap((pages: Page[]) =>
Expand All @@ -58,7 +59,8 @@ if (isMainThread) {
config,
ignorePages: workerData.ignorePages,
pageExtensions: workerData.pageExtensions,
serialiser
serialiser,
namespace: workerData.namespace
});
// In the main thread we would freeze the filesystem here, but since we throw it away after sending it to the parent process,
// we don't bother freezing
Expand Down
121 changes: 104 additions & 17 deletions packages/plugins/src/SharedConfigPlugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,17 +43,31 @@ export interface SharedConfigPluginOptions {
* It then exports a JSON file (name: `options.filename`) into each directory with the merged config for that level
*/
const SharedConfigPlugin: PluginType<SharedConfigPluginPage, SharedConfigPluginOptions> = {
async $afterSource(pages, { ignorePages, pageExtensions }) {
async $afterSource(pages, { ignorePages, pageExtensions, config, namespace }) {
const isNonHiddenPage = createPageTest(ignorePages, pageExtensions);
let finalSharedConfig;

const indexPagesWithSharedConfig = pages.filter(
const indexPages = pages.filter(
page =>
path.posix.basename(page.fullPath, path.posix.extname(page.fullPath)) === 'index' &&
page.sharedConfig !== undefined &&
isNonHiddenPage(page.fullPath)
);

const indexPagesWithSharedConfig = indexPages.filter(page => page.sharedConfig !== undefined);

if (indexPagesWithSharedConfig.length === 0 && indexPages.length > 0) {
const rootPath = indexPages[0].fullPath;
const applyNamespaceSharedConfig = {
[`${namespace}~~${rootPath}`]: {
paths: indexPages.map(indexPage => indexPage.fullPath),
rootPath,
namespace
}
};
config.setData({ applyNamespaceSharedConfig });
Copy link
Contributor Author

@DavieReid DavieReid Aug 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the config object is persisted across plugin lifecycle events. So this source is writing to the global config in the hope another source that does have a shared config will apply it.

This is picked up in the afterUpdate event

return pages;
}

for (const page of indexPagesWithSharedConfig) {
if (finalSharedConfig === undefined) {
// first shared config we have found so seed the finalSharedConfig
Expand Down Expand Up @@ -113,24 +127,97 @@ const SharedConfigPlugin: PluginType<SharedConfigPluginPage, SharedConfigPluginO
}

// apply closest shared config
let closestSharedConfigIndex = 0;
for (const pagePath of indexPagesWithoutConfig) {
for (let i = 0; i < sharedConfigFiles.length; i++) {
if (isWithin(sharedConfigFiles[i], pagePath)) {
closestSharedConfigIndex = i;
if (sharedConfigFiles.length > 0) {
let closestSharedConfigIndex = 0;
for (const pagePath of indexPagesWithoutConfig) {
for (let i = 0; i < sharedConfigFiles.length; i++) {
if (isWithin(sharedConfigFiles[i], pagePath)) {
closestSharedConfigIndex = i;
}
}

const sharedConfigFile = path.posix.join(
path.posix.dirname(String(pagePath)),
options.filename
);

const closestSharedConfig = path.posix.resolve(
path.dirname(String(pagePath)),
sharedConfigFiles[closestSharedConfigIndex]
);
config.setAliases(closestSharedConfig, [sharedConfigFile]);
}
}
},
async afterUpdate(mutableFilesystem, { sharedFilesystem, globalConfig, namespace }, options) {
const { applyNamespaceSharedConfig } = globalConfig.data;

const sharedConfigFile = path.posix.join(
path.posix.dirname(String(pagePath)),
options.filename
);
if (applyNamespaceSharedConfig === undefined) {
// there is no source that exists that has told us it needs to share a namespace shared-config
return;
}

const closestSharedConfig = path.posix.resolve(
path.dirname(String(pagePath)),
sharedConfigFiles[closestSharedConfigIndex]
);
config.setAliases(closestSharedConfig, [sharedConfigFile]);
// find all the entries that match the namespace the plugin is running against
const namespaceSharedConfigs: {
paths: string[];
rootPath: string;
namespace: string;
}[] = Object.keys(applyNamespaceSharedConfig)
.filter(key => {
const keyNamespace = key.split('~~')?.[0];
DavieReid marked this conversation as resolved.
Show resolved Hide resolved
return keyNamespace === namespace;
})
.map(key => applyNamespaceSharedConfig?.[key] || []);

for (const namespaceSharedConfig of namespaceSharedConfigs) {
if (await mutableFilesystem.promises.exists(namespaceSharedConfig.rootPath)) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to guard against the actual write of the shared config.json that is a copy. It might trigger afterUpdate again but not 100% sure

// a source does need a namespace shared config but the source running this plugin is the source that needs it
// so we don't need to do anything here
continue;
}

for (const applyPath of namespaceSharedConfig.paths) {
if (!(await sharedFilesystem.promises.exists(applyPath))) {
sharedFilesystem.promises.mkdir(path.posix.dirname(String(applyPath)), {
recursive: true
});
}
let parentDir = path.posix.join(path.posix.dirname(String(applyPath)), '../');
let closestSharedConfigPath = path.posix.join(parentDir, options.filename);

while (parentDir !== path.posix.sep) {
// walk up the directories in the path to find the closest shared config file
closestSharedConfigPath = path.posix.join(parentDir, options.filename);
if (await mutableFilesystem.promises.exists(closestSharedConfigPath)) {
break;
}
parentDir = path.posix.join(path.posix.dirname(String(closestSharedConfigPath)), '../');
}

const aliasSharedConfigPath = path.posix.join(
path.posix.dirname(String(applyPath)),
options.filename
);

if (
(await mutableFilesystem.promises.exists(closestSharedConfigPath)) &&
!(await sharedFilesystem.promises.exists(aliasSharedConfigPath))
) {
console.log(
`[Mosaic][Plugin] Source has no shared config. Root index page is: ${namespaceSharedConfig.rootPath}`
);
console.log(
'[Mosaic][Plugin] Copying shared config ',
closestSharedConfigPath,
'-->',
aliasSharedConfigPath
);
await sharedFilesystem.promises.writeFile(
aliasSharedConfigPath,
await mutableFilesystem.promises.readFile(closestSharedConfigPath)
);
}
}
}
}
};
Expand Down
Loading
Loading