Async bootstrap via extendExpressApp

[franky47]

I need to run some async tasks when the server starts, and so far the best place I've found to do runtime initialisation is in extendExpressApp.

Since it's called in an async context, it could be awaited in createExpressServer.ts to allow async operations there:

keystone/packages/core/src/lib/server/createExpressServer.ts

Lines 84 to 94 in 81e663d

	if (config.server?.extendExpressApp) {
	const createRequestContext = async (req: IncomingMessage, res: ServerResponse) =>
	createContext({
	sessionContext: config.session
	? await createSessionContext(config.session, req, res, createContext)
	: undefined,
	req,
	});
	config.server?.extendExpressApp(expressServer, createRequestContext);
	}

My question is: does this seem like a good idea, or is there a better place to do this kind of async startup work?

Edit: PR opened: #7370

[dcousens]

I think the pull request #7370 has merit on it's own right, and we'll look at that separately.

My question is: does this seem like a good idea, or is there a better place to do this kind of async startup work?

I think before we can answer this, we need to understand what kind of startup work you have in mind?

Without any additional context, I might ask if you found config.db.onConnect?

[franky47]

Thanks for merging it quickly, much appreciated!

My bootstrap sequence looks like this:

1. Check that the required environment variables are set. I do this at server launch rather than in the config file's top level to ensure that it does not interfere with the postinstall script, which imports and runs the config. I have a monorepo setup where the keystone backend is postinstalled even for building the front-end (to generate GraphQL type definitions), but the front-end deployment does not have access to all the backend environment variables, so the runtime check can only happen when we truly boot the backend server.
2. Connect to a Redis instance for cache, auto-expiring data (email verification tokens, password reset tokens) and rate-limiting
3. Generate email templates with Handlebars
4. Run pending data migrations (prisma migrate deploy runs before the app starts, but we may need to migrate data using some form of logic). Requires access to the Prisma client. This is a hypothetical step I haven't automated yet (still doing it manually, which is not ideal).

I had not found config.db.onConnect, it seems like a great spot to add some of those steps, thanks for pointing it out!

How does it relate in terms of order of execution with running extendExpressApp and the server starting listening?
My current setup (with a synchronous extendExpressApp) has a data race on the email templates being built, Redis connecting and the server being ready for requests, if awaiting config.db.onConnect ensures that all the async bootstrap is finished before the server accepts requests that would solve all my issues.

[emmatown]

config.db.onConnect will run and be awaited before extendExpressApp runs and the server starts accepting requests so you can use it to do your bootstrap

[dcousens]

This was added in #7370, with an extendHttpServer counterpart in #9026