NX-cloud: Cache node_modules when using Nx Agents

[vorlonbuster]

Hi.
Recently our organization started using Nx agents in our CI to speed up things.

This lead to an unexpected problem.

Some backstory.
We are using font awesome for our icons, this means that npm install fetches packages from font awesome's npm registry (https://npm.fontawesome.com/). This is not a problem in itself. A token is required and all works well.
One clean npm install with no caching downloads roughly 50MB (I think, depends on what you have included in your project).

Font awesome measures bandwidth so these clean installs add up. Standard pro bandwidth limit is 4GB. This means that 80 clean install will reach that max limit and you must wait to the start of next month to succeed with new clean install.

80 installs are quit a lot and haven't been a problem for us at all. Specially as we cache node modules on our CI servers.

When we started using Nx Agents we reached that limit within a day.

It seems that the agents being spun up all do a clean npm install every time. Please correct me if I'm wrong here and there is a solution for this. (almost) Nothing would please me more.

We use between 3 and 9 agents every time the CI need to build/lint/test stuff depending on the size of the job. This usually happens on PR, nightly builds, deployment etc.

So if 10 developers do 1 PR each day and each PR use 6 agents on average this means 60 clean installs right there. 10 nightly app builds using 3 agents and voila -> we are over the limit.

So my question is this:
Is there any way of caching node modules when we're using Nx agents?
If yes, hoorah, hit me!

if no, do you plan on supporting it?
If yes, hoorah, when?
If no, (sad face) we must abandon Nx agents.

Regards
Morten

[Cammisuli]

Hey @vorlonbuster, what agent launch template are you using? Are you using the preconfigured ones that we have defined here https://github.com/nrwl/nx-cloud-workflows/blob/main/launch-templates/linux.yaml, or something custom?

All the predefined ones have a specific step where it caches node_modules, as defined like so:

 - name: Restore Node Modules Cache
        uses: 'nrwl/nx-cloud-workflows/v4/workflow-steps/cache/main.yaml'
        inputs:
          key: 'package-lock.json|yarn.lock|pnpm-lock.yaml'
          paths: 'node_modules'
          base-branch: 'main'

The thing is that the predefined launch templates expect the base-branch name to be main. So if you are using that, and your base branch is master then you should define your own template (its as easy as copying the predefined one and then adding it to your repo under .nx/workflows/agents.yaml (read more here https://nx.dev/ci/reference/launch-templates#getting-started-with-custom-launch-templates)

If you're already using a custom launch template, do you have a cache step configured?

[vorlonbuster]

Hi @Cammisuli. thanks for the quick response.

We use the following setup (github actions):

      - name: Start CI Agents
        run: pnpm dlx nx-cloud start-ci-run --distribute-on=".nx/workflows/dynamic-changesets.yaml"

With the following dynamic-changesets.yaml:

distribute-on:
  small-changeset: 3 linux-medium-js
  medium-changeset: 6 linux-medium-js
  large-changeset: 9 linux-medium-js

The agents ar stopped manually at a later stage in the workflow:

      - name: Stop CI Agents
        run: pnpm dlx nx-cloud stop-all-agents

The name of our base branch is main.

So we are using pretty standard stuff straight out of your playbook as fas as I can tell. Have I missed something crucial here?

Regards
Morten

[WolfSoko]

Similar problem here.
node_modues install always took around 4 Minutes. Cache hit doesn't affect the time.
So it seems like the cache is not working correctly.
Cache settings are as described:

      - name: Restore Node Modules Cache
        uses: 'nrwl/nx-cloud-workflows/v4/workflow-steps/cache/main.yaml'
        inputs:
          key: 'package-lock.json|yarn.lock|pnpm-lock.yaml'
          paths: 'node_modules'
          base-branch: 'main'
      - name: Restore Browser Binary Cache
        uses: 'nrwl/nx-cloud-workflows/v4/workflow-steps/cache/main.yaml'
        inputs:
          key: 'package-lock.json|yarn.lock|pnpm-lock.yaml|"browsers"'
          paths: |
            '../.cache/Cypress'
            '../.cache/ms-playwright'
          base-branch: 'main'
      - name: Install Node Modules
        uses: 'nrwl/nx-cloud-workflows/v4/workflow-steps/install-node-modules/main.yaml'
        inputs:
          npm_legacy_install: 'false'
      - name: Install Browsers (if needed)
        uses: 'nrwl/nx-cloud-workflows/v4/workflow-steps/install-browsers/main.yaml'