Multiplatform native dependencies pre-build and how to require

[cliqer]

• Version: 22.10.5

• Electron Version: 12.0.2
• Electron Type (current, beta, nightly): current

• Target: Mac default (arm64, x64), nsis(x64)

We are on Apple M1 Macs building for both arm64 and x64 with node v15.12.0 as also for windows nsis x64.
This works great when not using native modules.

Though, in our case we have:

• robotjs version=0.6.3
• electron-acrylic-window version=0.5.5
• win32-displayconfig version=0.1.0
• bytenode.js that is bound to the current node version

and as so the app only works if compiled in their respective environment.

That creates many problems like:

• overwriting the latest-mac.yml
• auto-update issues because of the above
• Can't create a universal app for mac
• not able to automate the process

So the main question is:
How to pre-build these native modules (each on a specific computer) and then require them on build time with electron builder?

ps: We are aware that bytenode may misbehave but that is another issue that may be solved by just downgrading node to v14 and using rosetta to compile.

[mmaietta]

How to pre-build these native modules (each on a specific computer) and then require them on build time with electron builder?

This sounds like a great candidate for node-pre-gyp. Just to confirm, you're trying to lipo together a universal "fat" binary for MacOS and receiving an error message that the .node file is the same architecture? Or are you trying to rebuild the native addons and then store them each in a repo per os+arch?

[cliqer]

@mmaietta thank you for your reply.

Trying lipo together a universal "fat" binary for MacOS and receiving an error message that the .node file is the same architecture?

Being able to create a universal binary would be nice for the future but for the time being not required.

When we build both archs on a silicon mac the native one works as expected and the x64 opens but without displaying anything or any error in the console.

The opposite happens when we build on an intel based mac but there we get errors about robotjs.node not being the correct architecture even though the latest xcode is being used.

Is this because of the node version being used per machine?
On silicon, it is the arm64 v15.2.0 and on x64 v14.x

Or are you trying to rebuild the native addons and then store them each in a repo per os+arch?

Indeed this is the best option because we want to be able to build arm-x64-win at the same time from a silicon mac.
Can something like remoteBuild being used for this purpose?
Any other build commands that may help?

  "build": {
    "remoteBuild": false,
    "artifactName": "app-v${version}-${arch}.${ext}",
    "directories": {
      "output": "build"
    },
    "win": {
      "target": [
        {
          "target": "nsis",
          "arch": "x64"
        }
      ]
    },
    "squirrelWindows": {
      "msi": true
    },
    "nsis": {
      "oneClick": false,
      "allowToChangeInstallationDirectory": false,
      "createDesktopShortcut": "always",
      "license": "license.html"
    },
    "mac": {
      "hardenedRuntime": true,
      "entitlements": "electron/entitlements.mac.inherit.plist",
      "entitlementsInherit": "electron/entitlements.mac.inherit.plist",
      "target": [
        {
          "target": "default",
          "arch": [
            "x64",
            "arm64"
          ]
        }
      ]
    }

The procedure could be something like:

• go to each machine (intel mac-windows) and run node-pre-gyp inside each of the modules.
  Any better solutions?
• Save these on separate folders as for electron-builder to be able to understand and create a new repo.
  How does electron builder try to find which native .node to use per arch?
• load the custom repos on the silicon mac and build without rebuilding those native modules.

Also, how can we force those links as to find the binaries?:

electron-builder --macos --x64
  • electron-builder  version=22.10.5 os=20.3.0
  • loaded configuration  file=package.json ("build" field)
  • writing effective config  file=build/builder-effective-config.yaml
  • rebuilding native dependencies  [email protected], [email protected], [email protected] platform=darwin arch=x64
  • install prebuilt binary  name=robotjs version=0.6.3 platform=darwin arch=x64
  • build native dependency from sources  name=robotjs
                                          version=0.6.3
                                          platform=darwin
                                          arch=x64
                                          reason=prebuild-install failed with error (run with env DEBUG=electron-builder to get more information)
                                          error=prebuild-install info begin Prebuild-install version 5.3.6
    prebuild-install info looking for cached prebuild @ /Users/admin/.npm/_prebuilds/dc29d9-robotjs-v0.6.3-electron-v87-darwin-x64.tar.gz
    prebuild-install http request GET https://github.com/intermedia-net/robotjs/releases/download/v0.6.3/robotjs-v0.6.3-electron-v87-darwin-x64.tar.gz
    prebuild-install http 404 https://github.com/intermedia-net/robotjs/releases/download/v0.6.3/robotjs-v0.6.3-electron-v87-darwin-x64.tar.gz
    prebuild-install WARN install No prebuilt binaries found (target=12.0.2 runtime=electron arch=x64 libc= platform=darwin)

Any suggestions are greatly appreciated :)

[cliqer]

@mmaietta it turns out that the no-screen issue is due to bytenode.

When removed, both mac versions worked correctly when compiled on M1.

It would be lovely though if you could shed some light on how to also build or import the native dependencies for the windows build.

[mmaietta]

Curious, I think someone either asked for this feature or confirmed that it worked, but can you try using an arch macro in copying the .node files? Something like this could reference prebuilt binaries that you committed into your repro.

        {
            from: `./bin/${process.platform}-\${arch}-${NODE_COMPILER_VERSION}`, // example:   ./bin/darwin-arm64-85/name.node
            to: nodeAddonDir,
            filter: "*.node"
        }

Alternatively, my project's approach is to rebuild the node modules on every build, as it's a simple setup and minimally impacts build time. My config looks like this:

const nodeAddonDir = process.platform === "darwin" ? "MacOS" : ""; // Copy native addons to process.exec root dir for rebranded electron
extraFiles: [
        {
            from: "build/Release",
            to: nodeAddonDir,
            filter: "*.node"
        },
        {
            from: "node_modules/node-mac-permissions/build/Release",
            to: nodeAddonDir,
            filter: "*.node"
        }
    ],

[cliqer]

Thank you for your help, @mmaietta. Figuring these out and learning new stuff every day.

Closing this now as it has to do with when bytenode runs and under which architecture when packaging a universal build.