yarn bootstrap errors on MacOS while building the canvas dependency

[eatyourgreens]

Describe the bug

yarn bootstrap fails on MacOS Sonoma when it tries to run post-install builds for the canvas package. My local build environment is:

• MacOS Sonoma 14.7 on an Apple M3 Pro
• yarn 1.22.22
• node 20.13.1
• python 3.11.5

Here's the build output with the error.

[5/5] 🔨  Building fresh packages...
[10/12] ⠄ @newrelic/native-metrics
[-/12] ⠄ waiting...
[12/12] ⠄ core-js
[4/12] ⠄ canvas
error /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas: Command failed.
Exit code: 1
Command: node-pre-gyp install --fallback-to-build --update-binary
Arguments: 
Directory: /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas
Output:
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using [email protected]
node-pre-gyp info using [email protected] | darwin | arm64
node-pre-gyp http GET https://github.com/Automattic/node-canvas/releases/download/v2.11.2/canvas-v2.11.2-node-v115-darwin-unknown-arm64.tar.gz
node-pre-gyp ERR! install response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.11.2/canvas-v2.11.2-node-v115-darwin-unknown-arm64.tar.gz 
node-pre-gyp WARN Pre-built binaries not installable for [email protected] and [email protected] (node-v115 ABI, unknown) (falling back to source compile with node-gyp) 
node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/Automattic/node-canvas/releases/download/v2.11.2/canvas-v2.11.2-node-v115-darwin-unknown-arm64.tar.gz 
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | arm64
gyp info ok 
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | arm64
gyp info find Python using Python version 3.11.5 found at "/Users/jimodonnell/anaconda3/bin/python3"
gyp info spawn /Users/jimodonnell/anaconda3/bin/python3
gyp info spawn args [
gyp info spawn args '/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/Users/jimodonnell/Library/Caches/node-gyp/20.13.1/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/Users/jimodonnell/Library/Caches/node-gyp/20.13.1',
gyp info spawn args '-Dnode_gyp_dir=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/Users/jimodonnell/Library/Caches/node-gyp/20.13.1/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.'
gyp info spawn args ]

/bin/sh: pkg-config: command not found
gyp: Call to 'pkg-config pixman-1 --libs' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack at ChildProcess.<anonymous> (/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/lib/configure.js:271:18)
gyp ERR! stack at ChildProcess.emit (node:events:519:28)
gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 23.6.0
gyp ERR! command "/Users/jimodonnell/.nvm/versions/node/v20.13.1/bin/node" "/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--update-binary" "--module=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/build/Release/canvas.node" "--module_name=canvas" "--module_path=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/build/Release" "--napi_version=9" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v115"
gyp ERR! cwd /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas
gyp ERR! node -v v20.13.1
gyp ERR! node-gyp -v v10.0.1
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/Users/jimodonnell/.nvm/versions/node/v20.13.1/bin/node /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --update-binary --module=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/build/Release/canvas.node --module_name=canvas --module_path=/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/build/Release --napi_version=9 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v115' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/@mapbox/node-pre-gyp/lib/util/compile.js:89:23)
node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1105:16)
node-pre-gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:305:5)
node-pre-gyp ERR! System Darwin 23.6.0
node-pre-gyp ERR! command "/Users/jimodonnell/.nvm/versions/node/v20.13.1/bin/node" "/Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build" "--update-binary"
node-pre-gyp ERR! cwd /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/canvas
node-pre-gyp ERR! node -v v20.13.1
node-pre-gyp ERR! node-pre-gyp -v v1.0.11
node-pre-gyp ERR! not ok 
Failed to execute '/Users/jimodonnell/.nvm/versions/node/v20.13.1/bin/node /Users/jimodonnell/zooniverse/front-end-monorepo/node_modules/node-gyp/bin/node-



error Command failed.
Exit code: 1
Command: /Users/jimodonnell/.nvm/versions/node/v20.13.1/bin/node
Arguments: /Users/jimodonnell/zooniverse/front-end-monorepo/.yarn/releases/yarn-1.22.22.cjs install --frozen-lockfile
Directory: /Users/jimodonnell/zooniverse/front-end-monorepo/packages/lib-react-components
Output:

To Reproduce

yarn bootstrap and the build fails during the yarn install --frozen-lockfile step.

I'm not sure if there's something specific to my build environment that breaks the install just for me. node-canvas has to be compiled from source on Apple CPUs.

Additional context

To work around this, install with install scripts disabled. This is good security practice anyway, to guard against supply-chain attacks that install malware on your laptop from NPM.

yarn install --ignore-scripts --frozen-lockfile
yarn bootstrap

I found this in the node-canvas Readme:

By default, pre-built binaries will be downloaded if you're on one of the following platforms:

• macOS x86/64 (not Apple silicon)
• Linux x86/64 (glibc only)
• Windows x86/64

I'm trying to bootstrap the monorepo on an M3 Macbook, so that explains why it's trying to compile node-canvas from source.

[eatyourgreens]

I think #6349 introduced this.

[eatyourgreens]

To be honest, the subject viewer is never rendered on the server, so node-canvas isn't really needed in the build. On a personal level, I'm also not sure about the ethics of supporting Automattic at the moment.

[goplayoutside3]

@eatyourgreens see instructions for MacOS installation of canvas in lib-subject-viewer's Readme and the node-canvas issue.

[eatyourgreens]

Thanks! The brew install shouldn't be necessary, because those Node binaries are never used.

[eatyourgreens]

--ignore-scripts fixed the build for me. I don't think any of the post-install scripts are needed on MacOS.

[eatyourgreens]

The Docker builds use --ignore-scripts, and the GitHub workflows too, so I've opened #6393 to add it to the bootstrap scripts. That will fix this by not building the canvas binaries.

Subject viewer tests run fine in Node, even without those binaries. I wasn't expecting that, to be honest.

[eatyourgreens]

After a bit of experimentation, it turns out you can completely remove canvas from the Volumetric Viewer without breaking the tests or the storybook. It isn't used in the browser and the tests aren't using it to render <canvas> in Node either.

TBH, given that the subject viewers are only ever run in browsers, never in Node, I'd test the Volumetric Viewer with Playwright rather than JSDOM. I've been using Playwright for some WebGL/canvas stuff I'm working on, and it just works out of the box. It also allows me to run my tests in Firefox, Chromium and Webkit, which is really useful.

[eatyourgreens]

#6446 will close this.