Deploy / build failure on Mac M1 Arm

Hi, not sure what has happened to my project but for some reason I cannot deploy it locally this evening. It’s been working fine until now, although I think there was a message about updating node modules when I opened it up this evening.

I’m using NodeJS, Docker container, and a sqlite3 database.
On Mac OSX 12.0.1 with Wappler 4.4.5

When I click deploy I’m getting this error:

=> ERROR [5/5] RUN npm install --no-optional                                                                                                    7.5s
------                                                                                                                                                
 > [5/5] RUN npm install --no-optional:                                                                                                               
#9 2.650 npm WARN deprecated node-pre-gyp@0.11.0: Please upgrade to @mapbox/node-pre-gyp: the non-scoped node-pre-gyp package is deprecated and only the @mapbox scoped package will recieve updates in the future                                                                                                                                                 
#9 3.857 npm WARN deprecated tar@2.2.2: This version of tar is no longer supported, and will not receive security updates. Please upgrade asap.       
#9 3.911 npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142                          
#9 4.077 npm WARN deprecated har-validator@5.1.5: this library is no longer supported
#9 4.163 npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
#9 6.570 
#9 6.570 > sqlite3@5.0.2 install /opt/node_app/node_modules/sqlite3
#9 6.570 > node-pre-gyp install --fallback-to-build
#9 6.570 
#9 6.697 node-pre-gyp WARN Using needle for node-pre-gyp https download 
#9 7.163 node-pre-gyp WARN Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.2/napi-v3-linux-arm64.tar.gz 
#9 7.163 node-pre-gyp WARN Pre-built binaries not found for sqlite3@5.0.2 and node@12.22.9 (node-v72 ABI, glibc) (falling back to source compile with node-gyp) 
#9 7.289 gyp ERR! find Python 
#9 7.289 gyp ERR! find Python Python is not set from command line or npm configuration
#9 7.289 gyp ERR! find Python Python is not set from environment variable PYTHON
#9 7.290 gyp ERR! find Python checking if "python" can be used
#9 7.290 gyp ERR! find Python - "python" is not in PATH or produced an error
#9 7.290 gyp ERR! find Python checking if "python2" can be used
#9 7.290 gyp ERR! find Python - "python2" is not in PATH or produced an error
#9 7.290 gyp ERR! find Python checking if "python3" can be used
#9 7.290 gyp ERR! find Python - "python3" is not in PATH or produced an error
#9 7.290 gyp ERR! find Python 
#9 7.290 gyp ERR! find Python **********************************************************
#9 7.290 gyp ERR! find Python You need to install the latest version of Python.
#9 7.290 gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
#9 7.290 gyp ERR! find Python you can try one of the following options:
#9 7.290 gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
#9 7.290 gyp ERR! find Python   (accepted by both node-gyp and npm)
#9 7.290 gyp ERR! find Python - Set the environment variable PYTHON
#9 7.290 gyp ERR! find Python - Set the npm configuration variable python:
#9 7.290 gyp ERR! find Python   npm config set python "/path/to/pythonexecutable"
#9 7.290 gyp ERR! find Python For more information consult the documentation at:
#9 7.290 gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
#9 7.290 gyp ERR! find Python **********************************************************
#9 7.290 gyp ERR! find Python 
#9 7.291 gyp ERR! configure error 
#9 7.291 gyp ERR! stack Error: Could not find any Python installation to use
#9 7.291 gyp ERR! stack     at PythonFinder.fail (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:307:47)
#9 7.291 gyp ERR! stack     at PythonFinder.runChecks (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:136:21)
#9 7.291 gyp ERR! stack     at PythonFinder.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:179:16)
#9 7.291 gyp ERR! stack     at PythonFinder.execFileCallback (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:271:16)
#9 7.291 gyp ERR! stack     at exithandler (child_process.js:315:5)
#9 7.291 gyp ERR! stack     at ChildProcess.errorhandler (child_process.js:327:5)
#9 7.291 gyp ERR! stack     at ChildProcess.emit (events.js:314:20)
#9 7.291 gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:274:12)
#9 7.291 gyp ERR! stack     at onErrorNT (internal/child_process.js:470:16)
#9 7.291 gyp ERR! stack     at processTicksAndRejections (internal/process/task_queues.js:84:21)
#9 7.292 gyp ERR! System Linux 5.10.76-linuxkit
#9 7.292 gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64/node_sqlite3.node" "--module_name=node_sqlite3" "--module_path=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64" "--napi_version=8" "--node_abi_napi=napi" "--napi_build_version=3" "--node_napi_label=napi-v3"
#9 7.292 gyp ERR! cwd /opt/node_app/node_modules/sqlite3
#9 7.292 gyp ERR! node -v v12.22.9
#9 7.292 gyp ERR! node-gyp -v v5.1.0
#9 7.292 gyp ERR! not ok 
#9 7.294 node-pre-gyp ERR! build error 
#9 7.295 node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64 --napi_version=8 --node_abi_napi=napi --napi_build_version=3 --node_napi_label=napi-v3' (1)
#9 7.295 node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/opt/node_app/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
#9 7.295 node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:314:20)
#9 7.295 node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1022:16)
#9 7.295 node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
#9 7.295 node-pre-gyp ERR! System Linux 5.10.76-linuxkit
#9 7.295 node-pre-gyp ERR! command "/usr/local/bin/node" "/opt/node_app/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
#9 7.295 node-pre-gyp ERR! cwd /opt/node_app/node_modules/sqlite3
#9 7.295 node-pre-gyp ERR! node -v v12.22.9
#9 7.295 node-pre-gyp ERR! node-pre-gyp -v v0.11.0
#9 7.295 node-pre-gyp ERR! not ok 
#9 7.295 Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64 --napi_version=8 --node_abi_napi=napi --napi_build_version=3 --node_napi_label=napi-v3' (1)
#9 7.418 npm ERR! code ELIFECYCLE
#9 7.418 npm ERR! errno 1
#9 7.423 npm ERR! sqlite3@5.0.2 install: `node-pre-gyp install --fallback-to-build`
#9 7.423 npm ERR! Exit status 1
#9 7.423 npm ERR! 
#9 7.423 npm ERR! Failed at the sqlite3@5.0.2 install script.
#9 7.423 npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
#9 7.427 
#9 7.427 npm ERR! A complete log of this run can be found in:
#9 7.427 npm ERR!     /root/.npm/_logs/2022-01-19T18_24_52_996Z-debug.log
------
executor failed running [/bin/sh -c npm install --no-optional]: exit code: 1
ERROR: Service 'web' failed to build : Build failed
Error Launching Services!

The error seems to refer to python, but both python and python3 commands are working from terminal.

Any suggestions on how to fix this?
Thanks

As you are probably using SQLite as database on Mac with M1 - this module doesn’t support M1 natively yet without recompilation.

For that it needs to have python installed.

Thanks George - I am indeed using an M1 Mac. Although it has been working fine until now.

Do you have any suggestions on how to fix it?
Should I be using the standard Mac build of wappler and running it with Rosetta?
Or is this issue something to do with sqlite/npm etc ?

I see that you are using older node that is not for M1 and probably intel version.

You should update the the latest node LTS version that has arm M1 support as well.

Hi George,
I can see what you mean now about node version - when Wappler is trying to build it references v12.22.9

However when I run these commands on terminal it seems to be the up to date versions:

node -v 
v17.4.0
npm -v
8.1.2

How to do I change which version of node that wappler is using?

Well Wappler just uses what is available on your system. So those should be fine.

Still having troubles running NodeJS? I have pretty much the same on my MacBook Pro M1 max and runs great.

Hi George,
I'm still having some trouble with all of this.

I've actually completely wiped and reset my M1 Air.
From a factory fresh install I've done the following steps

  • Install google chrome
  • Enable rosetta via terminal command:
    /usr/sbin/softwareupdate --install-rosetta --agree-to-license
  • Install xcode command line tools via terminal command:
    xcode-select --install
  • Download Wappler for Apple Silicon
  • Create a new project using Docker and NodeJS
  • Complete the system setup suggested by Wappler which installs Brew and Docker Desktop
  • Try to deploy the project - and then get hit with this error:

.
ERROR [5/5] RUN npm install --no-optional 12.3s
[5/5] RUN npm install --no-optional:
#9 11.12
#9 11.12 > argon2@0.28.3 install /opt/node_app/node_modules/argon2
#9 11.12 > node-pre-gyp install --fallback-to-build
#9 11.12
#9 12.00 [argon2] Success: "/opt/node_app/node_modules/argon2/lib/binding/napi-v3/argon2.node" is installed via remote
#9 12.02
#9 12.02 > sharp@0.29.3 install /opt/node_app/node_modules/sharp
#9 12.02 > (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
#9 12.02
#9 12.09 sharp: Installation error: Use with glibc 2.28 requires manual installation of libvips >= 8.11.3
#9 12.09 sharp: Please see sharp - High performance Node.js image processing for required dependencies
#9 12.21 npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.3.2 (node_modules/chokidar/node_modules/fsevents):
#9 12.21 npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.3.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"arm64"})
#9 12.21
#9 12.22 npm ERR! code ELIFECYCLE
#9 12.22 npm ERR! errno 1
#9 12.22 npm ERR! sharp@0.29.3 install: (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
#9 12.22 npm ERR! Exit status 1
#9 12.22 npm ERR!
#9 12.22 npm ERR! Failed at the sharp@0.29.3 install script.
#9 12.22 npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
#9 12.23
#9 12.23 npm ERR! A complete log of this run can be found in:
#9 12.23 npm ERR! /root/.npm/_logs/2022-01-20T21_37_00_324Z-debug.log
failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c npm install --no-optional]: exit code: 1
Error Launching Services!

I have also then wiped the Wappler install as per the guide below and downloaded the standard Apple Mac version of Wappler and created a new project - same issue.

The only thing I haven't done yet is manually install node / npm but judging by the output from wappler it seems to have installed it itself? Maybe when Homebrew was installing?

I'm really at a loss here as my Mac install is totally fresh and can't see how there would be any incompatibilities that would make this deploy fail.

Well it wasn’t really necessary to fully reset your Mac.

The docker problem with sharp and arm is easily fixed in the docker options when you choose to use debian 11

I will see if we can enforce that on the UI specially for M1 arm processors as it is only needed there.

See also:

Hi Geroge,

Thanks for the tip about the sharp/docker issue - this was fixed when I followed your instructions to change to Debian 11 Bullseye and the project deployed.

I then added a new database connection to a basic sqlite database file. Then when I click deploy I am back to the original error with sqlite.

ERROR [5/5] RUN npm install --no-optional 18.5s
[5/5] RUN npm install --no-optional:
#9 4.079 npm WARN deprecated node-pre-gyp@0.11.0: Please upgrade to @mapbox/node-pre-gyp: the non-scoped node-pre-gyp package is deprecated and only the @mapbox scoped package will recieve updates in the future
#9 8.210 npm WARN deprecated tar@2.2.2: This version of tar is no longer supported, and will not receive security updates. Please upgrade asap.
#9 8.337 npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
#9 8.494 npm WARN deprecated har-validator@5.1.5: this library is no longer supported
#9 8.530 npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See There’s Math.random(), and then there’s Math.random() · V8 for details.
#9 13.61
#9 13.61 > argon2@0.28.3 install /opt/node_app/node_modules/argon2
#9 13.61 > node-pre-gyp install --fallback-to-build
#9 13.61
#9 14.22 [argon2] Success: "/opt/node_app/node_modules/argon2/lib/binding/napi-v3/argon2.node" is installed via remote
#9 14.23
#9 14.23 > sharp@0.29.3 install /opt/node_app/node_modules/sharp
#9 14.23 > (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
#9 14.23
#9 14.31 sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.11.3/libvips-8.11.3-linux-arm64v8.tar.br
#9 17.60
#9 17.60 > sqlite3@5.0.2 install /opt/node_app/node_modules/sqlite3
#9 17.60 > node-pre-gyp install --fallback-to-build
#9 17.60
#9 17.67 node-pre-gyp WARN Using needle for node-pre-gyp https download
#9 18.05 node-pre-gyp WARN Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.2/napi-v3-linux-arm64.tar.gz
#9 18.06 node-pre-gyp WARN Pre-built binaries not found for sqlite3@5.0.2 and node@12.22.9 (node-v72 ABI, glibc) (falling back to source compile with node-gyp)
#9 18.19 gyp ERR! find Python
#9 18.19 gyp ERR! find Python Python is not set from command line or npm configuration
#9 18.19 gyp ERR! find Python Python is not set from environment variable PYTHON
#9 18.19 gyp ERR! find Python checking if "python" can be used
#9 18.19 gyp ERR! find Python - "python" is not in PATH or produced an error
#9 18.19 gyp ERR! find Python checking if "python2" can be used
#9 18.19 gyp ERR! find Python - "python2" is not in PATH or produced an error
#9 18.19 gyp ERR! find Python checking if "python3" can be used
#9 18.20 gyp ERR! find Python - "python3" is not in PATH or produced an error
#9 18.20 gyp ERR! find Python
#9 18.20 gyp ERR! find Python **********************************************************
#9 18.20 gyp ERR! find Python You need to install the latest version of Python.
#9 18.20 gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
#9 18.20 gyp ERR! find Python you can try one of the following options:
#9 18.20 gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
#9 18.20 gyp ERR! find Python (accepted by both node-gyp and npm)
#9 18.20 gyp ERR! find Python - Set the environment variable PYTHON
#9 18.20 gyp ERR! find Python - Set the npm configuration variable python:
#9 18.20 gyp ERR! find Python npm config set python "/path/to/pythonexecutable"
#9 18.20 gyp ERR! find Python For more information consult the documentation at:
#9 18.20 gyp ERR! find Python GitHub - nodejs/node-gyp: Node.js native addon build tool
#9 18.20 gyp ERR! find Python **********************************************************
#9 18.20 gyp ERR! find Python
#9 18.20 gyp ERR! configure error
#9 18.20 gyp ERR! stack Error: Could not find any Python installation to use
#9 18.20 gyp ERR! stack at PythonFinder.fail (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:307:47)
#9 18.20 gyp ERR! stack at PythonFinder.runChecks (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:136:21)
#9 18.20 gyp ERR! stack at PythonFinder. (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:179:16)
#9 18.20 gyp ERR! stack at PythonFinder.execFileCallback (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:271:16)
#9 18.20 gyp ERR! stack at exithandler (child_process.js:315:5)
#9 18.20 gyp ERR! stack at ChildProcess.errorhandler (child_process.js:327:5)
#9 18.20 gyp ERR! stack at ChildProcess.emit (events.js:314:20)
#9 18.20 gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:274:12)
#9 18.20 gyp ERR! stack at onErrorNT (internal/child_process.js:470:16)
#9 18.20 gyp ERR! stack at processTicksAndRejections (internal/process/task_queues.js:84:21)
#9 18.20 gyp ERR! System Linux 5.10.76-linuxkit
#9 18.20 gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64/node_sqlite3.node" "--module_name=node_sqlite3" "--module_path=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64" "--napi_version=8" "--node_abi_napi=napi" "--napi_build_version=3" "--node_napi_label=napi-v3"
#9 18.20 gyp ERR! cwd /opt/node_app/node_modules/sqlite3
#9 18.20 gyp ERR! node -v v12.22.9
#9 18.20 gyp ERR! node-gyp -v v5.1.0
#9 18.20 gyp ERR! not ok
#9 18.20 node-pre-gyp ERR! build error
#9 18.20 node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64 --napi_version=8 --node_abi_napi=napi --napi_build_version=3 --node_napi_label=napi-v3' (1)
#9 18.20 node-pre-gyp ERR! stack at ChildProcess. (/opt/node_app/node_modules/sqlite3/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
#9 18.20 node-pre-gyp ERR! stack at ChildProcess.emit (events.js:314:20)
#9 18.20 node-pre-gyp ERR! stack at maybeClose (internal/child_process.js:1022:16)
#9 18.20 node-pre-gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
#9 18.20 node-pre-gyp ERR! System Linux 5.10.76-linuxkit
#9 18.20 node-pre-gyp ERR! command "/usr/local/bin/node" "/opt/node_app/node_modules/sqlite3/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
#9 18.20 node-pre-gyp ERR! cwd /opt/node_app/node_modules/sqlite3
#9 18.20 node-pre-gyp ERR! node -v v12.22.9
#9 18.20 node-pre-gyp ERR! node-pre-gyp -v v0.11.0
#9 18.20 node-pre-gyp ERR! not ok
#9 18.20 Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/opt/node_app/node_modules/sqlite3/lib/binding/napi-v3-linux-arm64 --napi_version=8 --node_abi_napi=napi --napi_build_version=3 --node_napi_label=napi-v3' (1)
#9 18.31 npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.3.2 (node_modules/chokidar/node_modules/fsevents):
#9 18.31 npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.3.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"arm64"})
#9 18.31
#9 18.32 npm ERR! code ELIFECYCLE
#9 18.32 npm ERR! errno 1
#9 18.32 npm ERR! sqlite3@5.0.2 install: node-pre-gyp install --fallback-to-build
#9 18.32 npm ERR! Exit status 1
#9 18.32 npm ERR!
#9 18.32 npm ERR! Failed at the sqlite3@5.0.2 install script.
#9 18.32 npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
#9 18.34
#9 18.34 npm ERR! A complete log of this run can be found in:
#9 18.34 npm ERR! /root/.npm/_logs/2022-01-21T08_11_59_184Z-debug.log
failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c npm install --no-optional]: exit code: 1
Error Launching Services!

Again this is with Wappler (Intel Mac version) and Node 16 and npm 8.
Python3 is available (default mac install) and I've tried the suggestion in the output of

npm config set python "/usr/bin/python3"

and still no luck.

There must be some underlying issue with sqlite build process that is not just unique to my machine?

well Ben - you seem to hit every possible roadblock with the new M1 Arm architecture :slight_smile:

If you use Docker - it creates a whole new Linux Arm environment for you. And unfortunately SQLite doesn’t work there out of the box - the module writers don’t support that yet.

But if you don’t use docker as you initially started and have nodeJS and SQLite installed locally on the M1 - then it should be all fine. As I instructed you earlier. So no need for docker - it complicates and makes local development more heavy.

You need Docker only when you also want to use bigger databases like MariaDB/Postgress. Not MySQL as they also run behind in support on Arm.

Sorry that it may seem all complex - but the new Apple M1 Arm processor is just too brand new and not all open source projects have fully embraced it yet and some run behind - hence the installation problems. With Wappler we try to ease the pain by automating things as much as possible but we are also dependent unfortunately of some of those external modules.

See:

I swapped over to MariaDB due to issues with SQL on M1, it now works like a dream and was very easy to set up on the server.
Not sure if that is an option for you.