Long Deploy Time (Docker)

Eric_F · February 25, 2023, 4:42am

How long does it usually take to deploy a website?

When I first deployed my site to DigitalOcean it only took maybe 30 secs.
I added Traefik and redeployed and it took maybe 2 min.
I just now updated 1 server action and redeployed, and it took over 10 min(657 sec).

Why is the time increasing so much? My Wappler project folder is only 2.4MB.

And the website is down during the deploy which will be a problem if the time keeps going up.

This part

took up maybe 9 of the 10 mins.

Any idea if there is something wrong or does it usually take this long?

Apple · February 25, 2023, 6:17am

The long time to deploy is likely due to temporary external factors, some third-party remote server may be answering slowly during the npm install step.

However, this step could certainly be optimized so it wouldn’t run at each deploy attempt, but rather just when package.json is updated

I’m not at the computer right now, I could modify the Dockerfile tomorrow or so to optimize such, but then it’s possible by then the problem is already “solved” on its own because the third-party remote server(s) may be fast again

Eric_F · February 25, 2023, 6:39am

Hoping it’s just temporary, I will try to deploy it again sometime over the weekend and check.

I am not using Docker locally though, so I’m not sure how I would go about modifying the live Docker file.

And regarding not using Docker locally, could that causing any problems with speed, or would it still have to install and do the same things if I used Docker?

Apple · February 25, 2023, 6:48am

Your local environment wouldn’t affect the speed of your remote deployment regarding the current issue

In this case, you could also be affected by the speed of npm install locally, using Docker or not would be the same - but usually you don’t deploy on local, so such npm install was already run when you first created your Wappler project

jmartland22 · February 25, 2023, 9:14am

I had similar issue. Server needed a reboot and it made it go alot quicker. If you ssh in to your server at least on mine it said it needed a reboot. To do that go to resource manager.

Eric_F · February 26, 2023, 12:57am

Just tried it and I gave up after waiting 30 min

That seems way too long even if something is slow to respond.

Eric_F · February 26, 2023, 1:00am

I checked and it did say I needed a reboot. I rebooted and it said it was successful but then gave me an ECON error when I tried to deploy.

Restarted Wappler and checked and it didn’t say it needed a reboot so I tried to deploy but deploy is currently at 7 min already.

Eric_F · February 26, 2023, 1:20am

Now if I try to reboot the server I get

Rebooting server SkinsClickCom ...
Error: read ECONNRESET at __node_internal_captureLargerStackTrace (node:internal/errors:464:5) at __node_internal_errnoException (node:internal/errors:594:12) at TCP.onStreamRead (node:internal/stream_base_commons:220:20)
Error: Connection lost before handshake at makeError (C:\Users\Eric\AppData\Local\Wappler\resources\app\node_modules\ssh2\lib\protocol\utils.js:142:15) at C:\Users\Eric\AppData\Local\Wappler\resources\app\node_modules\ssh2\lib\client.js:693:13 at Socket.<anonymous> (C:\Users\Eric\AppData\Local\Wappler\resources\app\node_modules\ssh2\lib\client.js:755:7) at Socket.emit (node:events:390:28) at TCP.<anonymous> (node:net:687:12)
Reboot of server SkinsClickCom completed
Rebooting server SkinsClickCom ...
Reboot of server SkinsClickCom completed
Error: connect ECONNREFUSED 142.93.119.120:22 at __node_internal_captureLargerStackTrace (node:internal/errors:464:5) at __node_internal_exceptionWithHostPort (node:internal/errors:642:12) at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1161:16)

Would this be related to long deploy time?

Apple · February 26, 2023, 3:42am

Unlikely, seems the errors are due to connection failures due to the server rebooting (if you reboot the server, then it doesn’t answer any connections for a few seconds)

Can you try to find a Dockerfile file inside the .wappler directory? It should be in your targets, in a folder “Live”, if you paste it here I can modify it

Eric_F · February 26, 2023, 3:49am

This file?

Apple · February 26, 2023, 3:50am

Close, should be near! The file is really named “Dockerfile”, check the app or web folders

Eric_F · February 26, 2023, 3:52am

Here it is

FROM node:16-bullseye-slim

ARG NODE_ENV=production

ENV NODE_ENV $NODE_ENV

ARG PORT=3000

ENV PORT $PORT

EXPOSE $PORT

RUN mkdir /opt/node_app

COPY ./ /opt/node_app/

ENV PATH /opt/node_app/node_modules/.bin:$PATH

WORKDIR /opt/node_app

RUN npm install --no-optional --production --no-package-lock

CMD [ "nodemon", "./index.js" ]

is this something I would have to do on all my projects?

George · February 26, 2023, 5:56pm

A post was split to a new topic: Optimize docker NodeJS deployments

Eric_F · February 26, 2023, 4:04am

Getting


[+] Building 1.7s (6/10)
[+] Building 1.8s (7/10)
[+] Building 1.8s (8/10)
 => [internal] load build definition from Dockerfile                                                                                             0.7s 
 => => transferring dockerfile: 394B                                                                                                             0.2s 
 => [internal] load .dockerignore                                                                                                                0.7s 
 => => transferring context: 34B                                                                                                                 0.1s 
 => [internal] load metadata for docker.io/library/node:16-bullseye-slim                                                                         0.7s 
 => [1/6] FROM docker.io/library/node:16-bullseye-slim@sha256:97de7e6381540b593c55cd628a44ddffacdbadf9d3ba2f56b56f4913d8f95541                   0.0s 
 => [internal] load build context                                                                                                                0.2s 
 => => transferring context: 8.40kB                                                                                                              0.2s 
 => CACHED [2/6] RUN mkdir /opt/node_app                                                                                                         0.0s 
 => [3/6] WORKDIR /opt/node_app                                                                                                                  0.1s 
 => ERROR [4/6] COPY package.json package-lock.json ./                                                                                           0.0s 
------
 > [4/6] COPY package.json package-lock.json ./:
------
failed to solve: failed to compute cache key: "/package-lock.json" not found: not found
Error Launching Services!

Apple · February 26, 2023, 4:05am

Hm… Sorry about that, try this:

FROM node:16-bullseye-slim

ARG NODE_ENV=production
ENV NODE_ENV $NODE_ENV

ARG PORT=3000
ENV PORT $PORT
EXPOSE $PORT

RUN mkdir /opt/node_app
ENV PATH /opt/node_app/node_modules/.bin:$PATH

WORKDIR /opt/node_app

COPY package.json ./
RUN npm install --no-optional --production --no-package-lock

COPY . .
CMD [ "nodemon", "./index.js" ]

Also, repeat the step of switching to local environment, and switching back to Live

Eric_F · February 26, 2023, 4:06am

Looks to be working. Thanks for your help! Will report back when deploy is done.

Eric_F · February 26, 2023, 4:24am

Got this

[+] Building 956.8s (8/10)
 => [internal] load .dockerignore                                                                                                                0.8s 
 => => transferring context: 34B                                                                                                                 0.1s 
 => [internal] load build definition from Dockerfile                                                                                             0.8s 
 => => transferring dockerfile: 376B                                                                                                             0.2s 
 => [internal] load metadata for docker.io/library/node:16-bullseye-slim                                                                         0.2s 
 => [1/6] FROM docker.io/library/node:16-bullseye-slim@sha256:97de7e6381540b593c55cd628a44ddffacdbadf9d3ba2f56b56f4913d8f95541                   0.0s 
 => [internal] load build context                                                                                                                0.1s 
 => => transferring context: 8.40kB                                                                                                              0.1s 
[+] Building 956.8s (9/10)
 => [internal] load .dockerignore                                                                                                                0.8s 
 => => transferring context: 34B                                                                                                                 0.1s 
 => [internal] load build definition from Dockerfile                                                                                             0.8s 
 => => transferring dockerfile: 376B                                                                                                             0.2s 
 => [internal] load metadata for docker.io/library/node:16-bullseye-slim                                                                         0.2s 
 => [1/6] FROM docker.io/library/node:16-bullseye-slim@sha256:97de7e6381540b593c55cd628a44ddffacdbadf9d3ba2f56b56f4913d8f95541                   0.0s 
 => [internal] load build context                                                                                                                0.1s 
 => => transferring context: 8.40kB                                                                                                              0.1s 
 => CACHED [2/6] RUN mkdir /opt/node_app                                                                                                         0.0s 
 => CACHED [3/6] WORKDIR /opt/node_app                                                                                                           0.0s 
 => [4/6] COPY package.json ./                                                                                                                   0.0s 
 => ERROR [5/6] RUN npm install --no-optional --production --no-package-lock                                                                   955.4s 
------
 > [5/6] RUN npm install --no-optional --production --no-package-lock:
#0 2.419 npm WARN config optional Use `--omit=optional` to exclude optional dependencies, or
#0 2.419 npm WARN config `--include=optional` to include them.
#0 2.419 npm WARN config
#0 2.419 npm WARN config     Default value does install optional deps unless otherwise omitted.
#0 2.419 npm WARN config production Use `--omit=dev` instead.
#0 952.9 npm notice 
#0 952.9 npm notice New major version of npm available! 8.19.3 -> 9.5.1
#0 952.9 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.5.1>
#0 952.9 npm notice Run `npm install -g npm@9.5.1` to update!
#0 952.9 npm notice 
#0 953.2 npm ERR! code ECONNRESET
#0 953.5 npm ERR! network aborted
#0 953.5 npm ERR! network This is a problem related to network connectivity.
#0 953.5 npm ERR! network In most cases you are behind a proxy or have bad network settings.
#0 953.9 npm ERR! network 
#0 953.9 npm ERR! network If you are behind a proxy, please make sure that the
#0 953.9 npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
#0 954.8 
#0 955.0 npm ERR! A complete log of this run can be found in:
#0 955.0 npm ERR!     /root/.npm/_logs/2023-02-26T04_06_12_668Z-debug-0.log
------
failed to solve: executor failed running [/bin/sh -c npm install --no-optional --production --no-package-lock]: exit code: 1
Error Launching Services!

Apple · February 26, 2023, 5:33am

Sounds like it’s not slow anymore, now it’s completely broken

It’s not a fault of the Dockerfile I supplied, you can keep using it

Would you maybe be able to deploy the VPS in another location? The location you’re currently using might be having a network problem. You could always open a ticket but then you’d probably need to spend time with the diagnosis of the problem with them

Eric_F · February 26, 2023, 6:24am

I think the problem may have to do with portainer/traefik.

I made a brand new project and made a new digital ocean server through the resource manager with Frankfurt as the location(old one has NYC). I was able to deploy fine and it only took 30 secs.

I changed the text and deployed again and it worked and took a little over 30 secs.

One last test took around 30 sec again

I used an old domain I had on DigitalOcean and added portainer and traefik. They both installed correctly

After they were installed I deployed to my URL rather than IP and it worked

Site was visible online with SSL.

I changed text again and redeployed and it is currently almost at 10 min

This is the same that happened to my original site. Before portainer and traefik it deployed fine and so did first deploy after installing them, but any further deploys take forever.

Not sure if I am doing something wrong or if there’s a problem with DigitalOcean and/or Wappler.

Update, it just finished deploying and took 17ish minutes

I will try another deploy to test but I assume it will take over 30min like with my original site.

Another update, the redeploy took around 6 mins

and I tried again and it broke after 9ish min

So I am assuming the problem is with portainer/traefik since the 3 deploys before adding them took 30sec and other than the first deploy after installing them, they all took 5+ and/or gave an error.

Any ideas or am I better off @ing someone on the team or filling a bug report?

Apple · February 26, 2023, 7:07am

It’s likely the Wappler team will need the log file npm spat out. However, it might be a bit tricky to get it, as it’s inside the Docker container rather than the VPS itself. Here’s the command you need to run on the remote server (Terminal/SSH [can use the one built-in in Wappler]):

find / -path "*/root/.npm/_logs/2023-02-26T06"

The output of that command might take a minute or two, whatever is the output run:

tail -50 log_location

Of course, replace the last argument (log_location). That command (tail) will give the output of the last 50 lines of the log file

You can then reply to this topic. @ patrick is probably the best person to check this, but I’m not sure if he actually receives notifications, so it’s better to tag @ Teodor