Deploying via github webhooks with seperate database targets

[iamsoops]

Good morning,
I did raise this query in response to a Heroku issue, so apologies if you have seen elsewhere. However, the issue isnt the same.

I am self hosting (coolify) and deploy directly from github (either manually or via webhook). Coolify then deploys docker container and sets up the proxy & certs etc.

What I cant seem to understand his how to manage the database connections across the branches within github to align with my prod, staging and dev.

Please let me know if this can be achieved. I am open to an alternate approach to deployment if this doesnt align to Wappler. I have now developed my MVP in dev and am holding off pushing to staging as I dont want to do something now that backs me into a corner as I develop further

Here is my target setup:

Production:
Project on coolify linked to main branch of my repo - Manual deploy (following merge of staging)
Target DB: productoin databse on remote postgreql instance

Staging:
Project on coolify linked to staging branch in repo - Auto deploy via webhook when dev is merged to staging.
Target DB: staging database on remote postgresql instance

Dev:
Local build on laptop
Target DB: local postgresql instance.

Really welcome any thoughts and thanks for taking the time to read this

[Apple]

@iamsoops

What I cant seem to understand his how to manage the database connections across the branches within github to align with my prod, staging and dev.

You'll want to modify your project to read the DB credentials from an environment variable ($_ENV), and you specify those variables in your Coolify control panel for each "target" (production/staging). Basically, you'll deviate from Wappler's standard configuration and use environment variables for specifying database configuration.

Maybe you can get some ideas from here:

My Dockerfile for hosting Wappler NodeJS on Caprover

Dockerfile: FROM node:16-bullseye-slim ARG NODE_ENV=production ENV NODE_ENV $NODE_ENV ARG PORT=3000 ENV PORT $PORT EXPOSE $PORT RUN apt update && apt install jq -y ENV PATH /opt/node_app/node_modules/.bin:$PATH WORKDIR /opt/node_app COPY package.json . RUN npm install --no-optional COPY . . RUN jq '.options.connection.host = "{{$_ENV.DB_HOST}}" | .options.connection.port = "{{$_ENV.DB_PORT}}" | .options.connection.user = "{{$_ENV.DB_USER}}" | .options.connection.password = "{{$_ENV.DB_PA…

[Hyperbytes]

@Apple

You'll want to modify your project to read the DB credentials from an environment variable ($_ENV), and you specify those variables in your Coolify control panel for each "target" (production/staging). Basically, you'll deviate from Wappler's standard configuration and use environment variables for specifying database configuration.

That's tweaked my interest Apple, would defining the target values in Wappler server connect settings do the same (which is my norm now) or would that not work in this case?
Not a scenario I have ever used.

[Apple]

I think so! If you can inject a $_ENV variable there. It's been a while since I last did this. I know the db.json configuration file accepts it

[iamsoops]

This is really helpful thanks apple and hyperbytes. I'm going to have a look at this tonight and see if I can make it work with the approach of inserting the variable into the json in the connection file if I have understood that correctly.

Assume on top of this I will need to manually manage schema deployments into staging and prod at each release as wappler will be unaware of the remote databases.

[Apple]

@iamsoops

Assume on top of this I will need to manually manage schema deployments into staging and prod at each release as wappler will be unaware of the remote databases.

The are two DB configs, one for the web app, one for the Wappler editor. You can configure the Wappler editor to connect to the remote database. "Schema deployments" still need to be managed through the Wappler editor, as currently there's no way to programmatically apply changes on Git deployment.

[iamsoops]

OK, thanks apple.

Now you have given me a pointer on the approach I have found this helpful blog post which is a starter for 10:

Deploy Node.js applications on a VPS using Coolify

Will report back on how I get on

[Hyperbytes]

Yes, i now always place connection settings in ENV variable within server connect settings, find it less glitchy than putting settings into Globals-> Database Connections which sometimes don't seem to refresh properly

Perhaps that may make a good video, will add it to the queue

[iamsoops]

That would be great hyperbytes.
Your videos have been such a great help in getting my head around wappler.

[kfawcett]

If you're still having trouble. You might be able to hire @tbvgl. He is familiar with Coolify and has set it up for multiple projects.

[Hyperbytes]

@iamsoops

That would be great hyperbytes.
Your videos have been such a great help in getting my head around wappler.

Managing Database connections via $_ENV variables and "manuipulating" Database Manager for use with production data

In this video we explore using the Server Connect settings "environment" tab to define database connection settings for workflow globals. We also look at how to "manipulate" the database manager to edit production settings and view/edit production data (not recommended by Wappler team but can be done) [Using $_ENV variables to define database connection settings & "manipulating" the Database Manager.]

[iamsoops]

Hyperbytes, thats awesome. Thanks so much for preparing.

Ive had a bit of a disaster with my own attempt prior to your video and after an evening of tinkering had to roll back to my backup. I ultimatly stood up a cloudpanel instance and have got the site up there initiallly.

When I have some time I'll have another go now I have your video and see if I can join the dots with coolify. If I can, I'll summarise back here for anyone else.