Asking for help from docker experts re custom extensions issue

Hyperbytes · August 5, 2025, 10:29am

Background for info
As many of you know, i do not use docker day to day, personally prefering the VPS/FTP route.

However, recently i found myself in need to the use of UUIDs as primary keys in a project and found the known issue with knex() and MySql not returning UUID identities on insert problematic.
So i decided to go the Docker route and use PostgreSQL.

The Problem
All went without issues until I went to install one of my own server extensions.
The extension appeared to install when installed form the Project/Extensions tab but i could not see the extension in the Server connect list.
I then went to check for the node module and noticed that there is no node_modules folder when using docker (intrigued as to why it's not necessary)

After several restarts i manually installed the files in the extensions/server_connect_modules folder which worked correctly but is far from ideal.
Is there something additional i need to do to make my extensions compatible with Docker?
Can't believe nobody from the 100s of downloads has highlighted this issue to me.
(not yet tested in production, development only)

guptast · August 5, 2025, 12:06pm

Hi Brian,

This is what I understand from building and deploying web apps via Docker in Wappler.

When deploying a project in Docker (locally or remotely), the node modules are installed via the package.json file. Basically, everytime a project is deployed via Docker, it installs the packages.

There are two files that drive this process: docker-compose.yml and Dockerfile. Both files are available here .wappler > targets > target_name > docker-compose.yml and .wappler > targets > target_name > web > Dockerfile.

I don't think this is related to Docker or the absence of node_modules folder. I have come across this issue where the extension was installed correctly, but it didn't add any npm modules and files in the extensions/server_connect_modules folder that were needed to run this extension. I had to manually copy the files and add the npm module in the package.json file before re-deploying the docker project.

At the time, I thought it was something wrong with my setup, but it seems it may be an issue or a bug with the installation of extensions itself.

Hyperbytes · August 5, 2025, 12:28pm

Thanks
did check docker-compose.yml and Dockerfile after finding this post.

Exactly as I have just found

In the above post I ahve just found JonL suggests a fix which @George later reports as "improved"
My dockercompose.yml does have that recommendation added

'../../../extensions:/opt/node_app/extensions'

but still not working:

services:
  web:
    volumes:
      - '../../../app:/opt/node_app/app'
      - '../../../lib:/opt/node_app/lib'
      - '../../../views:/opt/node_app/views'
      - '../../../public:/opt/node_app/public'
      - '../../../extensions:/opt/node_app/extensions'
      - '../../../db:/opt/node_app/db'
      - '../../../certs:/opt/node_app/certs'
      - '../../../public/userdata:/opt/node_app/public/userdata:rw'
    ports:

George · August 30, 2025, 11:26am

Hi @Hyperbytes,

I see you are raising a good point here about the docker based projects and the installation of the custom Wappler extensions.

There is indeed an extra step involved in the packaging of your custom extension to solve this problem.

You have to include an additional copy files post install script within the extension itself that is run on install and copies the required files in the extensions project folder and from there they are distributed to the rest.

See as example the extension from @tbvgl wappler-advanced-logger - npm

and specifically in the project.json:

"scripts": {
        "postinstall": "node scripts/copyFiles.js"
    },

and the in the scripts folder in copyFiles.js:

const fs = require('fs-extra');
const path = require('path');

const srcDir = path.join(__dirname, '../server_connect');
const destDir = path.join(__dirname, '../../../extensions/server_connect');

const subDirs = fs.readdirSync(srcDir, { withFileTypes: true })
    .filter(dirent => dirent.isDirectory())
    .map(dirent => dirent.name);

for (const subDir of subDirs) {
    const srcSubDir = path.join(srcDir, subDir);
    const destSubDir = path.join(destDir, subDir);

    if (!fs.existsSync(destSubDir)) {
        fs.mkdirSync(destSubDir, { recursive: true });
    }

    fs.copySync(srcSubDir, destSubDir, { overwrite: true });
}

This generic copyFiles.js can be enhanced also the copy any app_connect files as well if the extension requires them.

So try adjusting your extensions with this and see how it works.

We should add this instructions then also the the instructions for packaging custom extensions.

Hyperbytes · August 30, 2025, 12:15pm

Will add it to the queue, only about 26 to update!

George · August 30, 2025, 12:26pm

Maybe you can come up with more universal copy script than the one above, one that works for as well server-connect as app_connect extensions and their folders.

With App Connect extensions it is a bit more challenging because there might be some additional js/css files that need to be copied from within the node_modules. Those are usually referenced in the copyFiles key in the components.hjson and should go inside the extensions folder as well and the copyFiles key should be changed to refer as src the extensions as root.

so actually App Connect extensions like from @Roney_Dsilva like https://www.npmjs.com/package/@cdmx/wappler_ac_ag_chart need to be changed as well.

Hyperbytes · August 30, 2025, 12:35pm

Will take a look when i have some time, fortunately i dont currently have any app connect extensions out in the wild!

Bought some note paper for my todo list as its getting long.

Hyperbytes · August 30, 2025, 4:15pm

@George
hate to trouble you.
tried a test using this npm

i have added the script command to package.json

{
    "name": "@hyperbytes/wappler-summernote-word-formatting-cleaner",
    "version": "1.0.1",
    "description": "Strip word specific formatting from text",
    "license": "MIT",
    "author": {
        "name": "Brian English - hyperbytes"
    },
    "scripts": {
        "postinstall": "node scripts/copyFiles.js"
    },
    "keywords": [
        "wappler-extension",
        "server-connect",
        "module",
        "summernote",
        "word"
    ]
}

Added a folder "scripts"

containing copyFiles.js exactly as you suggested

Updated npm then installed to a test docker project via project/externsions but i am not seeing any files installed to extensions/server_connect/module on install

If any use, here is a log file. I can see a reference to the instansion installation but dont know exacly what i should be seeing

wappler.zip (4.3 KB)

Have also manually run project updater
Can you see what am i doing wrong?

Hyperbytes · August 31, 2025, 7:40pm

Just bumping @George before post ends up a km down the posts!

George · September 1, 2025, 2:47pm

You have the copyFiles.js script wrong, the path:

const destDir = path.join(__dirname, '../../../extensions/server_connect');

should be:

const destDir = path.join(__dirname, '../../../../extensions/server_connect');

because you have an additional @hyperbytes folder.

Now the files are copied under an extensions folder under node_modules while it should be in your web root.

Hyperbytes · September 1, 2025, 3:09pm

Thank you so much, I did supsect that may be something like that but hadnt had a chance to do tests (been at eye hospital this morning, no going blind - yet!)
I will get all my extensions updated asap and look at extending that script to include app connect extensions as a thank you.

Will also add that to the appropriate docs topic for reference.