Writing Custom Modules and Formatters (NodeJS)

Writing your own Formatters

Location

All formatters are being placed in the extensions/server_connect/formatters folder. A single file can contain multiple formatters and you can have multiple files. Best practice is to group formatters that belong together in a single file. For example you to keep all array specific formatters in a file called array.js .

File Structure

Here a sample of how such a formatter file looks like.

exports.uppercase = function(str) {
  return str.toUpperCase();
};

exports.lowercase = function(str) {
  return str.toLowerCase();
};

Only methods added to exports will be available as formatter. Naming a formatter the same as one bundled with Server Connect will override it.

Writing your own Modules

Location

All modules are being placed in the extensions/server_connect/modules folder. The name of the file is also the module name and the methods exported are the action names. Best practice is to group actions together under a module and not create a file for each action.

File Structure

Here is a sample of how a module file looks like.

exports.setvalue= function(options, name) {
  return this.parse(options.value);
};

Only methods added to exports will be available as actions. Naming the file the same as an build-in module will override it. The options parameter will contain all the options and the name is the name of the action step. The this.parse(...) can be used to parse expressions.

Creating a slugify formatter

NodeJS uses the CommonJS module specification. Formatters are just simple functions that accepts the value to which it is applied to as first parameter and the transforms/formats that value to something else. You export the methods to make them available as formatter. The modules needs to be placed inside the extensions/server_connect/formatters folder.

Let us create a slugify formatter step by step.

The supplied slugify formatter with Server Connect is not very advanced, it doesn’t take care of unicode very well and also you would like some other replaces depending on the language used.

There is a slugify package in npm that has a lot of features, so let us use that. For more information of the slugify package visit https://www.npmjs.com/package/slugify.

First you will need to install the package in our project. Open the console in Wappler and execute the following command: npm i slugify. When it finished installing we should be able to use it.

Now we will start creating our custom formatter. First make sure that we have a folder formatters created under the app folder. The app folder is where all user based code is placed, we will never want to edit files in the lib folder since these can be replaced by Wappler.

We create a file called slugify.js inside the extensions/server_connect/formatters folder. The code for the file will be very simple, we start by requiring the slugify module.

const slugify = require('slugify');

Next we will create our formatter method, the first parameter is always the value the method is added to. For example with an expression like {{ myStr.slugify() }} myStr is passed as first parameter to the formatter. Since the slugify method already accepts a string as first parameter, we don’t need to wrap it in a function and can export it directly like:

exports.slugify = slugify;

How about when we have extra parameters we want to pass. We will accept a parameter locale and call the formatter slugifyLocale. It will be called like {{ myStr.slugifyLocale('de') }}.

exports.slugifyLocale = function(str, locale) {
  return slugify(str, { locale });
};

So the complete slugify.js file now contains:

const slugify = require('slugify');

exports.slugify = slugify;

exports.slugifyLocale = function(str, locale) {
  return slugify(str, { locale });
};

That’s all.

4 Likes

@patrick, can we use asynchronous code? I am guessing not yet, but wanted to check.

You can use async in modules, not in formatters.

2 Likes

Please make a video showing how

Adriano, the docs are already pretty detailed!

yes, but if there was a video it would be much better, at the time of dmxzone there were always videos
ok more is what we have thank you Teodor

Hi @patrick / @Teodor ,

Can you please elaborate on these points:

  1. Where to put the extensions folder? In the source folder or in the public folder? Or somewhere else?
  2. What should be the extension of the modules file - .js or .ejs or something else?
  3. What is setvalue in your example for module?
  4. What does this.parse do? I tried Googleing, but could not find anything.
  5. Can you please share a complete module example like you have for formatter?

In PHP, I once created a custom module, which was an extension of the file download module.
I don’t understand what an action is here. When using the custom module in PHP, I just added file download in UI, went into the server action JSON and changed the name/params. For NodeJS, how to use the custom module/action without creating a hjson?

More info: I am trying to create a custom module which returns Base64 of an uploaded image. So I will just pass the file path, and it should return me Base64 encoded string using this package.

The extensions folder is just in your project root.

The module extension depends on the server model - for NodeJS it is just .js

1 Like

@sid I kindly ask someone who managed to make a module, show in a video how to do it step by step.

I believe it is not as simple as it seems, I did several unsuccessful tests

1 Like

It would be pretty simple i’m sure. The post is just lacking some information. Some more details would help to get it just right.
I hope I can post something, if I get it working before you do. :sweat_smile:

Should be pretty straight forward to do, if you need more info just keep asking and @patrick can also extend the docs with those questions afterwards

Got the module to work. :slight_smile:
Will configure the custom package now.
Will also try the .hjson part soon.

@patrick Was able to get Wappler to call the custom module from server action, but the file path I am supplying is not working. Its unable to find the file.

The value I am passing is /public/uploads/abc.png which is valid.
image

I am getting a no response. Is there a specific way to access the public folder stuff from the custom module js file?
I tried to add multiple ../ - hoping it will change path to root, and then try to access the file, but that did not work either.

P.S. I understood the method & action thing I asked earlier. So please ignore that question.

You need to convert the path to system path first. Here the exists action of the fs module with a little modification to use it in the extensions folder.

const fs = require('fs-extra');
const { toSystemPath } = require('../../lib/core/path');

exports.exists = async function(options) {
  let path = toSystemPath(this.parseRequired(options.path, 'string', 'fs.exists: path is required.'));

  if (fs.existsSync(path)) {
    if (options.then) this.exec(options.then, true);
    return true;
  } else {
    if (options.else) this.exec(options.else, true);
    return false;
  }
}
3 Likes

An update on the questions asked above:
George answered 1 & 2.
3. What is setvalue in your example for module?
A - Its the name of the action.
5. Can you please share a complete module example like you have for formatter?
A - I will try to share it myself, now that I am able to at least run the module

Still unanswered:
4. What does this.parse do? I tried Googleing, but could not find anything.

About the module name and action name, its clear now too. A step in server action is an action, which is inside a parent module. This module is basically the file where action definition resides.

The file path thing is also working now as expected. Thank you Patrick. :slightly_smiling_face:
One typo is the require path for path will be ../../../lib/core/path.

Next: .hjson.

1 Like

Just a note for those that already had custom formatters for nodejs.

I had to change manually /lib/formatters/index.js for backwards compatibility. I defined custom formatters in lib/formatters/custom.js

So you index.js would look something like this.

const collections = require('./collections');
const conditional = require('./conditional');
const core = require('./core');
const crypto = require('./crypto');
const date = require('./date');
const number = require('./number');
const string = require('./string');
const custom = require('./custom');

module.exports = {
  ...collections,
  ...conditional,
  ...core,
  ...crypto,
  ...date,
  ...number,
  ...string,
  ...custom
};

I will be migrating them to the current approach but just in case someone notices that something broke with 3.4.1

This file will be overwritten in future updates so make sure you migrate your formatters to the new approach or remember to discard changes(if you are using GIT) or redoing the change.

1 Like

The function runs in the context of Server Connect and has access to several of its internal methods. this.parse is like dmx.parse in App Connect, it parses an expression and returns the result. The input can also be a complete object, it will then check all the properties if it contains a string with an expression to parse and parses them all.

Other maybe handy methods/properties:

// parse an option or throw an error if not a valid string
this.parseRequired(options.value, 'string', 'parameter value is required.');
// parse an option, a default value will be used if it is not a valid string
this.parseOptional(options.value, 'string', 'default value');
// parse an option, like above a default value will be used if it is not set, it doesn't check the type
this.parseOptional(options.value, '*', 'default value');

// execute sub steps/actions
// always set the second parameter to true
await this.exec(options.then, true);

// express req object
this.req
// express res object
this.res

// set session variable
this.setSession(key, value);
// get session variable
this.getSession(key);
// remove session variable
this.removeSession(key);

// set cookie
this.setCookie(name, value, opts);
// get cookie
this.getCookie(name, opts);
// remove cookie
this.removeCookie(name);

I think these will be the most important ones. These only applies to custom modules, they are not available with formatters.

4 Likes

Edit: Or that :point_up_2:

It’s an internal function from Wappler. It parses data we send from the UI so that the framework understands how to treat it. Is it a string, a number, a reference to a variable, an sql query, etc

Tried out .hjson. Its amazing. :exploding_head:
Stuck with output though:

  1. There is no output checkbox. How to enable that? Enabling from the option up top does work, but having a checkbox would be great.
  2. The output is not showing up. The response in the custom module is just a string. Do I have to wrap it up in a JSON saying {"path": response}? With dataPickObject set to true, this does show up in the picker, but there is no actual output.
 dataScheme: [
    {name: 'path', type: 'text'}
  ],

Just have an extra field in your variables with optionName: ‘output’ , like this:

{ name: 'myActionOutput', optionName: 'output', title: 'Output', type: 'boolean', defaultValue: false }
1 Like