Extending App Connect: Formatters

Writing your own Formatters

Formatters are used in expressions like a method for a specific data type. The data types available in App Connect are array, boolean, null, number, object, string and undefined.

Using the formatter in an expression is done like {{ "Hello World".uppercase() }}, using the formatter on a type that is not a string will result in a warning in the console and the formatter will return undefined. So {{ (123).uppercase() }} will return the warning Formatter uppercase in expression [(123).uppercase()] doesn't exist for type number and will return undefined, this is because 123 is a number and not a string.

Formatters are sync, you can’t use async functions.

Syntax

dmx.Formatter({data type}, {formatter name}, {formatter function})
dmx.Formatters({data type}, {object with formatter name as key and function as value})

The first argument of the formatter function is always the value where it was added on, the other arguments are the ones passed as the arguments in the expression. For example a formatter function for append will use function(str1, str2), in the expression {{ "Hello".append(" World") }} str1 will be Hello and str2 will be world.

Samples

Here is how the full append formatter looks like:

dmx.Formatter('string', 'append', function(str1, str2) {
  return str1 + str2;
});

Adding multiple formatters:

dmx.Formatters('number', {
  'add': function(a, b) {
    return a + b;
  },

  'substract': function(a, b) {
    return a - b;
  }
});

Special Global Formatters

There are some formatters defined as type global, they are currently not available from the UI in Wappler. The global formatters are like functions and called like {{ run(flow, param) }}. The run formatter is used when you add a flow to an event handler.

Other already available global formatters are json and log, they are not available in the UI but are included in the App Connect source. The json formatter converts an object to a json string and the log formatter logs the object to the console, this can be handy for debugging your data.

Here is how the log formatter looks like:

dmx.Formatter('global', 'log', function(obj) {
  console.log(obj);
  return obj
});

Advanced formatters

Sometimes you want to do more advanced operations like looping over data and using a subexpression to manipulate that data. The this object references the component where the expression is runed from. The components are seen as scope objects that contain their own data and references their parents. I will not go deep in on how the component is working, but we will use it to create a new data scope for our subexpressions.

Here a sample for a map formatter:

dmx.Formatter('array', 'map', function(arr, expr) {
  var scope = this;
  arr = dmx.repeatItems(arr);
  return arr.map(function(item) {
    return dmx.parse(expr, new dmx.DataScope(item, scope));
  });
});

The formatter works like {{ [1,2,3].map('$value - 1') }} and it will return [0,1,2].

The first argument arr is the input array [1,2,3]. The function dmx.repeatItems will convert this array, the items will become objects like { $value: 1, $index: 0 }. If the input item was an object it will keep its properties like with the repeater component.

The arr.map is the native Array.map method. In the callback function we return the result of the subexpression as the new value. We use dmx.parse to parse the expression that was given as a string to the formatter. The second parameter for on dmx.parse is the scope it should work on, when not given it will use the app scope. We create a new scope with our current item and use the component scope as the parent.

7 Likes

Will these formatters show up in the pickers/formatter UI?

We will be adding hjson support soon for App Connect extensions similar to Server Connect

3 Likes

Has Hjson support been added for App connect extensions or is no longer in the pipeline?

Hi @patrick. I use complex formatting with a large amount of data. Due to the fact that the formatting is synchronous, this greatly affects the rendering at runtime. Is it possible to do something similar in formatters: How to use async functions with Array.filter in Javascript - Advanced Web Machinery, to achieve asynchronous formatting execution?

The formatters should be used just for simple data formatting, when you have more complex transformations you should do that in a flow (which are async) and then use the output data from that.

If I run an asynchronous flow inside which synchronous formatters will be used to convert data, will this solve the problem? If not, how can data be transformed inside the flow without using formatters (complex array filtering)?

You really shouldn’t use formatters for heavy data conversion - but flows and then use the new flow data output.

Eventually we will be making app flows even run in webworkers - so completely offload them. So this is the way to go.

1 Like

I understood that. But I didn’t quite understand exactly how it is supposed to transform the data inside the flow so that heavy data transformations remain asynchronous? Can you explain this point?

I’m looking forward to it. I have a lot of things that need webworkers. This would help make applications and interfaces as smooth and productive as possible, even in very complex tasks.

Flows are async, you can create a custom action step that returns a promise. If you use a page flow you can access its data as each other component on your page. You can run the flow when page loads or on a specific event.

Sample action that would run a heavy task in a webworker

dmx.Action('myTransform', function(options) {
  return new Promise((resolve, reject) => {
    const worker = new Worker('myWorker.js');
    worker.postMessage(options.input);

    worker.onmessage = (e) => {
      resolve(e.data);
      worker.terminate();
    };

    worker.onerror = (e) => {
      reject();
      worker.terminate();
    };
  });
});

myWorker.js

onmessage = (e) => {
  const input= e.data;

  // Do some heavy javascript
  const output = transform(input);

  // send data to client
  postMessage(output);
};
4 Likes

Thanks a lot for that, Patrick! If I can make it work, I can do a lot of the right things. However, I’m having a little difficulty trying to make it work.

So, what and how did I do:

  1. In the js directory, I created the mytransform.js with the following content:
dmx.Action('myTransform', function(options) {
  return new Promise((resolve, reject) => {
    const worker = new Worker('js/myWorker.js');
    worker.postMessage(options.input);

    worker.onmessage = (e) => {
      resolve(e.data);
      worker.terminate();
    };

    worker.onerror = (e) => {
      reject();
      worker.terminate();
    };
  });
});

Please note that I changed the URL of the worker in the code, because I will also create it in the js directory: const worker = new Worker('js/myWorker.js');

On the page I connect this file: <script src="js/mytransform.js"></script>

  1. In the js directory, I created the myWorker.js with the following content:
onmessage = (e) => {
  const input= e.data;

  // Do some heavy javascript
  const output = 555;

  // send data to client
  postMessage(output);
};
  1. I created a page flow: 1
  2. I created a test variable: 2

It remains unclear to me how to run a custom action myTransform inside the flow_test page flow, and then, after executing, get the result and set it to the test_worker variable. I tried to run it as a javascript function, but it didn’t work out:

An error occurs when starting page flow.

Could you explain what I’m doing wrong and how to do it right?

I guess the action has to be added in code view. Just check how other steps are called and add your custom one. You have created a custom action that is added as a step to the flow. Not a js function that is called in the run JS step.

The dmx.Action is to create your own custom actions, we currently don’t have official extension support for the client-side, so a UI is not available.

For your test is probably easier to create a normal function like:

function myTransform(input) {
  return new Promise((resolve, reject) => {
    const worker = new Worker('js/myWorker.js');
    worker.postMessage(input);

    worker.onmessage = (e) => {
      resolve(e.data);
      worker.terminate();
    };

    worker.onerror = (e) => {
      reject();
      worker.terminate();
    };
  });
});

And then call that JavaScript function with the Run JavaScript flow action.

1 Like