How to concatenate two arrays into one in Server Actions? (NodeJS)

Ah i see :slight_smile: well we deliverable disallow running any JavaScript in expressions because this can cause security issues.

Indeed it can.

That’s the reason why I selected VM2 as module for it.

1 Like

Aha I see, good idea indeed instead of just eval.
Good to be sandboxed.

1 Like

Thanks for kick-starting! I’ll take a look at it soon, some scary words there, “lexer” :smiley:
(no idea why we need to go such deep to the point of parsing actual strings)

My idea is more like allowing a snippet of JS, like this:

let array1 = [1, 3];
let array2 = [4, 5];
return array1.concat(array2);

Can’t we just call vm.run(whatever_in_textarea)? And in Wappler it would be a textarea instead of single-line input of code.

Pseudo-code of module:

function runCustomCode(snippet) {
    // Wrap snippet inside function()
    let code = "function() {" + snippet + "}()";
    // Run function, and return function result to Wappler
    return vm.run(code)
}

Yes, you can call whatever you want from the text area but wouldn’t you want to reference values stored in Server Connect via the Server Bindings?

Yes, so I need to call this.parse(snippet)? Or do I need that whole token parser thing?

The parse() function will only work with expressions that make sense within SC framework’s scope.

Mixing JS code and SC bindings is not parseable with parse(). We would need to build a new parser for this kind of expressions which is why you can see me calling the lexer function.

Unless you want to have some dynamic generated functions I would just suggest creating custom modules/formatters instead of evaluating javascript.

exports.concat = function(arr1, arr2) {
  if (!Array.isArray(arr1)) arr1 = [arr1];
  if (!Array.isArray(arr2)) arr2 = [arr2];
  return arr1.concat(arr2);
}

Indeed @patrick @George if it’s just for that that’s the perfect solution but it can become cumbersome at some points to create formatters and modules for each single small thing. Specially if there are infinite ways of using a function(think .map())

I was thinking about actually creating a feature request to create a very advance nodejs module that allows running any type of javascript while being able to parse SC expressions. All this from within secured sandbox.

Sometimes you just need to quickly map an array with a custom function or you want to loop through something without having to go through the hassle of creating a custom module/formatter and the hjson that is sometimes very rigid as to what it accepts as options.

What do you think? Would you guys be open to create such custom action by creating a new parser function that can identify what are SC bindings and what is vanilla JS code and execute it in a safe sandbox(i.e. vm2)?

I believe the great majority of low-code solutions out there have some sort of component that allows the end users to run custom JS and inserting their own bindings and I have to assume they do this by using a sandbox.

2 Likes

I can relate to this. In one of my projects, I have around 7 formatters, and most of them are just one single JS function - but I must admit - all of them have some custom tweak.

Not sure how the proposed solution will work, but it sounds like a good tool to have in Wappler’s arsenal.

1 Like

Check:

What about just passing an Object to the custom module and the snippet textarea, and then return the Object modified? Sounds like a good compromise between passing SC variables to inside the sandbox and getting them back. With an Object we can pass practically anything we want without worrying about token parsing and stuff, what do you think?

A Wappler bonus would be to have syntax highlighting in such module textarea

I don’t quite get you? What does the object contain?

Assuming the array concat example, you create this object in a Server Action:

// obj
{
    arr1: [1,2], 
    arr2: [3,4]
}

Module user input variable name:

obj

Module user input snippet (textarea):

return obj.arr1.concat(obj.arr2); // returns an array [1, 2, 3, 4]

The module returns a variable (in this case, an array). It could also return an Object or whatever you need

I guess we could even expand whatever inside obj to the global scope within the sandbox:

return arr1.concat(arr2);

Yes, but why would I want to perform that action in SC?

Isn’t that the equivalent of creating a Set value action in SC and assigning it a [1,2,3,4] value directly?

If the values are static and known to us we don’t need to execute any javascript to return the final result, right?

A different way of putting it. The purpose of this would be to run custom JS code that interacts with the rest of SC, specially the bindings that include dynamic data, right? Like data that comes from a DB, an API, a SC action, etc

Yes! I thought we were on the same page. I’m referring to the module you’ve created where you run the sandbox. The point is an easier way to pass dynamic data to inside the sandbox, and get it back to Wappler, avoiding the whole token parse thing

1 Like

Ok. I got the feeling that you might be missing some piece from the whole SC puzzle and I am not doing a great job at explaining which one it is.

Every SC flow is just one JSON file that contains information that will be read and processed by the SC framework. So when you add data to any type of field in a SC action you are just adding data to the json file. This means that when we use the “Run custom JS” step action and fill the code field you are adding the expression that will be evaluated, and not the data itself(unless it’s static).

But if the two arrays that I am concatenating come from a dynamic source(i.e an API) it will store in the json file two expressions in moustache notation {{this}} and {{that}}. SC will then read the json file and when it gets to that part it will evaluate those expressions and for that it uses a parser which in turn uses the lexer.

Don’t get me wrong. I understand your approach by separating data from the JS string and I’ll iterate on that idea tomorrow but I think it’s not as easy as it seems because eventually you need to parse the binding.

Edit: On second thoughts you might be on to something :slight_smile: I’ll spare some time tomorrow for sure.

1 Like

I couldn’t wait. I had to try your approach. I also remembered about the grid control. It’s perfect for this.

image

image

image

Great suggestion buddy!

1 Like

Yes.

You can either create a specific custom formatter as suggested by patrick being the most elegant solution for Wappler.

exports.concat = function(arr1, arr2) {
  if (!Array.isArray(arr1)) arr1 = [arr1];
  if (!Array.isArray(arr2)) arr2 = [arr2];
  return arr1.concat(arr2);
}

Or you can go with the RunJS custom module which will achieve exactly the same by running custom JS but implies running an additional step. With the formatter you can just chain the function where you need it.

Both solutions are more performant than the workaround you found out.

1 Like

Just test it, that’s really awesome and easy to use, thank you very much!
It definitely will help me a lot in work and save a lot of time and effort.

Why has nobody made this tool before? :slight_smile:

1 Like