Intelligent Merge (MailMerge) custom server connect module (node)

Mailmerge type functionality has, in the past, required fixed .replace() actions.
Intelligent Merge takes this to the next step allowing matching against the query schema via markup style [tags] embedded within text.
This can form the basis of a simple email templating engine for example.

(NOTE this does not work with a single record query. For paged queries select the data object i.e. query.data).

take an example query output:

query: [
{
people_id: 1,
surname: "English",
forname: "Brian",
email: "brian@hyperbytes.co.uk"
},
{
people_id: 2,
surname: "Smith",
forname: "John",
email: "john@hyperbytes.co.uk"
}
],

Intelligent merge with select the required record, split the response into its keys and values into an array . (for example key="surname", value="English" for the specified record (e.g. record 0) (remember counting starts at 0, not 1)

Intelligent Merge will the scan the input document for any occurrence of that "key" enclosed within square brackets (e.g. [surname]) and substitute the corresponding value (e.g. "English")

The process is entirely automatic

Example later

The extension can be found at:

Install in the normal was as outlined in this post:

After installation don't forget to perform a full restart of Wappler.

Intelligent Merge can be found under File Management

Add the component and name it

image

Now add the document text. This can be static, from a dynamic source or $_ value sent from app connect
In this example i will simply use static text which for ease i will add to a set value stage for convenience.

image

Note the embedded field names within the text

** Note Do not start the text with "[" or it will fool the wappler parser and it will assume the input is an array leading to module failure.

I add this to the action.

image

I the define a record set using a standard query

image

this record set will contain the following data

I add this to the action

image

Lastly whe add the offset within the record set to merge with, this can be static or dynamic for example where used in a repeat, "$index"

In this simple example i will add merge to record offset 2 (Bill Jones)

I check the Output checkbox to output the result.

image

I run the extension in a browser to see the output.

I see the output of the text and the selected data record

{merge1 "Hi, Bill Jones - your record number is 3. All corresponsnce will be sent to bill@hyperbytes.co.uk"
},

This component can also be used within a repeat to merge multiple records

Notice the record offset is now set to $index, the value of the current iteration of the repeat.

I run the action in the browser again and see the output of the merged text at each stage of the repeat

{
repeat: [
{merge1: "Hi, Brian English - your record number is 1. All corresponsnce will be sent to brian@hyperbytes.co.uk"},
{merge1: "Hi, John Smith - your record number is 2. All corresponsnce will be sent to john@hyperbytes.co.uk"},
{merge1: "Hi, Bill Jones - your record number is 3. All corresponsnce will be sent to bill@hyperbytes.co.uk"},
{merge1: "Hi, Ken Wilson - your record number is 4. All corresponsnce will be sent to ken@hyperbytes.co.uk"},
{merge1: "Hi, Jim Walters - your record number is 5. All corresponsnce will be sent to jim@hyperbytes.co.uk"}
]
}

This extension is primarily targeted at Mailmerge however has many more uses such as personalising text in user areas by applying the merge to content used within a app connect page.

Simple Email templating can be achieved by using a form with a text area/ summernote area and setting the module document source to the sent $_POST value.

5 Likes

Excellent extension Brian, I'm just finishing off my SQL for my own mailer and aim to test this with a mailout soon - I'll report back any findings..

1 Like

I would suggest doing a test run with the actual email stage disabled and checking output in dev console to be sure

1 Like

I'm paranoid about this stuff Brian so will definitely be testing it a lot, after testing out, the initial mailout will be circa 600 with a later one planned for 2000 and a potential for 20,000+ at a later date. I'll report back any findings at every stage.

I have a feeling if it works OK, this extension will be really popular.

Am considering a futher module which can be used just to extract all the keys from a query.
This could then be used within a select or similatr mechanisn to insert the dynamic values to mitigate possible typos but got a few other things to do first

1 Like

Compared with the overhead if a send mail step
i think the merge action overhead will be insignificant.
As every field in the query if checked for merge, limiting the query fields to only the ones needed will optimise performance.

1 Like

I've just tested and used it and its an absolute doddle to use. I'm using it in a repeat with the message inside one of my standard email templates. It all works perfectly.

Thanks for that great work Brian :slight_smile:

1 Like

Hi all, can I kindly ask what's the difference between this and using query - repeat - action?
Sorry if it's something obvious, think I'm not seeing right/understanding it :frowning_face:

I guess you probably dont realise what this is capable of.

Just pass a query and a template with markup embedded to the function, it does a full mailmerge. No further user input required, no using .replace() etc. All matched automatically based on the query scema.

Sorry, or maybe i am not understanding you.

Do you mean difference between using this component outside or inside a repeat?

Thanks Brian,
For example, the mailer thing and the example from the other topic:

Hi [[Recipient]],
Just a quick email to let you know that [[Company]] needs to log in to update their country info, we currently have you listed in [[Country]] .......

If you use a query, a repeat, and then send mail step/api send step, what's the difference?

Like:

This can not be done with this?:

Yes, it can be done on the backend like that and that's a great way for fixed messages, however, you can’t do that in the front end with a user generated text.

What this allows is for the front end user to be able to compile a message that is injected with database fields, thus allowing a highly personalised message to be sent to each recipient.

This could be done by using .replace() server side function but that’s laborious, this extension is only a couple of clicks to set up and is very quick and easy to implement.

Once I’m finished setting my project up I’m doing a usage case on it to show it in action, this one will also use AI for help with compilation.

1 Like

Your technique works perfectly when used solely within the server connect environment. Unfortunately it does not work if the text is sent as a post variable from a content page which is the primary driver for this extension.

The module effectively simulates what you suggest by parsing a text string within server connect which has been passed from app connect. It is a simple server connect version of dmx.parse() type function in app connect.

you cant just do:


and expect the embedded {{ }} sections to parse

2 Likes

Ohhhh now I get it, what a nice tool!
Thanks for this, and both with @TMR for the patience and taking the time to explain me! :slight_smile:

2 Likes

Just an anecdotal update: I've just sent 2600 emails using this and there was not noticeable time penalty during the send. In fact, I'd say it was slightly faster than merging the message on the back end (something not possible for this application but worth noting anyway).

1 Like

This looks excellent @Hyperbytes. Any chance of a PHP version?

Although moving to Node is on most of our roadmaps, we still have historical but very active projects on PHP where rebuilding just isn't viable.

:wink:

Certainly wasnt in the game plan for me. Didnt plan to do any PHP extensions.

I will take a look, if it Is easy then i will see what i can do as a favour to you.

Javascript works so much better when using JSON than PHP.

3 Likes