RabbitMQ Queues for NodeJS

Wappler Extensions
#1

This custom module for server connect allows to send out RabbiMQ Queues.
This comes in handy when you want to issue a queue that is consumed as a trigger by a third-party API eg: those created in n8n.

Requirements

  • Functioning RabbitMQ instance, with hostname specified within the Wappler server config. uses the default port.
  • Wappler project using Node server model

Installation

  • In your project folder, create /extensions/server_connect/modules (if it does not yet exist)
  • Unzip the source code into /extensions/server_connect/modules (2 files)
  • Refresh the Server Actions panel (restarting Wappler is also an option)
  • The required libraries will be installed automatically upon use and the next deployment
  • You should now have a RabbitMQ Queues group in your available actions list for server workflows

Actions

All actions require a queue name and hostname.

Add Job

  • Adds a job to a queue with Parmaters supplied

Example Use case to Trigger N8N workflows

image
image1692Ɨ527 129 KB

3 Likes
#2

Thanks for sharing!

I havenā€™t heard of RabbitMQ before, could you share an example of how you use that and this extension?

#3

hi @karh,
Sure.

Similar to Redis Bull Queue RabbitMQ is also a dedicated message broker, Mostly, RabbitMQ outperforms Redis and guarantees message delivery with the help of message durability and acknowledgments.

The module that I posted would come in handy when you have integration with external tools such as n8n, sometimes a complex functionality requires complex workflows which can be early constructed and maintained in workflow/API-building tools such as n8n, Node-RED, etc.

The use case is that we would sometimes need to Integrate wappler with such tools, having a RabbitMQ Integration would help you to pass on requests to backend Workflows built in those tools.
Consider for example a complex workflow wherein you do several checks against certain parameters, with conditional flows across which a request would pass through, although such a flow can be created in wappler, maintenance would become a headache when issues occur with the logic involved, asin a certain check fails, then you would need to go through each step checking input and outputs. In such cases, I prefer to have the flows built-in tools such as n8n wherein the troubleshooting/maintenance becomes really easy, you can visualize the data passing across the flow and easily determine where things went wrong.

For the Demo setup, you would require to have a RabbitMQ instance preferably in Docker
ex: docker run -d --hostname rabbit --name rabbit rabbitmq:latest

Join it on the same docker network that wappler is running on.
Have n8n or any other tool that you wish to use connected on the same docker network
ex:docker run -d --name n8n -p 5678:5678 n8nio/n8n

access the WebUI on http://localhost:5678 add RabbitMQ trigger node in n8n flow.

Now create a API in wappler and add the ā€œAdd jobā€ step from the RabbitMQ Queues group.
add hostname: rabbit, and any queue name you wish to use ex: test

In n8n flow add the same queue name for the trigger, and execute the flow in n8n, it will start listening for test requests.

Run the wappler api that you created, and the request would get received on n8n side.

For Wappler Internal use cases I would recommend using šŸ¦¬ Bull Queues for Node by @mebeingken

#4

This is a great write-up! Appreciate that you took the time for such an extensive reply.
I am personally not working with n8n or Node-RED, but Iā€™m wondering if RabbitMQ (or Redis bull queues) might have a use case for another challenge Iā€™m facing: Iā€™m catching a ā€˜payment completeā€™ webhook from Stripe. And this server action then does what youā€™re saying: itā€™s a complex workflow that does a lot of steps based on conditions.
Sometimes a step fails, and it then also stops executing the rest of the steps.

Right now I donā€™t have a place where I can:

  1. See which ones failed
  2. Debug where they failed, so I can fix it
  3. Replay that server action

I decided to put Hookdeck between Stripe and our serverā€¦ but now Iā€™m curious if RabbitMQ would be a good solution for this. Would you please share your thoughts?

#5

Queues are great to hand-off long running tasks, tasks that might need to be retried, etc. So yes, handing the process off to a queue might make sense for you. If you have RabbitMQ installed and operational already, use it. If not, bull queues is self-contained in the extension, and thus probably easier to get you going.

1 Like
#6

Oh thanks for the input. I tried your extension a while ago to get a feel for it, but didnā€™t grasp it yet fully. And didnā€™t feel a need for it. Will play around with it again :smiley:

#7

Updated Link