🦬 Bull Queues for Node

Wappler Extensions
#47

I think with the addition by @tbvgl of environment variables to specify the Redis connection, it can be anywhere

1 Like
#48

It works with remote docker. You can use ENV variables for the connection or the connection string in server connect. I’ve added parsing for credentials like pass and user from Wappler globals, so if you define an external connection inside server connect, then you are good to go. If you use ENV variables, then that will overwrite the global settings.

2 Likes
#49

Thanks @mebeingken . This is very useful for background processes.

I followed the steps however I’m having this error after adding a job and creating a queue:
Error loading process file /opt/node_app/extensions/server_connect/modules/bull_processor.js. Parser.parseValue is not a function

image
image933×627 53.2 KB

What did I miss?

#50

Hey Kevin,

It looks like you are attempting to do a library file processor. Try instead to setup the queue as an API type and use the Add Job API action.

It has been a while since I have used the library method other than a quick test, but I use the api style all the time.

Let me know how it goes.

#51

It works now using Add Job API
image

Thanks @mebeingken

1 Like
#52

Just a quick heads up: I’m refactoring the full extension and rewriting much of the code in preparation for the release of the new major version 2.0.

This is a major version because there will be breaking changes. The release is expected to be in the coming week and has many interesting new features.

So if you want to prepare for version 2.0, then here you go:

  1. Library jobs will be depreciated. So if you currently use library jobs, you must replace them with API jobs before upgrading.

  2. Logging is entirely outsourced to https://community.wappler.io/t/advanced-logger-extension/.
    So this extension will be required for v 2.0. This will allow you to configure logging globally instead of on a per-extension basis and allow us to implement a lot of logging into the bull extension.

4 Likes
#53

Isn’t using library jobs better than API? I have not run any performance tests, but API calls would take a few more ms than executing library actions, no?

#54

The API add job action was born from the library action not working 100% of the time. And then people started reporting trouble making the library action work, so we feel it is best to deprecate it.

If somebody wants to refactor that action so it works all the time, I’m happy to bring it back.

1 Like
#55

Also just a heads up, we will be bringing more extensibility improvements in Wappler this week.

Server Connect extensions can then be also packaged and published to NPM (or directly from github) and automatically installed in Wappler. So no longer there will be a need for manual file copy.

3 Likes
#56

I got the library actions to work, but there is no upside to using them for bull jobs. The only speed difference was due toAxioss calling the app URL, so DNS needed to be resolved.

Wappler has a bunch of middleware processing requests. These are helpful, so API routes are better than library actions for our use case.

I’m modifying the processor to use localhost if in a docker environment. That will get rid of the delay in resolving DNS and guarantees the job runs on the current node if you run a swarm or kubernetes.

2 Likes
#57

I did think of that, but did not have enough time to test it out on the said project. Definitely better to have.

I don’t use Wappler’s docker deployment. Are you planning to use some Wappler reference to identify… or an ENV variable?

#58

I’m using an ENV variable so that even non docker users can use this approach if their app supports it.

#59

nvm I got it to work by using the app directly:

const app = new App({ method: `POST`, body: jobData, session: {}, cookies: {}, signedCookies: {}, query: {}, headers: {} });
            const actionFile = await fs.readJSON(`app/api/${action}.json`);

            await app.define(actionFile, true);

This will allow us to support get requests, sessions etc as well in a later version and we don’t need to worry about localhost anymore :smiley:

1 Like
#60

To change from lib to action, just create the action and set params instead of GET or POST?

#61

Yeah, but libs work a little different under the hood and I still don’t see a single good reason to support them for bull jobs.

1 Like
#62

It appears that the newly updated Redis modules in 5.8 have broken this extension functionality.
Queues are not getting created, nor any errors returned

#63

Can somone have a look into this and confirm if they too are facing the same issue

#64

@Roney_Dsilva Are you having issues with this version or 🦬 Bull Queues for Node Version 2 ?

#65

Hi @tbvgl, Am having issues with this version, will check with your version as well.

#66

@tbvgl, Your version works fine. Its not affected by the upgrade.

1 Like