Redis Toolkit Extension - Collecting Feedback and Ideas

[tbvgl]

Hi all,

I’m working on a YouTube series for advanced Wappler workflows, as announced here: https://community.wappler.io/t/youtube-channel-for-advanced-wappler-workflows-extensions

I’m planning to release a couple of extensions alongside the videos. I’m a huge Redis fan, so one of them is a suite of Redis actions. I have already implemented the following:

And I’m planning to add a few more functions:

• Get all keys in a Redis database: This function can retrieve all the keys in a specific Redis database. The user can provide the database number as an input.

• Get all key-value pairs in a Redis database: This function can retrieve all the key-value pairs in a specific Redis database. The user can provide the database number as an input.

• Expire a key in Redis: This function can set an expiration time for a specific key in Redis. The user can provide the key, the expiration time (in seconds), and the database number as inputs.

• Rename a key in Redis: This function can be used to rename a key in Redis. The user can provide the old key name, the new key name, and the database number as inputs.

• Check if a key exists in Redis: This function can be used to check if a specific key exists in Redis. The user can provide the key and the database number as inputs.

The Redis connection can be defined in server connect or in your ENV.

Please let me know if there are any other Redis features you would like to see in this extension.

[Apple]

1. Support for custom increment/decrement (e.g.: INCRBY instead of INCR)
2. Get expiration TTL (TTL or PTTL)
3. Custom Redis command execution

What’s the difference between Insert JSON and Update JSON?

Looking good!

Edit: This is probably way more uncharted territory, but it would be great to have a Cache block where you could throw in any (e.g.) Database Query step(s) inside and the JSON of such block would be stored into Redis. I kind of hinted this here in the following links:

Content Caching for Node.js with Redis

I want to cache an API Action step (request to a third-party API), not the whole Server Action workflow. Is this possible? I would suggest creating a Cache block, where you could then place the API Action (or other steps) inside it. All return variables inside such block would be put in an Object and thrown to Redis. For next runs, the Cache block would pull the Object from Redis instead of running all inside steps. I would also recommend to add “Cache Time” and “Erase cache” button to the Cach…

(for the next link, don’t bother reading the whole technical details, read just the explanatory paragraphs)

Custom block to insert steps inside

Hello, Could I create a Server Connect extension/module like the Try/Catch? I’ve checked the Try/Catch code but I don’t know how to make the hjson. I see some pretty funny possibilities with modules that allow steps inside. Like building a Cache block or throwing everything into a Queue for later processing. Now, a question for the Wappler team: Method trycatch is executed, with the argument “options” being an object containing “try” and “catch” (see picture on right) Line 180, trycatch call…

In the end, George released the Group step

[tbvgl]

1. That’s already implemented with the current counter:
   
2. Good idea!
3. Good idea!

@Apple

What’s the difference between Insert JSON and Update JSON?

It lets you update values inside a JSON.

Let’s say you have this JSON:

{"myobject": {"apples":1,"bananas":2}}

And then you send this to the action:

{"myobject": {"bananas":10}}

Then it will update the JSON in Redis to:

{"myobject": {"apples":1,"bananas":10}}

@Apple

Edit: This is probably way more uncharted territory, but it would be great to have a Cache block where you could throw in any (e.g.) Database Query step(s) inside and the JSON of such block would be stored into Redis. I kind of hinted this here in the following links:

I didn’t check the links, but the insert action might do all the things you want since it automatically parses the JSON and it can read from values & files:

[tbvgl]

It’s getting a little out of hand @Apple

But it solves all your caching stuff and a lot more.

[sid]

Looks great @tbvgl. We very recently implemented Redis with Bull Queue for a project and have very little knowledge of Redis - so not using it for any SAs. I had posted a few questions some time ago trying to understand it a bit more, but that didn’t pan out. Maybe this extension and your videos will help.

[karh]

Awesome @tbvgl!!
Because of @Apple I am quite intrigued by Redis, it seems like it’s a great solution/improvement to certain issues.

However, just like Sid, barely using it because I barely understand it.

So I’m very excited for your YouTube series + extensions as it looks like you’re doing exactly what I need: someone to break it down, explain it and make it more accessible in Wappler.

Thank you!

[scalaris]

@Adetoyinbo1

[George]

Actually it will be also very useful to know in which cases you are going to use Redis and it is for simple json storage or more a db like scenarios?

Because we had some ideas to make the database connector, that is now more relation db based, to also work on doc based, NoSQL databases. In that cases collections will be threatened the same as tables in the relation world and queried upon.

[Elias]

In my case i am interested to have a vector DB for word embeddings, hence REDIS could be one option to do this

[tbvgl]

@George It depends on the project. I use Redis, for example, to store permissions when a user logs in at a project with a complex role editor or caches specific actions in a Wappler server connect workflow.

In another Wappler project, I use Dragonfly to parse and store 20k JSON objects and resync them every 10 seconds. These JSON objects contain known software vulnerabilities. I then parse all 20k objects against a request for software with versions. This comes close to what usually would be done in a relational database because each software has versions, and I need to parse if the current version is affected and if the name matches. Since a vulnerability affects multiple versions, I use Semver to parse all possible options and compare them against the database. The scan took 20-30 seconds with a relational database, and the sync took minutes. With Redis/Dragonfly, the sync takes 150ms, and a scan takes 80ms.

Making the database connector compatible with NOSQL would be a great addition to Wappler. I have a good friend who is a senior engineer at MongoDB in SF and would be happy to jump on a chat/call to help/give feedback.

PS: I sent you a dm about using the Wappler logo in my YouTube thumbnails. No hurry, but if you have a sec, please check.

@Elias, I’m planning to add vector functions to the extension as well.

[George]

@tbvgl

Making the database connector compatible with NOSQL would be a great addition to Wappler. I have a good friend who is a senior engineer at MongoDB in SF and would be happy to jump on a chat/call to help/give feedback.

No problem we already have the most work done as we did similar things for CounchDB connector.

The biggest challenge for NoSQL databases in Wappler is that we always need schema to populate the data pickers in Wappler. So you would always have to define tables (collections) in the database manager with structure first.

[tbvgl]

Sounds good! I’ll ping my friend from MongoDB and ask him if he has any ideas for automated schema generation.

But imo it would be fine to manually define the schema.

[Alexander_Paladines]

Hi,
I’m trying to implement a function that avoid users to login twice into backend, alowing to chose after the second login a modal with the options:

• Kill the first session and keep login in current session.
• Kill all sessions.

Right now the available option that I have it to check in database a boolean, 1=login, and in the server action on every login check this condition if = 1, then user interaction with above 2 options, else, normal secure login.
But all this no kill the first session or all possible open sessions, so, I think that using Redis make the proccess better, keeping in redis the securityId to verify current logins, and been able to delete that value from redis to kill session, is that right?

Maybe any update about the release of this extension?

Thanks in advances @tbvgl

[tbvgl]

The extension is available now Wappler Redis Toolkit