Currently I run all my matches between talent and jobs on a separate ‘systems’ app. But, if a user updates their preferences then I want to update those matches in real time. The logic brainstorm is when/how.
As I want it to be in real time, then I don’t want to use an action scheduler.
I also don’t want to run a new matching process (as it’s resource heavy) with every change, considering they may make 2/3 changes if not more.
I thought of a flag using a var that changes from 0 to 1, any of the update APIs would change this.
I could then do an inline flow on any link click (e.g. back to /jobs) that checks if this is 0, or 1, if it is 1, then I run the matching process api.
However, this isn’t fool proof as if a user types in the URL, or goes ‘back’ then no link click happens and the matching process will not happen.
Another approach, perhaps I could use a Global with a condition on the page URL that would be impacted by new searches (e.g. /jobs) but then I’d need to change the f/e flag approach, and also i’ve found globals to be unreliable (currently).
I’ll continue working through the problem - just thought I’d chuck on here for the bigger brains that are around!
Currently i’m trying a page flow, with the var flag. On load of the page, the flow will auto run, which means I can do the check on specific, relevant pages but, the page is loading before the flow/match process finishes.
My pick? Action scheduler every minute to run the matches for talent/jobs that have changed (var is 1)
If you have lots of free time, you can look into using the Bull queue extension for NodeJS, make it run 3 seconds after the save, and only compute the match if the var is 1 (and make sure the queue only runs one job at a time, or otherwise lock so subsequent jobs for the same talent/job are skipped if one is already in-progress)
Do a mixture. Updates add a flag in the db that is cleared when the match is updated. Run the matching using a flow but have a server-side scheduler to catch any missed from click/ back/other errors
Hey Apple! I’m a serial startup founder and now with 2 young kids (below 4) = I have zero free time. This app is my new venture, however so it’s important to get things right, and plan for some scale, especially considering who i’m targeting.
I have tried to get Bull Q’s up and running (using the recent extensions, but found too many errors - noted, and provided to the developer). I am thinking about building my own version, to get it running as I believe the schedulers I’m using will eventually not be able to handle the volume.
I am looking for something a little more instantaneous, e.g. if a user makes a change and then immediately heads to /jobs, the page will have loaded the old jobs, and they won’t match the new preferences. I think I ‘may’ be on the right path, doing the check if the user visits a relevant page, I just haven’t nailed that yet.
I was thinking that if I went Globals, I’d add the flag to the DB instead of as a var - That’s an interesting idea, to run both the flow as well as a backend check to ensure it’s not missed, good idea thank you
I’d probably create a library action that checks for change flags and runs the matching if found. This would be added before listing the jobs in any SC API file that depends on matches
Nice one! I’m entrepreneur-minded, we can chit-chat sometime Hopefully someday I can make the life of Wapplers less miserable when it comes to hosting and scaling web applications.
Why don’t you just compute the matches when the user saves the new preferences? So this way you don’t complicate with checking what link the user clicked and stuff… Maybe I’m missing the feel of your UI, is there a save button at all? Or is it saved as soon as the user clicks a checkbox?
If you’re concerned about the saving speed, run a separate server-connect just to compute the match after the preferences (1st server connect) are saved
In the jobs page, if the user goes to it too fast (var is 1), show a message indicating the matches are still being computed, and to refresh in a few seconds… or use a fancy loader animation and keep re-fetching the jobs till the var is 0
thank you @bpj - this took me in the right direction - and so obvious now I think about it!
This is what I did:
For every update api, I also update the flag from 0 to 1. I use a library action for this that I add to the update apis with a condition to check the delete / update actually happens.
I then created a library action for the matching process. There are two pages that load the matches, the /home and /jobs - I’ve now added a query at the beginning of both of the API’s that load the jobs, to check for the flag. If it’s one, then it’ll first run the match, and then load the jobs as normal. At the end of the match process I then update the flag from 1 to 0.
This is working perfectly, I’ll re-assess if the load becomes too much (as the match is quite a complex custom sql) but this is great, so thank you.
I don’t have a ‘save’ action on the preferences, instead I have an intuitive interface with buttons for most options, that add/remove on each press. Thus, running a complex matching process each time would be too resource intensive. The goal was to avoid that, and only run the match if they navigate back to view the new matches - which I think I’ve now achieved quite well.
I do use spinners/show/hide for things like data loading, and when this goes live it’ll become much slower than in my local environment or on staging - so those will be important.
BullQ’s is something I want to move forward with, but i’ve put it on the TDL until I’m close to live, and i’ll work on building my own implementation.
Hopefully someday I can make the life of Wapplers less miserable when it comes to hosting and scaling web applications.