Best way to query external data API when F/E has 29+ fields that can be used simultaneously

Wappler General Need Help
1

Hey all,

This complex part of our project is rearing it’s head this week - so need to get into it.

I was hoping to leverage the forum’s expertise to get started with the right Wappler components to execute this.

Basically, we have a significant dataset that we host on ElasticSearch in AWS. (over 300m records).

I want to connect to this via API - and return the results in a paginated list form.

We have over 29 filters currently (on our existing platform) that query this dataset, and when each one is chaned it automatically returns the revised set of results - so essentially all 29 filters can be used at the same time to get a really narrow result.

The data is returned from the ElasticSearch API in Json, which I’ll then map to each result. We also need to be cautious on how fast a user can add api calls - e.g. we have a list of checkboxes that dictate company size - what we currently do is wait until they have finished - so a slight delay on the call - and then make the call in case they are checking multiple quickly.

We then return and store the external ID’s of these results in ‘project’ folders if the user saves these - so they can then be called upon when they play with the project folders.

I notice there is: (and I am reading through the docs, this is just such a critical part of the platform - hoping to get additional guidance from this forum).

API Data Source
API Action - this ‘seems’ to be the way to go, for what I need as the element to use?
API Form
JSON Data Source

I’d also love some advice on whether or not it’s recommended to run these client side or server side? I see Wappler has the option for both.

I know this is a bit more complex than the usual request - so appreciative of any thoughts/feedback/guidance on this.

2

So far - I’ve decided to run the API action on the server side only.

Now I’m looking at how to make this work with so many queries via an API.

I’m assuming the following can handle this:

Query Manager

  • Add all 29 + filtering options as a query parameter by clicking ‘Define Query Params’ and entering these as variables.

Then, each filter is it’s own ‘form’ - and on submit, I set the new value to that respective Query Param in the Query Manager. Right after setting the value, I then tell the form to load the API again with the new parameters.

Would I be on the right path here? Are variables the appropriate items to use - considering each record that is returned is JSON data? Should I be using arrays instead of variables in the query manager?

3

Hey @TomD - I know you’ve set up really complex queries to an external call - wondering you wouldn’t mind sharing some guidance ?