Dynamic JSON APi Queries

Wappler General Need Help
1

Hi Wappler team,

So I’m at a complex part of my project we are transferring to Wappler.

Right now, we have an API that calls ElasticSearch. ES uses DSL Query language - so we need to send what appears to me quite complex nested JSON queries. This is all handled on the SC side - no API source etc. on the front end.

I have 29 + form fields which hold the users inputs, they have the ability to use multiple queries in each field, use all 29+ form fields, or just use some currently.

In our current build (built by dev team) when they use a field, the whole JSON section of that field with nested requirements is then added to the full JSON query dynamically.

So, if they only use one field, it’s a fairly short query, if they use 5 then the other 4 get added one at a time each time calling ES with the new query.

Here is an example single query:

GET /_search
{
  "query": {
    "bool": {
      "filter": {
        "script": {
          "script": {
            "source": "doc['num1'].value > params.param1",
            "lang": "painless",
            "params": {
              "param1": 5
            }
          }
        }
      }
    }
  }
}

I have the connection returning results from ES - but have not been able to figure out if it’s even possible to do this in Wappler as per our current setup.

@Teodor @George - sorry for tagging but I really need to understand if the above is even possible in Wappler, if it is - how I would do it and if it’s not - what would be a Wappler workaround? (without custom code).

On the SC side there is the query builder - but this appears mostly sql if i’m right - I thought to potentially add the JSON for each field in the value section but that caused an error. I have tried using the JSON type and entering the full (and working query) from our existing app - but it doesn’t cause any result changes.

Please help! I don’t want to go too far if it’s not possible.

2

Maybe @patrick can check this.

Are you using NodeJS for server connect?

3

Thanks @George - yep using NodeJs on Docker.

@patrick please let me know your thoughts, and if this is even possible?

4

Normally a GET request doesn’t have data in the body, our API action doesn’t support that currently. I can add it and post the update here for you to test. Updating the API would allow you to send the query and get results, but we don’t have a way to build that query in server connect. Building the query would require some custom code, I can help you with that when we know if the actual API call works.

1 Like
5

Yes please @patrick that would be amazing, thank you!!!

6

Here a first try, place the file in lib/modules.

api.zip (1.1 KB)

7

Hey @patrick

So updated this, the actual GET call with the JSON implemented has worked, and returned relevant results - what’s odd now though, is only some of the schema is displaying on the page.

E.g. some fields I pick with data picker return data in a repeating group fine, others from the same repeating source display nothing…

Initially when I first updated the api file, the API call that was working fine, still returned results but instead of the data that it had before, it was failing and just displaying 10’s of [objects]

8

Hey @patrick - just bumping this in case you didn’t see it

Would also appreciate the guidance around the custom code

9

Could you give a sample of what the api should return and what server connect returns.

10

Hi @patrick returning back to this today.

So what I need to be able to construct is a JSON section dynamically applid to the JSON query to ES via API Connect on the server side.

I have a basic static JSON query returning data after the latest update, but when I choose a field from the schema I’m getting odd responses.

E.g. {{_source.summaries}} works fine.

But when I use >{{_source.emails}} - I just get a string of ‘objects’ e.g.:
Screen Shot 2020-09-02 at 8.10.57 am

And then some other fields return nothing e.g.

{{_source.names[0].first_name+’ '+_source.names[0].last_name}}

Can you help me to understand how/if this is possible in Wappler.

I am querying ElasticSearch service via an API. To query ES I need to query in JSON.
I have over 29 filters, and each time a new filter is used, it should add that section of required JSON to the total JSON query.

Then next, if the above is a bug or if I am doing something wrong to get incorrect results rendered.

11

Looks like the emails is an array with objects in them. You can try to debug the values in the console using dmx.parse('_source.emails'). When user the expression in the html it get converted to strings, object will then become [object Object]. You could also use {{json(_source.emails)}} in the html to get the output as a json string.

12

Where would I do this Patrick? I’m looking at all the formatting options, and cannot see a way to manipulate the ‘_source.emails’ item in the schema.

13

I have to admit, the biggest frustration I have right now - is just not knowing whether or not something Will or Will Not work within Wappler, and/or if there is simply a bug, e.g. the bugs that exist with the API with node.

I have no idea if this (dynamic query to ES), and for example the Google autocomplete API ‘should’ be working, if they are possible to work, or if it’s a bug.

14

Hi,

Did you find a solution to the dynamic query issue?

I’ve had the same requirement this week and have managed to construct complex EL queries in Server Connect. If you want any feedback on that let me know. Happy to help.

-John

image
image963×825 58.4 KB