26+ set values will not be a performance issue from what I understand. The repeats are usually very fast too.
But, it will definitely be a maintainability issue.
I would suggest to use a custom server side formatter here.
Something that would take list of fields to remove from supplied json object.
Thanks @sid.
@patrick any chance you can advise what this would look like as a custom formatter please?
I will take a look at it tomorrow at work, can you tell me what server model you are using (asp/php/node) and what the expected output should be.
I think you are almost there, with the set value steps in the repeater it should work, but can be lot of work if you have a lot of data to output.
You have your api call and query. Repeat PDTSearchMail.data.hits.hits and in the repeat use Set Value steps to output the desired values like {{_source.fullname}}. You can use conditional output in the set value like {{condition ? _source.price : null}}.
Alternative is a custom module/formatter, but for that I need more information about the data and the expected output.
Hey @patrick thank you so much!
Server model is NodeJS.
Our API Call returns over 26 data points, with about 6/7 of these are nested. Some are nested with values and some are nested with objects and values.
We want to control the output schema. One schema will have all data returned to the front end, and the other will have about 5 fields NOT sent to the front end.
We determine which schema should be sent to the front end per record based on a check in ou internal DB.
Each document returned by ES has a unique id, this is source.id . We store the id in our DB.
When a source.id matched the id in our DB we want to show the full schema. If a record does not have a match in the DB we do not want to show the limited schema.
The issue I have found here, is that I need to display the data in a repeat, so I need a parent array to display. If I use the set value steps, then I cannot select them all at once, so I need to chose the core repeat array. BUT, when I use pdtseach.hit.hit - the repeat holds the entire array schema output.
If I use pdtsearch.hit.hit._source then I can choose the output fields, but it shows the aforementioned error.
I was really optimistic with this, it seems to be the perfect solution. But if I use 26 SetValue’s for the data, then that doesn’t work in my repeat group as I need to select the one array as the expression for the repeat on the FE. Then the Repeat in SC actually holds the full schema. This would work perfectly if I could choose what fields the repeat will output and then use a condition and set the values of the information that should be shown in the true condition. But it’s not quite there due to not being able to choose what appears in the repeat.
I have sent you two messages, one with the preview response (how it is returned to us by ElasticSearch) and also the full JSON response of the actual data itself. I sent it in a private message as there is come confidential data within the schema.
Essentially, if you look at the data, IF a _source.id from the ES response matches the internal DB query then we send the full schema to the FE (ideally we can still choose which fields, to save resources).
IF a _source.id does NOT match the internal DB query then we send the full schema EXCLUDING: work_email
phone_numbers (array).
emails (array)
mobile_phone
profiles
linkedin_url
twitter_url
github_url
facebook_url
Noting that we are displaying at least 15 records per page, so some records will match and some won’t - so we have to complete these actions per record.
What then happens, is the user can choose to reveal contact information. This then creates a new DB entry, and we re-run SC to go back through the process.
I have this all working with a FE condition, but the confidential info still appears, so we need to handle and process the data on the backend.
This is literally 2nd last task before we complete this complex project, and we can go to production in 1 to 2 weeks from completing this, so I REALLY REALLY appreciate your help on this @patrick
This is where I got to with sid’s advice, but came across the repeat issue which might stop this from working in it’s current form due to the above mentioned reasons.
This is all very useful… I think I’m having some similar issues with using nested repeats and trying to access nested arrays returned from an API.
@patrick if you’re still looking to this i had some info here and am still unable to properly access the value v which is nested in two arrays as per the below post.