I assume its there for the sake of simplicity.
You can just reuse it again as 1st source in the next step and add third collection as 2nd source.
Creating a custom extension to make this more powerful should not be a big task either.
Yes you are right. It can be used this way, but as you said, it would be much more useful to have multiple data sources.
You start with two data sources - and it creates a new merged data source that you can again merge with other data sources - so you can repeat it as many times as you wish.
You are just manipulating the data in various steps till you get it right. So adding/removing columns merging various source etc. - See it as a data transformation pipeline.
Yes @George
However, I thought it would be more comfortable to use the component 1 time in the UI. We already can reach the result we want in this way, there is no problem at all 
These new options are honestly going to make a massive difference in my life.
Having workflows where I may save data into my database for say products chosen in a shopping cart, and then need to get availability of those products from a third party API, and date options from a second API, and then pass all that data in a single step to a payment provider is currently a bit of a pain, while these tools are going to cut down multiple step procedures to far more streamlined ones.
Thank You Wappler
3 Likes
This is a great new ability, Join Results of 2 Data Collection.
It does make me curious about how all of you Developers that develop such products & order details apps, whether clothes or restaurant orders normally store the pertinent categorical data.
How often do you store the broad product category tables in a SEPARATE DATABASE from the Order Details tables?
I’ve copied JOIN queries from different Tables that all exist in the same database to another Results table in the same database or I’ve saved the results into another database altogether.
But I never had to query a table of products from a separate database than my query for the matching order details.
Do developers here have example stories of Querying more than ONE DATABASE source to make a Join for In common value tables?
1 Like
I don’t think it’s something that happens often.
It’s however often to use some external API or database and want to join it with some data coming from your database.
For this post Two data sources, one output
I was using data from a corporate database to retrieve real time job numbers, etc. but the core of the app used its own set of tables.
2 Likes
@NewMedia, it depends on you scenario. Take an example like this.
You have a travel company that sells boat tours, surfing trips, and volcano walks in a country like Hawaii, they are a full service operation that you can also book your flights through, as well as book the any of the 3 tours online.
The flight availability is held by the Airline, as well as the price of the flights, so you need a way to get that data, either by dealing with the single airline itself, or more commonly dealing with an API provider that already deals with all the airlines and stores all the information for you to access in their API.
In my example someone like Amadeus would hold all this information.
The tours themselves once you get to Hawaii may be operated by many third party companies, so lets say the surfing tours are operated by Company A, and the boat tours by company B.
You need a way to find the latest pricing as well as availability from each company, most of these companies may be smaller, self owned businesses and would not have their own APIs, however they would more than likely deal with a booking system, so lets say one uses Bookeo, another uses Rezdy, and a third uses Activitar.
Now the person who is booking through your website also needs a place to sleep while in Hawaii, so you may also sell accommodation at the same time, and again some accommodation could be a big hotel chain, and others may be a bed and breakfast, most of these companies would also not have their own APIs in place, however they would have some type of booking system, such as Siteminder with Booking Button or Nightsbridge. You again need their pricing and availability to make a booking.
Now on your own website, the person selects some dates to travel, they are presented some options, like do they want to do 1 tour only, or many tours, how many people are going, what are their ages, do they want to include flights, do they want to include accommodation.
If they chose your full service then you would need to pull the data from possibly 8 different API sources into a single source to pass to either PayPal or Stripe, and save all that data back to your database at the same time.
This is one example of where something like this can make your life far easier, not to even mention you could also add 2 more parameters to all this complexity, such as currency and language, there are just so many sources, that need to form a single transaction in the end of the day.
Example 2, which is less common, but there are instances I have found where a single company has more than one website, that both sell the same product, with slight variation, normally this is when one product is aimed at a local market, and the other is more international facing.
So if I were selling mobile phones in India, and that was my local country, I may be able to sell the handset as well as the data and airtime contract in a single package, but if I were selling the handset to a person in America, I could only sell the device itself and not the contract.
In such a case I would possibly have 2 websites.
The local website database would be the “Main” database, holding all the handset information and specifications as well as image links, this would also hold all the contract information needed.
The international site, does not need to replicate all the product data, it pulls it from the “Main” database, and ignores all the contractual tables, but uses it’s own local database to store it’s own data.
Both of these examples I have provided are real world examples that I have had to deal with on more than one occasion myself.
4 Likes
Amazing !
Gee, Paul, Hmmm— that all sounds so farfetched!
Thank you!
I have my newest app all prototyped now!
1 Like
Not farfetched at all, just take a look at Expedia, Amadeus, in fact most OTA’s (Online Travel Agents), and fortunately for all of us, pretty easily achievable with Wappler, even before this update.
2 Likes
Great feature indeed.
So, I was just checking this out and wanted to do a Dynamic Paging with it, but I didn’t dig deep in the forum search yet to see if anyone had talked about it yet so forgive me if it was already covered.
But how would one go about making the “Join Results of 2 Data Collections” work with the Bootstrap 4 Paging Generator.
As it Stands right now the “Join:set” does not return a Offset, Limit, Sort and Direction to apply Dynamic Paging.
( Wappler Documentation - Applying Dynamic Paging)
if one wants to list just a few record details from the join tables in a table repeat on a backend page for CMS users to skip through.
What I tried so far:
-
I turned two regular query, “Query Type” from Multiple Records into a Paged Records to try and get the Get values for Offset, Limit, Sort and Direction but that didn’t work because on the client side the “Join:set” data didn’t show up in my table repeat after I did that change, likewise for the dynamic paging there is no query with the Get values holding (Offset, Limit, Sort and Direction) to pick from.
-
I even try tried apply the Get value manually from the good old Get variable inside the “Input” so the Offset would show in the server connect parameter but that didn’t work because I still wasn’t able to pick it up from the Dynamic Paging “Data Source”
If this is not something that dynamic paging was meant to work with then I will just default to my old ways of regular table “Inner Joint.”
let me know. I am tagging these guys below as well just incase they ran into this later as well.
@psweb @mebeingken @Teodor @max_gb @ben
The paging generator expects you to use a paged query.
If you can use table joins in the query builder, then just use table joins. This feature is for a bit more different situation, where for example you are using two different sources (not necessarily db tables from the same database).
Good to know. I will delete the regular queries and start with paging query and see how it goes. Thanks @Teodor for the quick reply.
I’m beginning to realise the huge potential for this feature. What’s the performance impact? Is it going to require a lot of processing if you build it up using many different sources?
There is no performance impact, actually you might even gain performance if you filter data.
Just note that each data transformation ste duplicates the data and then manipulates it.
So if you have like 1 million records and do a couple of transforms, then you might need more memory.
But other than that you will probably don’t notice any difference in performance.
1 Like
That’s great to know. It would still be better to do joins in queries if everything is coming from the same database, though?
Yes of course as then the database will do the heavy lifting that it is optimized to do
2 Likes
Hi,
Is possible to apply condition in the JOIN Transformation?, I mean, if apply a WHERE condition in query of Collection 2 it doesn’t affect the generate table in UI because the condition only apply to the query of one collection and not to the entire JOIN with both collections.
1 Like