Custom middleware(routing) to discriminate URL parameters

======== TEMPLATE BUG FORM ========

Wappler Version : 3.2.1
Operating System : Mac
Project: NodeJS, MySql, Docker.

Expected behavior

What do you think should happen?

When a user visits the URL - they should be shown the correct page.

Actual behavior

What actually happens?

An incorrect page is shown, even when the route is setup correctly.

URL Path:

Routing Panel:
Screen Shot 2020-09-01 at 8.34.55 am

route meta on the page in Wappler:

Rendered path in the source code of the incorrect page:

The only way I have found to be able to fix this, is to copy the information in the route in ‘Routes’ and then recreate the same again.

But then other pages incorrectly display.

I have about 90 pages that are dynamic and use the route path (url parameters) to generate all page data, so this is pretty important.

No errors are displayed so I don’t know how to debug this myself @patrick.

1 Like

Do you have a route with path /:service/:function/:country/:state/:city and was this route above the other route? Then it is not a bug, it will return the first route that it matches, and that route matches the url.

I do yes, but it’s a totally different path and page.

One is /staffing-agencies/:service/:country/:state/:city

The other that you mention

:service/:function/:country/:state/:city

A completely different page and page path is chosen

But /staffing-agencies/ matches /:service/ since :service is a parameter and matches everything until the next /. It sees the staffing-agencies as service parameter.

/staffing-agencies/ is not a parameter on the URL, the first parameter is /:service/

These are two different pages, one has /staffing-agencies/ as the beginning of the URL and the other does not.

I have pages with multiple parameters that need to return the appropriate page, not just the first parameter.

You can see the route below, ‘staffing-agencies’ is not a parameter, and so the rendering page should at least load /staffing-agencies/ but it doesn’t, instead it’s loading the route below:
Screen Shot 2020-09-01 at 9.13.29 am

Screen Shot 2020-09-01 at 9.14.34 am

You can see the page it should load is also very different:

Screen Shot 2020-09-01 at 9.16.19 am

Screen Shot 2020-09-01 at 9.16.43 am

In what order are the routes? The route paths are being parsed on the server when it receives a request and then checks the url which page it should load by the first route it matches. The meta tag on the page does nothing else then getting the parameters out of the url, you will then see the service parameter will contain staffing-agency.

If I have 2 routes like:
/:firstname/:lastname pointing to the page user
/detail/:product pointing to the page product

I visit url /john/doe then it will load the page user, the parameters will be firstname=john and lastname=doe.

Now I visit the url /detail/wappler, I expect it to load the page product, but it doesn’t. It loads also the page user, the parameters will be firstname=detail and lastname=wappler.

Changing the order of the 2 routes fixes the problem since the product route is more specific then the user route. The parameters match all, it becomes a regex like /\/([^/]*)\/([^/]*)/.

Hey @patrick - ok, so a little concerned about this - so I appreciate your help here.

How do I identify the order of routes? What is the best way to ensure that the correct page is being loaded based on the url path?

I have about 90 pages all with route parameters, each one loads a unique (or should) page that the SC loads specific data based on the page url.

The order of the routes is simply the order you see them in the routing panel from top to bottom.

So make sure the more concrete routes are on top.

Also as a good tip - try to make your routes with unique prefix describing their purpose. Like /product/id

Don’t use too much similar parameters in the same order without having unique prefix as you will have multiple routes that match

Check this guide for naming conventions:
https://restfulapi.net/resource-naming/

And maybe some additional information about routes:

https://expressjs.com/en/guide/routing.html

Thank you @George - I’ve checked those out just now. I believed that I have done this correctly, a per this section ’ Use forward slash (/) to indicate hierarchical relationships’ on https://restfulapi.net/resource-naming/

But if it’s not possible to have a completely dynamic directory based around the url parameters (and in nearly all cases multiple parameters) i’m a bit lost with all the work done on this directory…

Here is an example of my structure, I’m not sure how I would sort what is or what is not concrete as they all play a role in the directory.

This to me looks relatively correct and each route points to it’s own uniquely built page.

Can you confirm if what I am trying to achieve is in fact not possible within Wappler only?

This actually is a general topic how routes work, not really Wappler specific.

In your case however you have a lot of ambiguous routes because you have fully parametrized all route segments!

So for example:

/:service/:country
And
/:service/:function

Are two exact same routes with two parameters, so your website does not know which one to call as they are both expecting two general parameters, so only the first will be used always.

It doesn’t matter that you named the variables differently - they are just names for placeholders. Essentially the two routes in that case are exactly the same - just a arbitrary path with two sediments of any kind…

So you should really make your routes much more strict by specifying static parts so that there is clear resolution.

:frowning: I’m really not sure how to do this then. We are transferring an existing site that uses this exact URL path for it’s dynamic directory - so the URL will always change.

I can’t make the path stricter as the url path is made up of all those variables.

Looks like this isn’t a bug as Patrick mentioned, so you can remove it - just looks like I may not be able to achieve what I need to.

If the site is using the exact same path it’s because one path part is fixed.

/:a/:b is the same as /:c/:d

This means that if the other site has

/football/spain
/football/england
/football/italy
/basketball/us
/basketball/italy
/basketball/greece
…
/tennis/switzerland
/tennis/sweden
/tennis/argentina

You can be sure that the routes are
/football/:country
/basketball/:country
…
/tennis/:country

And not /:sport/:country

Why don’t you do something like this or any other route that leaves no ambiguity.
/service/:service/country/:country
/s/:service/c/:country
/s/:service/f/:function

Jon

1 Like

Hey JonL

Unfortunately that would mean changing the URL paths, which is what we I’m trying really hard to avoid.

There are so many variations, I can’t see there only being one parameter on each path.

We have Industry, Function, Services, City, State, Country - so the most complex full path could have service/industry/function/country/state/city

There are over 140 industries, thousands of functions, only 8 services, and thousands and thousands of cities/states/countries globally.

This would surely mean creating a completely unmanageable amount of routes if we had fixed elements for each?

A couple of examples of sites that have large directories are:
https://www.avvo.com/all-lawyers/ca/bakersfield.html

and https://www.upcounsel.com/business-attorneys-california

the url paths change based on the type of attorney, and the location be it state or city.

It wouldn’t change the amount of routes you implement.
You are just “prepending” a path part so that the routing engine is capable of disambiguating.

/:service/:function is the same as /:service/:country

But /s/:service/f/:function is not the same as /s/:service/c/:country and you only added two fixed path parts to the above routes. Those fixed path parts won’t change ever and are enough for the routing engine to understand they are different.

Ok, I understand what you are suggesting now - thank you for clearing it up.

It does however mean that the url structures will change and therefore impact pre-existing traffic (considerable) volumes changing the url path - and this is 100% what I’m trying to avoid doing.

The two examples above don’t don’t appear to be doing this, and considering they would also have thousands of variables I can’t see it making sense to have fixed paths?

Pretty devastated if I can’t make this work somehow :frowning:

Don’t worry. There are always options :slight_smile:

You can leave one route called /:service/:cf and in the SC file just query against a table that contains all the possible countries or functions and if you get a match for a country or a function you know what other backend logic to perform. It’s just and If/else matter.

You can also define “redirect” routes in NodeJS with the last update. So you can redirect existing routes to new ones.

No sure if it will help in your case but it’s good to know :slight_smile:

Thanks @George but that won’t really help, we need to keep the same paths - however, that is helpful to know for other reasons.

@mgaussie That could be a good option too. Not actually using the redirect in the routing but the SC action step.

So you receive in /:service/:cf those hits that are processed by a specific SC file that queries database to check if it is a country or a function.

If country
redirect to /s/:service/c/:country
else
redirect to /s/:service/f/:function

That way you keep old links working and you start migrating traffic to two better constructed routes. Eventually you can remove /:service/:cf route when you see the traffic that comes through there is minimal.

Thank you Jon, I really appreciate your input here.

I am really trying to find a solution that does not result in changing the URL paths, short or long term.

I was looking at routing and using a ‘controller’ which I don’t fully understand or know if is at all possible in Wappler / NodeJS, but I believe that’s what is used on the existing project - so in Wappler terms that would be I guess taking the parameters from the URL, checking the DB and then telling the F/E what to show, so perhaps a single page and a combination of partials (noting they used PHP and a Symfony templating engine).

Worst case scenario for us is changing the url structure, so trying to find a solution that avoids that as we likely already have to do that for the profile part of the pages (the old team use an underscore in the url path for the persons id, but this is also causing issues in wappler so will likely need to move to a hyphen).

I’d be happy to share the existing project with you in private message JonL as a fyi, so you can see what I’m referring to.