What is the best approach for storing OAuth2 tokens?

Aah, sorry. I have (so far) created two SC Action Files: authorise and getInvoice.

authorise looks like this:

and getInvoice is like this:

It seems that I only need to include the Provider action (which was set up as linked so I can select the action from the dropdown just like I do with Security Provider) and then the API action. The authorise action is only in the ‘authorise’ SC file and I’ve yet to determine if that ever needs running again.

I probably should add the Security Provider to the ‘authorise’ SC action file so will likely do that.

Thanks @sitestreet, the API action you got there - apiGetInvoice, are you able to show that after masking data? Sorry for the trouble.

No problem. Here it is:

1 Like

Thanks bunch mate, glad it works for you… All the best!

I guess, we will have further convo on this when you reach dealing with refresh_token, i’m skeptical about it working automatically, as the refresh token is within the Authorizer & not the Provider…

It’s been a mega helpful collaboration today for me to get as far as I have.

One interesting thing…

I have a page which is showing the invoice data pulled in from Xero. I created in App Connect a Server Connect action. This is working fine but if the token has expired (it’s 1800 so should be 30 mins) then the data isn’t pulled in. That’s pretty much how I would have expected it.

BUT…

If I browse to the Server Connect action file (…dmxConnect/api/…) then it works fine and outputs all the JSON from Xero. If I then refresh the main page it works again. Very strange and I’m not sure why!

I now need to work out how I can open up a Xero login page (what is shown when the Provider is first used if there are no tokens) to refresh everything in this scenario. Not sure I’ve explained this very well :frowning:

[EDIT] It seems to be working and just needs another refresh so that explains why viewing the .php file directly works but the page doesn’t. So the connection to Xero seems to be holding out.

Ideally, I’d like a check to be made if the connection to Xero is fine and working. If it’s not, open the authorise script (…dmxConnect/api/…) in a new tab, let them go through the process and then close the tab. Or just redirect to it and then redirect back again. Or am I going about this the wrong way?

@sitestreet, don’t want to put you off reg refresh_token, it might be working as expected… haven’t tested it out yet.

Just read your message again, with regards to the flow, patrick’s suggestion & screenshots here might be helpful, check it out! As well as @psweb’s ones above!

1 Like

I’m making very good progress. The authorising is working, it’s taking me to the Xero login and then redirecting back once authenticated. I’m now working on the more detailed approach shown in this thread by @psweb. Is this a good summary of the workflow?

When logging in:

  • Check if Xero token is already in database
  • If it is…
    • Check if access token is still valid (compare the date/time field)
    • If it is…
      • Authenticate
    • If it is not…
      • Run API Post to refresh the token
      • Update tokens and expiry date/time in user record
      • Authenticate
  • If it is not…
    • Do initial authentication which redirects user to Xero login page
    • Save tokens and expiry date in user record

Yup pretty much looks like a brief run down on how i did it, the only thing is, and it may just be your wording compared to mine, but there is no Authenticate as such so more like this.

  • Check if Xero token is already in database
  • If it is…
    • Check if access token is still valid (compare the date/time field)
    • If it is…
      • Run your API actions with your existing stored access token
    • If it is not…
      • Run API Post to refresh the token
      • Update tokens and expiry date/time in user record
      • Run your API actions with your newly stored access token
  • If it is not…
    • Do initial authentication which redirects user to Xero login page
    • Save tokens and expiry date in user record

Only the 2 Authenticate lines were adjusted.

1 Like

Marvellous. Yes, I think my wording made it confusing.

Huge thanks again, it really is appreciated. I feel SO much more comfortable working with APIs and OAuth2 in Wappler just after spending today on it using your info and @Akayy’s.

1 Like

Yeah, I have to admit, although the API stuff is well documented, I had major confusion at first, probably took me about 4 days to start feeling pretty comfortable working in it, and now to be honest I don’t know why I was confused to begin with, haha, it somehow makes sense to me all of a sudden in the last 2 weeks.

I still have some questions looming in my head, biggest one being, if I only used the default “Session” API over the “Self Maintain” option, and the initial authentication retrieved the refresh token, and I never stored that token, then during that one browsing session the refresh tokens would work, but what if the user logged in 2 months later when that session was totally gone, and Google is not going to send a new refresh token even if you re authenticate, then how would it ever refresh on the next browser session 2 months later without knowing the refresh token.
Mind Blown, can’t figure that one out, which is why i stored everything in a database and opted for “Self Maintain” just could not understand, and still don’t.

2 Likes

Yep, that’s exactly how I’ve felt about major parts of Wappler. I struggle and struggle and then the penny drops and it’s easy.

1 Like

@psweb, I had a look at your workflow, and I don’t think you need the refresh step “apiGetRefreshToken”, the step that follows (calling the provider) will automatically check the expiry of the access_token and automatically use the refresh_token to fetch a new access_token. @sitestreet

You don’t really need the if else within the box:

Also, there might be a security flaw here… lets say, you login with ur email address, token fetched, persisted in the database… and you closed ur browser…

I, now, open my browser and used your email address, your flow will let me in with ur token, isn’t it? Sorry i could be wrong, Im not a techie, so might miss some fundamentals here!

I understand when token handling is using “Session”, automatic refreshing of token makes sense as it will be for the current user, but when token handling is using “Self-Maintain”, I don’t understand how this will work.

Yes, this script does not run on page load, I have an action scheduler on the dashboard that checks the expiry time every 5 minutes and if the expiry time is less than 5 minutes away from expiry then it runs the “apiGetRefreshToken” script to make sure the user remains logged in.

So my real workflow for this is

  • Login with email address (token-handling-login.php)
    • Check database for matching email address
    • Condition (if Email Exists)
      • THEN get refresh token, pass to self maintain non linked provider, update db record, redirect to dashboard.
      • ELSE use non linked session provider, authorize, insert db record, redirect to dashboard

The silly part with this script is it will get a refreshed access token even if they decide to go back to the login screen while they are still using a already valid access token and it will get a new one anyway. I can not see many people doing that but even if they did the one extra API call is not too terrible so I did not bother.

Now the user is redirected to the dashboard, the page auto runs a script to display some information.

  • Display Info (display-info.php)
    • Query database to get token
    • Add queried result to provider
    • API Action to retrieve profile information

While on this dashboard page the Action Scheduler is running a simple database query to check the expiry time vs the real time and when there is only 5 minutes left on the access token expiry it runs the refresh token script

  • Refresh token (token-handling-refresh.php)
    • Db query to get refresh token
    • API action refreshes the access token
    • Adds the new information into a “linked” self maintain provider
    • Updates the db record with the new information

And that is really it, the login runs, then displays info, and every 55 minutes refreshes the token.

1 Like

You are quite right, I am still in a very developmental stage of this right now, so in the real app, the user will first create an account with security, and then access their various Google accounts from within the secure area, where they can manage multiple gmail email accounts to remove duplication and merge 2 email accounts into one.

As I say, still early days, lots of stuff to work out when i finally put all the parts together.

1 Like

This is such an excellent thread and I’m sure will be really helpful to everyone grappling with OAuth2 and APIs generally. :slight_smile:

2 Likes

@Akayy and @sitestreet, just out of interest with the APIs you are both connecting to does the refresh token get sent more than once, i mean if you do another authorise then does it send a new or existing refresh token to be used.
If you would not mind checking with something like this added to your code i would be grateful. Just want to see whats stored inside the session.

<?php
	// Start the session
	session_start();

	// Show session variables
	print_r($_SESSION); 
?>

The reason I ask is this. As I have said before with Google OAuth, it sends a refresh token only once, and never ever again, you can use providers and authorize steps etc. but you never get a new refresh token.

So I did a test just using session storage. The Access token expires after 3600 seconds (1 hour)
I set a very simple login with an OAuth2 Provider and an OAuth2 Authorise step which redirects me to a simple dashboard page showing a little retrieved data from the API.

I then waited and every 10 minutes or so refreshed the dashboard page, after an hour I continued refreshing and saw the session was updated with a new access token and still had the same refresh token as before, i continued running this for 3 hours to fully test and all worked perfectly.

The I opened a new browser Safari instead of Chrome, and went through the same login process, however this time the Google Account was already authorized to use the App from the above Chrome test.
So it ran fine and gave me an access token only, the refresh token was empty, which is what i expected, I refreshed every 10 minutes again and it ran perfectly for the first hour, however when it needed to handle the auto refresh, it could not as it had no refresh token to send to get the new access token.

So my conclusion is that if you are using an API that only sends a refresh token once in a lifetime, then you would have to use a database to store that token, and you would have to self maintain it, or your access token refreshing will only ever work the first time the user works on the site, and if they use it a couple days later they are going to not see their data after an hour without having to authorize each hour.

1 Like

Hi @psweb. My thinking at the moment is that the refresh token given by Xero has no expiry so I am storing it in the database and will use it to get a new access token when the current one has expired (theirs has a 30 minute expiry time) but I’ve not yet implemented all that. I’m aiming to crack through it today so I’ll happily put your code in and confirm my findings later.

1 Like

I checked at my end, I get a new refresh_token every time authorize runs… haven’t tested auto-refresh yet, I haven’t got that far, just clear on the theory at the moment, will implement and test it out in the next day or two, will share my results.

What if you re-did this test back in chrome (may be in a separate incognito window)? Do you get only access token?

Even if the authorize didn’t send a refresh token, you would have persisted the first refresh_token for the user and feeding this in Provider settings as yours is self-maintain token handling, so refresh should automatically work.

1 Like

Yes, only the access token, just looked now, and the refresh token is no longer there.

Yes, that is exactly why I opted to use the self maintain, however this was just a test to see if I could use the session provider exclusively rather than any self maintain providers, which I cant, I am kind of forced to use self maintain providers with the way the Google OAuth API works.

So in my mind the basic difference is this
If your particular API you are using only provides a non expiring refresh token at birth and never ever again then you would have to use a database with self maintain provider.

If your particular API you are using provides an expiring or non expiring refresh token at birth as well as on authorize then you could probably skip the entire database and self maintain provider and use the simpler session provider.

2 Likes