Remote Database Changes

Just wanted to pick up on a topic which has had a few people asking over the months on the community.

Maybe I missed it, so I am asking in the coffee lounge just incase.

Is there a reason why we can not use Database Manager with Remote targets, as an example for me personally I use Digital Ocean, and allow Wappler to create the Database for me most of the time remotely too, so its all Wappler, but when I try create a new table or anything on the remote target I get the notification: Making database changes is only allowed on local development targets.
So it leaves me using Navicat or whatever for that part of it.

I do understand I would ideally be doing everything locally and then applying my changes to my remote target, however depending on your workflow, I find it easier to just do it all remotely and have a single source to deal with.
Is there a possibility we could have an override in the preferences for people like me who understand I am doing it wrong, and still want to do it regardless.

1 Like

As a secondary side not to this, what is your own workflow. Would love to see how some users, especially those using Docker, find it easiest. Like.
Step 1: New Project.
Step 2: Setup local target as Docker, add a database.
Step 3: Work local, until your client wants to start seeing a few versions of what you doing, or you need the community to see a piece of code you stuck with and you now need your project remote.
Step 4: Create a remote target, with a database.
Step 5: Change target to remote. Push your entire local database to the remote, Deploy, test on remote.

Once the client now does all their checks and gives you a bunch of alterations and ideas.
Step 1. Change back to local target, make changes, edit database schema if needs be.
Step 2: Switch to remote again, apply database schema changes, deploy.

Just interested to see if thats your sort of normal workflow, if not using GIT as well in the procedure.

Never used Docker ā€¦ but I also never use the Database Manager. I do everything on the remote database with Navicat. I don;t trust Wappler not to overwrite/delete live data.

It took me a while to trust it too, but database manager has got a lot better and more powerful over the versions, I especially like the way it does nested table structures, makes it much easier in my opinion than creating all the foreign keys and indexes etc. directly in Navicat, and if you stick to the Wappler naming conventions then you can still make a manual table in navicat and Wappler will recognise it in their same nested table structure.

For me, my issue is often I work on projects where the client is super involved, therefore I have a staging remote server as well as a live remote server, I work entirely from the remote staging one, and when the client is happy we roll out the changes to the live remote.
The issue with this is that both are still remote and therefore I am not able to adjust with database manager easily.

@sid works similar, but with far more GIT than myself, I am still a real GIT newby. Sid do you still cheat the docker compose file or something to get around this.

For what itā€™s worth, I can make changes to a remote DB hosted on a Linode Managed DB cluster using the Wappler DB manager.

I imagine the difference is that that particular DB i make changes on is set up as a ā€˜staging targetā€™ i believe, or possibly Dev targetā€¦but i can definitely do it.

I imagine you could always make the changes in a dev environment in DB manager and then use then ā€˜Applyā€™ the changes in the production manager using the Wappler DB manager

1 Like

Thank Philip, thats interesting indeed, is this on Wappler 5?
I just tried switching the Remote Staging Server target from Development, to Staging, as well as Production, and none of the Usage modes seem to allow me to edit inside database manager.

I just tried now in Wappler 5 (only upgraded today) and it still worked.

Iā€™m in my Dev target, which is Docker, but i have an additional (apart from the standard local/docker DB connection) direct DB connection to a Linode Managed DB and iā€™ve set up a few tables in there.

1 Like

Okā€¦so maybe my memory is wrong, perhaps iā€™ve only ever made the changes in a Dev targetā€¦iā€™d need to test on the staging target i have set up

This one? If so what is the question?

@Jon, question is, is there a way to use Wapplers Built in Database Manager with a remote database, instead of just local.

Data: yes
Schema: nopity nope and forā€¦wellā€¦reasons. If remote is not dev that is.

So even though mine is a remote server, but its a staging server, set to usage of staging in the project settings, i still canā€™t, damn. Surely it would be best then to allow db schema editing for those that are set to Staging or Development usage, I understand if set to Production, probably not a great plan.

Part 2 of my question then is, what is your workflow for new projects, do you really work totally local until production and then go live?

If you change the schema of a staging or production database you will encounter issues when you do a change in dev and try to perform a migration.

If itā€™s a personal project I go with the winds. Whatever feels right at that specific moment. Sometimes I end up with a mess, others I save a lot of time by improvising.

If itā€™s a well defined project for others I stick to what is asked even if it sucks balls big time.

1 Like

I donā€™t use Wappler DB Manager at all, except to right-click and ā€œrefresh schemaā€ on tables so I can peacefully build queries using Wappler query builder, after I have made changes to the DB schema using DBeaver.

As for editing the DB schema from Wappler DB Manager, I tried it just onceā€¦ a few months ago, and it was a nightmare. At that time I just had Dev target.
But as Philip suggests, it should work with dev targets. I donā€™t think it will work with targets marked as staging as staging is treated as prod as well.

I have seen many improvements in DB Manager since I last tried, along with nested tables, but the whole UX is very limiting for my flow, and I am still doing manual changes in both Dev & Prod DBs.

One thing that is great with Wapplerā€™s DB Manager is migrations - and if the latest list by Jonas about feature requests to implement is worked on by Wappler - option to write migrations manually will be great for our Git workflow.

1 Like

Iā€™m following this topic with interest, as this has been a difficult subject to wrap my brain around.

Could you expound on this, or link if there is a resource? TY!

Do you mean JonLā€™s list?

Canā€™t seem to find any docs. @Teodor would know if there are any for sure.

The idea is the any change you make with Wapplerā€™s DB Manager to the schema, a migration file is mantained by Wappler locally and using a table in the connected DB.
I donā€™t have much knowledge about what exactly is inside the migration data, but the flow is that when you switch to a production target, you can just ā€œapply changesā€, and all changes made in Dev so far, will get applied automatically - keeping both DBs in sync - Schema-wise - always.

Yes.

1 Like

The idea of the database manager workflow is the following:

  • You have a development target, on a development server, with a development database. You work on your development database and do everything there.
  • You have a production target, where your live version of the site/app is. Thereā€™s also the production/live database which stores your real data and structure.

You use the development target to add new stuff, pages, change database structure or data. Then you can test and verify it is working fine - as you donā€™t want to break anything your live site/app testing there ā€¦
After you have tested and verified everything is working fine on the dev target, you publish your files to the production target. The same way after you verified everything is fine with your database structure changes you switch to remote target and apply the changes there.

1 Like

Thanks @Teodor, that all makes perfect sense to me, and thats how i use it currently, but in my situation, there is one difference, my development target is also going to a remote docker container hosted with Digital Ocean, and is not just on localhost.
Because it is set to remote & development, database manager will not let me use it like a local & development target.
I am hoping the database manager can rather be restricted to the usage parameter of development vs staging vs production, rather than it is currently, where it restricts based upon local or remote.

2 Likes

Wait what? It should be restricted by environment flag(development, staging, production) not if it itā€™s remote or local.

Is this not the case?

Thats what it seems to be to me @JonL, my development is already set to a usage of Development, but its a remote machine, and the database manager just gives a notification saying I can not edit a remote database, only local.
Would you mind testing to confirm? But I am 99.9% sure.