Hi @George,
I’ve got to a point where I need to start offloading manual test so I’ve started learning and configuring cypress.
I am at the point of modifying .wappler/targets/Docker Local/docker-compose.yml to add it there and start fiddling with it.
As I know you guys had testing features in the roadmap I want to ask if there is anything coming in the short term in this front. I wouldn’t want to start writing tests if you already have something coming soon.
We don’t have the automated testing on the short term roadmap Jon.
It is also unclear yet the best way to implement it.
So it is great if you do some ground work to find out the best practices and we can implement those later.
So go ahead - dig in and keep us informed!
So far it looks good.
So cypress is based on mocha and a headless browser to perform the tests you write. You also get an electron app that will run your tests visually so you can see what’s happening. It currently supports electron browser, chrome, edge and firefox.
The docker version is very simple to run but if you want to use the electron app you need an X11 server so that the docker container can communicate with it and you can interact with the electron app from your desktop. It’s not ideal and prone to errors.
Eventually I reverted to the standard npm package that runs the electron app and that works perfectly.
Writing tests is quite easy and I totally see Wappler doing the heavy lifting. You just need to leverage the ability to interact with the elements that Wappler already knows. Users wouldn’t have a hard time with selectors. And then just translate the API to Wappler terminology for users.
In regards to UI the current approach that flow takes would probably suffice. So a few actions at the beginning to select from like Go to, select element(from DOM), click, type, assert, etc And then some additional depending of the chaining. Cypress allows chaining so Wappler could easily take advantage of that.
To summarize. I think that Wappler would just need to take care of the configuration files and translating a similar structure like in flows to a specific js file that can be interpreted by Cypress.
Then it just needs to allow running a test via npm/npx so an electron app opens and runs it for the user. That would cover the basics.
I think big part of the effort to write tests is interacting with the elements. And Wappler alredy does a great job at that. Add a similar UI as the with flows and users could be writing and running tests in no time.
Good idea about the flow-alike interface to define your tests! Seems indeed very logical and natural.
Ideally we would also like to have to whole test runner within Wappler and not use the Electron runner, but seems cypress doesn’t support NWJS as runner yet but we might be able to tweak it.
Maybe you can setup a simple test project to show how you have integrated it and we can start building from there? A simple github repo with a demo site will do just fine.
That would be very nice indeed and I first thought of that. But I kind of like the option of being able to have the test runner in a second display and Wappler on the main one. The test runner will automatically pick up changes on save and rerun the test for you. So it’s nice to see in my second display the rerun the moment you save the test file.
Maybe you could integrate it to keep branding consistency but run it in a second window of Wappler? That way we could keep the “test flow” window open and work on it while we see the test run in realtime. Great time saver.
Consider it done. I am currently writing tests for the whole register, login and recover password process. Once I finish it I will upload it to a new repository and add you as collaborators.
The more I learn cypress the more I love it.
For developers with no time to test manually(all of us? :D) a small investment of our time will save hours and days in the future. This is true for any automation test software but with Cypress you even save more time as it’s sooooo easy to use.
Thanks for this Jon!
Do I need to build anything into my design to aid the test process, or do I just apply it to an existing unchanged design?
It’s agnostic so no dependencies with your wappler project.
You just need to install an npm package in your project and write the tests. For practical purposes I have installed the package in my wappler project (/node_modules) and I am writing my tests also inside my project folder as I am using GIT and I want to source control these test files.
Here you have the code of two tests I’ve written. You can see how easy and readable the code is. The complexity doesn’t come from the language but from how thorough you want your tests to be.
I start with the simple things as Wappler already makes sure that we are not able to make too many mistakes. I will be iterating over my basic test files each time a bug is found. Also called regression testing.
context('When a password is typed', () => {
it('shows a strength indicator', () => {
cy.get('#inputpassword')
.type('password')
cy.get('#passwordstrength')
.should('be.visible')
});
});
context('When password has been breached', () => {
it('shows a popup with information', () => {
cy.get('#inputpassword')
.type('password')
.blur()
cy.get('#breachmodal')
.should('be.visible')
});
});
This is one queries my database to make sure a user has been created. It also makes use of an SMTP API service for testing purposes. It creates mailboxes via API and can query contents also. Intended for email validation purposes.
context('When form is submitted and all fields are filled correctly', () => {
it('creates a user in the database', () => {
cy.newEmailAddress().then(({ emailAddress }) => {
cy.get('#inputemail')
.type(emailAddress)
// assert that email address was entered
cy.get('#inputemail')
.should("contain.value", "@mailslurp.com")
cy.get('#inputpassword')
.type("fdsajh893ydsfh29")
cy.get('#checkterms')
.check({ force: true })
cy.get('#checkprivacy')
.check({ force: true })
cy.get('#registerAccount')
.submit()
cy.wait('@registerAccount')
.should('have.property', 'status', 200)
cy.task('queryDb', "SELECT email FROM mpc.user WHERE email = '" + emailAddress + "'")
.then(results => {
expect(results).to.have.lengthOf(1)
})
});
});
});
And for every test in this file I create some routes so that Cypress can wait on xhr calls before moving to the next step and I also will wait always for a translation to be downloaded from the server.
beforeEach(function () {
cy.server()
cy.route('POST', '/dmxConnect/api/account/registerAccount.php')
.as('registerAccount')
cy.route('GET', '/assets/locales/**')
.as('translation')
cy.visit('/register')
cy.wait('@translation')
.should('have.property', 'status', 200)
}
Thanks for the details Jon!
I’ll be lost in all the development for a couple of months, but will watch some videos about it in the meantime so I am up to speed as I need it.
Just finished my first e2e automatic test using Cypress for all my authentication process.
It includes:
Now I can safely move on to another feature knowing that if a Wappler update or myself break this part I will get a notification before deploying to production.
@JonL Thank you for all the info! Thanks to you I’ve set up cypress today and started to set up some basic tests.
Still confused about the examples they use to reset and seed the database so you don’t always have to login through the UI to test things that are behind the login.
But this is already great, thanks!
It is indeed a great tool. I am glad there is someone else using Cypress to run tests. As far as I know we are the only ones from the forum currently doing it.
to less manual testing and more building 
Sort of. It’s part of a few projects but as they are quite stable and don’t require new functionality its just there.
In my latest project I have a deadline that doesn’t allow me to integrate automated tests. But once I deliver the first work package I will squeeze cypress in again. It’s a time saver in the long run.
Well this shouldn’t be a problem anymore. I can see this working very well in a tab now. Maybe for 5.0
Unfortunately Cypress discontinued their Electron integration.
Note: At this time, Cypress is no longer supporting testing Electron.js applications, and are no longer publishing the binaries referenced in this post. Thank you for your interest, and check out our Roadmap for information on upcoming features from Cypress.
https://www.cypress.io/blog/2019/09/26/testing-electron-js-applications-using-cypress-alpha-release/
but who knows - maybe in the future.
Just having done a Ruby on Rails course I was wondering on this subject, too. Automating testing seems like a very powerful concept! Do you still make these tests with Cypress? Or doesn’t it work with Wappler 4, anymore?
Not focused on testing right now but my cypress test scripts still work.
Cypress is fantastic. Its use is not integrated into the editor so there is no change from version three to four.