I’m trying to understand our options for quickly evolving our solution towards a continuous deployment environment, at least to the continuous delivery step anyway, but having strong automated tests is critical for this to work. I don’t see any information here or in the Appfarm docs about creating these tests though.
Do any of you have experience building a CI/CD pipeline with Appfarm, or even external test suites that could test solutions after deployment? Or is everyone relying 100% on manual testing for now?
Hi Matt! We are researching this topic at the moment internally, but we are in a very early phase, so it’s hard to provide a meaningful estimate or roadmap.
Many projects rely on Appfarm’s testing regarding the components’ inner workings, but the functionalities within solutions are still mostly tested manually.
I believe some attempts at test automation have been made on a couple of projects though, with tools like TestProject. Antoine, would you be able to share your experiences with this approach? @lequeuxa
Yes automatic testing would be great in Appfarm but indeed, quite difficult to implement.
What we did at ControlC is to create log table in which we create records at each critical step so we know what is happening.
We also sometimes test manually our solutions (having somehow built test scenario’s that we run manually through the user interface).
We tried to find other solutions but manual testing remained the easiest approach and we did not spend much time on this.
I did not know TestProject which seems very interesting.
I used to build Rspec first like you probably did but definitely think TDD is difficult to implement in Appfarm solutions, it could maybe be done for critical actions etc.
We do indeed have 2 Rspec enthusiasts on the team researching the topic, myself included
We’ll definitely keep you posted when we have something more concrete to share.
Also interested in this topic. Any news about automated testing? I’m researching Playwright for end-to-end testing at least. As our solution grows, manual testing is a pain.
Playwright is pretty nice, and is also the tool we use internally for e2e regression testing for the client.
Some things you should know if you want to use Playwright to test Appfarm apps:
Enable Password Login in the Environment you want to test in and read Playwrights documentation on re-using the authentication state: Authentication | Playwright. This is required if you want to test an authenticated app. I can share some more details on our setup here if needed
You can define custom attributes on Containers in Appfarm. Playwright uses data-testid for finding elements on the page. Recommend you use those. Go to UiContainer > Custom Attributes and add an attribute with the name data-testid and a value that you want to use in the tests. Makes tests less flaky.
You want to increase the default navigation and test timeout in your Playwright Config, since Appfarm apps might take longer to load than regular websites.
We don’t have an built-in automated testing framework, so at the moment Playwright might be your best option for this.
but definitely think TDD is difficult to implement in Appfarm solutions, it could maybe be done for critical actions etc.