Automated vs manual UI testing?
How do you choose whether to use automated UI testing vs manual UI testing for a new web application being developed for the first time? For automated UI testing I am talking about using test automation frameworks such as Selenium in which you write code to simulate user interaction with the GUI (user clicks and data entry). Should you use automated testing from release 1 when the product GUI and features are not yet finalized and users may ask for UI changes? Or do you do manual testing for the first few releases until the product is stable and then do automated testing? Any good practices or guidelines for when and how to make that choice?
Discussion
(2 comments)- 3 months agoSenior Software Engineer [L5]
There are a few questions you can ask yourself to inform your decision better:
- How many users do you currently have?
- How many features do you currently have?
- What is the impact if anything breaks?
If there aren't a lot of users using your web app, then you should focus your efforts on either building out critical features or acquiring new users.
If there aren't a lot of features to test, then it should be quick to make sure you aren't accidentally breaking any existing features when you commit new code.
If the impact when something breaks is not severe, then you can tolerate some bugs if it means you are pushing the business forward without too many regressions.
It can be easy to underestimate how much work is involved with automated testing. It can be easy to set up your first few automated tests, and it even seems magical in the beginning. But, you end up having to develop and maintain your automated testing framework. There is also a maintenance burden of maintaining your automated tests (can be flaky or change often).
When you reach a stage where your product becomes critical and any downtime can really impact the customer, you can start building out a lean automated testing framework with tests for the critical paths.
4 Likes ![Senior Software Engineer [L5]](/_next/image/?url=https%3A%2F%2Ffirebasestorage.googleapis.com%2Fv0%2Fb%2Ftech-career-growth.appspot.com%2Fo%2Fcompany_logos%252Fgoogle_logo.png%3Falt%3Dmedia%26token%3Dded981a0-71c9-42a4-a03e-a2299a0a11b8&w=96&q=75)
3 months agoAgree with Charlie - it all depends on the life expectancy of the UI code you are writing. If it has no users and pivots are likely, there's no real value in adding UI tests. When you serve millions of users, tens and maybe hundreds of developers are working on the same code base, and regressions are harder to detect (for the simple reason that not all developers are experts at the product behavior anymore), automation is great and enables faster development because it allows developers to be confident in their changes.
That being said, there are actually several different types of UI automation (from simpler, unit-test-esque screenshot tests designed to test UI rendering and simple interactions to heavier end-to-end browser tests like Selenium). The heavier the tests are, the more cautious you should be about implementing them.
1 Like
