Automated Web UI Testing: Best Practices, Challenges, & Tools

What Is Automated UI Testing?

Automated web UI testing is one more way to shift left quality validation within a development team’s workflow. As with any automated process, it should run without oversight from a human. While other Parasoft tools implement automation across things like CI/CD pipelines, our Parasoft Selenic tool tackles UI automation testing. This is where testing is performed via an automation tool rather than relying on manual testing.

Various test scripts perform different tests that return as pass or fail. From there, dev teams can approach bug fixes and other changes to the software.

This blog answers the following questions:

1. What is automated UI testing?
2. Is automated UI testing worth it?
3. What are common automated web UI testing challenges?
4. What is self-healing test automation?
5. What are UI testing best practices?
6. What are the best automated UI testing tools?

Is Automated UI Testing Worth It?

Any dev team looking to improve workflow and shorten release cycles should consider automated UI tests. Manual testing will always play a role in development, but automated tests ensure a higher quality minimum baseline.

Moreover, it reduces costs, ensures actionable results, and streamlines the entire review process.

Common Automated Web UI Testing Challenges

Writing UI test scripts can be time-consuming with the work loaded into the front of the cycle. But the hard work pays off down the road; however, UI scripts are inherently brittle. The majority of testing is done via Selenium test scripts which can be written in many languages such as Java, Python, and C#.

Key Challenges

• Knowledge-heavy. Whoever is writing the scripts has to have technical knowledge and experience with the subject.
• Time-consuming. One test can take 5 – 10+ minutes to run due to the processes. Load browser > execute tasks like a human > set up and tear down tests > data loading, and so on.
• Maintenance. Modern apps have dynamic frontend UI. When a page loads and then the same page reloads, those element locators and links may change under the hood as to how they’re defined. The wait conditions can break the test. If a page takes longer to load, that returns a broken test.
• Handling multiple errors. Complex scenarios mean huge data volumes. Sifting through this can be cumbersome.
• Troubleshooting. Finding out what broke a test can be difficult depending on the frequency of issue and time spent fixing it.
• Dynamic apps. Changes are always being made to web applications, as well as to apps themselves due to rapid/agile development. Tests must keep up from a maintenance perspective AND a results perspective.

Test automation can run nightly, but the tests could take something like 6 hours to run. A sprint to sprint plus release candidate means failing tests can hold up the release and reports will come back with failures. This doesn’t mean that the app is broken; instead, it indicates that the tests could be broken.

Teams can remediate this with self-healing so a test can fix itself and not delay the release. Another option is a “codeless solution”, using object repositories, leveraging elements that don’t complicate UI testing, and relying on best practices for UI testing.

What Is Self-Healing Test Automation?

At its core, self-healing test automation proactively makes object changes that can solve automation script breaks. Normally, if an object is changed beyond its configurations, an automated script may no longer recognize it. This would shut down the process.

With self-healing automation, the UI test scripts can continue to run while also sensing the changes. This removes the need for human intervention and further automates web UI testing.

Automated UI Tests Best Practices

Since the UI is what users touch and see, automating testing to shorten release cycles is crucial. As with all things in development, there are unyielding best practices that all teams can follow to better ensure a great ROI from their automation investment.

1. Follow a consistent naming convention.
2. Review which test cases should be automated.
3. Create quality test data.
4. Keep tests independent.
5. Don’t rely just on one type of testing.
6. Avoid sleep when you should actually pause a UI test.
7. Not all tests must be used in all target browsers.
8. Look into headless browser testing.
9. Consider making use of a BDD framework.
10. Use data-driven vs. repeated tests.
11. Keep tests simple.
12. Take screenshots to improve failure investigation.
13. Leverage the right automated testing tools

Top 10 Web UI Test Tools: Automated Web UI Testing

At least a dozen brand new UI test automation tools have surfaced in the last few years. Since every tool has its own focus and strategy, it can be hard to know where to start. But there are 10 essential web UI testing tools; you just have to discern which one has the features your team requires.

As any UI tester could contest, UI testing is relatively straightforward, as long as nothing in your GUI changes, but the problem is… things change all the time. Depending on the solution you’ve chosen for UI testing, changing conditions can either be a revolutionary experience with self-healing and AI locators, or an abysmal failure of convoluted manual workflows.

UI Testing Tools & Features

In an effort to achieve the former vs the latter, I’ve reviewed the top 10 UI testing tools that you should look at.

1. Parasoft Selenic
2. Katalon
3. Selenium IDE
4. mabl
5. TestIM
6. Functionize
7. Perfecto
8. TestCraft
9. Squish
10. AutonomIQ

Here’s a quick chart that covers each tool’s features.

Furthermore, Selenic, Katalon, and Selenium IDE have basic reporting. All of the other automated UI testing tools featured advanced dashboards. Below, we review each tool in-depth to explain features as they apply to automated web UI testing.

There are also some key questions to ask yourself when evaluating UI testing tools that ensure you end up with a tool that’s the right fit and that can achieve what you need. Whichever tool you choose, all of these UI testing tools have great usability features and have put the user experience at the forefront.

1. Parasoft Selenic

Designed for: Existing (or new) Selenium users who are spending too much time creating, updating, and maintaining their UI tests. Also for those who don’t want to be locked into a vendor-specific tool.

Pricing: Annual Subscription for Project and/or Teams

Pros:

• Smart recorder makes it easy to create Selenium tests that use the page object model
• Can execute self-healing on existing Selenium tests and recommends smart locators on existing Selenium tests
• Integrates directly into your CI/CD pipeline by leveraging your existing Selenium scripts or creating new Selenium scripts
• Provides test impact analysis technology to automatically identify which Selenium tests need to be executed in the CI/CD pipeline to validate new code changes
• Provides customer support at every level (not just enterprise)
• BDD supported (Cucumber). Focuses on the maintenance of the underlying Java (JUnit and TestNG) Selenium test code

Cons:

• There’s no free license for this commercial offering, but a free trial is available.
• This product is still relatively new. The current version supports Java, JUnit 4 & 5, TestNG, Cucumber, Eclipse, and IntelliJ. Other support is under consideration for future releases.

Key takeaways: AI-powered recommendations for your existing Selenium tests make it easy to add this solution to your existing Selenium practice right away. As evidenced in Gartner Peer Insights, one of the best benefits of the solution is Parasoft’s world-class customer support, which has been recognized countless times and has supported over 30 years of software testing product innovation.

2. Katalon

Designed for: Users looking for a middle ground between codeand codeless testing tools.

For a free tool, Katalon does a lot, and is used heavily by system integrators for UI testing. Its recorder plugs into the Chrome browser so you can generate test cases, and then the tool builds them in the Katalon IDE using the page object model, so they are highly maintainable. At any point you can execute your tests directly in their IDE, or export them to many different types of test scripts.

Pricing: Free for the basic Katalon Studio (with paid plugins available as subscriptions through the marketplace). Enterprise support is available (but can get quite expensive)

Pros:

• Recommends smart locators
• Uses the page object model (but only for the tests generated and managed within the IDE)
• Has a self-healing capability (via a paid plug-in)
• Ability to export to many different types of test scripts
• BDD supported. Feature files execute Katalon test code

Cons:

• Although you can export your tests as Selenium and other test scripts, once you’ve exported you lose all the usability of Katalon and the export doesn’t include the page object model, which means the tests become hard to reuse and maintain
• Their customer support is only available at the enterprise level, and is very expensive
• Doesn’t directly integrate into your existing execution framework (uses a proprietary framework, with CI integrations available as plugins)

Key takeaways: Users can start using Katalon without having any technical knowledge, creating test cases quickly thanks to the keywords in the program. It’s free to get going with Katalon Studio, but to get the benefit of some of those exciting innovative technologies, you’ll have to upgrade or purchase paid plugins.

3. Selenium IDE

Designed for: Users that are looking for a UI driven record-and-playback tool.
Selenium IDE is an open-source project that harnesses the power of Selenium into a Chrome plug-in, and makes it available for free. Getting started with Selenium IDE requires no additional setup other than installing the extension on your browser, aligning with the project’s driving philosophy of providing a tool that’s easy to use and gives instant feedback.

Pricing: Free

Pros:

• Users can export any created test into a script, making it an incredibly powerful tool for UI test creation
• Offers multiple locator strategies for each element it records
• Has a self-healing feature (but doesn’t work on Selenium scripts outside of the Selenium IDE, and isn’t as reliable as self-healing features from other solutions)

Cons:

• Does not fit directly into your CI/CD process
• Buggy and a blackbox (when something doesn’t get recorded you are pretty much stuck and have to export out into code to figure out what’s going on)
• Doesn’t use the page object model when building tests
• While you can export your tests in many different languages, there’s no ability to import at this time
• While they offer CI integrations, it is not directly integrated into your existing framework
• No built-in support for BDD

Key takeaways: Selenium IDE is simple to use, giving users the ability to rapidly create tests against their Web UIs. It is very Selenium friendly and will feel natural for those accustomed to the framework.

4. mabl

Designed for: Teams who aren’t interested in writing tests and don’t want to establish a UI testing practice, meaning that you hand over the majority of the work to mabl.

mabl has a very different approach to UI testing – it’s less about tests and more about journeys. You log into their website, define a journey through your application, and then that’s it. They don’t really give you a test case or anything to really work on. From that point, they will allow you to periodically run that test and receive results about its successful execution.

Pricing: Tiered pricing based on number of journeys executed

Pros:

• Recommends smart locators (but it’s a bit obscured from the user)
• Self-healing feature works very well
• SaaS solution; easy to access and get started
• Scalable pricing model

Cons:

• Doesn’t use the page object model when recording journeys
• Uses its own framework, and there is no import and export of the test scripts
• While CI integrations exist, you can’t directly integrate the solution into your existing framework
• If something doesn’t work the way you intended, it’s really difficult to get access to “code” to configure or manipulate
• The focus on “journeys” makes the creation of “functional validation” (i.e. test with assertions) feel awkward
• SaaS-only solution; no on-premise deployment and no ability to access an application not available in the public internet
• No built-in support for BDD

Key takeaways: mabl helps users improve the speed and quality of their release pipeline, with sophisticated auto-healing that keeps automation stable, regardless of changes to the application’s UI. mabl does what it does and the human takes a back seat to its prescribed analytics, so essentially you hand over your journeys to mabl and hope for the best. For some, that is great, but for others, that might be handing off too much control.

5. TestIM

Designed for: Organizations looking for an out-of-the-box solution for UI testing that focuses on execution paths for complex workflows.

TestIM is a SaaS application, created by developers who realized that while they were spending time and energy maintaining automated testing environments, they were still anxious about how a simple bug fix might break another part of the application. So they created this incredibly easy-to-use solution for UI testing.

Pricing: Annual subscription based of number of tests executed
Pros:

• Tests are incredibly easy to create, with an intuitive user interface
• Recommends smart locators, along with a great strategy for maintaining them
• Has an impressive self-healing capability as part of their Smart Locator technology

Cons:

• Doesn’t use the page object model in their tests
• Lacks ability to import/export test scripts
• Users can’t get access to code or take tests out of TestIM
• Vendor-locked framework — scripts are in the TestIM framework
• Unclear support for BDD

Key takeaways: It’s very easy to get access to the TestIM technology, and very easy to create a test case that works right, out-of-the-box. The sleek user experience makes self-healing look like magic (If I want a solution to improve my own user experience, I can probably trust a solution with a great user experience, itself, right?), but the main challenge with TestIM is that there’s no way to get access to the tests. Since all of the tests are in the TestIM framework, you’re at the mercy of their infrastructure, business model, and whatever happens with the company in the future.

6. Functionize

Designed for: Manual testers or non-technical testers who are getting into BDD (they call it “ALP”) as a function of UI testing.

Functionize provides all the features you would expect from a full web UI testing tool, but with some additional advancements around locator strategy, including an autonomous testing pitch where tests can be created by simply using the application. This is all accomplished in their proprietary framework.

Pricing: No pricing information publicly available

Pros:

• Excellent visual display of test flow
• Recommends multiple locators for elements
• Has a self-healing capability
• Alternative approach to traditional BB (using proprietary NLP). Could be a Con if you are looking to do traditional BDD (i.e. Cucumber)

Cons:

• Doesn’t use the page object model
• Vendor-locked framework
• No import/export of test scripts

Key takeaways: Functionize’s AI seems to be capable of delivering on the organization’s promise to remove test churn, increasing the accuracy and efficiency of testers. But beware of vendor-lock here, as you consider how you might bring it into the flow.

7. Perfecto

Designed for: Users looking for a one-stop-shop for UI and mobile testing. Because it’s scriptless, it is best suited for those looking to not write code.

Perfecto has been around for a while, and they’ve heavily focused on mobile testing, but they have a nice UI testing ecosystem that enables creation, maintenance, cloud execution, and reporting.

Price: Between $99 and $3588+ for an annual subscription

• Uses the page object model (but only in their generated tests)
• Provides multiple locators for recorded web elements
• Recently introduced self-healing (although it’s a little unclear what it does or how well it performs, and therefore how useful it is)

Cons:

• Test scripts are in a proprietary TCL-based language
• You can’t import your own Selenium scripts
• While they offer CI integrations, it is not directly integrated into your existing framework
• BDD supported through 3rd party implementation, Project Quantum

Key takeaways: Perfecto provides solutions around maintaining and writing test scripts, managing and validating tests, and debugging defects. Perfecto’s UI testing solution gives users the option of scripted or scriptless creation, and they have recently added new features around maintenance and self-healing.

8. TestCraft

Designed for: Users who do not want to write code and are looking for advanced analytics to continuously validate their tests.

TestCraft is a codeless Selenium test automation platform, with AI technology and unique visual modeling claiming to enable faster test creation and execution while eliminating test maintenance overhead.

Pricing: No pricing information publicly available

Pros:

• Uses the page object model (but only in their generated tests)
• Recommends smart locators, called “smart bindings”
• Self-healing is a part of their smart binding technology
• A built-in scheduler allows users to execute test flows periodically over time

Cons:

• Uses a proprietary framework
• You can’t import/export test scripts
• Doesn’t directly integrate into your existing framework (CI integrations are available as plugins)
• BDD supported through 3rd party implementation, Project Quantum

Key takeaways: TestCraft is a powerful SaaS application that enables testers to create fully automated test scenarios without coding. Users record their test cases on the UI and from that, TestCraft will create a model for every test flow, with a visual indication of the different paths that your test will execute. This makes it approachable for users that have complex usage patterns.

9. Squish

Designed for: Users who need to test GUIs and human machine interfaces (HMIs), especially across multiple platforms. Squish automates testing for Windows, Java, QT, .NET, Web, iOS, Android, and more.

Pricing: A perpetual floating license can be purchased for €4,990. Free trial available.

Pros:

• Extensive cross-platform testing support
• Powerful optical character recognition (OCR) capabilities
• Reliable object identification and verification independent of screen layouts
• Good support for scripting languages

Cons:

• May need improvements for end user training and technical support, though most feedback is favorable
• Some issues reported with adding new objects to the object map

Key takeaways: With support for both GUI and regression testing, Squish uses property-based object identification to evaluate the web UI. If you’re using a BDD workflow, you can integrate Squish into your IDE. Automation support for Chromium is also available.

10. AutonomIQ

Designed for: Users looking for an interesting hybrid between code and codeless solutions. (There’s code, but it’s aided by natural language processing.)

AutonomIQ has a powerful message around AI and machine learning, but what does it do? The tool provides a three-step process for UI test automation. It begins by discovering your application through its AI, and then building test cases from that discovery process. The third step is using their analytics engine to monitor and maintain the test cases.

Pricing: No pricing information publicly available

Pros:

• Strong NLP, AI, and ML
• Uses the page object model (but only in their generated tests)
• Recommends smart locators, through their Chrome Plug-in
• Self-healing in their execution
• Alternative approach to traditional BDD (using proprietary NLP). Could be a Con if you are looking to do traditional BDD (i.e. Cucumber)

Cons:

• Uses a proprietary framework
• You can’t import test scripts
• Doesn’t directly integrate into your existing framework (CI integrations are available as plugins)
• Not entirely proven technology

Key takeaways: It seems like this technology has it all and does it all. So why is it at the very bottom of my list? Because I haven’t been able to find a customer of theirs with any real case studies or usage. But if their solution can get to the potential results, it could be quite disruptive.

Use Parasoft Selenic For UI Testing

If you, like many other Selenium users, have found yourself dedicating too much time to crafting, maintaining, and updating your UI tests, it’s time to add another tool to your arsenal.

Consider Parasoft Selenic to improve your automated web UI testing processes and reduce test maintenance. It comes backed by Parasoft’s 35+ years of software testing product research and innovation.