With another major upgrade around the corner (it isn’t always there), it is time to get ready to roll out the latest new features ServiceNow offers. To minimise disruption to end users, we need to test the system after upgrading it in a sub-production environment to mitigate any risk of upgrading the production environment. Your boss might ask you to “run your regression tests”, – but what are they and why do you need them?

Why do I need regression tests?

Upgrading to the latest ServiceNow release is exciting, with new features and improvements that can help make our work lives more manageable. But one crucial thing to keep in mind during the upgrade process is regression testing.

Regression testing is when we re-test the system’s existing functionality after making changes or updates. This is important because it ensures that the new release didn’t break anything that was working before. We don’t want to fix one problem and then create ten more!

Creating regression tests can be time-consuming and requires a lot of effort. One way to make it more manageable is to prioritize the most critical functionality of the system first. Another way is to use automation tools to speed up the process. This can help reduce the workload on the testing team and free up their time for other essential tasks. While upgrades are provided at no additional cost, the human cost of upgrades can take its toll. What free tools can I use to assist?

Free, automated testing tools for ServiceNow

Automated Test Framework (ATF)

ServiceNow’s Automated Test Framework (ATF) needs no introduction. Whether you are a customer, administrator, developer or product owner, chances are you have heard of ATF and know what it does. Can’t you just use that to do all your regression testing? Like most things, the answer is not that simple.

Constraints

While ATF is an incredibly powerful tool for testing applications on the ServiceNow platform, it’s essential to be aware of its limitations, especially when upgrading to the NextExperience UI. Currently, ATF and Regress do not support the NextExperience UI, meaning that ATF support is only available for the classic UI and legacy workspaces. This is a significant constraint for companies already adopting workspaces looking to upgrade to the latest version of ServiceNow.

While ATF support is still available for both the Polaris and classic views, it’s important to note that automated tests still need to be created in the platform. This means that developers need to put in the effort to create test cases that can be run automatically, which can be time-consuming and challenging. However, this effort is well worth it in the long run, as it can save significant time and effort in the testing process.

Finally, it’s worth mentioning that UI tests still need to be run in a browser. While this might seem obvious, it’s an important reminder that ATF can only test what it can see in the browser. This means it’s essential to thoroughly test all aspects of the application, including any back-end processes that might not be visible in the UI. While ATF is an incredibly useful tool for testing applications on the ServiceNow platform, it’s essential to be aware of its limitations, especially when upgrading to the NextExperience UI.

While there are some constraints, other supplementary tools can assist and address some of the limitations.

ATF Test Generator and Cloud Runner

ServiceNow’s Automated Test Framework (ATF) Test Generator and Cloud Runner is a powerful tool for ServiceNow administrators looking to streamline their testing process. The Test Generator enables users to easily create test cases for their ServiceNow platform, which can be run automatically using the Cloud Runner.

By analyzing instance behavior and workflows, the test generator intelligently creates tests and test suites that seamlessly integrate with ServiceNow’s ATF framework. And if that’s not enough, you can even execute these tests using the Cloud Runner option, which runs tests on a headless browser hosted by ServiceNow.

One of the most significant benefits of using the ATF Test Generator and Cloud Runner is reducing testing time and effort. Rather than manually creating and executing tests, administrators can use the Test Generator to generate tests quickly and then run them automatically using the Cloud Runner. This saves time and ensures that the tests are executed consistently and accurately, reducing the risk of errors or oversights.

Constraints

ATF Test Generator and Cloud Runner is a powerful tools for generating regression tests, but there are a few constraints to be aware of. First, it’s unavailable on PDIs, so you’ll need to work on a customer instance to use it actively. Additionally, it doesn’t currently generate tests for catalog items, although this may be added in a future release.

Another thing to keep in mind is that while the tool takes snapshots of default views, you’ll still need to create your end-to-end tests manually. Additionally, generating tests using cloud runner on ServiceNow’s infrastructure may violate your customer’s security policy, so check before using it.

The ATF Test Generator and Cloud Runner is only supported on Tokyo patch 5+ or Utah, so ironically, you’ll want to make sure your instance is current before trying to use it. It’s also important to note that while being a store app allows for a more rapid cadence of features (the ServiceNow store has monthly releases), particular features may depend on the latest release, so keeping your system up to date is still important. You can check the release notes for more information on updates and new features.

See Test Generator in action on Platform Academy Session #27 – November 10th, 2022 – ATF Test Generator and Cloud Runner

Regress

Regress is a custom ServiceNow application I built to assist with the creation of ATF Tests. Once installed and configured in your instance, the application translates the current user’s context, form changes and actions to ATF Tests and Test Steps that can be used as and executed as your regression tests. While recording, records and relationships are tracked to seamlessly tie navigation to related records together, such as Requested Items and Catalog tasks, with the option to even create tests for all emails triggered from records touched. By running through your regression tests manually with Regress enabled you can fill the gaps currently left by ServiceNow’s ATF Test Generator, such and Catalog Items and end to end testing, to generate a comprehensive suite. Using it with Cloud Runner, you can run your tests through a headless browser to execute these tests.

Regress can be downloaded for free on ServiceNow Share.

Constraints

Regress is a custom application for ServiceNow that allows users to create ATF tests by recording user inputs and translating them into test steps. However, it is limited by the same constraints as ATF and the ServiceNow API. It only supports recording tests for user actions and UI components that are available through Client Scripts, UI Scripts, Script Includes, and Business rules. Regress is also not compatible with NextExperience Workspaces and requires users to manually test. It is important to note that Regress is not an official ServiceNow product and does not come with any warranty or support. Therefore, it should be used at your own risk.

Conclusion

Using a combination of ATF and free tools such as ATF Test Generator and Cloud Runner (Store App) and Regress (Share App) you can create a comprehensive list of regression tests that can be executed without human touch to prepare your systems for major upgrades and releases by ensuring all your ServiceNow instances behave to your predefined specification. If you are willing to invest further, there is also a number of paid tools that offer similar functionality.

Regression testing is critical to the upgrade process, and we must not overlook it. It ensures the new release is stable and doesn’t break any existing functionality. While it can be a lot of work, we can find ways to make it smarter and faster by prioritizing critical functionality and using automation tools. With proper regression testing, we can confidently move forward with the upgrade and enjoy the benefits of the latest ServiceNow release.

ChatGPT was used to assist with the writing of this blog.