---

Have you ever found yourself in this situation:

🙋

I know I have.

I usually don’t like doing Data Fixes and found myself back in the position recently. Running Data Fixes can be an incredibly stressful experience for any administrator or developer.

If it goes wrong, you can irreparably destroy your customer’s data. 
If it goes right, you are a hero (unless the underlying cause of the data fix was your fault!)

We all have war stories of cheeky Data Fixes going horribly wrong. I once heard of a 3-day marathon P1 where every single ticket in the customer’s system was closed due to a Background Script having unintended side effects!

Thankfully, the NowPlatform today has a number of new tools at your disposal to almost entirely eliminate the Risk of performing scripted Data Fixes, so you can be the superhero – with no Stress!

In this blog, I will provide some background on the API that is often used for Scripted Data Fixes, the main methods (Fixed Script vs Data Fix), and some tips and tricks I have learned along the way.

GlideRecord Update API

Whether you are using a Fix Script or a Background Script (I will go into these later) the API used is usually the same.

Example

The Automatic Script to Close Incidents after 5 days has stopped working, and as a result, the monthly report due tomorrow is showing incorrect results. Your development team made a change to the script last sprint that must have broken it.

You must remediate the situation and set all affected Incidents to Closed. 

Consider the common form of an update script below:

function run() {
    var gr = new GlideRecord('incident');
    gr.addEncodedQuery('state=6');
    //gr.setWorkflow(false);
    //gr.autoSysFields(false);
    gr.setValue('state',IncidentState.CLOSED);
    gr.updateMultiple();
        
}
run();

Let’s go through the key API calls 

setWorkflow(false)

Sometimes you don’t want Business Rules to execute and fire notifications and update other records.
When setting false as the first parameter, set to true, the Business Rule engine is disabled. This may come in useful in certain situations but always use with caution – it also disables Notifications and the SLA engine, as well as Business Rules.

This could be used for data fixes performed on string fields, where there is no desire for any notifications or business rules to be triggered.

autoSysFields(false)

Using this method prevents metadata fields from being updated. By metadata, I mean audit fields (such as Updated). Use this if you don’t want all your records to be shown as updated by the user who ran the script, as shown below:

This can be used in conjunction with setWorkflow() for a truly “silent” update on records.

update()

There is nothing stopping you from using the standard GlideRecord query/loop/update pattern when performing a data fix. 

var gr = new GlideRecord('incident');
gr.addEncodedQuery('state=6');
gr.setWorkflow(false);
gr.autoSysFields(false);
gr.query();

while (gr.next()) {
   gr.setValue('state',IncidentState.CLOSED);   
   gr.update();
}

However, you might want to consider using a more purpose-built API for this task.

updateMutliple()

This may run faster and more consistently than the standard query/loop/update pattern, and with fewer lines of code. Limiting the execution to 50 records, with Business Rules disabled, performance is shown below:

While the sample size is too small to draw conclusions, it would appear that updateMultiple() is faster and more predictable.

You might prefer using the update() method instead so you can log the results and the records that are queried. I would argue that logging each record processed is will slow down your fix script significantly and is actually unnecessary (see Background Script).

Now that we have gone through the API – how do we go about running this code in a controlled manner?

Running a Scripted Data Fix

Fix Script

With the name ‘Fix Scripts’, you would think it is the perfect solution for Data Fixes. I had moved on from Background Scripts years ago and swore by Fix Scripts. Fix Scripts lets you package your script for execution in a controlled manner. 

Unloadable

If the ‘Unloadable’ flag is checked, the Fix Script is added to your currently selected Update Set.
This allows the Script to be easily transported across your ServiceNow environment, where you may wish to test the script on different Data Sets.

Run in Background 

When ‘Run Fix Script’ is selected, you are prompted with the option to run the script in the background, so you can continue working, or in the foreground – in your current session.
This is quite helpful when you are trying to multi-task, as most ServiceNow administrators often do.

For more details on Fix Scripts, consult the Product Documentation.

Rollback

Fix Scripts can leverage Rollback functionality via a hidden feature.

See KB0761248 – How to enable and use rollback functionality for Fix Scripts

Background Script

Unlike Fix Scripts – and despite its name – Background Scripts are run in the foreground – in your current session. This will tie up your session, so you can’t navigate around until the script has returned.
You don’t have the option to run them in the background, which is quite ironic.

With that out of the way, I’ll admit that I hadn’t touched Background Scripts in years. I found Fix Scripts and other scripting tools such as Xplore and VSCode much easier to work with.
I have recently started using it again, as I discovered the Rollback functionality is available when using Background Scripts. This means you can rollback if it all goes horribly wrong (but you should still test first).
Initially, it looks very tacky and very YOLO (You Only Live Once).

Just paste your script in and away you go! 

Run Script

Let’s run our script from our example and walk through the different functions on offer.

Firstly, a summary is provided of all records updated. Pretty neat!

Execution History

When clicking ‘available here’, you are presented with the Script Execution History, and log output saved for reference.

Affected Records Summary

This is not limited to the code that is run – a full report of records updated is also provided.

You can click through into lists to see the exact records that were updated.

Below shows the results for incident:

If you used .setWorkflow(false) you will see only updates to the Incident Record – as all subsequent updates were triggered by Business Rules, which is disabled.

Record for rollback

I was once asked by a Configuration Manager before running a Data Fix – “Can it be backed out?”. I then realized – Fix Scripts can’t easily be backed out. I knew that ATF had introduced Rollback functionality into the tool, and was keen to find out how it could be used.

This is where it gets really cool.

At the bottom of the form, the Rollback Script Execution Related Link becomes available.

This completely undoes the damage your script has done – including any and all updates to other records that occurred in the transaction.

EDIT: An important note on rollback

Audit history is not rolled back, so the Activity Log and audited history will be out of sync with the current record state. As a result, this cannot be relied upon as a backout plan in production.

I’ve raised the Idea Rollback to include removal of audit data to get this fixed in future

Honorable Mentions

Execute in sandbox

No records are updated when running a script with this option enabled.
This restricts the use of update() and updateMultiple() API.

Execute as Scriplet

I was unsure on this one, but the ever knowledgable @Brad Tilton had an answer (as always)

“… Scriplets are functionality added in Paris that give you a scripting environment that doesn’t load the ordinary scripting scopes you would have in a server-side script. It is isolated from script includes and most APIs and just has access to the vanilla rhino javascript.”

For further discussion – Any insights on what Scriptlet Function is?

I won’t go into further details on this one… it does not add value to this conversation.

For more details on Background Scripts, see the Product Documentation. 

You can also run Background Scripts in VSCode.

Testing

I would like to call out the importance of testing when running scripted data fixes. Using Background Scripts, we now know that it is possible to test multiple times. However, this doesn’t make it OK to just build your script in Production and rollback until it is right! Rollback is not a valid backout plan for a Data Change!

Always test in sub-production first, and test often!

Do not just do positive testing – do negative testing too. Make sure other records that you did not intend to update are left alone.
For example, if you are closing all Resolved Incidents, make sure Incidents in the other States are not updated too!

Fix Script or Background Scripts?

Back to the meme – which red button should I press? 

	Fix Script	Background Script
Run in background	✅	❌
Saved in Update Set	✅	❌
Rollback	✅*	✅
Sandbox mode	❌	✅
Script Auditing	❌	✅
Reporting of records changed	❌	✅
Readable log output	❌	✅

 *see https://hi.service-now.com/kb_view.do?sysparm_article=KB0761248

EDIT: It has come to light that you can enable Rollback in Fix Scripts, which somewhat changes my original conclusion. See the KB article above.

While Background Scripts has it’s negatives, they are easily defensible.
Not being able to run in the background isn’t a huge deal – just open up a session in your non-preferred browser and run the code from there. Nor is the inability to Save to Update Set. Your Script can always be saved as a Fix Script to transfer via Update Set, then copied into the Background Script when the time comes.

What do you think?

 

I hope that after reading this blog, your next Data Fix change will go smoothly and not be as stressful.