Data-Enabled: Post Nudge Campaign Results

Data-Enabled: Post Nudge Campaign Results

A lot of Organizations have adopted things like CIS Controls and part of those controls are around patching and vulnerability management. Depending on what version of the CIS controls you are using, this could be CIS 3 or CIS 7. This can already be confusing as the controls are very similar between versions but the have moved places in the CIS framework. One thing I think most tech and security professionals can agree upon, is that patching often to the latest operating systems is a great way to avoid your fleet of devices from being exploited. Known vulnerabilities have a shelf life, and bad actors know that some organizations do not patch, or patch extremely slow. So, they develop attacks against known published vulnerabilities.

The good news is that this is very easy to mitigate, and all you need to do is to update your operating system and applications across your Org’s fleet of devices. This blog post will be on the specific topic of updating macOS.

Original Problem

We originally faced a problem, and that was that softwareupdate on macOS has had some nasty known bugs in the last recent years. While, Apple has patched them at least one time according to the recent release notes, many IT professionals in our communities have still experienced issues with updating macOS.

MDM software updates lack some features around UI/UX and have inconsistent results in our testing. We first tried to use them to auto update Mac Minis running digital signage and our conference rooms. Given that these were static devices that were always on the network, and we had very inconsistent results we decided to not use MDM commands to update the OS of our fleet. To add fuel to the fire, our MDM vendor also had an issue where if their binary collected available software updates from macOS, it could potentially hang the agent. Thus, leaving the device in a non-managed state. So, we disabled the Software Update collections, and got rid of all existing software update workflows since this bug was impacting other things. Our existing dialog box workflow which relied on this data no longer worked. What were we to do? Wait for vendor fixes, rebuild our entire workflow, or maybe migrate to a different tool or method to get our desired end results? As a quick and dirty solution I hard coded OS versions into a Python script I had written to get us by while we assessed our situation.

Enter Nudge

Nudge is an open source tool written by a community member to solve this problem. Nudge is an independent tool that can operate on its own, and is not dependent upon MDM to work. The app runs with a launch agent that will prompt the user to update their OS on a regular interval. Since software updates on macOS have been problematic for the past couple years folks have been looking at alternative ways to accomplish this goal. Out of the box Nudge is honestly a great tool. We did not find much of anything wrong with the default settings it came with. Couple that with the fact that Nudge allows the end user to control when they get “Nudge’d” we felt that giving each end user full control over when Nudge will pop up and ask you to update was the best option. You cannot please everyone in these type of decisions, so giving everyone their own opinions to be enabled is a good route to try.

Here is what our Nudge screen was prototyped as, our final production one was slightly tweaked from this:

You can see in the circled part of the bottom right an employee can choose when they want to be notified next and this puts all the control in their hands. The headsup display that Nudge gives a human is also a combination of minimal yet very informative. Which we also found perfect. If you click on the custom option it brings up your calendar where you can choose the next time on your calendar when you want to be notified.

So my team, and I deployed Nudge to our devices, demoed it to leadership, and took it through our process to release it to our fleet of macOS devices. We also launched a small campaign of Slack announcements and emails to our coworkers informing them we were deprecating Big Sur and older OSes completely. We were going to get our entire fleet up to 12.4 to combat these software update bugs that are impacting other things.

Post Campaign Results

The best part about being data-enabled is that we already collect, ship, and ingest all the data from our organization. We are enabled to use the data in our everyday work life, and we share it freely among our teams and departments. I can query OS update trends, what departments patch the fastest, what department patches the slowest, what locations update macOS faster, so on and so forth. With additional data sources I can also filter out contingent workers, interns, terms, new hires, legal/term hold devices, primary and secondary devices, and a plethora of other things that can be highly useful when wanting to look at data and get a clear context of what your data means.

example query in Snowflake:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

with macos_devices as (
    select 
  SN.substatus as type 
, JI.EVENT:emailAddress::varchar as USER
, JI.EVENT:osVersion::string as OS_VERS
, JI.EVENT:serialNumber::varchar as SERIAL_NUMBER
-- little logic to add "conference room" as a department to conference room computers
, IFF(JI.EVENT:emailAddress::varchar = 'foo@acme.com' AND JI.EVENT:department::varchar = '', 'conference room', JI.EVENT:department::varchar) as department
, WEBHOOK:eventTimestamp::varchar::timestamp_ltz as event_date
FROM 
  "DB"."SCHEMA"."JAMF_INVENTORY" as JI
  inner join "DB"."SCHEMA"."SERVICENOW_ASSETS_V" as SN on upper(SN.SERIAL_NUMBER) = JI.EVENT:serialNumber
  where type in ('primary', 'secondary')
  )
select
count (distinct serial_number)
, event_date::date
, os_vers
from macos_devices
-- just modify the versions you want displayed here
where os_vers in ('12.3.1', '12.3.0', '12.4.0')
group by 2, 3
;

example of a full dashboard:

Just being enabled to filter out any exited employee from data is very powerful, and I show the difference in the data in the above screenshot. With the power of Snowflake, data sharing and being data-enabled this is just the tip of the iceberg on data you can dive into. This will give you more insight of your IT assets and your fleet of computers than you have ever had before. It is pure magic as some say. This is what it is like when you are data-enabled, IT professionals can get fine-tuned data with the proper context applied.

Data can also help showcase results. What we do know as a matter of fact, is that we did our first Nudge campaign to the entire Org with the release of 12.4. Our campaign to upgrade the fleet when 12.4 was for all Macs in our Org. So, we were also leveraging Nudge for a major OS upgrade from Big Sur to Monterey since we knew Big Sur had existing bugs that were not being addressed.

First we can look at our Big Sur versions over time. This was a downtrend of data due to the fact we were pushing for our fleet to update to Monterey. We had experienced some bugs in Big Sur with software updates, and with other tools. So, we had been campaigning with our previous workflows and tools to get folks to upgrade to Monterey.

Big Sur Versions over time:

Early Adoption of Monterey over time:

Monterey 12.3.x to 12.4 adoption:

macOS 12.4 Adoption rate from Nudge:

Conclusions

Looking at the historical data, which we can keep at scale in the data cloud, we can observe that our adoption rates did seem to happen faster over time than our previous campaigns to update macOS. Of course this data represents the results of my team’s labor and not necessarily the perception our coworkers on how the new tool worked. The big takeaway for me personally is how much this impacted major OS upgrades since we have been trying to phase out Big Sur here. Nudge really helped us achieve this goal, and we were pretty impressed with the results. I am also happy to say that while we did get minimal feedback on Nudge, it was almost all positive. The non-positive feedback was also mostly neutral and not really negative at all. Users really loved the concept of them being in full control of when they get notified to update and being able to defer when they choose. There is additional data you can collect with Nudge as well. Like tracking how long folks wait to update, how many times they click on the defer button, and more. However, I will not be covering any of that data in this blog post. Overall, our Nudge campaign seems to have been one of our most successful OS update campaigns we have observed. It has also been one of the smoothest I have witnessed personally in my career. I feel this tool is fantastic, and it requires very little customization to meet the end goals of user experience and results.