by Paul Caiazzo
An effective patch management strategy is one of the foundations of an organizational cybersecurity strategy. However well understood this may be in theory, many organizations struggle to implement a good patch management program.
In this post, we discuss the importance of strong patch management, patch management best practices, and how to implement a program that effectively manages the risk of unpatched, vulnerable software.
What Is Patch Management?
All software has bugs. Whether these are caused by design flaws or implementation flaws, the sheer amount of code in systems that we use every day is bound to contain errors.
In his book Code Complete, Steve McConnell explores the average rate of errors in programming and estimates that the average program will have between 15 and 50 errors for every thousand lines of code. For reference, the average iPhone app has over 20,000 lines of code or an estimated 300-1000 errors, some of which are exploitable vulnerabilities.
The type and level of risk associated with any one of these bugs could range from minor nuisance instability to the potential for a major data compromise.
Patches are software or firmware updates issued by a program’s developer designed to fix identified flaws in a program. Typically, this occurs after the flaw has been identified as an exploitable vulnerability, meaning that applying these patches in a timely fashion is critical to the security of the organization.
More by Paul Caiazzo: Using MITRE ATT&CK Framework for Beyond-Checkbox Cybersecurity
Why Don’t Companies Patch their Software?
Companies may not regularly patch their systems for a number of reasons. A lack of technical staff can make updating an application widely used by the organization an intimidating proposition to those without a technical background. Undermanned IT staff can become busy with problems perceived to be more important, and only focus on patching in a best-effort cadence. Additionally, some updates can cause performance issues or impact stability and operability, and thus are often put off to avoid dealing with the complications associated with updating.
Even large well-funded organizations with dedicated IT staff struggle with patching, for a variety of reasons. Legacy systems may require a specific version of an application, and patching or upgrading that application may not be possible. This creates a persistent vulnerability within an environment that must be handled through compensating controls like network segmentation to isolate the vulnerable system as much as possible.
Additionally, organizations often apply sound patch management strategies to Operating System level patches and other more easily maintained software, such as the Microsoft Office Suite, but struggle with patching other third-party applications – most notably web browser plug-ins. This is typically due to the lack of good centralized configuration and patch management capabilities, and it presents a significant risk.
Many attackers’ most commonly-leveraged TTPs rely on a web browser exploit to gain initial access, and as such patches applicable to the browser and associated plugins are critical to maintain.
Regardless of the reasons why an organization may procrastinate when it comes to patching, it’s a critical process that needs to be in place. Effort spent developing and executing a sound patch management program can save hundreds of thousands of dollars in damage and hundreds of hours in recovery from a problem caused by a vulnerability or software failure.
Why Is Patching Software Important?
Patches are typically issued after an exploitable vulnerability has been discovered by the community or disclosed by the originating vendor for a piece of software or firmware. After a vulnerability is acknowledged, it is not uncommon for malicious actors to try to exploit it within the window between learning about it from its public disclosure and when the majority of the public have applied patches.
Making this window as small as possible is important for organizational cybersecurity.
The highly-publicized 2017 Equifax data breach is an example of the dangers of a poor patch management strategy. As many as 143 million US customers had their personal data exposed in the breach. The cause of the breach was a failure to patch a known vulnerability in Apache Struts. A patch for the vulnerability had been available for two months before Equifax’s breach.
Hackers had been exploiting the vulnerability starting just days after the patch was released, demonstrating that failure to patch quickly can endanger an organization (and its customers). Equifax’s poor patch management policies opened it up to one of the most significant breaches in history.
More by Paul Caiazzo: Cloud Security Using Defense in Depth
Implementing a Good Patch Management Strategy
A good patch management strategy ensures that patches are applied in a timely manner and will not negatively affect operations. The key steps within the patch management program are governance, testing, change management, and patch rollout.
Defining Governance for Patch Management
Like most components of a security program, patch management strategy governance boils down to risk tolerance and managing risk down to a level acceptable to the organization.
Understanding this is imperative because the level of risk tolerance will directly correlate to the timeliness of patching required by a specific organization. Patches should always be installed as quickly as possible, but since resources available to perform the requisite testing and rollout are always limited, addressing patches by the risk associated with a specific patch yields an alignment between resource allocation and reduction in risk.
An organization with an extremely low-risk tolerance may require Critical patches to be applied within 72 hours, whereas an organization with higher risk tolerance may allow up to 30 days to apply the same patch. Tailoring the patch management program to the needs of the organization helps a program succeed, and also raises the conversation to a level that can address sustainability and resourcing should the organization struggle to keep up.
All organizations should identify the requirements for patch timeliness in a matrix based upon the severity of the vulnerability the patch is designed to fix. Low severity vulnerabilities may never be patched by some organizations, whereas high or critical severity vulnerabilities must be patched quickly.
For highly risk-averse organizations, we recommend the following:
• Critical Severity – 72 hours
• High Severity – 7 days
• Medium Severity – 30 days
• Low Severity – 60 days
Extenuating circumstances often arise that force the organization to miss their marks. In these cases, a well-governed organization documents the risk and applies a compensating control like network segmentation.
Once the organization’s governance model has been defined, the organization next must consider the process – testing, change management, and application rollout.
More by Paul Caiazzo: CISO Advice: Operating to a Cybersecurity Gold Standard During Crisis and Beyond
Testing Your Patch
The second step in a patch management strategy is testing.
Patches are designed to improve the system or software; however, the developer cannot test against every possible use case and build environment. Depending on the specifics of your organizational network, some patches may break functionality, meaning that testing is vital before deployment.
Testing should be performed in an isolated test environment, ideally a virtualized mirror of your production environment. By testing in an environment identical to production, it’s possible to identify and correct potential issues before they affect production systems.
Change Management
Once testing is complete, the well-governed organization’s change management procedures are activated.
Change management ensures communication with the relevant business stakeholders potentially impacted by the patch. This process allows for the type of collaboration required to avoid potential stability issues which may arise from the patch, and also establishes clearly defined rollback plans to be executed in the event a problem arises. Additionally, good change management enables coordinated scheduling of patches, which is usually a key element of successful patch cycles.
In the case of critical patches requiring tight timelines for deployment, organizations will often consider these as pre-approved emergency changes which follow an adjusted change management procedure designed to facilitate speed to rollout.
Patch Application Rollout
Once testing and change management procedures are complete, patches should be deployed in accordance with the patch governance model the organization has defined, but generally as soon as possible.
While it may be feasible to do this manually in small environments, an automated patch application process is preferred. Automation ensures that patches are deployed as quickly as possible and consistently applied across the network. The reduced workload on IT staff allows them to respond to specific patching issues quickly.
Automated patch management enables not just efficient rollout, but also more readily obtained metrics regarding completeness of application to the population of affected devices. It also enables reporting on the meantime to remediate (MTTR), a key metric CISOs should track to gauge the effectiveness of the IT security program.
Patch Management is Vital to Cyber Security
Software developers commonly issue patches to fix vulnerabilities in their software. Patch testing and deployment may seem like a low priority next to monitoring and incident handling; however, it is vital to organizational cybersecurity.
The development and publishing of a patch by the software vendor creates the necessary evil of notifying malicious actors of a potential vulnerability, and they commonly seize the opportunity to search for and exploit vulnerable systems.
By developing a sound strategy that enables a timely and sustainable patch management process across an environment, an organization can minimize the probability of a data breach or regulatory non-compliance due to unpatched software.
Empower Yourself to Act Quickly When an Attack Occurs. Download the e-book.
- The six phases of a comprehensive IR plan
- The five steps required for getting started
- The factors and entities you need to consider when writing your IR plan
- How to practice your plan for responding to an incident
Paul Caiazzo, Senior VP of Security and Compliance
Paul brings his wealth of cybersecurity experience to guide Avertium customers through challenging security problems while keeping business goals and objectives at the forefront. His primary focus is on business development, partner and client engagement, and other strategic initiatives.