OT Vulnerability Management: A Deep Dive

May 23, 2022

OT vulnerability management is the process of systematically mitigating exploitable weaknesses within industrial control systems. We explore what that means and strategies to achieve this efficiently.

Table of Contents

  1. What Is OT Vulnerability Management?
  2. IT vs. OT Vulnerability Management
  3. The Importance of Context for OT Vulnerability Management
  4. How to Identify Vulnerabilities
  5. Vulnerability Prioritization for OT
  6. Patching Challenges in OT
  7. Validating Patches or Remediations for Vulnerabilities
  8. Best Practice Recommendations
  9. OT Vulnerability Management Resources

What Is OT Vulnerability Management?

Operational technology (OT) vulnerability management is the process of systematically mitigating exploitable weaknesses within industrial control systems (ICS). Every environment is different, but good OT vulnerability management has three components: system context, vulnerability indexing, and mitigation efforts. Understanding system context indicates where vulnerabilities might exist. Vulnerability indexing indicates what vulnerabilities might exist. Mitigation efforts seek to assure those vulnerabilities do not impact operations.

This vulnerability management workflow is recognized across the cybersecurity industry and is described in detail by the OT Cyber Security Alliance. Yet, OT vulnerability management is not a simple task. Although the process can be stated succinctly, it is complicated in practice. For example, equipment changes, new threat vectors, and patching challenges all complicate the management process. Fortunately, common principles, best practices, and integrated solutions have been developed and can aide in the establishment of efficient vulnerability management practices in operational environments.

IT vs. OT Vulnerability Management

Before considering the specifics of OT vulnerability management, it’s important to consider how it differs from IT vulnerability management. IT and OT operate in separate environments with unique equipment. These core differences are reflected in their subsequent approaches to vulnerability management. IT operates inside environments such as corporate offices and includes devices like personal computers, phone systems and printers. IT systems operate on standardized equipment. This may increase the likelihood of known vulnerabilities, but simultaneously makes it easier to have predictable windows of downtime during low usage times.

OT systems, on the other hand, provide the backbone for physical processes such as energy production, manufacturing and other industrial settings, which means they often require continuous operation and are more sensitive to periods of downtime. As a result, OT environments incur greater costs when changing or updating equipment. OT systems are uniquely configured and specialized to a control system’s particular needs. This has the potential to obscure vulnerabilities, but also makes patching more difficult. Therefore, OT vulnerability management tends to prioritize threat prevention over frequent patching cycles.

The Importance of Context for OT Vulnerability Management

Avoiding outages and maximizing threat prevention first relies on building contextual information. The main context of an ICS is composed of system assets and the activity taking place inside those assets. This maps directly onto the core practices of asset inventorying and network monitoring – key aspects of OT vulnerability management. Both practices can be combined to provide a baseline image of a system’s threat surface. Additional system context should be provided by insight into network and production level dependencies. Which production processes rely on shared assets? Do such overlapping dependencies result in heighted vulnerability?

The greater extent to which OT endpoint data can be mapped and integrated into the overall contextual picture, the better. Each year, numerous exploits and security incidents arise simply because of insufficient situational awareness. Every subsequent stage of OT vulnerability management relies upon the completeness of contextual information. The success and failure of each subsequent step in the vulnerability management process is contingent on the initial investment into contextual knowledge.

How to Identify Vulnerabilities

After establishing context, it’s possible to begin identifying specific vulnerabilities within a given industrial control system. ISO 27002 defines a vulnerability as “a weakness of an asset or control that could potentially be exploited by one or more threats.” Such weaknesses may arise from software bugs, insecure password systems, or various forms of malware propagation. Effective risk-based vulnerability management, however, generally emphasizes exploits arising from software bugs, as they cover a significant area of most organizations overall threat surface. Of course, it’s impractical for individual organizations to individually discover software vulnerabilities within equipment purchased from various vendors. Instead, security operators maximize their effectiveness by relying on shared advisories propagated by vendors and public sector initiatives.

These advisories are often referred to as common vulnerabilities and exposures (CVEs). CVEs document known weaknesses and system exploits, as well as known patches. According to the Global Cybersecurity Alliance, companies such as Microsoft have become among the largest producers of CVEs. Additionally, public initiatives such as the NIST's NVD and CISA's Known Exploited Vulnerabilities Catalog provide additional sources for vulnerability indexing. Effectively combining system context with these databases results in effective OT vulnerability management.

Vulnerability Prioritization for OT

Once identified, security operators must determine a framework for prioritizing vulnerabilities. The best approaches will distinguish between the type and severity of risk associated with a given vulnerability. Types of risks can be divided into safety, legal, and business. Safety risks involve threats to employees, end users, and the public. Limiting the potential for physical harm will always be a priority across industries. Legal risks, on the other hand, could arise from industry-specific regulatory standards. Meeting compliance reporting requirements and security standards may be of concern to specific types of industrial control system operators. Finally, business risks can be both monetary and reputational. Vulnerabilities that delay production will result in financial losses and have the potential to decrease consumer confidence.

Within these types, vulnerabilities can be further divided into their level of severity. The most authoritative metric is the Common Vulnerability Scoring System (CVSS). This index provides a common scale for interpreting the urgency of various exploits and is used within the NIST database of vulnerabilities previously mentioned.

There is no universal model for correct prioritization. Each organization must use contextual information about their own assets to determine an effective OT vulnerability management model that considers vulnerability types against severity in a manner reflective of the business’ priorities. As an example, if an affected asset uses a serial connection, there is little chance that a vulnerability would be exploited remotely. However, some vulnerabilities are easy to exploit and sitting on a critical, IP-connected asset and should be mitigated right away.

Patching Challenges in OT

Once a framework for prioritization is established, mitigation efforts can be implemented. Patching is the most common of these strategies but comes with challenges. As discussed, published CVEs and other documentation share methods for updating systems to eliminate vulnerabilities. But these databases are not able to take into consideration the specifics of your own ICS environment. Therefore, the challenges of resource constraints, availability requirements, and local hardware dependencies must be overcome to maintain effective patching procedures.

Resource constraints are likely to be seen most within available man hours. Maintaining records of relevant patches and the status of individual assets can become a very labor intensive process. Furthermore, in some cases, patching requires operators to be physically near the various assets. Even when patches are identified, the appropriateness of implementation will need to be weighed against availability requirements. Patching may require system downtime and will increase the chances of production disruption. Depending on the severity of the threat and possibility of alternative mitigation efforts, this may or not may not be advantageous.

While patches are designed to eliminate specific vulnerabilities, they do not consider the specifics of a given ICS. As a result, unique interactions between local hardware may prevent patch implementation, leading to a requirement for alternate forms of mitigation. Each of these considerations factors into the challenging question of patching.

Validating Patches or Remediations for Vulnerabilities

Once a patch is implemented, the process doesn’t stop there. Given the unique configuration of various OT systems, patches have the potential to cause unintended consequences within the larger control system. For this reason, it may be appropriate to troubleshoot and validate patches before pushing updates to an entire system. Additionally, having validated backup configurations is advantageous in case patches need to be reverted.

Patches, especially many at once, can have damaging consequences if reversion is not an option. Finally, investigating alternative mitigation efforts will be necessary if a system patch is unavailable or not appropriate for a given network configuration. Alternate solutions range from equipment replacement to added security layers, such as monitoring for configuration changes on an endpoint.

Best Practice Recommendations

Given the complexity of OT vulnerability management, it is important to streamline the workflow. At least three practices can help ensure vulnerability management is implemented effectively:

  1. Invest in robust asset inventorying: Vulnerability management is only as effective as the context it operates within. Investing in complete, continuously updated asset inventory is critical to effective vulnerability mitigation.
  2. Review management priorities: As production processes, legal regulations, and consumer interests change, it may be necessary to revise priorities. Keeping an updated framework of critical aspects that accurately reflect production processes is essential to achieve effective vulnerability management.
  3. Combine automation with manual processes: Important aspects of vulnerability and patch management lend themselves to partial automation. Learning to integrate these solutions into the OT vulnerability management workflow can multiply the effectiveness of your team and increase the resilience of OT networks.

OT Vulnerability Management Resources

Third-party solutions exist, and are increasingly necessary, to aid organizations in effectively implementing OT vulnerability management. Organizations with large asset inventories should seriously consider these integrated solutions. With Industrial Defender’s approach to comprehensive OT asset data collection, our platform is able to provide an accurate, complete, and up-to-date view of known vulnerabilities across your OT environment.

Another useful resource is the OEM patch management solution offered jointly by Industrial Defender and FoxGuard Solutions. The solution automates the patch inventorying process and tracks the release of recent patches, empowering operators to make smarter patching decisions. You can view how the solution works here.

Learning how to maximize efficiency within OT vulnerability management is the key to success. With so many challenges and so much complexity involved in this process, being strategic about what you fix first and how you fix it is important to avoid getting lost in an ocean of vulnerabilities.