Threats to software supply chains became a public issue when the U.S. GAO (Government Accountability Office) called out four national security agencies over the inadequacies of their supply chain security practices. However, the impact of targeted supply chain attacks didn't really hit the papers until eight years later, with the SolarWinds/Orion hack. That exploit gave hackers backdoor access to networks at places such as the Pentagon, Homeland Security, the State Department, plus a good bit of the Fortune 500. Which in turn gave Cozy Bear, also known as the Russian FSB, access to all the lovely secrets on those networks. So, given the implications of that egregious breach of public confidence, I probably don't need to tell you that we have a problem with software supply chains.
The thing that makes the SolarWinds hack newsworthy, and it is indeed extraordinary if national security is important to you, is that it is the first notable example—or at least, the first one we've heard about—of the very thing that the GAO warned Congress about back in 2012. Which is the pathetically inadequate control that the software industry has over its lengthy and intricate supply chains.
Software enables everything, so its compromise threatens the very basis of our society. But then again, software is invisible, which makes it almost impossible to assure with any confidence. Worse, large software systems are built bottom up from top-down designs, meaning the system is integrated in tiers out of simple modules at the lowest level of abstraction up to the final product at the top. And since the aim is to get the job done as quickly and cost-efficiently as possible, the work itself is often offshored.
As a consequence, we find ourselves relying on a highly complex and abstract product that is built through multiple levels of control involving a range of societies and cultures, some of whom aren't necessarily our friends. And to make the matter even more interesting, it's hard to draw intelligent conclusions about the trustworthiness of the denizens of any supply chain since managerial oversight rarely extends past the sub-contractor level. This means that the customer, or even the prime contractor, doesn't really know what's happening at the bottom of a five- or seven-tier supply chain. And so, every customer of every large system is at the risk of buying a proverbial pig in a poke.
In the case of SolarWinds, the malware was injected into the Orion system through a routine update. The after-incident investigation centered on company offices in Eastern Europe, particularly Poland and Belarus, where engineers had access to the compromised update servers. However, every large system is created by integration. So, the dangling potential of some adversary embedding malicious code into your organization's software via the production and sustainment processes ought to keep you up night.
The obvious question is, what shall we do? The U.S. National Institute of Standards and Technology (NIST) has issued guidance for effective supply chain risk management (SCRM). That's in the form of NIST SP 800-161r1, which contains 17 potential categories of controls and points to NIST 800-53 as the source for implementing them. This is essentially the FIPS 200/FISMA process which—to say the least—imposes another expensive layer of complexity on a process that is already highly complex. Nevertheless, NIST has an alternative model.
Although NIST IR-7622 was never an official standard, its humble advice is both realistic and easy to understand by regular managers. It is built around 10 foundational practices that any organization can implement to enhance their supply chain risk management posture. These 10 best practices are:
Uniquely Identify Supply Chain Elements, Processes, and Actors: In simple terms, your first step always has to be to establish top-to-bottom visibility into the entire supply chain. That's because it is hard to manage risk without knowing what its components, processes, and practices look like.
Limit Access and Exposure within the Supply Chain: The problem is that the components of the product can be manipulated as they are integrated up the supply chain. So, the aim is to monitor and enforce access privileges. That's why it's critical to make certain that every access rests on the principle of least privilege.
Establish and Maintain the Provenance of Elements, Processes, Tools, and Data: Every component is important in a system. Therefore, the organization needs to keep a strict accounting of where each of its components originated, its change history, and who might have had access. That will enable greater traceability and managerial control should an adverse event occur.
Share Information within Strict Limits: Information sharing up and down the supply chain is a necessary part of risk management. That can include data about components, individuals, and organizations, as well as details about a component. Consequently, the sharing process must be based on a well-defined and commonly understood sharing agreement that has been worked out among all participants.
Perform SCRM Awareness and Training: Supply chain risk management knowledge isn't innate. It has to be conveyed and reinforced by a formal knowledge dissemination program. That's why a complete program of awareness and training, focused on propagating supply chain-specific best practices is so necessary.
Use Defensive Design for Systems, Elements, and Processes: In essence, defensive design considers all potential failure modes and the factors that might cause them and then builds in response and recovery processes to address each contingency. So, building a supply chain process on defensive design principles strengthens it against attack and aids recovery. This is always done by plan.
Perform Continuous Integrator Review: The integrator accepts modules from lower tiers and assembles them into a product to be passed to a higher tier. The relationship clearly requires a high degree of visibility and traceability up-and-down the supply chain. Therefore, continuous integrator review is an essential practice for managing the assurance of components as they move up the integration ladder.
Strengthen Delivery Mechanisms: Delivery of a routine update was the point where the SolarWinds hack took place. Thus, it is important to have explicit mechanisms in place that are geared to catching and remediating any compromises at the point of delivery. The aim is to ensure that no unauthorized modifications or tampering are transmitted in the delivery.
Assure Sustainment Activities and Processes: Products have to be upgraded, patched, or replaced during their useful lifecycle. But that change process is an opportunity for subversion. For instance, it was a maintenance update that delivered the malware to the Orion system. Therefore, a well-defined and robust sustainment process has to exist, which is aimed at limiting opportunities for compromise during the construction and operation of the product.
Manage Disposal and Final Disposition Activities throughout the System or Element Life Cycle: Poor disposal practices can lead to unauthorized access. But those are not often factored into supply chain risk management plans. Consequently, there must be formal consideration that ensures that all elements, tools, and documentation that are being retired are disposed of safely and securely.
In summary
Since the path to compromise is far too easy and obvious, it goes without saying that we need to get supply chains under control. The principles I've discussed here aren't difficult to understand, nor are they hard to implement. But it takes a willingness to take the first step, and that originates with accepting that a problem exists and knowing that there are simple, practical mechanisms for dealing with it. That is what I have provided here.
These 10 principles are just common sense. But if they are fleshed out in the form of a well-defined and systematic supply chain risk management process, they can be a powerful deterrent to any adversary who is hell-bent on loading your software with malicious code. The decision to do that is up to you.
