IMPORTANT ADVISORY
The following blog post addresses a recently uncovered major cybersecurity attack that was spread through an update to the SolarWinds Orion network monitoring software. This attack has major implications for both iON clients and any organizations using SolarWinds Orion.
FireEye refers to the backdoor as SUNBURST. They are tracking the campaign as UNC2452. Microsoft has labeled the attack “Solarigate” in Windows Defender (the latest Windows Defender update detects and blocks this attack).
Background
- SolarWinds is a software company best known for its systems management tools used by IT professionals
- Among the most widely used SolarWinds products is Orion, a Network Management System (NMS)
- The Orion NMS has broad capabilities for monitoring and managing systems including:
- Servers
- Workstations
- Network Devices
What Happened
- On December 13, 2020, Chris Bing of Reuters reported that the United States Treasury Department had been compromised by a sophisticated adversary.
- Using background sources, Washington Post reporter Ellen Nakashima later confirmed that:
- The Treasury Department breach was perpetrated by an Advanced Persistent Threat Group dubbed APT29 (aka Cozy Bear), a group known to be part of Russia’s national foreign intelligence service, the SVR.
- FireEye, a top U.S.-based cybersecurity firm, was also breached using this method (this incident was first reported by the Washington Post on December 8, 2020).
- FireEye confirmed that other victims included North American government, consulting, technology, telecom, and energy companies.
- The attackers apparently modified SolarWinds source code causing the malware to be distributed in an update of the Orion software released in March and June. Since the malware was embedded in a legitimate release from SolarWinds, it was signed with a valid digital certificate issued by Symantec.
- While this is not the first instance of state-backed APT groups targeting software vendors or masquerading as updates to deploy malware payloads, this attack appears highly sophisticated:
- It uses anti-analysis countermeasures
- The APT group appears to have used specific infrastructure for each victim, rendering the attack largely invisible in terms of conventional network-based Indicators of Compromise (IOCs).
What to Do If Your Organization Uses a SolarWinds NMS
CISA’s Emergency Directive to Mitigate the Compromise of SolarWinds Orion NSM (directed at U.S. government agencies but useful to any organization) is available here:
https://cyber.dhs.gov/ed/21-01/
Additionally, FireEye has released a set of countermeasures on GitHub, available here:
https://github.com/fireeye/sunburst_countermeasures
To Network Administrators and Security Officers, iON recommends the following:
- If you run SolarWinds Orion versions 2019.4 through 2020.2.1 HF1, assume it is compromised.
- Further, until more about the attack campaign is known, do not assume that only the published versions have been compromised.
- Immediately disconnect or power down SolarWinds Orion products, versions 2019.4 through 2020.2.1 HF1, from your network.
- Block all traffic to and from hosts, external to the enterprise, where any version of Solar Winds Orion software has been installed (i.e., take a Zero-Trust networking approach).
- In most cases, even East/West netflow will be of limited value since the NMS talks with so many devices.
- Identify and remove all threat actor-controlled accounts and identified persistence mechanisms.
- Update your SolarWinds platform as soon as possible to version 2020.2.1 HF 2 (scheduled release date: Tuesday, December 15). Furthermore, we recommend that you perform a complete rebuild of the server on which Orion is installed (both the operating system and the NMS) if the time and effort required is reasonable.
If your SolarWinds Orion NMS is compromised, the following are likely also at risk:
- Any data monitored, backed up, or managed by Orion, including access credentials and configurations.
- Any system monitored, backed up, or managed by Orion.
- Firewalls rule sets are also at risk of having been changed.
In response, we recommend focusing your search for past Indications of Compromise (IOCs) in the network and application logs for at-risk systems instead of focusing on current network activity. This is because the attacker infrastructure will have been eradicated by both the mitigations listed here and the Windows Defender update, so previous activity records will best indicate the extent of a compromise. Also consider:
- The attacker is clearly OPSEC aware and will likely have changed any file system-based IOCs.
- The attacker is probably already retooling, so do not anticipate finding any active IOCs for SUNBURST malware.
- FireEye noted that this code does not overlap with other malware.
NOTE:
FireEye has pointed out three important traits of the malicious code:
a. The malware checks file system time stamps to ensure the compromised version of Orion has been deployed for 12-14 days before it goes active.
b. This effectively prevents the use of malware sandboxes and other instrumented environments to detect the malware.
a. Unless your malware sandbox machine is joined to a domain, the malware will not execute.
b. Are your malware sandboxes (or other instrumented environments) domain joined?
3. DNS Resolution and IP Address Checks
a. If the malware resolves a domain to a private IP address, the malware will not execute (most malware sandboxes intercept DNS and point traffic to themselves for analysis).
What to Consider if You Do Not Run SolarWinds Orion
If your organization does not use Orion but does run another NMS, we advise increased caution and vigilance. Network monitoring software has broad access to your internal network, making it a high-value target for attackers.
We therefore recommend reviewing your network logs for any anomalous activity by your NMS. In particular, look for any communication on the Internet with unknown or suspicious hosts.
We also advise restricting NMS permissions and access (account and network access) as much as possible.
Conclusions
The discovery of this exploit in SolarWinds’ Orion NMS, and the apparent global espionage campaign it enabled, has prompted a great deal of discussion in the cybersecurity community. One figure we hold in high regard, Alex Stamos of the Stanford Internet Observatory, suggested that this incident has exposed a critical flaw in the way organizations conduct IT vendor risk management, calling it “incredibly expensive and mostly useless” in most instances, and “when decent, it happens too late in procurement.” We are inclined to agree, although we contend that an element of trust will always be implicit regardless of how much diligence a buyer puts into their purchase of an IT/IS product or service.
Case in point, a forensics expert at the SANS institute decompiled the .NET code of the compromised Orion software and determined that its source code was compromised in ways that should have been easy to detect with a cursory evaluation, but SolarWinds compiled and signed it anyway. He concluded that the vendor’s shortcoming in this instance was primarily a failure of source code change control and auditing. Considering this expert’s opinion, can any vetting process ever provide certainty to the buyer that a vendor will consistently audit their code adequately before compiling and signing it?
As for FireEye, we do not believe their breach was the result of carelessness or lack of vigilance on their part. Ultimately, this was an advanced attack that was ingeniously embedded and disguised in the Orion source code by a party authorized to modify it. We believe FireEye’s vigor in investigating and providing steps for mitigation have demonstrated their commitment to owning the fact they were breached and being a part of the solution.
In more general terms, this discovery drives home the importance of core security precepts that iON consistently preaches in sessions with our clients:
- The importance of proper network segmentation
- The need to reduce network privilege/access to the least privilege necessary for both users and tools/services
- The importance of understanding normal baseline activity on your network and the ability to detect unusual behaviour
Following these precepts both limits the impact of these types of attacks and accelerates your ability to determine their extent.
Sources and Additional Reading
iON credits the following articles as sources for this post.
This is a rapidly developing situation and we strongly recommend Network Administrators and Security Officers of organizations affected by this attack stay abreast of the latest developments.
The following links are good places to get additional detail and receive further updates:
https://www.sans.org/webcasts/emergency-webcast-about-solarwinds-supply-chain-attack-118015
https://twitter.com/KimZetter/status/1338305089597964290
https://twitter.com/KimZetter/status/1338389130951061504
https://twitter.com/KimZetter/status/1338389141080350720
https://msrc-blog.microsoft.com/2020/12/13/customer-guidance-on-recent-nation-state-cyber-attacks/
https://twitter.com/razhael/status/1338267165221396480
https://twitter.com/C_C_Krebs/status/1338348746426474498
https://thehackernews.com/2020/12/us-agencies-and-fireeye-were-hacked.html
