Colonial Pipeline Hack
Revisiting the facts after a week of news
One of my articles on Data Breaches.
Free Content on Jobs in Cybersecurity | Sign up for the Email List
Any time a major cyberattack occurs, news organizations of all types rush to get out the news and report on the available facts to get the “scoop”. In my classes, I warn students to wait a few days before coming to a conclusion as often the facts evolve as new information emerges. Let’s take a look at the big picture now that we have more facts, starting with this one:
The pipeline was restarted as of Wednesday, May 12th.
If you are out of gas at your local station it is on the way.
Now let’s start from the top:
On May 7, the Colonial Pipeline Company learned it was the victim of a cybersecurity attack. We have since determined that this incident involves ransomware.
The Colonial Pipeline took systems offline:
In response, we proactively took certain systems offline to contain the threat, which has temporarily halted all pipeline operations, and affected some of our IT systems. Upon learning of the issue, a leading third-party cybersecurity firm was engaged, and they have launched an investigation into the nature and scope of this incident, which is ongoing. We have contacted law enforcement and other federal agencies.
Standard Incident Response Involves Shutting Down Systems Infected with Malware.
When systems are infected with malware a series of steps are taken to limit the damage and restore systems. SANS Institute has an acronym for this process. PICERL. This acronym stands for:
- Preparation
- Identification
- Containment
- Eradication
- Recovery
- Lessons Learned
The phase of containment involves taking systems offline so they don’t infect other systems. If you want to understand why this is important, read about the damage caused by the WannaCry or NotPetya ransomware that spread around the world and took down of systems, including hospitals, in over 300 countries. You may also want to read Countdown to Zero Day: Stuxnet and the Launch of the World’s First Digital Weapon by Kim Zetter, the world’s first cyber weapon, and what it did to industrial control systems in Iran.
How an organization “shuts down” systems may vary depending on the risk and type of systems involved. An organization may block that system from any network access while they investigate. They may also completely shut down the system. Either way, the system does not have network access to infect other systems during containment.
The next phase, eradication, means wiping the malware from the system or completely rebuilding the system. You might think, why rebuild the system? Just get the malware off and run it again? However, some malware embeds itself into systems in ways that make it impossible to eradicate without rebuilding the system. In any case, this process takes time.
After the malware is wiped from the system, the organization moves into the recovery phase and restarts the systems. After the systems are restarted and evaluated to ensure the malware has truly been eradicated. Typically, business personnel need to test and ensure the system is functioning properly.
Colonial Pipeline involved law enforcement and a third-party company.
Often an organization will involve law enforcement in the process, and sometimes companies that specialize in cybersecurity incident response. In this case, Colonial Pipeline wrote:
Upon learning of the issue, a leading, third-party cybersecurity firm was engaged, and they have launched an investigation into the nature and scope of this incident, which is ongoing. We have contacted law enforcement and other federal agencies.
Why does it take so long?
It sounds like many systems went offline. Some have reported that these were “only billing systems” not “operational systems” so they should have kept shipping gas. I can tell from experience working at an oil company when I got out of college that it is not that simple.
First of all, the company is not going to give out free gas. I know people would love that, but be realistic. They need to be paid to stay in business and continue to operate. Do you want the pipeline to exist and be able to provide gas in the future? It has to be restarted in such a way the business can survive. The concept of determining how long a business can take systems down and not go out of business is called business continuity planning (BCP).
More importantly, a lot of these systems are interconnected. I don’t work at this pipeline but, likely, you can’t just flip one system on or off without affecting other systems. For example, there may be checks that prevent gas from flowing unless the billing systems operate a certain way. These checks and reconciliation processes are designed to prevent mistakes and detect incidents such as the one that occurred.
Finally, the security organization has to make sure the malware doesn’t escape and infect other pipelines in the United States, customer systems, or operational systems within Colonial Pipeline. If the malware is still there and in any way can impact the operational systems, something worse could happen. Refer to my link to the book on Stuxtnet above or read one of the other books in my list of cybersecurity book reviews. Sure, in a perfect world, there are no connections between the administrative and operational systems. But in the case of an attack on an electric grid in Europe, one bastion host (also known as a jump host) allowed attackers to bypass segmentation.
Whether or not the company pays the ransom, they need to restore systems. They may be rebuilding from clean operating system installs and code, trying to get around and eradicate the malware, restoring from backups, or working with the criminals to restore systems after paying the ransom. Kim Zetter wrote a great article about restoring systems after paying a DarkSide ransom I found after writing this post so adding it here. It gives some additional perspective into why restoring systems takes so long in the event of a ransomware attack.
Here’s the Colonial Pipeline statement on the matter of safely restarting the pipeline:
The Colonial Pipeline operations team is developing a system restart plan. While our mainlines (Lines 1, 2, 3 and 4) remain offline, some smaller lateral lines between terminals and delivery points are now operational. We are in the process of restoring service to other laterals and will bring our full system back online only when we believe it is safe to do so, and in full compliance with the approval of all federal regulations.
Part of the restoration of the pipeline included physically inspecting the pipeline according to this statement:
Tuesday, May 11, 5:15 p.m. — Consistent with our safety policies and regulatory requirements, Colonial has increased aerial patrols of our pipeline right of way and deployed more than 50 personnel to walk and drive ~ 5,000 miles of pipeline each day.
In this case, because this is critical national infrastructure and distributing a substance that could cause a major explosion, some federal regulations apply as indicated by the statements above.
While restoring systems the pipeline delivered gas on a limited basis.
Sunday, May 9, 5:10 p.m. — The Colonial Pipeline operations team is developing a system restart plan. While our mainlines (Lines 1, 2, 3 and 4) remain offline, some smaller lateral lines between terminals and delivery points are now operational.
Monday, May 10, 12:25 p.m. — Segments of our pipeline are being brought back online in a stepwise fashion, in compliance with relevant federal regulations and in close consultation with the Department of Energy, which is leading and coordinating the Federal Government’s response.
Monday, May 10, 7:50 p.m. — We can now report that Line 4, which runs from Greensboro, N.C., to Woodbine, Md., is operating under manual control for a limited period of time while existing inventory is available.
Tuesday, May 11, 5:15 p.m. — Colonial Pipeline continues to make forward progress in our around-the-clock efforts to return our system to service, with additional laterals operating manually to deliver existing inventories to markets along the pipeline. Markets experiencing supply constraints and/or not serviced by other fuel delivery systems are being prioritized. […] Since our pipeline system was taken offline, working with our shippers, Colonial has delivered approximately 967,000 barrels (~41 million gallons) to various delivery points along our system. This includes delivery into the following markets: Atlanta, Ga., Belton and Spartanburg, S.C., Charlotte and Greensboro, N.C., Baltimore, Md., and Woodbury and Linden N.J.
Thursday, May 13, 9 a.m.- Colonial Pipeline has made substantial progress in safely restarting our pipeline system and can report that product delivery has commenced in a majority of the markets we service. By mid-day today, we project that each market we service will be receiving product from our system.
The Department of Transportation’s Federal Motor Carrier Safety Administration issued a regional emergency declaration to provide emergency transportation for fuel.
By execution of this Emergency Declaration, motor carriers and drivers providing direct assistance to the emergency in the Affected States in direct support of relief efforts related to the shortages of gasoline, diesel, jet fuel, and other refined petroleum products due to the shutdown, partial shutdown, and/or manual operation of the Colonial pipeline system are granted relief from Parts 390 through 399 of Title 49 Code of Federal Regulations except as restricted herein.
Those restrictions relate to weight limits and bypassing weigh stations to get gas to destinations more quickly, among other things.
The shortage was caused by panic buying, not an actual shortage.
Colonial Pipeline completely restarted Thursday, May 13, 4:40 p.m.
Thursday, May 13, 4:40 p.m. — Colonial Pipeline has continued to make substantial progress in safely restarting our pipeline system. We can now report that we have restarted our entire pipeline system and that product delivery has commenced to all markets we serve.
The malicious actors were identified as a group named DarkSide
Likely while analyzing the malware activities and details in communications and networking while coordinating paying a ransom, the FBI was able to glean enough information to determine the source of the attack. The attackers were identified as a group called DarkSide out of Russia. According to public statements by President Biden and others, they do not yet have a link between this group and the Russian government.
A company called Elliptic says it identified the Bitcoin wallet that received the ransom.
Colonial Pipeline paid a ransom
The Colonial Pipeline paid a ransom of $5 million. Paying ransomware criminals only encourages them to continue to exploit systems and demand payments in the future. Most of the time I think companies should not be allowed to pay and insurance companies should not be reinforcing this behavior. However, in this case, critical infrastructure was impacting our American way of life in a substantial way. Additionally, some politicians, social media campaigns, and news organizations were inciting hysteria and making things worse. For this incident specifically I feel that paying the ransom was the right thing to do because it addressed the immediate problem and I suspected that there would be repercussions as a result — more so than ever in the past. The attackers crossed a line.
In the worst breach of American national security via a cyberattack in the SolarWinds Hack last December, it appeared that no action was taken in response to the infiltration of systems for months that sent sensitive information to Russia. In this case, paying the ransom may have helped identify the attackers and shut down their operations with the help of law enforcement.
DarkSide lost access to infrastructure and funds
One way or another, it appears the group was affected. The systems used by the attackers are shut down. Their bitcoin wallet used for ransomware payments is empty. The exact details as to whether the group went out of business or someone seized their funds are not entirely clear. It is clear that their business is no longer operational and other ransomware groups have been impacted by a domino effect. Check out these articles for more information.
Preventing Future Attacks
What have we learned from this attack? The same things cybersecurity professionals have been saying for years. As I mentioned before you can check out the cybersecurity books I’ve reviewed for more information on cybersecurity history and that this is nothing new. It’s finally happened and a critical infrastructure attack has had a major impact on the average American.
For this specific attack, FireEye has provided some specific indicators of compromise (IOCs) that organizations can use to determine if they have been compromised.
But what organizations need to understand is that these IOCs will be quickly out of date and unhelpful in preventing future attacks because they will change. Organizations need to understand and get back to cybersecurity fundamentals, as explained in my book, Cybersecurity for Executives in the Age of Cloud if we want to stop these breaches. We can extrapolate from many attacks the common patterns that always apply and the tactics so commonly used that can help us identify and stop data breaches using higher-level principles than specific IP addresses or code in a particular piece of malware.
It also helps to have security professionals on staff:
Alternatively hire outside consultants when gaps exist. Many people do not want to work directly for a company (such as myself) but are happy to provide consulting services.
Social Media and News Hysteria
For the past week, I’ve been monitoring Social Media, politicians, the affected company, and various news sources. All I can say is that I am very concerned for our national security in this country. Instead of focusing on the root problems, politicians and news organizations were creating false narratives and pointing fingers to try to inflict political damage at the expense of the American people.
The first thing I noticed was a social media propaganda engine freaking people with photos of long lines at gas stations. Images showed people filling up gas cans and plastic bags to hoard gas. These images gave people the impression that if they didn’t get gas now they would never be able to get any, when in fact, the pipeline stated it would be operational by Friday. It was.
When I read the first reports about the pipeline, I recommended that my boyfriend fill up his tank with gas. I didn’t tell him to rush or fill up extra cans and store them in the basement. Just fill up when he got a chance. Why?
NOT because of the cyberattack — because I predicted two things:
1.) The “Toilet Paper Effect” (You know what I’m talking about.)
2.) Gas price gouging taking advantage of this short-lived problem.
Yep, just as expected, but worse after the fans were flamed by crazy social media posts creating false narratives trying to inflict political damage instead of focusing on the real problem — we need better cybersecurity in this country to protect critical infrastructure to prevent problems like this in the future and worse.
An appropriate response addressing the root problem:
Regarding those bags of gasoline:
Aligning the attack with Biden’s energy policy. His policy wouldn’t help and people would know that if they understood cybersecurity and the root problem here. Unless you get electricity completely off the grid, a cyberattack can affect you just like it did in the case of this pipeline. Read Sandworm. The root problem here is ransomware attacks by criminals that need to be dealt with and companies that don’t properly secure systems.
Comparing the Colonial Pipeline attack to voting system cyberattacks. There was no indication of a cyberattack successfully infiltrating voting systems. This is a statement with no proof to back it up.
Blaming Biden for shutting down the pipeline. I already explained above that Biden did not shut down the pipeline and why that happened. These statements came from politicians who made no comment on the cyberattack or provided any information about what they were going to do to address the core problem that led to people not being able to buy gas.
Tweets asking people if they missed Trump now. The propaganda campaign that immediately ensued seemed so completely coordinated that it made me wonder about involved of others in this attack. The talking points across the board were so in sync and the messages were targeted and repetitive. We’ll have to watch as the investigation continues to see if any new information comes to light.
People were complaining that gas prices had gone up. Yes, when you panic buy prices go up, just like they did when toilet paper was in short supply during the pandemic. That is due to the law of supply and demand. I suspect prices will drop now that the pipeline is operational again. Wait and see. If they don’t your local gas station is taking advantage of you.
Side note those saying prices were going up prior to the pipeline issue: When no one was driving while holed up over Covid people likely didn’t buy gas. Demand was low. Prices went down. People got vaccinated. They started driving more (including me!) Prices go up. Supply. And. Demand. You can see the peak of cases in the graph below. Cases started falling sharply as soon as people started getting vaccinated. Gas prices likely started going up about the same time.
Posts were blaming Biden for the cyberattack. I already mentioned we had a much worse cybersecurity breach of national security in December. The response to that breach was nearly nothing from the federal government. In the case of the Colonial Pipeline Attack, the attacker’s infrastructure was taken out in days, they are out of business, and I suspect this is not over. Keep watching. Biden also issued an executive order on cybersecurity.
Security professionals need to help educate the general public
When people get tired of ridiculous statements and politics they tend to tune it out. The problem is that when you tune it out these thoughts and propaganda campaigns exist in a vacuum and spread. For those working in cybersecurity I ask you to get out there and inform people on social media making inaccurate statements, not with attacks, but with information. People don’t understand how cybersecurity works and are easily misinformed and swayed to believe things that are not true. This lack of understanding could lead to even more insecure systems in the future.
Educate yourself on cybersecurity fundamentals
For the general population, ask your elected officials what they are going to do to protect systems from cyberattacks like this. Make sure they are focused on the root problem and providing a solution. Learn something about cybersecurity. I didn’t see one question about cybersecurity in conjunction with national security in the presidential debates. We need elected officials who understand it and will protect our country in the era of cyberwar and rampant cyberattacks.
Follow for updates.
Teri Radichel | © 2nd Sight Lab 2021
About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight LabNeed Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for PresentationFollow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab
