Advice from the pros: How to deal with a data breach

I’ve a confession to make: I hate writing about data breaches. They’re so frequent these days that I’m sure we’ve all been harmed by multiple breaches per person, whether we’re aware of it or not. I’ve a feeling that for every data breach that’s reported in the media, there are hundreds that people never hear about or the company isn’t even aware of.

The data breach phenomenon will escalate over time. So the more infosec professionals can help educate the businesses they work for and implement best practices, the more we can protect them from the worst effects of a breach.

In the Security Bytes series, I share senior security professionals’ most savvy advice. This time I spoke with cybersecurity professionals who were courageous enough to share their data breach stories with me. Each professional had to deal with a data breach incident firsthand. I’ve kept some details anonymous given the sensitive topic. But the insight here can help you respond to data breaches more effectively.

The question:

What did you do when you were faced with a critical data breach?

Daniel Ruf, Developer and Security Consultant

First responses are critical to contain a data breach. Daniel Ruf was contracted to work for one anonymous project owner. Ruf described the first thing that happened that led to the discovery of a data breach. His quick and careful actions are a great example of how to handle a breach.

“One evening, through a system monitoring alert, I became aware of a 99 percent CPU usage of our root server which impacted the availability of other websites using the same server. I informed the project owner of the hacked website, then started my first analysis. Next, I blacklisted the IP address of the attacker, blocked system access, then killed the suspicious processes. I archived as many files as I could from the project owner’s web server for further analysis and conducted a system integrity check, then informed the project owner.

“The culprit? A hacked CMS instance. I wasn’t informed about the creation of this instance, which then used different webshells to start attacks, including cryptojacking (illegal cryptomining) which used up all our available computing resources.”

The major factors that led to the breach were the use of weak and reused passwords and an overall lack of robust security measures. Don’t let this happen to you!

Teagan M, Security Practitioner, Founder Green Duck Consulting, LLC

Teagan described the data breach she dealt with. This is a common way that people can have their data compromised at work. Smaller companies and start-ups must be careful to be more thorough about their security, and realize that it’s not just a problem that can be resolved with technology alone.

“An authorised source had access to an email environment that was unauthorized and lasted, the first time, for four weeks. This company was very disorganized and treated security as a compliance issue instead of an actual business function.

“The credentials to the user account were most likely captured by a phishing attack, and the attackers set up an email forward rule to send all emails to a Gmail account controlled by the attacker. The second time it happened, it was the same thing but I caught it much sooner. It was after the second event the company finally followed my recommendation to implement 2FA (two-factor authentication). That event still resulted in an exposure which required the company to report the incident under state data breach laws.

“This was a small company that was growing fast. They didn’t have the leadership in place they needed, nor did they want to properly compensate the necessary security talent. They burned through many security analysts due to weak and inexperienced security C-level management and a desire to appease audit requirements instead of securing the business.

“They considered security to be a technology problem and continued buying services and equipment with no clear plan about how to integrate or manage any of it. They took their strategy from sales reps instead of properly planning their own path.

“I worked closely with the C-level leadership to try to improve their program and was met with resistance the entire time. They’d give security ‘lip service’ but no action or empowerment. Things started to change slightly towards the end of my tenure there, but by that point I’d been burned out and needed to leave to preserve my own mental state.”

This naivety Teagan dealt with is one of the reasons why data breaches are so alarmingly frequent.

‘BM‘, Former Infosec Analyst

It’s important to remember that large companies need to improve their data security too. BM shares a useful lesson to help anyone who secures data in any industry.

‘BM’ previously worked as an analyst, security engineer and later a consultant, for multiple companies. They’ve dealt with post-breach data forensics, incident response and triage work. ‘BM’ described a breach they dealt with.

“When I was a consultant, I was sent to a Fortune 500 company for an incident response event. Their system was completely compromised and the attacker was threatening the CEO of the company to expose the breach. We immediately jumped on a flight.

“For a full week, my manager and I reviewed the security logs and developed a remediation plan for the company. Although the company was very valuable, they had a relatively small IT staff. Much of our work was educating the team about exactly what happened, reviewing their logs, and working with them to upgrade their infrastructure. For example, they were running older versions of Windows on their entire network.

“Fortunately, the incident was determined to be minor. It was pretty much a ‘script kiddie’ using easily available software to exploit unpatched versions of Windows. This turned into a full week of over twenty-hour days and numerous conversations to determine what could’ve been done better.”

Over twenty-hour days? That’s ridiculous! It’s important to educate all of your employees about security awareness. And also to make more effective use of your security staff – don’t let it become a crisis management situation for external consultants to resolve.

Sameep Agarwal, Information Security Consultant and Penetration Tester

I’ve spoken with Sameep before about his data breach experiences – he’s experienced several in his career. This story illustrates how personal conflicts can interfere with incident response and why it should be overcome.

“A server was provided by the vendor for hosting a specific web application in a test zone. One day, flags were raised by the security team over an untested and unverified application. It was decided that since the hosting was being set up in a hurry, server hardening activity, which requires at least five working days, couldn’t be completed. So internet access to the test zone server was never allowed in the initialization.

“Since the vendor representative wasn’t present at the client’s location, server access was requested for remote administration. This was also raised as a security concern, but was overridden by the federal agency.

“Operations on the targeted server started. After three days, a few updates from Microsoft were installed without verification of the update bundle. The virtual server was residing on obsolete out-of-life hardware taken from a previous government project which had many critical flaws. Since the physical server was out of life, it was not recorded in the federal agency’s inventory. This meant intrusion prevention and anti-malware clients were not installed but it was accessible on their network infrastructure.

“The attacker installed Telegram’s desktop application, and joined the Iranian hacker group on Telegram. They started downloading applications for anonymity like proxy tools, VPN and bots. Later, the attacker added many dictionaries and combo lists containing common passwords for applications for cracking social media accounts like Instagram and gaming accounts like Fortnite.

“The affected server traffic wasn’t directed through the firewall because of exceptions made by the federal agency override. And because of the exception, the data breach couldn’t be easily detected.

“The intelligence agency investigation concluded that the blue team (defensive security specialists) made many exceptions based on personal relations to please the federal agency head. The key security findings of the red team (security testing specialists) were ignored time and again. The investigator suggested a deep probe on the intentions of insiders, and the psychology which allowed the attacks to happen, even after prior threat intelligence was available a month in advance.”

People are often the weakest link in security. It’s important to have IT staff who have integrity and the right level of security awareness training.

How to protect your business from data breaches

Data breaches aren’t purely technological in nature, they’re also problems caused by people. The dangers posed by cyber-attackers is significant, from ‘script kiddies’ to advanced persistent threats (APTs). None of this surprises me. What did surprise me was how the businesses my interview subjects worked with worsened the data breach problems through a lack of personal integrity, or sometimes just through carelessness.

Businesses need to start taking security more seriously by spending money, time and effort to better protect their, and by virtue their customers’, data.

I’m hopeful that by sharing these stories with you, cybersecurity professionals will be better prepared for the (not so) good, the bad and the ugly of data breach incident response.