Incident Response – Lessons to be Better Prepared
by Jeffrey Reich, Senior Information Security Instructor, CIAS
Incident Response means different things to different people. For the past 40 years, my experiences have taught me that incident response addresses the question: “How well did you prepare?” Your response is directly related to your preparation. This supports the adage that an ounce of prevention is worth a pound of cure—or response.
As a consultant, I have dealt with incidents that ranged from significant annoyance to life-threatening and everything in between. In each engagement, I have found similar conditions. Let’s take a look at five basic lessons that I learned while mitigating these incidents to help you prepare and prevent a potential cyber incident.
Lesson One
No matter how difficult it may seem to quantify every asset in your organization, it is always easier to do before you have to respond to an incident.
What are your IT high value assets and where are they? Many organizations struggle with this question. To help you get started, read the article “Have you identified your high value assets?”
Additionally, the CIAS is currently piloting a program to help communities identify their high value assets, so I encourage you to reach out to them for assistance if needed.
You will need to determine a methodology to find all of the assets in your organization, quantify them and assign a value to each. Once you have started the process, protect the investment of time and effort that you are spending by having a system to track all of these assets and update that inventory continuously.
When you know what you have and what it’s worth, you can take the right steps to protect what is most important.
Good tip: People are your most important asset. Yes, sometimes that is just a management feel-good statement; even so, make it the foundation of any program that you run. Most things can be replaced, people cannot.
Lesson Two
Deal with what’s already broken.
As you identify all of your assets, learn from the people that deal with them every day. It is very likely that they know more about it than you and they know what is working well and what isn’t. Asking what sort of failures occur—and why—will give you the insight to take steps to avoid those failures. Implement processes as a result of that and you may have prevented a future incident.
As you address known and repeated failures, use your inventory insights to find single points of failure. Even when a given failure has not occurred yet, expect that every piece of hardware will break at some point; every software program or application will fail; a dependency on a given individual will face a day when that individual is not available.
All of this information gives you the opportunity to prevent incidents by removing single points of failure. This can be done by duplicating a piece of hardware, modifying a process to work without the availability of software, and ensuring that at least two people are able to perform any given critical task.
Lesson Three
Plan for the outcome, not the cause.
As you begin writing your incident response plan, work on avoiding the trap of predicting potential failures, attacks, etc. You may be surprised when the cause of an incident is something that you did not predict and did not expect. Consider focusing on the results of incidents rather than the cause. I am not ignoring the value of root cause analysis (RCA). Rather, I’m simply stating that, in the immediate term, dealing with outcomes is more straightforward than dealing with causes, which may not be known.
According to the Cybersecurity and Infrastructure Security Agency (CISA), today’s risk landscape includes:
- Insider Threats
- Acts of Terrorism
- Cyber Attacks
- Extreme Weather
- Pandemics
- Accidents or Technical Failures
Realistically, you cannot address every cause listed within each threat. As an example, of the six risks listed above, I can think of four that, when realized, could each cause you to not be able to enter your office building for an undetermined amount of time. These include, and are not limited to, Acts of Terrorism, Extreme Weather, Pandemics, and Accidents or Technical Failures.
You will want to determine the cause (more on this in Lesson Five). First, however, you must deal with the situation of not being able to enter your office building. Your plan should address how to deal with “no office.” As of 2022, we have the experience of the previous two years of remote work to help us through that one.
Continue building your plan for outcomes that are related to your most important assets. You could use some CISA Cybersecurity Incident & Vulnerability Response Playbooks as templates. They can be found at CISA.gov. Keep in mind that not every scenario would apply to you.
Lesson Four
No plan survives first contact – Helmuth von Moltke.
Now that you have your plan, let’s see how it works. There are a few ways to test your plan, each with its own level of investment and risk.
| Test Type | Investment Risk |
| Read Through | Minimal time/may miss key scenarios |
| Table Top | Measurable portion of a day, or more/easy to digress into non-relevant discussions |
| Parallel Test | Run systems and processes to test in parallel environment/May be expensive |
| Full-Failure Test | Full interruption of operations/Expensive |
Each one of these tests brings many benefits. Some Table Top Exercises available from CISA can be found at https://www.cisa.gov/cisa-tabletop-exercises-packages. After each test, always find what does not work and revise the plan accordingly. Repeat your tests regularly. The more your processes change and evolve, the more you need to test.
Lesson Five
After an incident, when you do not know what caused it, you will suffer it again.
Most importantly, when your plan does encounter first contact, Follow the steps of Containment, Eradication & Recovery and Post-Incident Activities, which include Root Cause Analysis (RCA). In RCA, you identify the problem, analyze the problem and identify the underlying cause of the problem. This can be the most painful activity after an incident. Proper RCA always tells you the one event, process or trigger that occurred that, had it been prevented, would have prevented the incident from occurring. Find the RCA method that works best for you and your organization. Some common methods are:
This list is not exhaustive, but it provides some popular methods. Once you have a defined the root cause, you can implement mitigation or avoidance and avoid a repeat incident.
While this article does not provide a complete list of everything needed for incident management, it does get you on the road to success in identifying your assets, developing a plan and successfully recovering from whatever the world throws at you.
For guidance or assistance in how to develop your organization’s incident response plans, email cias@utsa.edu.
