Friday, January 04, 2019

Living Without SOCs

So yesterday I tweeted about a conversation I had recently when a CSO asked me if they should build and staff a SOC (security operations center) in addition to an existing operations team. I said not really, because having done this, I tend to agree with the thinking coming out of places like Netflix and Slack on the subject - a nontraditional SOC yields better results.

A traditional SOC is a dedicated space that tried to look like the military command center in the movie "WarGames" - big screens on the wall with data displays that try to visualize threats and security things, flashing and blinking lights, and serious looking analysts staring at two or more monitors filled with alerts and things. When an important alert arrives, the analysts turn on the flashing red light and start an "incident" which often means herding people into a meeting or conference call where people are asked to gather information or take action. The conference call / meeting approach is partly for communication, partly for coordination, and partly a means of getting things done.

This can be very inefficient. Senior engineers typically don't want to be "goldfish" on display in a SOC reading alerts all day and junior engineers don't have the experience or knowledge to investigate and root cause incidents - hence the conference calls, which are a means to obtain assistance from more senior operations or engineering people. At the same time, the cost of having two different operational teams, in a 24/7 world, can be astronomical.

Another approach is to empower operations teams to handle security incidents themselves by investing in security resources there. What does that mean? I would start by identifying a leader. For an operations team new to security, where no one is the heir apparent, I would hire someone to be a security lead who can mentor the team and help them learn to work security issues. In my experience, security leads are self-selecting through some innate instincts and are not waiting for someone to start their training or prompt them to do so. I have not had super-great results trying to train someone to be a security lead who has shown no previous interest in the subject and I don't think this is a productive approach. In some cases, there are one or more team members who are already on the way to being self-taught in security and we need only encourage this by supporting learning and enrichment activities. 

People learn about security in different ways and I believe it is best to allow people to choose their own path. Some people learn by going to conferences and / or taking structured classroom training. Some like to learn by participating in hackathons and capture-the-flag events. Some like to read books or take self-paced training and implementing new found knowledge as they go. For me, as with many people, it has been a combination of all of these things, so having the flexibility to let people identify what kinds of education and enrichment activities they need is key. Funding and supporting such activities is key - a security team needs access to continuous learning to be successful and if your culture cannot support meaningful learning and enrichment activities, security team performance will suffer.

When an existing dev/ops team becomes empowered to investigate and root cause security issues, there are numerous benefits. Velocity is increased and the conference calls can be replaced with asynchronous chats which, in addition to being less harmful to productivity, and facilitating collaboration and information sharing, enhance the team's ability to learn by providing a written record of solutions to issues that can become a knowledge base.

So no, I don't think we must have a traditional SOC - at the end of the day the metrics that matter are MTTK (mean time to know) and MTTR (mean time to resolution) and a virtual SOC can achieve higher velocity on these.

No comments: