RSA 2012 was my first RSA. I used a vendor code to get an expo-only pass. I’ll be attending again since it’s a good chance to visit vendors, customers, prospects and online buddies in a short amount of time.
I had the chance to speak briefly to Richard Bejtlich, whose NSM principles we borrow from at our company when giving advice to customers. I recently revived my interest in metrics so I asked about metrics. He said to track two metrics: incidents per unit time and elapsed time to resolution per incident. An incident is whatever your organization thinks it is.
I was as a breakfast on Wednesday morning with a panel of about 8 CISOs from large corporations. Afterward I asked a couple of them which KPIs they use or have developed. I know the sample size is small, but it appears that they all develop their own indicators and don’t share with other CISOs or are in the process of developing them. The reason for this is no company wants anyone to see how much money they spend on security or risk they tolerate compared to their peers. Seems to me that they wouldn’t be giving their peers any advantage but perhaps there is a social stigma to sharing this stuff – like admitting you have herpes.
During the breakfast, a few of the CISOs also mentioned that security ROI is a waste of time and just to look at TCO. I’m of the opinion that you can’t avoid at least a gut ROI calculation. If you calculate the TCO, then decide it is worth purchasing, you just calculated the ROI right? My need to call it something else since it isn’t an investment per se. Paying for security is usually trying to avoid a probable loss. Like paying extra for a car with airbags
These same CISOs also said quantitative risk management is a waste of time and unnecessary. I don’t think you can escape at least a gut risk calculation and that the quantitative and qualitative are just different ends of a spectrum rather than binary options.