Comments on: In the land of the blind..

By: Adam

Adam — Tue, 01 Jul 2008 11:39:34 +0000

Hi Russ,
Thanks for your comments, and I’ll follow up more after I’ve looked at the additional data. I did want to clarify that I’m not claiming the CSOs were unaware of their own state, but of the state of others. I’ve added update 2 to explain that.

By: Russ Cooper

Russ Cooper — Tue, 01 Jul 2008 10:30:59 +0000

Adam, First, thanks for the kind words about our study. It is definitely our intention to make more data available, and on a more regular basis. With regard to our measuring CSO response,I think CSO’s are being done a disservice by typifying them as not knowing, in adequate detail, their patching solutions. In our experience, they are not so far removed from the front lines that they are as subjective as is being suggested in your post. Further, our contact wasn’t always, or only, CSOs. We asked to speak to the “key person responsible???, and in most cases we felt we did. As for defining 1 – 5 differently, we don’t think it would have necessarily been any better. We have consistently found that key responsible people often think they’ve achieved 100% deployment when, in fact, they haven’t. Consider both the “unknown unknowns??? and “Error??? sections of the study as explanations of why they think they’ve achieved 100%. Knowing whether you’re “truly??? a 1 or a 5 will always be subjective, even if it’s based on reports your patching solution spit out for you. You can’t blame a patching solution for telling you 100% deployed if it is unaware of the 6 computers over there that no one knew existed. Further, consider that we have three separate studies coming to the same conclusion. The “Countermeasures Effectiveness Study??? you referenced is but one. In each of the three we found similar results despite asking the question in three different ways (via the different surveys or analysis of the investigator.) The “Sasser Study,??? also referenced in my article, asked what percentage of systems were patched before the outbreak. The “Data Breach Investigations Report??? used forensics for its determination. As for more information about the “Countermeasures Effectiveness Study??? itself, I’d invite you to examine it in more detail via the IEEE Security & Privacy published article: “Is Information Security Under Control???? This article can be found in the January/February 2007 edition, written by Wade Baker and Linda Wallace. Wade was also a co-author of the Data Breach Investigations Report, and is a member of the Verizon Business RISK Team. The Survey Methodology is more fully explained there. Our goal, however, wasn’t to be the most precise in terms of comparing reality to what a CSO thinks, but to demonstrate that the perceived effectiveness of patching may be misplaced, or not even clearly understood. If you think that AV alone is going to solve your malware problem, then you’ve misplaced your trust. If you think that a Firewall is all that you need to prevent intrusions, think again. If you think that purchasing a “patching solution??? is better than implementing relatively simple controls (that don’t require a new product); you’re overlooking a great chance to reduce risk and save money. Cheers, Russ Cooper

By: Pete

Pete — Mon, 30 Jun 2008 21:02:23 +0000

I was under the impression that there were two separate sets of data at play here. One was the 1 through 5 rating, but Jeff’s point was in regards to the length of time that a patch was available prior to exploit for the incidents in question, wasn’t it?

By: Davi Ottenheimer

Davi Ottenheimer — Mon, 30 Jun 2008 19:39:51 +0000

Yes, very annoying that the “data” is really just a measure of self-perception.
I have run numerous engagements where the perception of executives is so far removed from the reality of security, that the PCI DSS is a welcome breath of fresh air to the conversation.
Patching not only makes a lot of sense, but the PCI incident response/investigation teams obviously ferret out the root cause (pun not intended) on their own. When breaches are no longer due to missing patches then they will surely update the PCI DSS appropriately.

By: Blake

Blake — Mon, 30 Jun 2008 08:14:31 +0000

Sounds like one could as well make the argument that CSO’s who are happy with their patching programs don’t have a better security outcome than CSO’s who feel their programs could be improved. The lesson has nothing to do with patching and everything to do with CSO attitudes, which is what Verizon measured.