Threat Modeling and Risk Assessment

Yesterday, I got into a bit of a back and forth with Wendy Nather on threat modeling and the role of risk management, and I wanted to respond more fully.

So first, what was said:

(Wendy) As much as I love Elevation of Privilege, I don’t think any threat modeling is complete without considering probability too.
(me) Thanks! I’m not advocating against risk, but asking when. Do you evaluate bugs 2x? Once in threat model & once in bug triage?
(Wendy) Yes, because I see TM as being important in design, when the bugs haven’t been written in yet. 🙂

I think Wendy and I are in agreement that threat modeling should happen early, and that probability is important. My issue is that I think issues discovered by threat modeling are, in reality, dealt with by only a few of Gunnar’s top 5 influencers.

I think there are two good reasons to consider threat modeling as an activity that produces a bug list, rather than a prioritized list. First is that bugs are a great exit point for the activity, and second, bugs are going to get triaged again anyway.

First, bugs are a great end point. An important part of my perspective on threat modeling is that it works best when there’s a clear entry and exit point, that is, when developers know when the threat modeling activity is done. (Window Snyder, who knows a thing or two about threat modeling, raised this as the first thing that needed fixing when I took my job at Microsoft to improve threat modeling.) Developers are familiar with bugs. If you end a strange activity, such as threat modeling, with a familiar one, such as filing bugs, developers feel empowered to take a next step. They know what they need to do next.

And that’s my second point: developers and development organizations triage bugs. Any good development organization has a way to deal with bugs. The only two real outputs I’ve ever seen from threat modeling are bugs and threat model documents. I’ve seen bugs work far better than documents in almost every case.

So if you expect that bugs will work better then you’re left with the important question that Wendy is raising: when do you consider probability? That’s going to happen in bug triage anyway, so why bother including it in threat modeling? You might prune the list and avoid entering silly bugs. That’s a win. But if you capture your risk assessment process and expertise within threat modeling, then what happens in bug triage? Will the security expert be in the room? Do you have a process for comparing security priority to other priorities? (At Microsoft, we use security bug bars for this, and a sample is here.)

My concern, and the reason I got into a back and forth, is I suspect that putting risk assessment into threat modeling keeps organizations from ensuring that expertise is in bug triage, and that’s risky.

(As usual, these opinions are mine, and may differ from those of my employer.)

[Updated to correct editing issues.]

Outrage of the Day: Police Violence

When the LAPD finally began arresting those of us interlocked around the symbolic tent, we were all ordered by the LAPD to unlink from each other (in order to facilitate the arrests). Each seated, nonviolent protester beside me who refused to cooperate by unlinking his arms had the following done to him: an LAPD officer would forcibly extend the protestor’s legs, grab his left foot, twist it all the way around and then stomp his boot on the insole, pinning the protestor’s left foot to the pavement, twisted backwards. Then the LAPD officer would grab the protestor’s right foot and twist it all the way the other direction until the non-violent protestor, in incredible agony, would shriek in pain and unlink from his neighbor.

It was horrible to watch, and apparently designed to terrorize the rest of us. At least I was sufficiently terrorized. I unlinked my arms voluntarily and informed the LAPD officers that I would go peacefully and cooperatively. I stood as instructed, and then I had my arms wrenched behind my back, and an officer hyperextended my wrists into my inner arms. It was super violent, it hurt really really bad, and he was doing it on purpose. When I involuntarily recoiled from the pain, the LAPD officer threw me face-first to the pavement. He had my hands behind my back, so I landed right on my face. The officer dropped with his knee on my back and ground my face into the pavement. It really, really hurt and my face started bleeding and I was very scared. I begged for mercy and I promised that I was honestly not resisting and would not resist.

Go read My Occupy LA Arrest, by Patrick Meighan and Listener Privacy

It turns out that it’s very hard to subscribe to many podcasts without talking to servers. (Technical details in the full post, below.) So I took a look at their privacy statement:

Podtrac provides free services to podcasters whereby Podtrac gathers data specific to individual podcasts (e.g. audience survey data, content ratings, measurement data, etc). This podcast data is not considered personally identifiable information and may be shared by Podtrac with member advertisers. (“Podtrac Client Privacy Statement,” undated, unversioned.)

It’s not clear to me who doesn’t consider what they collect to be personal data, because the passive voice is annoyingly used. So I’ll ask: precisely what data is collected? And under what set of laws or even perspectives is the data they’re collecting is not considered personally identifiable? For example, are they collecting IP addresses, which I understand are PII in the EU?

Enquiring minds with privacy officials might want to ask those officials.

Continue reading

Twitter Weekly Updates for 2011-12-04

  • New School blog "'Its Time to Learn Like Experts' by @jayjacobs" #
  • RT @dmolnar Help me shop for furniture #
  • RT @moxie__ WhisperSystems has been acquired! < Congratulations! I hope it leads to great things for Twitter privacy #
  • RT @tsastatus A few new features, and a bunch of status updates, at, more info on the blog #
  • New blog: "Telephones and Privacy" #
  • New blog: "We Robot: The Conference" #
  • New bloggage: "Telephones and Privacy" #
  • RT @securityninja @realex_tracy everyone that walks past my desk has to play with the elevation of privilege cards, eye catching! #
  • New blog: Big Brother Watch Report on Breaches (ht @PogoWasRight) #
  • NPR has a story on FAA/SSA violations of Privacy act going to Supreme Court #
  • RT @hvyboots: So apparently, Carrier IQ is on iPhones too #
  • RT @Jim_Harper Looking for Sen. Schumer's statement re: TSA tracking of cell phones. Anyone know where I can find it? #
  • New School blog post: "The Future of Work is Play" #
  • RT @srbaker Film producers: if your film isn't on Netflix, I'll find it on BitTorrent. You might as well make some money on it. #
  • RT @AdasBooks Today is such an exciting day! The first big news is that our webstore is now live! #
  • RT @ggreenwald Surreal: Sen Feinstein had an amendment to say: you can't imprison US citizens w/o charges – it failed: #
  • .@rmogull I don't think it's hypocrisy, I think people are tired of privacy disempowerment, confusion and the short end of the stick #
  • Oops, violated my own medical privacy, darnit! Can I be in datalossdb now? 🙂 #
  • RT @lorrietweet Speaking today at Silicon Flatirons Privacy Economics event – live stream available < Yay, looks great! #
  • RT @WeldPond It should be in the public's best interest to allow researchers to tell us what the software on our phones does. #carrieriq #
  • Julie Cohen's comments are frames embedded in information systems is really well stated. #flatirons privacy economics #
  • "Let's make sure the system is not set up to thwart consumer desire for privacy" Joseph Farrell @ silicon flatirons #
  • "What we call disclosures are often when someone wants to say they disclosed it but they don't want you to know it" Joseph Farrell #flatiron #
  • "We have effective disclosure, they're called ads" Joseph Farrell #flatiron discussing privacy "disclosure" #
  • RT @liorjs Here's the livestream for Economics of Privacy Conf. Catch @ericgoldman, @paulohm, @rcalo, @laguarda, etc. #
  • ~"Aleecia has a knack for stating the controversial in a non-controversial way" @rcalo #flatiron #privacy #
  • Great question from @rcalo to venture capitalist Seth Levine: Do you worry about privacy as a risk in investing? #
  • Did anyone mention the work on organ donation rates and defaults, tie it to privacy? #flatirons #
  • UStream is reaching out to 14 different websites as I listen to the Silicon #flatirons event on Privacy economics #
  • RT @kbcran #flatirons Thwarting facial recognition with Blade Runner makeup #
  • EyeLink has some really amazing & scary iris recognition tech. Way beyond what i thought was state of the art. Can't find link (@dmolnar?) #
  • Peter Swire has a depressing view of the impact of Sorrel vs IMS #flatirons #
  • Yay @hoofnagle for bringing up the compelled nature of speech in Sorrel. #flatirons #
  • Gävle Goat Gambit Goes Astray #
  • Did he just say "You won't have Herman Cain to kick around anymore"? #

Powered by Twitter Tools

Gävle Goat Gambit Goes Astray

Gavle Goat 2011
It’s a bit of a Christmas tradition here at Emergent Chaos to keep you informed about the Gävle Goat. Ok, technically, our traditions seem hit and miss, but whaddaya want from a site with Chaos in the name? You want precision, read a project management blog. Project management blogs probably set calendar reminders to kick off a plan with defined stakeholders, success metrics and milestones to ensure high quality blog posts. Us, we sometimes randomly remember.

But, but! This year, we actually have a plan with 8×10 color gannt charts with circles and arrows explaining how to set up a market to predict when the goat would burn.

We even have prizes.

Unfortunately, chaos (and flames) emerged, and the goat was burned before we set up the market.

You can read the full story of “Sweden’s Christmas goat succumbs to flames.”