A Mini-Review of “The Practice of Network Security Monitoring”

NSM book coverRecently the kind folks at No Starch Press sent me a review copy of Rich Bejtlich’s newest book The Practice of Network Security Monitoring and I can’t recommend it enough. It is well worth reading from a theory perspective, but where it really shines is digging into the nuts and bolts of building an NSM program from the ground up. He has essentially built a full end to end tutorial on a broad variety of tools (especially Open Source ones) that will help with every aspect of the program, from collection to analysis to reporting.

As someone who used to own security monitoring and incident response for various organizations, the book was a great refresher on the why and wherefores of building an NSM program and it was really interesting to see how much the tools have evolved over the last 10 years or so since I was in the trenches with the bits and bytes. This is a great resource though regardless of your level of experience and will be a great reference work for years to come. Go read it…

Seattle in the Snow

Seattle snow (From The Oatmeal.)

It’s widely understood that Seattle needs a better way to measure snowfall. However, what’s lacking is a solid proposal for how to measure snowfall around here. And so I have a proposal.

We should create a new unit of measurement: The Nickels. Named after Greg Nickels, who lost the mayorship of Seattle because he couldn’t manage the snow.

Now, there’s a couple of ways we could define the Nickels. It could be:

  • The amount of snow needed to cost a Mayor 10 points of approval rating
  • The amount of snow needed to cause a bus to slide down Olive way and teeter over the highway
  • 2 millimeters
  • Enough snow to reduce the coefficient of city road friction by 1%.

I’m not sure any of these are really right, so please suggest other ways we could define a Nickels in the comments.

Emergent Map: Streets of the US

This is really cool. All Streets is a map of the United States made of nothing but roads. A surprisingly accurate map of the country emerges from the chaos of our roads:

Allstreets poster

All Streets consists of 240 million individual road segments. No other features — no outlines, cities, or types of terrain — are marked, yet canyons and mountains emerge as the roads course around them, and sparser webs of road mark less populated areas. More details can be found here, with additional discussion of the previous version here.

In the discussion page, “Fry” writes:

The result is a map made of 240 million segments of road. It’s very difficult to say exactly how many individual streets are involved — since a winding road might consist of dozens or even hundreds of segments — but I’m sure there’s someone deep inside the Census Bureau who knows the exact number.

Which raises a fascinating question: is there a Platonic definition of “a road”? Is the question answerable in the sort of concrete way that I can say “there are 2 pens in my hand”? We tend to believe that things are countable, but as you try to count them in larger scales, the question of what is a discrete thing grows in importance. We see this when map software tells us to “continue on Foo Street.” Most drivers don’t care about such instructions; the road is the same road, insofar as you can drive in a straight line and be on what seems the same “stretch of pavement.” All that differs is the signs (if there are signs). There’s a story that when Bostonians named Washington Street after our first President, they changed the names of all the streets as they cross Washington Street, to draw attention to the great man. Are those different streets? They are likely different segments, but I think that for someone to know the number of streets in the US requires not an ontological analysis of the nature of street, but rather a purpose-driven one. Who needs to know how many individual streets are in the US? What would they do with that knowledge? Will they count gravel roads? What about new roads, under construction, or roads in the process of being torn up? This weekend of “carmageddeon” closing of 405 in LA, does 405 count as a road?

Only with these questions answered could someone answer the question of “how many streets are there?” People often steam-roller over such issues to get to answers when they need them, and that may be ok, depending on what details are flattened. Me, I’ll stick with “a great many,” since it is accurate enough for all my purposes.

So the takeaway for you? Well, there’s two. First, even with the seemingly most concrete of questions, definitions matter a lot. When someone gives you big numbers and the influence behavior, be sure to understand what they measured and how, and what decisions they made along the way. In information security, a great many people announce seemingly precise and often scary-sounding numbers that, on investigation, mean far different things than they seem to. (Or, more often, far less.)

And second, despite what I wrote above, it’s not the whole country that emerges. It’s the contiguous 48. Again, watch those definitions, especially for what’s not there.

Previously on Emergent Chaos: Steve Coast’s “Map of London” and “Map of Where Tourists Take Pictures.”

Your credit worthiness in 140 Characters or Less

In “Social networking: Your key to easy credit?,” Eric Sandberg writes:

In their quest to identify creditworthy customers, some are tapping into the information you and your friends reveal in the virtual stratosphere. Before calling the privacy police, though, understand how it’s really being used.


To be clear, creditors aren’t accessing the credit reports or scores of those in your social network, nor do those friends affect your personal credit rating. Jewitt asserts that the graphs aren’t being used to penalize borrowers or to find reasons to reject customers, but quite the opposite: “There is an immediate concern that it’s going to affect the ability to get a financial product. But it makes it more likely” that it will work in their favor,” says Jewitt. [vice president of business development of Rapleaf, a San Francisco, Calif., company specializing in social media monitoring.]

I’ll give Jewitt the benefit of the doubt here, and assume he’s sincere. But the issue isn’t will it make it more or less likely to get a loan. The issue is the rate that people will pay. If you think about it from the perspective of a smart banker, they want to segment their loans into slices of more and less likely to pay. The most profitable loans are the ones where people who are really likely to pay them back, but can be convinced that they must pay a higher rate.

The way the banking industry works this is through the emergent phenomenon of credit scores. If banks colluded to ensure you paid a higher rate, it would raise regulatory eyebrows. But since Fair Issac does that, all the bankers know that as your credit score falls, they can charge you more without violating rules against collusion.

Secretive and obscure criteria for differentiating people are a godsend, because most people don’t believe that it matters even when there’s evidence that it does.

Another way to ask this is, “if it’s really likely it will work in my favor, why is it so hard to find details about how it works? Wouldn’t RapLeaf’s customers be telling people about all the extra loans they’re handing out at great rates?”

I look forward to that story emerging.

2008 Breaches: More or More Reporting?

Dissent has some good coverage of an announcement from the ID Theft Resource Center, “ITRC: Breaches Blast ’07 Record:”

With slightly more than four months left to go for 2008, the Identity Theft Resource Center (ITRC) has sent out a press release saying that it has already compiled 449 breaches– more than its total for all of 2007.

As they note, the 449 is an underestimate of the actual number of reported breaches, due in part to ITRC’s system of reporting breaches that affect multiple businesses as one incident. This year we have seen a number of such incidents, including Administrative Systems, Inc., two BNY Mellon incidents, SunGard Higher Education, Colt Express Outsourcing, Willis, and the missing GE Money backup tape that reportedly affected 230 companies. Linda Foley, ITRC Founder, informs this site that contractor breaches represent 11% of the 449 breaches reported on their site this year.

I don’t have much to add, but I do have a question: are incidents up, or are more organizations deciding that a report is the right thing to do?

[Update: I wanted to point out interesting responses by Rich Mogull and Dissent.]

Laptops and border crossings

The New York Times has in an editorial, “The Government and Your Laptop” a plea for Congress to pass a law to ensure that laptops (along with phones, etc.) are not seized at borders without reasonable suspicion.

The have the interesting statistic that in a survey by the Association of Corporate Travel Executives, 7 of 100 respondents reported a laptop or other electronic device seized. Of course, this indicates a problem with metrics. It almost certainly does not mean a 7% seizure rate, as I’ve seen this inflated to. These seizures are such an outrageous thing that the people who have been subjected to them are properly and justifiably outraged. They’re not going to toss the survey in the trash.

I’m not sure how much I like the idea that Congress should pass a law to ensure that the fourth amendment is met. Part of me grits my teeth, as I think it should happen on its own. But if the courts aren’t going to agree, that probably has to happen.

Department of Justice on breach notice

data-breaches-carding-justice.jpg
There’s an important new report out from the Department of Justice, “Data Breaches: What the Underground World of “Carding” Reveals.” It’s an analysis of several cases and the trends in carding and the markets which exist. I want to focus in on one area, which is recommendations around breach notification:

Several bills now before Congress include a national notification standard. In addition to merely requiring notice of a security breach to law enforcement,200 it is also helpful if such laws require victim companies to notify law enforcement prior to mandatory customer notification. This provides law enforcement with the opportunity to delay customer notification if there is an ongoing criminal investigation and such
notification would impede the investigation. Finally, it is also helpful if such laws do not include thresholds for reporting to law enforcement even if certain thresholds – such as the number of customers affected or the likelihood of customer harm — are contained within customer notification requirements. Such thresholds are often premised on the large expense of notifications for the victim entity, the fear of desensitizing customers to breaches, and causing undue alarm in circumstances where customers are unlikely to suffer harm. These reasons have little applicability in the law enforcement setting, however, where notification (to law enforcement) is inexpensive, does not result in reporting fatigue, and allows for criminal investigations even where particular customers were not apparently harmed. (“Data Breaches: What the Underground World of “Carding” Reveals,” Kimberly Kiefer Peretti U.S. Department of Justice, Forthcoming in Volume 25 of the Santa Clara Computer and High Technology Journal, page 28.)

I think such reports should go not only to law enforcement, but to consumer protection agencies. Of course, this sets aside the question of “are these arguments meaningful,” and potentially costs us an ally in the fight for more and better data, but I’m willing to take small steps forward.

Regardless, it’s great to see that the Department of Justice is looking at this as something more than a flash in the pan. They see it as an opportunity to learn.

Security Prediction Markets: theory & practice

reckless-experimentation.jpgThere are a lot of great comments on the “Security Prediction Markets” post.

There’s a tremendous amount of theorizing going on here, and no one has any data. Why don’t we experiment and get some? What would it take to create a market in breach notification prediction?

Dan Guido said in a comment, “In security, SOMEONE knows the RIGHT answer. It is not indeterminate, the code is out there, your network is accessible to me and so on. There’s none of this wishy-washy risk stuff.”

I don’t think he’s actually right. Often times, no one knows the answer. Gathering it is expensive. Translating from “there’s a vuln” to “I can exploit it” isn’t always easy. For example, one of my co-workers tried exploit a (known, reported, not yet fixed) issue in an internal site via Sharepoint. Something in Sharepoint keeps munging his exploit code. I’ve even set my browser homepage to a page under his control. Who cares what I think, when we can experiment?

What would be involved in setting up an experiment? We’d need, in no particular order:

  • A web site with some market software. Is there a market for such sites? (There is! Inkling will let you run a 45 day pilot with up to 400 traders. There’s likely others.)
  • Terms & conditions. Some issues to be determined:
    1. Can you bet on your employer? Clients? Customers?
    2. Are bets anonymous?
    3. What’s the terms of the payoff? Are you betting company X has a breach of PII, or a vuln? Would Lazard count?
    4. What’s the term of a futures option? What’s the ideal for a quick experiment? What’s the ideal for an operational market?
    5. Are we taking singleton bets (Bank A will have a problem) or comparative (Bank A will have more problems than bank B.)
  • Participants. I think that’s pretty easy.
  • Dispute arbitration. What if someone claims that Amazon’s issue on Friday the 6th was a break-in? Amazon hasn’t yet said what happened.

So, we could debate like mad, or we could experiment. Michael Cloppert asked a good question. Let’s experiment and see what emerges.


Photo: “Better living…” by GallixSee media.

Security Prediction Markets?

In our first open thread, Michael Cloppert asked:

Considering the contributors to this blog often discuss security in
terms of economics, I’m curious what you (and any readers educated on
the topic) think about the utility of href="http://blog.cloppert.org/2008/05/market-based-approach-to-predict
ing.html" rel="nofollow">using prediction markets to forecast
compromises.

So I’m generally a big fan of markets. I think markets are, as Hayek pointed out, a great way to extract information from systems. The prediction markets function by rewarding those who can make better predictions. So would this work for security, and predicting compromises?

I don’t think so, despite being a huge fan of the value of the chaos that emerges from markets.

Allow me to explain. There are two reasons why it won’t work. Let’s take Alice and Bob, market speculators. Both work in banks. Alice thinks her bank has great security (“oh, those password rules!”). So she bets that her bank has a low likelihood of breach. Bob, in contrast, thinks his bank has rotten security (“oh, those password rules!”). So he bets against it. Perhaps their models are more sophisticated, and I’ll return to that point.


As Alice buys, the price breach futures in her bank rises. As Bob sells, the price of his futures falls. (Assuming fixed numbers of trades, and that they’re not working for the same bank.)

But what do Alice and Bob really know? How much experience does either have to make accurate assessments of their employers’ security? We don’t talk about security failures. We don’t learn from each other’s failures, and so failure strikes arbitrarily.

So I’m not sure who the skilled predictors would be who would make money by entering the market. Without such skilled predictors, or people with better information, the market can’t extract the information.

Now, there may be information which is purely negative which could be usefully extracted. I doubt it, absent baselines that Alice and Bob can use to objectively assess what they see.

There may well be more sophisticated models, where people with more or better information could bet. Setting aside ethical or professional standards, auditors of various sorts might be able to play the market.

I don’t know that there are enough of them to trade effectively. A thinly traded security doesn’t offer up as much information as one that’s being heavily traded.

So I’m skeptical.