My talk at Black Hat this year was “Elevation of Privilege, the Easy Way to Get Started Threat Modeling.” I covered the game, why it works and where games work. The link will take you to the PPTX deck.
Frank Pasquale follows a Joe Nocera article on credit scores with a great roundup of issues that the credit system imposes on American citizens, including arbitrariness, discriminatory effects and self-fulfilling prophecies. His article is worth a look even if you think you understand credit scores.
I’d like to add one more danger of credit scores: deceptive advertising. The way it works is that a bank advertises a great rate for those with “perfect credit.” What it doesn’t advertise is what the curve of credit scores versus rates looks like. There are two issues here. The first is that the market is inefficient, as figuring out what actual rates are often involves talking to a human, and usually disclosing enough personal information to make a fraudster drool. Inefficient markets favor the side with more information (the loan offerer) and lead to less trade than more transparent markets.
The second issue is that everyone is mislead by the headline rate. I’ve looked for data on what fraction of Americans are listed as having “perfect credit” or data on the distribution of interest rates people are really paying, and I’ve been unable to find it. For publicly traded companies, it’s sometimes possible to reverse engineer some of this, but not very much.
- Breath mints
Ariel Waissbein has been building security games for a while now. He was They were kind enough to send a copy of his their “Exploit” game after I released Elevation of Privilege. [Update: I had confused Ariel Futoransky and Ariel Waissbein, because Waissbein wrote the blog post. Sorry!] At Defcon, he and his colleagues will be running a more capture-the-flag sort of game, titled “Hide and seek the backdoor:”
For starters, a backdoor is said to be a piece of code intentionally added to a program to grant remote control of the program — or the host that runs it – to its author, that at the same time remains difficult to detect by anybody else.
But this last aspect of the definition actually limits its usefulness, as it implies that the validity of the backdoor’s existence is contingent upon the victim’s failure to detect it. It does not provide any clue at all into how to create or detect a backdoor successfully.
A few years ago, the CoreTex team did an internal experiment at Core and designed the Backdoor Hiding Game, which mimics the old game Dictionary. In this new game, the game master provides a description of the functionalities of a program, together with the setting where it runs, and the players must then develop programs that fulfill these functionalities and have a backdoor. The game master then mixes all these programs with one that he developed and has no backdoors, and gives these to the players. Then, the players must audit all the programs and pick the benign one.
First, I think this is great, and I look forward to seeing it. I do have some questions. What elements of the game can we evaluate and how? A general question we can ask is “Is the game for fun or to advance the state of the art?” (Both are ok and sometimes it’s unclear until knowledge emerges from the chaos of experimentation.) His blog states “We discovered many new hiding techniques,” which is awesome. Games that are fun and advance the state of the art are very hard to create. It’s a seriously cool achievement.
My next question is, how close is the game to the reality of secure software development? How can we transfer knowledge from one to the other? The rules seem to drive backdoors into most code (assuming they all work, (n-1)/n). That’s unlike reality, with a much higher incidence of backdoors than exist in the wild. I’m assuming that the code will all be custom, and thus short enough to create and audit in a game, which also leads to a higher concentration of backdoors per line of code. That different concentration will reward different techniques from those that could scale to a million lines of code.
More generally, do we know how to evaluate hiding techniques? Do hackers playing a game create the same sort of backdoors as disgruntled employees or industrial spies? Because of this contest and the Underhanded C Contests, we have two corpuses of backdoored code. However, I’m not aware of any corpus of deployed backdoor code which we could compare.
So anyway, I look forward to seeing this game at Defcon, and in the future, more serious games for information security.
In “Engineers Are People, Too” Adam Shostack will address an often invisible link in the chain between research on usable security and privacy and delivering that usability: the engineer. All too often, engineers are assumed to have infinite time and skills for usability testing and iteration. They have time to read papers, adapt research ideas to the specifics of their product, and still ship cool new features. This talk will bring together lessons from enabling Microsoft’s thousands of engineers to threat modeling effectively, share some new approaches to engineering security usability, and propose new directions for research.
A fair number of people have asked for the slides, and they’re here: Engineers Are People Too.
Over the last week, there’s been a set of entertaining stories around Blizzard’s World of Warcraft games and forums. First, “World of Warcraft maker to end anonymous forum logins,” in a bid to make the forums less vitriolic:
Mr Brand said that one Blizzard employee posted his real name on the forums, saying that there was no risk to users, and the experiment went drastically wrong. “Within five minutes, users had got hold of his telephone number, home address, photographs of him and a ton of other information,” said Mr Brand.
The key lesson is that your customers think about identity differently than you do, and trying to add it to a system is fraught with risk. (Don’t even get me started on the jargon “identity provider.”)
Back when I commented on David Cameron apologizing for Bloody Sunday, someone said “It’s important to remember that it’s much easier to make magnanimous apologise about the behaviour of government agents when none of those responsible are still in their jobs.” Which was fine, but now Mr. Cameron is setting up an investigation into torture by UK security services. (“
Britain Pledges Inquiry Into Torture.”
And yes, it’s certainly more fun to investigate the opposition, but…I’d really like to bring Mr. Cameron over here for a little while. Some investigations would do us, and our fight against al Qaeda, a great deal of good.
In looking at Frank Pasquale’s very interesting blog post “Secrecy & the Spill,” a phrase jumped out at me:
I have tried to give the Obama Administration the benefit of the doubt during the Gulf/BP oil disaster. There was a “grand ole party” at Interior for at least eight years. Many Republicans in Congress would have tried to block nominees for Interior who were committed to environmentalism. But the more I read about the controversy, the harder it gets to excuse current players for their actions.
Now, if you had told me six months ago that the Minerals Management Service was critically messed up, I might have searched a bit and said “sure, ok.” There are a lot of government agencies which are poorly run. Prioritizing between them is hard. Had you told me that their failure would cost a billion dollars, I’d have been more skeptical than usual.
Government is too big to clean out; at each level, you get appointees who are less likely to be interested in pursuing the President’s interest, and more likely to be interested in featherbedding. That’s not to say that all agencies are mis-run. There are still people out there who consider themselves civil servants who aim to run their agencies (or areas) well. I don’t have enough data to know what fraction of agencies are well run, but I expect that you could graph it and it would look a lot like a bell curve. Some good, some bad, most middling.
The agencies that are well run don’t get attention. The problems they face are ‘managed’ and don’t descend into crisis very much. Unfortunately it’s hard to tell, a-priori, if an agency is well run or lucky.
For any Administration to dig deeply into each of the government agencies could easily become an all-consuming issue. And it’s unclear if it would do any good. Agency executives are expected to be able to present a pleasant picture with a few things that need fixing.
This is a structural and systematic issue which emerges from how big government is and how much it tries to do. The only way to clean things up will be to reduce the size of government, so that prior oversight becomes a reasonable expectation.
Otherwise, we can look forward to the chaotic universe helping us discover where the problems emerge.
In CONGRESS, July 4, 1776
The unanimous Declaration of the thirteen united States of America,
When in the Course of human events, it becomes necessary for one people to dissolve the political bands which have connected them with another, and to assume among the powers of the earth, the separate and equal station to which the Laws of Nature and of Nature’s God entitle them, a decent respect to the opinions of mankind requires that they should declare the causes which impel them to the separation.
We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness. –That to secure these rights, Governments are instituted among Men, deriving their just powers from the consent of the governed, –That whenever any Form of Government becomes destructive of these ends, it is the Right of the People to alter or to abolish it, and to institute new Government, laying its foundation on such principles and organizing its powers in such form, as to them shall seem most likely to effect their Safety and Happiness. Prudence, indeed, will dictate that Governments long established should not be changed for light and transient causes; and accordingly all experience hath shewn, that mankind are more disposed to suffer, while evils are sufferable, than to right themselves by abolishing the forms to which they are accustomed. But when a long train of abuses and usurpations, pursuing invariably the same Object evinces a design to reduce them under absolute Despotism, it is their right, it is their duty, to throw off such Government, and to provide new Guards for their future security. —Such has been the patient sufferance of these Colonies; and such is now the necessity which constrains them to alter their former Systems of Government. The history of the present King of Great Britain [George III] is a history of repeated injuries and usurpations, all having in direct object the establishment of an absolute Tyranny over these States. To prove this, let Facts be submitted to a candid world.