The Evolution of Secure Things

One of the most interesting security books I’ve read in a while barely mentions computers or security. The book is Petroski’s The Evolution of Useful Things.

Evolution Of useful Things Book Cover

As the subtitle explains, the book discusses “How Everyday Artifacts – From Forks and Pins to Paper Clips and Zippers – Came to be as They are.”

The chapter on the fork is a fine example of the construction of the book.. The book traces its evolution from a two-tined tool useful for holding meat as it was cut to the 4 tines we have today. Petroski documents the many variants of forks which were created, and how each was created with reference to the perceived failings of previous designs. The first designs were useful for holding meat as you cut it, before transferring it to your mouth with the knife. Later designs were unable to hold peas, extract an oyster, cut pastry, or meet a variety of other goals that diners had. Those goals acted as evolutionary pressures, and drove innovators to create new forms of the fork.

Not speaking of the fork, but rather of newer devices, Petroski writes:

Why designers do not get things right the first time may be more understandable than excusable. Whether electronics designers pay less attention to how their devices will be operated, or whether their familiarity with the electronic guts of their own little monsters hardens them against these monsters’ facial expressions, there is a consensus among consumers and reflective critics like Donald Norman, who has characterized “usable design” as the “next competitive frontier,” that things seldom live up to their promise. Norman states flatly, “Warning labels and large instruction manuals are signs of failures, attempts to patch up problems that should have been avoided by proper design in the first place.” He is correct, of course, but how is it that designers have, almost to a person, been so myopic?

So what does this have to do with security?

(No, it’s not “stick a fork in it, it’s done fer.”)

Its a matter of the pressures brought to bear on the designs of even what (we now see) as the very simplest technologies. It’s about the constant imperfection of products, and how engineering is a response to perceived imperfections. It’s about the chaotic real world from which progress emerges. In a sense, products are never perfected, but express tradeoffs between many pressures, like manufacturing techniques, available materials, and fashion in both superficial and deep ways.

In security, we ask for perfection against an ill-defined and ever-growing list of hard-to-understand properties, such as “double-free safety.”

Computer security is in a process of moving from expressing “security” to expressing more precise goals, and the evolution of useful tools for finding, naming, and discussing vulnerabilities will help us express what we want in secure software.

The various manifestations of failure, as have been articulated in case studies throughout this book, provide the conceptual underpinning for understanding the evolving form of artifacts and the fabric of technology into which they are inextricably woven. It is clearly the perception of failure in existing technology that drives inventors, designers, and engineers to modify what others may find perfectly adequate, or at least usable. What constitutes failure and what improvement is not totally objective, for in the final analysis a considerable list of criteria, ranging from the functional to the aesthetic, from the economic to the moral, can come into play. Nevertheless, each criterion must be judged in a context of failure, which, though perhaps much easier than success to quantify, will always retain an aspect of subjectivity. The spectrum of subjectivity may appear to narrow to a band of objectivity within the confines of disciplinary discussion, but when a diversity of individuals and groups comes together to discuss criteria of success and failure, consensus can be an elusive state.

Even if you’ve previously read it, re-reading it from a infosec perspective is worthwhile. Highly recommended.

[As I was writing this, Ben Hughes wrote a closely related post on the practical importance of tradeoffs, “A Dockery of a Sham.”]

The Unexpected Meanings of Facebook Privacy Disclaimers

Paul Gowder has an interesting post over at Prawfblog, “In Defense of Facebook Copyright Disclaimer Status Updates (!!!).” He presents the facts:

…People then decide that, hey, goose, gander, if Facebook can unilaterally change the terms of our agreement by presenting new ones where, theoretically, a user might see them, then a user can unilaterally change the terms of our agreement by presenting new ones where, theoretically, some responsible party in Facebook might see them. Accordingly, they post Facebook statuses declaring that they reserve all kinds of rights in the content they post to Facebook, and expressly denying that Facebook acquires any rights to that content by virtue of that posting.

Before commenting on his analysis, which is worth reading in full, there’s an important takeaway, which is that even on Facebook, and even with Facebook’s investment in making their privacy controls more usable, people want more privacy while they’re using Facebook. Is that everyone? No, but it’s enough for the phenomenon of people posting these notices to get noticed.

His analysis instead goes to what we can learn about how people see the law:

To the contrary, I think the Facebook status-updaters reflect both cause for hope and cause for worry about our legal system. The cause for worry is that the system does seem to present itself as magic words. The Facebook status updates, like the protests of the sovereign citizens (but much more mainstream), seem to me to reflect a serious alienation of the public from the law, in which the law isn’t rational, or a reflection of our collective values and ideas about how we ought to treat one another and organize our civic life. Instead, it’s weaponized ritual, a set of pieces of magic paper or bits on a computer screen, administered by a captured priesthood, which the powerful can use to exercise that power over others. With mere words, unhinged from any semblance of autonomy or agreement, Facebook can (the status-updaters perceive) whisk away your property and your private information. This is of a kind with the sort of alienation that I worried about over the last few posts, but in the civil rather than the criminal context: the perception that the law is something done to one, rather than something one does with others as an autonomous agent as well as a democratic citizen. Whether this appears in the form of one-sided boilerplate contracts or petty police harassment, it’s still potentially alienating, and, for that reason, troubling.

This is spot-on. Let me extend it. These “weaponized rituals” are not just at the level of the law. Our institutions are developing anti-bodies to unscripted or difficult to categorize human participation, because engaging with human participation is expensive to deliver and inconvenient to the organization. We see this in the increasingly ritualized engagement with the courts. Despite regular attempts to make courts operate in plain English, it becomes a headline when “Prisoner wins Supreme Court case after submitting handwritten petition.” (Yes, the guy’s apparently otherwise a jerk, serving a life sentence.) Comments to government agencies are now expected to follow a form (and regular commenters learn to follow it, lest their comments engage the organizational anti-bodies on procedural grounds). When John Oliver suggested writing to the FCC, its systems crashed and they had to extend the deadline. Submitting Freedom of Information requests to governments, originally meant to increase transparency and engagement, has become so scripted that there are web sites to track your requests and departmental failures to comply with the statuatory timelines. We have come to accept that our legislators and regulators are looking out for themselves, and no longer ask them to focus on societal good. We are pleasantly surprised when they pay more than lip service to anything beyond their agency’s remit. In such a world, is it any surprise that most people don’t bother to vote?

Such problems are not limited to the law. We no longer talk to the man in the gray flannel suit, we talk to someone reading from a script he wrote. Our interactions with organizations are fenceposted by vague references to “policy.” Telephone script-readers are so irksome to deal with that we all put off making calls, because we know that even asking for a supervisor barely helps. (This underlies why rage-tweeting can actually help cut red tape; it summons a different department to try to work your way through a problem created by intra-organizational shuffling of costs.) Sometimes the references to policy are not vague, but precise, and the precision itself is a cost-shifting ritual. By demanding a form that’s convenient to itself, an organization can simultaneously call for engagement while making that engagement expensive and frustrating. When engaging requires understanding the the system as well as those who are immersed in it, engagement is discouraged. We can see this at Wikipedia, for example, discussed in a blog post like “The Closed, Unfriendly World of Wikipedia.” Wikipedia has evolved a system for managing disputes, and that system is ritualized. Danny Sullivan doesn’t understand why they want him to jump through hoops and express himself in the way that makes it easy for them to process.

Such ritualized forms of engagement display commitment to the organization. This can inform our understanding of how social engineers work. Much of their success at impersonating employees comes from being fluid in the use of a victim’s jargon, and in the 90s, much of what was published in 2600 was lists of Ma Bell’s acronyms or descriptions of operating procedures. People believe that only an employee would bother to learn such things, and so learning such things acts as an authenticator in ways that infuriate technical system designers.

What Gowder calls rituals can also be viewed as protocols (or protocol messages). They are the formalized, algorithm friendly, state-machine altering messages, and thus we’ll see more of them.

Such growth makes systems brittle, as they focus on processing those messages and not others. Brittle systems break in chaotic and often ugly ways.

So let me leave this with a question: how can we design systems which scale without becoming brittle, and also allow for empathy?

The Future Is So Cool

When you were growing up, 2014 was the future. And it’s become cliche to bemoan that we don’t have the flying cars we were promised, but did get early delivery on a dystopian surveillance state.

So living here in the future, I just wanted to point out how cool it is that you can detect extrasolar planets with a home kit.

A camera mounted on a clever set of hinges to track the sky

Read the story at IEEE Spectrum: DIY Exoplanet Detector.

L’Academie Gawker

Via Poynter, we learn that the word “massive” has been banned on Gawker.

We want to sound like regular adult human beings, not Buzzfeed writers or Reddit commenters,” new Gawker Editor Max Read says in a memo to the publication’s writers. Words like “epic,” “pwn” and “derp” are no longer welcome on the site. Read also says the word “massive” is “never to appear on the website Gawker dot com.”

The desire to sound like regular human beings is admirable, and Mr. Read is correct when he says that jokes made using strikethrough are generally not worth saving.

However, he seems to fall into a trap of believing that there is an hierarchy of language goodness which is removed from our social hierarchies. We’re not the French, with L’Acadamie française to define correct language, and to be ignored by Le Frenchmen dans on le weekends.

The observable reality is that language evolves as a result of a variety of pressures or opportunities. That is, language is emergent, not decreed. There is no authority who gets to declare what words a community uses (outside of NewSpeak, and even in Orwell’s world, normal people don’t use NewSpeak daily, because the words decreed by Big Brother didn’t serve their needs. Real language is inevitably chaotic and messy.

This is a massive pile of derp, and an epic mistake on Gawker’s part.

[Updated to add a strikethrough joke.]

3D-printed guns and the crypto wars

So there’s a working set of plans for the “Liberator.” It’s a working firearm you can print on a 3d printer. You can no longer get the files from the authors, whose site states: “DEFCAD files are being removed from public access at the request of the US Department of Defense Trade Controls.
Until further notice, the United States government claims control of the information.” Cue Streisand Effect.

My understanding is that the censorship order was issued under the ITARs, the “International Traffic in Arms Regulations.” Cory Doctorow has said “Impact litigation — where good precedents overturn bad rules — is greatly assisted by good facts and good defendants. I would much rather the Internet-as-library question be ruled on in a less emotionally overheated realm than DIY guns.” I think that’s reasonable, but recall that Shaw claimed that all progress depends on the unreasonable man.

Doctorow also refers to Bernstein, who did good work, but his lawsuit was the last nail in ITARs applying to crypto, not the first. (ITARs still do apply to crypto, but in ways that allow both open source and commercial software to ship strong crypto, which wasn’t the case in the 90s.) Me, I see lots of evidence that gun control doesn’t work any better than alcohol control or marijuana control. And I think that the regulatory response by the DoD is silly. (One can argue that the law gives them no choice, but I don’t believe that to be the case.)

So the right step was demonstrated for crypto nearly 20 years ago by Phil Karn. He filed a pair of “Commodity Jurisdiction Requests.” One for Applied Cryptography, a book, and one for a floppy disk containing the source code.

The State Department ruled that even though the book itself is “in the public domain” and hence outside their jurisdiction, a floppy disk containing the exact same source code as printed in the book is a “munition” requiring a license to export. It’s old news that the US Government believes only Americans (and maybe a few Canadians) can write C code, but now they have apparently decided that foreigners can’t type either!

In the past three years I have taken my case to all three branches of the federal government. Here is the full case history in the Executive and Judicial branches, including all my correspondence with the US State Department, the Bureau of Export Administration (BXA) in the Commerce Department, the US District Court for the District of Columbia, and the Court of Appeals for the DC Circuit.

I believe the analogy is obvious. The DefCad files are 2mb zipped, and the STL files can be opened with a variety of software. Unfortunately, STL looks to be a binary format, and it’s not clear to me after a few minutes of searching if there’s a trivially printed text format. But that’s a very low hurdle.

As Doctorow implied, reasonableness on all sides would be nice to have. But at home printing isn’t going to go away, and censorship orders are not a productive step forward.

[Previously here: “What Should a Printer Print?“]

Gamifying Driving

P90115441 highRes 640x419

…the new points system rates the driver’s ability to pilot the MINI with a sporty yet steady hand. Praise is given to particularly sprightly sprints, precise gear changes, controlled braking, smooth cornering and U-turns executed at well-judged speeds. For example, the system awards maximum Experience Points for upshifts carried out within the ideal rev range and in less than 1.2 seconds. Super-slick gear changes prompt a “Perfect change up” message on the on-board monitor, while a “Breathtaking U-turn” and a masterful touch with the anchors (“Well-balanced braking”) are similarly recognised with top marks and positive, MINI-style feedback.

For more, see “MINI Connected Adds Driving Excitement Analyser.”

Now, driving is the most dangerous thing most of us do on a regular basis. Most Americans don’t get any supplemental driving instruction after they turn 17. So maybe there’s actually something to be said for a system that incents people to drive better.

I can’t see any possible issues with a game pushing people towards things that are undesirable in the real world. I mean, I’m sure that before suggesting a U-turn, the game will use the car’s adaptive cruise control radar to see what’s around, even if the car doesn’t have one.

Can Science Improvise?

My friend Raquell Holmes is doing some really interesting work at using improv to unlock creativity. There’s some really interesting ties between the use of games and the use of improv to get people to approach problems in a new light, and I’m bummed that I won’t be able to make this event:

Monday Dec 17th – 7:15 to 9:15pm
835 Market Street, Rm. 619, Downtown San Francisco State University Campus

Register at
In advance- $15 At the Door- $20

What happens when you combine the playfulness of improvisation with
the rigor of science? The Life Performance Coaching Center which
leads people from all walks of life in a performance-based approach to
human development is pleased to host Dr. Raquell M. Holmes founder of
improvscience. Holmes has been bringing the discoveries in human
development and performance to researchers and educators in many areas
of science including biology and computing sciences.

In this exploration for scientists and those interested in creativity
and development, participants are introduced to what the
improvisational arts bring to science. Learning to build with the
contributions of others and see opportunities for improvisational
conversation helps us to take risks and discover new ways of seeing
each other and our work.

Come and play as we break down the social barriers that can inhibit
creativity, exploration and discovery.

Helen Abel, LCSW, has worked with people to develop their lives for
over 30 years as a social worker, therapist and coach. She is on the
staff of the Life Performance Coaching Center where she leads the
popular Playground series {link if available} where people learn how
to use their capacity to create, perform and play. As a life coach she
helps people access these same skills to develop creative and new
kinds of conversations with their friends, family and colleagues.

Dr. Raquell Holmes is Director of Outreach, Recruitment and Retention
at the Center for Cell Analysis and Modeling at University of
Connecticut Health Center. She helps biologists to incorporate
computing and computational resources into their teaching and
research. Community building and improvisational theater are explicit
components of the majority of her National Science Foundation funded
projects. She founded improvscience to provide scientists with
opportunities to develop skills in leadership, collaboration and
innovation. Since its inception improvscience has worked with over a
thousand professionals in Science, Technology, Engineering and
Mathematics education and research.

Regulations and Their Emergent Effects

There’s a fascinating story in the New York Times, “Profits on Carbon Credits Drive Output of a Harmful Gas“:

[W]here the United Nations envisioned environmental reform, some manufacturers of gases used in air-conditioning and refrigeration saw a lucrative business opportunity.

They quickly figured out that they could earn one carbon credit by eliminating one ton of carbon dioxide, but could earn more than 11,000 credits by simply destroying a ton of an obscure waste gas normally released in the manufacturing of a widely used coolant gas. That is because that byproduct has a huge global warming effect. The credits could be sold on international markets, earning tens of millions of dollars a year.

That incentive has driven plants in the developing world not only to increase production of the coolant gas but also to keep it high — a huge problem because the coolant itself contributes to global warming and depletes the ozone layer.

Writing good regulation to achieve exactly the effects that you want is a hard problem. It’s not hard in the “throw some smart people” at it sense, but hard in the sense that you’re generally going to have to make hard tradeoffs around behavior like this. Simple regulations will fail to capture nuance, but as the regulation becomes more complex, you end up with more nooks and crannies full of strange outcomes.

We as people and as a society need to think about how much of this we want. If we want to regulate with a fine-toothed comb, then we’re going to see strange things like this. If we want to regulate more broadly, we’ll likely end up with some egregious failures and frauds like Enron or the mortgage crisis. But those failures are entirely predictable: companies occasionally fake their books, and bankers will consistently sell as much risk as they can to the biggest sucker. For example, Bush administration’s TARP program or Seattle taking on $200 million in risk from a hedge fund manager who wants to build a new sports stadium. At least that risk isn’t hidden in some bizarre emergent effect of the regulation.

That aside, long, complex regulations are always going to produce emergent and chaotic effects. That matters for us in security because as we look at the new laws that are proposed, we should look to see not only their intended effects, but judge if their complexity itself is a risk.

I’m sure there’s other emergent effects which I’m missing.

Washington State Frees Liquor Sales: some quick thoughts

I hate to let an increase in liberty go by without a little celebration.

For the past 78 years, Washington State has had a set of (effectively) state-operated liquor stores, with identical pricing and inventory. Today, that system is gone, replaced by private liquor sales. The law was overturned by a ballot initiative, heavily backed by Costco.

This is an interesting experiment in letting a little chaos emerge. Unfortunately, it’s not really a transition to a free market, since there are all sorts of licensing restrictions on who may trade in the demon rum. However, there will initially be about 5 times as many legal retailers as were previously present.

The transition is going to be messy. There’s lots of licensed retailers who haven’t obtained inventory. There’s a thousand people who were voted out of their jobs. Change is often messy.

After the transition, I expect prices will be roughly the same because of taxes and fees. What I expect will be much better is the selection and variety, especially of locally produced products from folks like Oola and Pacific Distillery. Many of those businesses were seriously inhibited by the complex and chummy system that was present.

I also expect surprise and look forward to it.

So raise a toast to the slow unwinding of a very silly system of prohibition.