In Today’s Choicepoint Roundup, I mentioned that Richard Smith had found a number of issues with Choicepoint’s web sites. In discussion, Richard told me that the issues included (but were not limited to) robots.txt files and directory listings enabled.
The robots.txt standard is a way to tell search engines “please don’t go here.” That’s useful, if you have a section of web that’s database driven, and can result in infinite looping, of no value to either the search engine or your site. It can also be used to say, “please don’t index these ‘secret’ documents.” Now, those documents are not only not secret, but they’re now being pointed to, so someone gathering data can find it. They’re not an attacker, you chose to put that data on the web without any protection.
Similarly, directory listings being enabled may or may not be a security issue. You may want all users to be able to see all the documents in a directory. You might have made a mistake.
When building automated vulnerability scanners of any sort, these issues raise thorny questions. This applies across the spectrum, from Nessus-style credential-free scanners that look for known vulnerabilities, to Nikto, looking for classes of common implementation flaws, to static code analyzers like Splint. You’ll always find things which may or may not be ok in context. A system running a web server may be running your corporate web server, or it may be running, forgotten, on a developers desktop, full of flaws. That
strcpy(foo, bar) may never see attacker-provided data. The creators of these tools try to categorize and describe what they’ve found to help their users. Consultants offering a service around the tools can learn what questions to ask, to help sort through the issues faster, and focus on those that matter.
Similarly, an outsider looking at T-Mobile, Choicepoint, or PayMaxx suffers from trying to interpret what they see, perhaps trying to explain to a company that they’re not trying to hack the site, but that they stumbled across the issue.
I often see things which make me question “Hey, is there a serious security issue here?” and the answer is usually determinable within 5 minutes. Since I’m trying to do business with the company (which is why I’m on their site), I’d like the issue fixed.
Mature disclosure models need to improve not only the researcher side, but the way vulnerability reports are received. (“Press 9 to report a security problem with our web site.”)