Hacker's toolchest

May, 2000

Techniques and tools for penetration testing

Summary
There are hackers capable of penetrating almost any system. The good ones get paid for it. The bad ones pay for it. What is a hacker's approach to penetration testing? What tools do they use? In this column, Carole Fennelly asks noted security specialists Brian Martin, Mark Abene, and Rain Forest Puppy for their perspective. (3,000 words)

  WIZARD'S GUIDE TO SECURITY  
By Carole Fennelly

I recently heard of yet another penetration test in which the vendor charged $150,000 for two days of testing. It seemed pretty expensive to me, but I assumed that the testers must have brought in some major security gurus who ran uber-elite secret exploits against the systems. In fact, they ran ISS Scanner.

Don't get me wrong; ISS Scanner is a useful tool that tests for all known vulnerabilities. (I'm not picking on ISS; there are other tools like this as well.) It's just that for $150,000, I want Robert Redford testing my system. (Ok, who hasn't seen Sneakers?) It also occurred to me that there are hackers capable of penetrating almost any system. The good ones get paid for it.

What is a hacker's approach to penetration testing? What tools do hackers use? I decided to find out.

Hacker phobia

The term hacker has been abused by the media and by hacker wannabes to the extent that many people think all hackers are criminals. This is simply not true. For the purposes of this column, I'll define a hacker as "an expert with informal training."

While it's true that there are hackers who have violated laws and/or cannot be trusted, the same can be said for people who avoid the hacker label. We've all heard of insiders who, using their computers, rob a company blind. To make a hiring decision based solely on the label a person uses to describe him or herself is a copout.

In the following Q&A, I asked three well-known security specialists about their background and approach to penetration testing. The players include:

Brian Martin, a.k.a. Jericho:

Founder of Attrition.org, a popular hacker site with an eclectic mix of computer security tools, humor, rants, and the ever-popular Attrition Defacement Mirror. Brian presently works as a computer security consultant responsible for leading a security audit and penetration assessment team. In recent months, Brian's articles focusing on security issues have been widely circulated in corporate newsletters, in print magazines, and on the Internet.

Mark Abene, a.k.a. Phiber Optik:

Cofounder and president of Crossbar Security. Mark's teenage curiosity with phone networks prompted a federal judge to sentence him to a year in jail. While the experience deterred Mark from illegal hacking, time has proved that it wasn't much of a deterrence to others.

Rain Forest Puppy, a.k.a. RFP:

The bane of Microsoft. Best known for releasing the Microsoft IIS RDS exploit used by thousands of script kiddies. RFP is the author of the Web-scanning tool whisker, which can defeat intrusion detection systems. RFP is credited for recently revealing a backdoor inserted by Microsoft engineers that stated that "Netscape engineers are weenies!" RFP prefers to keep his professional identify anonymous.

Q: How long have you been doing penetration testing?

Brian: Professionally, for five years.

Mark: Professionally for over five years, but I first started penetrating systems when I was about 12.

RFP: Professionally with a company for about eight months, although individually as needed for about two years.

Q: Do you work for a company or independently?

Brian: I presently work for a company, although I've done both in the past.

Mark: Both. Currently, I'm the president of my own firm, Crossbar Security, Inc.

RFP: Company.

Q: Do your perceive yourself as a hacker-type or a computer security professional?

Brian: A bit of both. I try to be a well-balanced hybrid of the two.

Mark: One's experiences as a hacker-type are what make one a truly great computer security professional -- which is to say [that] it's one thing to be book smart or school taught and quite another to have years of hands-on experience defeating security systems.

RFP:
I'm a hacker-type, but I'm billed as a computer security professional because my clients are scared by the former. (Laughs)

Q: Do you see any problems with how penetration tests are typically performed? If so, what?

Brian: Yes. Most companies are charging an arm and a leg for canned security scanner output. While these scanners test for most vulnerabilities, they're always behind the curve in the latest exploits. The whole reason for having a penetration test is to test for everything, especially the latest vulnerabilities. If they can't do that, the client should just purchase their own copies of the scanners.

Mark: Typical penetrations are hardly thorough. They often rely on untrained auditors that use canned tools, which are only as good as their updates. They produce canned reports, devoid of actual useful information and hard facts. We go in after some large audit firm has done their thing, leaving the client with reason for doubt, and we totally penetrate their most sensitive systems. A truly valuable penetration study is a strategic operation, from start to finish.

RFP: Auditors who walk in and exclusively run ISS or CyberCop aren't doing a thorough job. The majority of the time, the auditors don't understand the security vulnerabilities that the scanners are finding.

Q: How can auditors prove that they've done a good job if they've been unable to find anything wrong in the allotted time?

Brian: By writing a comprehensive and well-written report that covers what was tested. I also believe that every network can be improved, so if they didn't find a single point of concern, no matter how trivial, they missed something.

Mark: If an auditor hasn't found anything, by de facto he's done a poor job. Any sufficiently large corporation is going to have security problems. Bureaucracy often unintentionally creates an environment where good security practices are overlooked. The larger the corporation, the more likely it is that holes are lurking. Disorganization and breakdowns in management breed security problems.

RFP: Auditors should document the list of vulnerabilities they checked and say something like, "I didn't find any problems, given the time frame and this list of checks." If the list of checks was small, an explanation of why those checks were picked would probably be sufficient. Obviously, more critical problems and popular vulnerabilities should take precedence.

Q: Do your clients ask you to sign a nondisclosure agreement?

Brian: Yes. And we require them to sign one as well.

Mark: Yes.

RFP:
For the most part, yes.

Q: How do you and your clients reach an agreement on the scope of the audit? Do you have a written Statement of Work before you proceed?

Brian: We always come to agreement and are very clear on what is within the scope. 99 percent of the time there's a written Statement of Work.

Mark: We always try to sell our clients on the largest scope possible, because we believe that a penetration greatly diminishes in value the more restrictions and off-limits areas are declared. Obviously, we both have to agree to something on paper, and we author our Statement of Work based on preliminary meetings with clients. Also, we treat each client separately, so our simulation will best suit their individual needs.

RFP: Typically, yes. However, the best interest is with the client, so if they wish to have extra systems audited, and time permits, we'd be happy to check them. This works well if you work with your client beforehand to firmly establish what they should and need to have done. Some companies have a sales guy/gal roll through, sell some service, and when the actual assessment team shows up, it turns out what was sold is not really what should be done.

Q: Do you write a report for the client yourself, or does your company write up your findings? Do you get final approval on the report?

Brian: For the most part, I write the entire report myself. After that, the rest of the team gives input, our managers give it a thorough review, and I get the final say.

Mark: One of the luxuries of running your own company is that you get to write your own reports. That way, nothing gets edited out that should be left in, and there are no misunderstandings.

RFP: I have personally done both; it depends on the particulars of that client.

Q: Do you tell your clients which tools you use?

Brian: If they ask. While it's no secret, we try not to list every single step and tool, because to do so would create a lot more work for us.

Mark: Always. We don't believe in crystal balls or sleight-of-hand, and we don't want our clients to either. Part of our final report is detailing exactly what tools and exploits were used, even demonstrating them after the fact for technical staff. We believe we're all on the same team, and teaching security awareness is the most valuable tool of all.

RFP: If they ask, we do tell, and, time permitting, offer to demonstrate and even teach them how to use ones that are available to them.

The toolchest

While many hackers have their favorite homegrown tools and scripts, some open source tools are freely available and quite useful. Which tool is most appropriate depends on what you're auditing. Brian Martin describes a few of these below. This is by no means a complete list (see Resources for more) and I hope to explore other tools in the future. Of course, these tools should only be used on a network that you're authorized to audit.

Just as you might question the wisdom of trusting hackers, you might question the wisdom of trusting hacker tools as well. The tools are open source and painstakingly reviewed by other hackers, often for egotistical reasons. If there's a back door or Trojan horse in the code, there will be a mad rush to be the first to post it on Bugtraq. From my experience, the hackers who write these tools are more concerned with maintaining a respectable technical reputation than in getting into systems that they couldn't care less about. That said, of course you shouldn't trust them, any more than you should trust code you get from a vendor. Review the code, verify the PGP signatures and checksum information, and test it on an isolated system. Do this for anything you get from a vendor as well.

Summary

So, do you hire a hacker to test your systems, or do you stick with the Big Five auditors? That's up to you to decide -- you run the risk of being owned either way. The alternative is to do it yourself. Then someday you may be considered a hacker. Just be sure to cover your assets.

Acknowledgments

Much appreciation to Brian Martin for writing the descriptions of the tools and for providing examples. My thanks also go out to Mark Abene and Rain Forest Puppy for their participation and contributions. I've learned a lot from this. Finally, I'd like to thank Lew Koch and Adam Penenberg for their review and suggestions.

Disclaimer: The information and software in this article are provided as-is and should be used with caution. Each environment is unique and readers are cautioned to investigate with their companies the feasibility of using the information and software in the article. No warranties, implied or actual, are granted for any use of the information and software in this article, and neither author nor publisher is responsible for any damages, either consequential or incidental, with respect to the use of the information and software contained herein.

Resources and Related Links
 
Copyright 2003, ITworld *-* Please direct questions and comments to Carole Fennelly. Permission is granted to quote, reprint or redistribute provided the text is not altered, and the author and ITworld are credited.