The HP Security Laboratory Blog
Protecting businesses from hackers through software & services that helps customers find & fix security vulnerabilities across the application life cycle.

ASC Products are "Production Ready"

A recent blog post from Jeremiah Grossman caused a bit of discussion this past week. Jeremiah was trying to create a much needed chart showing all the different vendors who have an application scanning offering be it SaaS or a product, and be it dynamic scanning or static/source scanning. WAF vendors were not included. He rated each offering against a set of criteria. One criteria "Production Safe" struck me as a vague metric. Certainly scanning production systems has additional challenges that scanning internal test systems do not. How easily a product can be configured to overcome these challenges is critical. However, there was no clarification about how the chart defined "Production Safe."


In the comments of the post, Jeremiah stated:


Adjusting scan speed, simultaneous thread, and ensuring the tests themselves do not have active payloads is most common. Also very important is having a person mark forms as safe for testing. I'd argue though that these steps are function of people/process and not a feature of the scanner itself. Hence, no check mark.


This last sentence concerned me. It seemed to imply that scanners are not "production safe"... it's the people that use them that make them "production safe" or not. So I emailed Jeremiah for clarification.


Billy:


I want to make sure I understand. Does your benchmark that "these steps are function of people/process and not a feature of the scanner itself" require a human driven SaaS offering to receive a checkbox under "Production Safe?" In other words by your criteria is it impossible for a product to be "production safe?"


Jeremiah:


I want to be careful as not to suggest my definition is set is stone. Im open to a better or more refined one. Having said that, I'd say if a scanner is testing forms blindly (for example / some links too) without human decision making, then this is inherently dangerous. As is injection executable code.  If however the forms are custom configured by "someone", then the tool can in theory run reasonably safely. This combo can be delivered by a service (SaaS/Consultant) in a package, but to the best of my knowledge products don't ship with enough AI to do it autonomously. I guess this is the crux of my distinction.


I responded:


Software is never going to, out of the box, "know" what it should and should not audit. It requires intelligence to properly configure and launch. Whether that intelligence is provided by the customer who is using the product or a man-behind-the-curtain using technology inside of  a SaaS offering should not matter. However you are making that  distinction.


Jeremiah:


I think the distinction matters a great deal from the customer perspective. The solution is production-safe upon pressing "Go" or  it isn't. Or maybe "production-safe" should have a qualifier like   ready-to-go production-safe or something.


Jeremiah later emailed me saying he was going round and round on his opinion and might have a few things wrong. Admittedly it is a tough point to articulate. Hopefully he will publicly clarify what "product safe" means. Regardless, the "how do you properly scan a production application" is an important discussion that the industry needs to have.


So let me state ASC's position: A scanner should be considered "production safe" if what it can and cannot crawl or audit can be configured and controlled by an informed user. We say unequivocally that HP's Application Security Center products are "Production Safe" as we provide a diverse group of settings that allow a user to:



  • Record macros to whitelist parts of the application to test

  • Step the product through a business process to focus the scanner to specific areas of your application

  • Use regexs to define what urls should or should not be crawled and/or audited.

  • Use regexs to define what HTML Forms should or should not be submitted or audited

  • Configure the number of threads used to crawl or audit the web server

  • Manually control speed through our proxy tool

  • Define file extensions or Mime-types to discard should the product inadvertently encounter them

  • Turn on/off execution of various technologies like Flash, JavaScript, etc.


Furthermore, from a customer experience point of view, we do everything we can to ensure our customers know how to use our products properly. We have a comprehensive 2 day training course that is included in nearly every product sale and additional training available. We have countless online videos, tutorials, and support documentation. We even offer a managed SaaS offering for ASC products. In short, we have multiple different efforts to ensure that our customers know how to use our products effectively from Day 1.


Remember: Scanning products should never be considered "fire and forget" tools. Care should always be used when configuring and launching a security assessment. This rule apples to application security scanners just as it applies to network security scanners and other security auditing tools. Any vendor who characterizes their products in a purely "fire and forget" fashion is irresponsible at best and flat out lying at worst.


The bottom line is as long as a tool has a diverse number of settings that allow you to easily control what it can and cannot be assessed, the tool can be used against production systems. An informed user, and not some animated piece of code makes something "Production Ready." Make sure you check for features to control the scope of the security assessment when evaluating any security tool.


 <-->

Comments
Anonymous(anon) | ‎09-02-2009 03:44 AM

Software testers increasingly have to deal with production systems. Some tests make sense only with production systems, such as Nessus-style vulnerability scanning. And an increasing number of systems is hard to reproduce in

Anonymous(anon) | ‎09-15-2009 06:42 PM

I am very  glad to see these lovely and beautiful detail.

Search
Showing results for 
Search instead for 
Do you mean 
About the Author


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation