Penetration Testing: The Art or The Science?

September 20, 2013

So, I have been penetration testing for a while now. Over the years, I have seen penetration testing evolve dramatically. Back in the days, tools were not as smart as they are now. Now, we have state of art tools (burp Pro, Net Sparker, HP Web Inspect to name a few) and to be brutally honest, these tools are only going to get better as time goes by. The application frameworks are also getting better. A widespread issue such as Cross Site Request Forgery, is not as prevalent as it used to be and that’s because the application framework are getting more security conscious then they were a few years ago.

In terms of penetration testing, people are spending lot more time in checking best security practices;

  • Cookies are marked as secure/http only
  • HTTP headers (x-frame, content security policy, HSTS, the list goes on…)
  • Usual SSL checks (ciphers, protocol, BEAST, CRIME etc)
  • Mixing HTTP and HTTPS
  • Auto-complete
  • yada yada yada
  • So, the point is everyone now a days follows a check-list to make sure all of this gets covered. I have no issues with check-list based approach and I understand that this is done to ensure that a level of consistency is achieved. My worry is that these check-lists are growing bigger by the day and we are making Pentesting a Science and killing the Art aspect of Penetration Testing.

    Penetration testing, to me has always been something which requires out-of-box thinking. My worry is, there are a number of things which you can’t document in a check-list, which are application and case specific and we need to educate the budding generation of penetration testers on this Art.

    If you report 20 best security practices but miss a critical business logic flaw, then its a #FAIL;

    To give you a more specific example to highlight this issue.

    Assessing The Forget Password Functionality

    So, these days apps send out a link with a token in it to reset your password. The link looks something like this:


    What tests do you perform on this:

  • check if the token is predictable (cryptographically insecure)
  • check the token is 1 time only
  • A few more tests (is it over SSL or HTTP etc)
  • check that you cannot use the token of 1 user to reset the password of another user. So you may try to generate a link:
  • http://host/resetpass.php?

    So, it turns out that in this 1 case, the app passes all these checks. Now, here comes the Art part. The Application validates that the token, but not quite as it should. It turns out that the application checks if the token is associate with an account on which reset password link has been generated rather than the actual email. So, in the above URL if the user ‘user2’ has not had a password reset initiated, the application will not process the request. However, if we initiate the reset process on both users (user1 and user2) then we can use the token for user1 to reset the link for user2.

    So, as an attacker all you need is someone’s email address.

  • You initiate a password reset on your account; get a valid token. Save this.
  • Initiate a password reset by submitting victim’s email in the forget password functionality.
  • Now, you can use your token to reset victim’s password and compromise his account.
  • Job Done!
  • The bottom line is, we need to strike a balance between the Science and the Art aspect of penetration testing. I get a feeling that currently the industry is moving away from the Art Aspect and pentesting is becoming more of a science. There are a couple of good presentations to get you motivated on business logic testing (1 of them by me and an ex-colleague):

    Get rich or die trying
    The Art of Exploiting Logical Flaws

    Thats my 10 cents for this week!

    Marketing Blurb
    If you would like to try out our Penetration Testing Services, contact us here!


    1 Comment

    • ePsiLoN says:

      “So, the point is everyone now a days follows a check-list to make sure all of this gets covered.”
      Well I think is that this is exactly where they are missing the point of the penetration testing. The truth is that nowadays to little people understand the difference between vulnerability assessment and penetration test and even if they understand they do not explain or inform their customers about that. And the honest explanation for this is that “penetration test” sounds more fancy 🙂 and is more sellable.
      But the idea of the PT is completely different form the one of the VA and people should now it.
      So … coming from this perspective I think that for real PT you still need out-of-the-box thinking.

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    This site uses Akismet to reduce spam. Learn how your comment data is processed.