Penetration Testing: The Art or The Science?

So, I have been penetration testing for a while now. Over the years, I have seen penetration testing evolve dramatically. Back in the days, tools were not as smart as they are now. Now, we have state of art tools (burp Pro, Net Sparker, HP Web Inspect to name a few) and to be brutally honest, these tools are only going to get better as time goes by. The application frameworks are also getting better. A widespread issue such as Cross Site Request Forgery, is not as prevalent as it used to be and that's because the application framework are getting more security conscious then they were a few years ago.

In terms of penetration testing, people are spending lot more time in checking best security practices;

  • Cookies are marked as secure/http only
  • HTTP headers (x-frame, content security policy, HSTS, the list goes on...)
  • Usual SSL checks (ciphers, protocol, BEAST, CRIME etc)
  • Mixing HTTP and HTTPS
  • Auto-complete
  • yada yada yada

So, the point is everyone now a days follows a check-list to make sure all of this gets covered. I have no issues with check-list based approach and I understand that this is done to ensure that a level of consistency is achieved. My worry is that these check-lists are growing bigger by the day and we are making Pentesting a Science and killing the Art aspect of Penetration Testing.

Penetration testing, to me has always been something which requires out-of-box thinking. My worry is, there are a number of things which you can't document in a check-list, which are application and case specific and we need to educate the budding generation of penetration testers on this Art.

If you report 20 best security practices but miss a critical business logic flaw, then its a #FAIL;

To give you a more specific example to highlight this issue.

Assessing The Forget Password Functionality

So, these days apps send out a link with a token in it to reset your password. The link looks something like this:


What tests do you perform on this:

  • check if the token is predictable (cryptographically insecure)
  • check the token is 1 time only
  • A few more tests (is it over SSL or HTTP etc)
  • check that you cannot use the token of 1 user to reset the password of another user. So you may try to generate a link:


So, it turns out that in this 1 case, the app passes all these checks. Now, here comes the Art part. The Application validates that the token, but not quite as it should. It turns out that the application checks if the token is associate with an account on which reset password link has been generated rather than the actual email. So, in the above URL if the user 'user2' has not had a password reset initiated, the application will not process the request. However, if we initiate the reset process on both users (user1 and user2) then we can use the token for user1 to reset the link for user2.

So, as an attacker all you need is someone's email address.

  • You initiate a password reset on your account; get a valid token. Save this.
  • Initiate a password reset by submitting victim's email in the forget password functionality.
  • Now, you can use your token to reset victim's password and compromise his account.
  • Job Done!

The bottom line is, we need to strike a balance between the Science and the Art aspect of penetration testing. I get a feeling that currently the industry is moving away from the Art Aspect and pentesting is becoming more of a science. There are a couple of good presentations to get you motivated on business logic testing (1 of them by me and an ex-colleague):

Get rich or die trying
The Art of Exploiting Logical Flaws

Thats my 10 cents for this week!

Marketing Blurb
If you would like to try out our Penetration Testing Services, contact us here!