Sunday, April 15

What does RPT Lacks for a Loadrunner User

    I have been a HP Loadrunner user most of the time for Performance/Load Test.  Off late, I ventured into task  to use IBM Rational Performance tester (RPT). I did had very pleasant surprises from this tool . Below are few of those to mention
  1. Automatic correlation feature got me quite amazed when it worked flawless all the time. RPT Uses Perl type of regular expression patterns to extract data from respone.  I was impressed with perl regular expression back in time when I first used Jmeter and still find they give robust way of defining pattern. RPT does all related to automatic extraction expression pattern, instance number etc. Thus makes life easy when creating a new script.
  2. Not just records thinktime, also records client processing delays . Hence represents right way of simulating secondary requests

  3. But I find that many features ,which are very common to Loadrunner or any scripting tool, are missing here in RPT. RPT does make maintaining scripts bit harder without simple scripting interface . Below are the few of those missing features that made RPT annoying to me at times.

    You cannot call a test within a test.
    This meant that I cannot refractor the common steps (like login, etc). This meant that I cannot maintain common http calls across the scripts. Frequent RPT users  advice another way on how to go about this.  They suggest to break up one test in many tests and then arrange them in performance test schedule.  I did manage to refractor this way, but this meant lot of overhead work of creating variables, assigning them .  And every time I make change in one test, then I need to go to schedule and execute it to verify. Wouldn't the world be easy if we could be able to call a test within a test. This would solve many small pestering problems like I just mentioned

    Cannot verify another test script, if there is already a performance schedule is running.
    While using Loadrunner, there were situation,  where I executed a test in controller for a Project and at the same time, verified the scripts from another project. This helped me to make use of those idle time when test execution happens in background. But in RPT, you cannot do that. You can execute one test at a time. If you executing a main test in RPT master, then you cannot verify another script

    No Vuser end method
    In Loadrunner, if one stops a test , then users exit gracefully and execute vuser end code. This is kind of feature is missing in RPT. Hence it means, when you have an application which requires user to logoff , there is no way you can achieve this, when you stop the test in middle. Users won't exit gracefully.
    I have an application which requires user to logoff else it won't allow to login back for another 10 mins. If I find for some reason that I need to stop the test, then I had to wait for 10 minutes, verify all those users are okay (imagine time and effort spent to verify 10000 users if I require to) and then restart the test. This missing feature has made me to loose lot of time in the middle of tests and is very annoying for all who  are involved in the tests

    Cannot stop particular user during execution
    This thing I used to frequently in Loadrunner, where in I find some user had corrupt data (because they did not logoff properly ) or I require to bring down a particular number of users  in a particular group , then I had means to pick and choose user(s)and then stop them. While in RPT, you can still stop users, but you cannot pick and choose particular user(s)  and then stop them. On this front, RPT just fares same as Jmeter which is sad. I admire Jmeter although it lacks this feature.

    Capturing response, data substitution on error or failure
    This capability was one of very good troubleshooting asset of Loadrunner. During a performance test with many users doing lot of interaction,  if an error or failure occurs, you would definitely like to know what was request made, response received, data susbituted etc ,and also at same moment itself you view them. RPT has bit complex logging method, which I still need to get my head around. But what I found ,when eyeball over it, there is no obvious way to get this. Only way is you have to log a datapool value(parameter in loadrunner), for which you have write a custom code and write the value in Test Log. This gets logged irrespective of failure or not. Imagine in Http request where you are sending quite number of correlated values plus datapool values, then you have write Java custom code to log all of these. Wish, it has some option which allows to capture response, data substitutions etc only on error.


Related Posts Plugin for WordPress, Blogger...