Stress/Load-Testing of Asynchronous HTTP/REST Services with JMeter

Although I have been using JMeter for stress- and load-testing of web applications a good few times it took us a while to figure out how to test asynchronous HTTP/REST based services with the tool. With us I mean a fellow programmer – Holger Staudacher, I have the honor to work with currently on a project – and my humble self.

While Holger developed restfuse based on the experience of doing functional and integration tests for the project mentioned above, we decided to use JMeter for stress- and load-testing. The main service of the software under test processes a data structure that is uploaded to a certain URL. If the upload process was successful an URL pointing to a resource containing the processing result is returned. The resulting resource is not available immediately – processing takes a while. So polling is used to retrieve the resource once it is available1.

Our goal was to measure the time it takes to upload the data structure, process it and download the result resource in one test run. Running such a test with multiple users concurrently should give us a fair impression of the throughput capabilities of the system. Does not sound too complicated, but…

…our first approach writing a test plan for the scenario described in the previous paragraph using the on-board capabilities of JMeter did not work out very well. Neither was the plan comprehendible nor – and that was worse – made the measurement results any sense. In particular clamping the upload request and the polling loops together with a transaction controller seemed to have some unexpected side effects with timers. So after a while of additional Google research I stumbled across the JavaSamplerClient API, which I did not know before.

There is an entry at stackoverflow.com that describes how to extend the AbstractJavaSamplerClient – an implementation of JavaSamplerClient – and use it within JMeter. So this was the way to solve our problem. We created an extension of AbstractJavaSamplerClient overriding runTest(JavaSamplerContext). Within that method we use HttpClient to perform the upload and poll requests. Once the processing result is successfully retrieved by a poll request all the header and content information is stored in an instance of SampleResult. The latter is returned by the overridden test sampler method for further processing by JMeter – quite straight forward2.

Once you have created a jar that contains a custom JavaSampleClient and put this into the lib/ext/ folder below your JMeter installation directory you can add a Sampler type Java Request to your Thread Group. This allows you to select and configure a custom sampler as shown in the picture below:

Using the the JavaSamplerClient made our test plan very simple and allowed us to use the common JMeter result measurement functionality as shown below examplary with the Graph Results view:

And of course the measurement results are now reasonable…

Since we had to fumble quite a while to get this done I thought our solution might be of interest for other people, too – which was the reason to write this post. But it would be also interesting to hear from you if there are easier solutions we did not notice. So feel welcome to provide feedback :-)


  1. We started using WebHooks but our customer had problems to convince the IT admins to open up the firewall…
  2. For the sake of a reasonble length of this post I skip any description of how to deal with unsuccessful requests – but most of our implementation work had to be done in that area…
Follow me

Frank Appel

Frank is a stalwart of agile methods and test driven development in particular. He understands software development as a craftsmanship based on a well-balanced mix of knowledge and the experience of the daily work.

fappel@codeaffine.com
Follow me

Latest posts by Frank Appel (see all)

Tags: , , , , ,

24 Responses to “Stress/Load-Testing of Asynchronous HTTP/REST Services with JMeter”

  • Jason Weden says:

    Check out this analysis I just found today related to this: https://github.com/excilys/gatling/wiki/Benchmarks

  • Gayatri says:

    What would you suggest for unit testing rest API – restfuse, or Jmeter?

    • Frank Appel says:

      Your question already contains the answer. For REST API testing I would use restfuse. JMeter comes into play if you want to stress-/load-test your application.

      • Anil says:

        Hello Frank,

        I too have similar requirement. can you please share the code yo have written.

        • Frank Appel says:

          I am sorry, but the post is based on work for a customer which is not open source. However given the description above it should not be that difficult to implement it by yourself.

  • Benamar says:

    Hello Frank,

    You said = ” The latter is returned by the overridden test sampler method for further processing”

    How return you the results of multiple requests in your runTest?

    thanks.

    • Frank Appel says:

      Benamar,

      I don’t know if I understand your question correctly. But if I remember it right the whole point of a sampler is to work on single request basis. Multiple requests will be executed by JMeter subsequently in a loop or with multiple threads concurrently. But for each request a single delegation call to the sampler is issued by JMeter. Therefore you actually don’t have to deal with multiple results in the sampler implementation.

      Regards
      Frank

  • Benamar says:

    Thank you for your quick response.
    I used a thread pool to execute the simultaneous requests in ‘runTest’ method , and I used result.addSubresult (result [i]) to aggregate the results.

    I can see the requests results in the results tree (as subresults) but the are absents in the reports!

    Do you have any solution to get the results of these simultaneous tests in reports ?

    Thank you.

    • Frank Appel says:

      I’m sorry but I don’t have a solution for this right at hand as I do not remember working with subresults.

  • DK says:

    I have a similar requirement but we are using WebHooks.
    So, once the request is processed our REST Service will POST a response back to the callback URL sent in the request.

    Can this be stress tested with JMeter using a similar approach?

    • Frank Appel says:

      Hm – I never did it, but I could imagine that it is possible to implement an AbstractJavaSamplerClient that serves as “server” for the callback URL. Such an implementation would block (something like ServerSocket#accept() instead of poll like described above) until the callback URL is triggered. But that’s just a spontaneous idea which sounds a bit low level at first glance – maybe there is a better solution around somewhere…

  • Shashank Pandey says:

    Can you please share the code for accepting the requests and sending a reponse. I just need a generic code for accepting a Student object. In my scenario the callbacks are giving me a jaxb object via a PUT body. I want to write that object in a file or on console. Please help (specially the connecting and returning the object part).

    • Frank Appel says:

      I am sorry, but as I already mentioned to anil the post is based on work for a customer that is not open source. So I am not able to post any part of it here.

  • Shashank Pandey says:

    And even the callbacks are being accepted via a rest interface

  • Hi, just wanted to mention, I loved this blog post. It was helpful. Keep on posting!

  • rinzenzandro says:

    I created a new project, exported it to a jar, put the jar in lib/ext of jmeter but I don’t see my class in “Classname” inside “Java Request” in jmeter.
    what am I doing wrong?
    does my jar need to have a special name? special package structure?
    do I need to rename my jar to ApacheJMeter_java.jar?
    do I need to add my class to that jar and recompile?

    • Frank Appel says:

      There is no need a for special name or package structure, as far as I know. The jar must contain your compiled custom JavaSampleClient of course. But otherwise I don’t know about any particularities.

  • Mandar says:

    I want to load test a Restful service which accepts XML file as an input parameter. Do you have any idea how I can use JMeter to load test it? Or how we can pass XML to Restful Webservice?
    Regards
    Mandar

    • Rüdiger Herrmann says:

      Did you have a look at the JMeter documentation (search for ‘xml’, the fourth or so hit)? It says that there is a ‘Post Body’ tab on the HTTP Request configuration panel that you can use to post raw content.

      HTH
      Rüdiger

  • Niharika says:

    Hey Frank
    I have a situation in my project and we are using JMeter for performance testing.
    We have RESTFul API and I need to load test these services.
    The intent of this test is to reach the threshold. This will require me to increase the user load at runtime gradually until the application breaks down.
    Could you please suggest how to go about this?
    So far I have been doing all the configuration in the script and then execute it. I have no clue how can I make JMeter to do this at runtime.
    Any help would be appreciated.
    Thanks
    Niharika

    • Frank Appel says:

      Hi Niharika,

      did you try to configure your thread group with an appropriate amount of threads and ramp up time for continously increasing the load? For more information on this you might have a look at Test Plan manual description, section 4.1 ThreadGroup.

      Hope that helps
      Frank

  • Niharika says:

    Yes Frank
    I generally do it for 5-10 threads for a duration of 1-2 hrs. Also used a constant throughput timer to stick to the throughput per SLA.
    So this sums up to 100k transactions, which my application is capable of handling.
    I want to test beyond these numbers and execute a test, where I can increase user load at runtime if I find the application does good.
    Was I able to explain my concern any better?
    Let me know if you know of any other tool which might help. I will do the research.
    Thanks

    • Frank Appel says:

      Now I think I understand. You want to change your test-plan configuration while it is executed. Well, I am not aware of a possibility to do that with JMeter (but that must not mean much…). However, why not running the test-plan multiply times with different load? Doing this in a way like a binary search algorithm, you should find the capacity threshold of you application reasonably fast.

      I mean run you app with a large load that crashes it pretty soon. For the next run decrease the load by half. If the app crashes, decrease load again by half else increase it to the middle of the current and the previous load. After a few iterations you should get a good impression what load your application can bare. This might work out even faster than looking for a tool. Good luck!


Leave a Reply