Stress/Load-Testing of Asynchronous HTTP/REST Services with JMeter

Monday, November 28th, 2011 by Frank Appel

Although I have been using JMeter for stress- and load-testing of web applications a good few times it took us a while to figure out how to test asynchronous HTTP/REST based services with the tool. With us I mean a fellow programmer – Holger Staudacher, I have the honor to work with currently on a project – and my humble self.

While Holger developed restfuse based on the experience of doing functional and integration tests for the project mentioned above, we decided to use JMeter for stress- and load-testing. The main service of the software under test processes a data structure that is uploaded to a certain URL. If the upload process was successful an URL pointing to a resource containing the processing result is returned. The resulting resource is not available immediately – processing takes a while. So polling is used to retrieve the resource once it is available¹.

Our goal was to measure the time it takes to upload the data structure, process it and download the result resource in one test run. Running such a test with multiple users concurrently should give us a fair impression of the throughput capabilities of the system. Does not sound too complicated, but…

…our first approach writing a test plan for the scenario described in the previous paragraph using the on-board capabilities of JMeter did not work out very well. Neither was the plan comprehendible nor – and that was worse – made the measurement results any sense. In particular clamping the upload request and the polling loops together with a transaction controller seemed to have some unexpected side effects with timers. So after a while of additional Google research I stumbled across the JavaSamplerClient API, which I did not know before.

There is an entry at stackoverflow.com that describes how to extend the AbstractJavaSamplerClient – an implementation of JavaSamplerClient – and use it within JMeter. So this was the way to solve our problem. We created an extension of AbstractJavaSamplerClient overriding runTest(JavaSamplerContext). Within that method we use HttpClient to perform the upload and poll requests. Once the processing result is successfully retrieved by a poll request all the header and content information is stored in an instance of SampleResult. The latter is returned by the overridden test sampler method for further processing by JMeter – quite straight forward².

Once you have created a jar that contains a custom JavaSampleClient and put this into the lib/ext/ folder below your JMeter installation directory you can add a Sampler type Java Request to your Thread Group. This allows you to select and configure a custom sampler as shown in the picture below:

Using the the JavaSamplerClient made our test plan very simple and allowed us to use the common JMeter result measurement functionality as shown below examplary with the Graph Results view:

And of course the measurement results are now reasonable…

Since we had to fumble quite a while to get this done I thought our solution might be of interest for other people, too – which was the reason to write this post. But it would be also interesting to hear from you if there are easier solutions we did not notice. So feel welcome to provide feedback

We started using WebHooks but our customer had problems to convince the IT admins to open up the firewall… ↩
For the sake of a reasonble length of this post I skip any description of how to deal with unsuccessful requests – but most of our implementation work had to be done in that area… ↩

Jason Weden says:

16. July 2012 at 16:34

Check out this analysis I just found today related to this: https://github.com/excilys/gatling/wiki/Benchmarks

Gayatri says:

13. August 2012 at 10:29

What would you suggest for unit testing rest API – restfuse, or Jmeter?

Frank Appel says:

13. August 2012 at 11:07

Your question already contains the answer. For REST API testing I would use restfuse. JMeter comes into play if you want to stress-/load-test your application.

- Anil says:
  
  22. October 2012 at 16:18
  
  Hello Frank,
  
  I too have similar requirement. can you please share the code yo have written.
  
  - Frank Appel says:
    
    23. October 2012 at 05:00
    
    I am sorry, but the post is based on work for a customer which is not open source. However given the description above it should not be that difficult to implement it by yourself.
    
    - Anil says:
      
      23. October 2012 at 13:37
      
      Thanks for your prompt reply
      I got it working ….

Benamar says:

7. February 2013 at 16:14

Hello Frank,

You said = ” The latter is returned by the overridden test sampler method for further processing”

How return you the results of multiple requests in your runTest?

thanks.

Frank Appel says:

7. February 2013 at 17:16

Benamar,

I don’t know if I understand your question correctly. But if I remember it right the whole point of a sampler is to work on single request basis. Multiple requests will be executed by JMeter subsequently in a loop or with multiple threads concurrently. But for each request a single delegation call to the sampler is issued by JMeter. Therefore you actually don’t have to deal with multiple results in the sampler implementation.

Regards
Frank

8. February 2013 at 08:56

Thank you for your quick response.
I used a thread pool to execute the simultaneous requests in ‘runTest’ method , and I used result.addSubresult (result [i]) to aggregate the results.

I can see the requests results in the results tree (as subresults) but the are absents in the reports!

Do you have any solution to get the results of these simultaneous tests in reports ?

Thank you.

Frank Appel says:

8. February 2013 at 17:07

I’m sorry but I don’t have a solution for this right at hand as I do not remember working with subresults.

DK says:

3. April 2013 at 13:47

I have a similar requirement but we are using WebHooks.
So, once the request is processed our REST Service will POST a response back to the callback URL sent in the request.

Can this be stress tested with JMeter using a similar approach?

Frank Appel says:

4. April 2013 at 11:33

Hm – I never did it, but I could imagine that it is possible to implement an AbstractJavaSamplerClient that serves as “server” for the callback URL. Such an implementation would block (something like ServerSocket#accept() instead of poll like described above) until the callback URL is triggered. But that’s just a spontaneous idea which sounds a bit low level at first glance – maybe there is a better solution around somewhere…

Shashank Pandey says:

21. May 2013 at 06:05

Can you please share the code for accepting the requests and sending a reponse. I just need a generic code for accepting a Student object. In my scenario the callbacks are giving me a jaxb object via a PUT body. I want to write that object in a file or on console. Please help (specially the connecting and returning the object part).

Frank Appel says:

24. May 2013 at 05:38

I am sorry, but as I already mentioned to anil the post is based on work for a customer that is not open source. So I am not able to post any part of it here.