Tips for Running Load Tests with Apache JMeter

The platform team at WB Games in San Francisco was recently tasked with load testing our services so that we could optimize our Node.js code. Here are a few of our learnings about load testing with Apache JMeter.

Apache JMeter is "open source software, a 100% pure Java application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.” JMeter is well documented, so we were up and running tests — both locally and using remote slaves — pretty quickly. The few things that caught us off guard, and caused subsequent strife, were the following:

Running JMeter locally

If you’re trying to test your system with more than ~500 threads using your own machine, you’re likely to encounter a few issues:

On Linux or OSX, you’ll want to run the command ulimit -n 65536 (or some number greater than the default of 1024) so that your machine can hold that many open “files” i.e network connections.
You’re also likely to encounter Java java.lang.OutOfMemoryError exceptions, at which point you’ll want to take a look at this post on StackOverflow.

Running JMeter on remote hosts

Remote JMeter masters and slaves can’t function across networks, which makes it impossible, so far as we know, to coordinate load generation from different data centers.

Because of this, if and when we want to run tests in production, we’ll have to aggregate the data from different regions separately.
We learned that it’s important to attach a “Constant Timer” to HTTP requests to limit the number of requests per interval each thread makes. In our case, forgetting to do so led to 1250 threads making requests as fast as they could, instead of (in our case) once per second.

We also used a JMeter plugin called Stepping Thread Group, which enabled us to specify ramp-up and ramp-down periods for our tests. Unless you’re explicitly trying to test what happens to your system during DDoS attack, you’ll want to check out the Stepping Thread Group plugin.

This is the first in what will be a series of very concise posts about load testing services built on Node.js, so that’s pretty much it! If there’s anything I left out that would be helpful to list here, please don’t hesitate to reach out to me (@elrasguno) on twitter or on github.