Load Testing Web Services with Grinder

I first came across Grinder a year or so ago but never got beyond installing it on my machine. The latest version, which is still in beta, has a lot of new features, including support for the Jython scripting engine. The latest version also has a console, which collects and displays timing information generated by the load tests.

First impressions have been very positive indeed. I was able to install and run a simple Jython-scripted example in less that 1 hour. After one day with Jython and Grinder I was able to leverage some existing JUnit test cases to do some ad hoc load testing on my web service.

Initial contact with Jython has also been very positive and I intend on learning more about this scripting language. I found the Introduction to Jython Part 1 and Part 2 tutorials, from IBM devWorks, very useful.

Below is a simple Jython test script, which can be executed by Grinder. (Check the Grinder Tutorial for more details)

from net.grinder.script import Test;
from net.grinder.script.Grinder import grinder;
from com.comware.wm.test.rest.load import RestWorkManagerSession;

log = grinder.logger.output;
test = Test(1, "Create, Accept and Complete");
class TestRunner:
    def __call__(self):
        session = test.wrap(RestWorkManagerSession());
        session.getMyWorkList();

This test basically leverages an existing Java class called RestWorkManagerSession (partially shown below) and makes a call to the Work Manager web service to retrieve my work list.

public class RestWorkManagerSession extends AbstractRestTestCase {
	       ...
    public WorkItems getMyWorkList() {
	String myWorkListUrl = getLocationUri() + _sessionId + 
                               "/workLists/myWorkList";
        try {
	    return getWorkList(myWorkListUrl);

	} catch (Exception exception) {
	    // convert to a runtime exception

	    throw new RuntimeException("getMyWorkList",
		  		       exception);
	}
    }
}

Grinder uses a property file to parametize particular aspects of a load test. In the property file you can configure the number of processes, threads and iterations for a instance of grinder. You can also configure sleep time between each iteration, which is useful for simulating web service transaction rates.

#
# Example grinder.properties
#
grinder.processes=1
grinder.threads=2
grinder.runs=10

grinder.jvm.classpath=build/classes;build/test
grinder.jvm.arguments=-Dpython.home=/applications/jython-2.1

grinder.useConsole=false
grinder.consolePort=6372

grinder.logDirectory=logs
grinder.numberOfOldLogs=2
grinder.logProcessStreams=false

grinder.initialSleepTime=500
grinder.sleepTimeFactor=0.01
grinder.sleepTimeVariation=0.005

grinder.script=src/test/load/grinder/CreateAcceptCompleteScenario.py

To run grinder you simply pass it the property file

  grinder.bat grinder.properties

The next step is to determine whether I can configure grinder to execute the following load test

• 10 users concurrently retrieving work lists every 5 seconds and

• 5 users concurrently creating work items every 10 seconds and

• 3 users accepting and completing work items every 15 seconds

Four years ago a collegue and I created openexec , which was supposed to do just that. This is what the configuration file looked like. (There is even some documentation).

<?xml version="1.0"?>
<OpenExecConfiguration>
  <Scheduler />
  <Logger />

  <Users>
    <User userName="chris" />
    <User userName="jima" maxConcurrentThreads="1" />
    <User userName="jimm" maxConcurrentThreads="2" />
  </Users>

  <Scenarios>
    <Scenario className="SleepExecutable" scenarioName="sleep100">
      <Properties>
        <Property name="sleep.time" value="100" />
      </Properties>
    </Scenario>
    <Scenario className="SleepExecutable" scenarioName="sleep1000">
      <Properties>
        <Property name="sleep.time" value="1000" />
      </Properties>
    </Scenario>
    <Scenario className="SleepExecutable" scenarioName="sleep10000">
      <Properties>
        <Property name="sleep.time" value="10000" />
      </Properties>
    </Scenario>
  </Scenarios>

  <UserScenarios>
    <UserScenario interval="500" delay="0" serialSceduling="false" 
      executionCount="20" userName="chris" scenarioName="sleep1000"/>
    <UserScenario interval="500" delay="15000" serialSceduling="false" 
      executionCount="20" userName="jima" scenarioName="sleep1000"/>
    <UserScenario interval="500" delay="40000" serialSceduling="false" 
      executionCount="20" userName="jimm" scenarioName="sleep10000"/>
  </UserScenarios>
</OpenExecConfiguration>

...the CVS logs show that it hasn't been touched for 4 years :-(

Jim Alateras is an independent consultant specializing in open source and emerging technologies.

oreillynet.com Copyright © 2006 O'Reilly Media, Inc.