ProductivityAndFun Read More about LoadUI

Command Line Execution

PDF Print E-mail
User Rating:  / 7
Rate this article: PoorBest 

Once created, you might well want to run your LoadTests from the command-line, maybe part of a continuous integration build or for monitoring the day-to-day performance of your services. soapUI provides both a command-line runner and plugins for maven to do this, but since the maven-plugin is covered elsewhere, we will focus on the command-line runner instead.

The runner is available in your soapUI\bin folder and appropriately named loadtestrunner.bat/.sh. It takes a bunch of arguments related to reports, properties, etc which can make the creation of the initial argument-list rather tedious (the command-line options are all covered in detail in the LoadTestRunner Reference, here we will just focus on the most common ones). Fortunately the UI includes a wizard for launching the loadtestrunner, which will help you both assess the output and create the command-line invocation string.

The "Launch LoadTestRunner" option is available from the LoadTest popup-menu:

launch-loadtestrunner-option

It brings up the following dialog:

launch-loadtestrunner-tab1

By default the dialog will select the corresponding LoadTest, but the TestSuite/TestCase/LoadTest drop-downs allow you expand the scope of the execution, for example setting all 3 to "<all>" will run all LoadTests in the project. By default soapUI uses the testrunner in the soapui\bin folder, change to another version (perhaps for testing purposes) in the TestRunner Path field.

The "Save Project" option ensures that the current Project containing the LoadTest(s) to run is saved before launching, so that any modification made in the UI apply to the command-line execution also.

The "Save After" option should be used when the executed LoadTest modifies some state in the project file that should be persisted, most notably if you are using a DataGen TestStep with a Number property that you don't want restarting for each run.

The Overrides tab allows you to override the endpoint, host and port used by all requests in the LoadTest;

launch-loadtestrunner-tab2

Also the configured Limit and initial ThreadCount can be changed to another one, allowing you to easily rerun the same LoadTest with different limits and loads. The security-related overrides apply at the request level, for example you might be changing host to a server that requires Basic or WSS authentication.

The Reports tab allows you to configure which reports that should be generated for your LoadTest results and where to put them;

launch-loadtestrunner-tab3

The last two options are disabled in the open-source version since they are specific to the reporting functionality in soapUI Pro. The "Report to Generate" field should contain the name of one of the corresponding reports available in the "Create Report" dialog:

create-report-names

Correspondingly, the "Report Format(s)" option should then contain a comma-separated list of formats to generate; for printable reports the available values are PDF, XLS, HTML, RTF, CSV, and XML, for Data Export they are XML and CSV.

Next up is the Properties-tab, allowing you to override any global, system or project properties when executing the LoadTest(s);

launch-loadtestrunner-tab4

The obvious use case is if you have parameterized your TestCase(s) somehow and need to change those parameters accordingly.

Last up is the Custom Args tab which allows you to add any custom arguments to the command-line invocation:

launch-loadtestrunner-tab5

Ok, now lets actually run the LoadTest by pressing the "Launch" button. A window pops up showing the command-line invocation and output of the LoadTestRunner as it executes, when the LoadTest finishes we see the following:

launch-loadtestrunner-output-2

All seems to have gone well, the log tells us the LoadTest Report has been created in the specified folder, opening it shows us the desired results;

loadtestreport-output

The above shows us one other (rather important) fact to be aware of; running tests from the command-line can be slightly faster (and thus give better results) compared to running the from the UI. If we run the above test in soapUI Pro we get:

loadtestreport-output-in-ui

which is about 7% below the TPS and AVG values measured from the command-line. The difference is due to the UI overhead imposed by updating the table, etc.

Let's run the exact same test from the console; as you could see in the first output screenshot, theloadtestrunner.bat file gets launched as follows:

launch-loadtestrunner-output-1

which we can copy-paste into notepad and get the following arguments:

loadtestrunner.bat -sCurrencyConvertorSoap TestSuite -cConversionRate TestCase 
-lSimple Strategy -r -fc:\dev\reports -RLoadTest Report -FPDF C:\Users\Ole\Documents\loadtest-soapui-project.xml

If this is the desired configuration, we can take this string and use it in other command-line invocations or when creating automated runs; here from the command-prompt (be sure to surround arguments containing spaces with quotes);

loadtestrunner-from-console-1

And off it goes!