Data Driven Testing with soapUI Pro

Before diving into the next installment of my series on new features in soapUI 3.0, let's back up a bit and have a look at the basics of Data-Driven testing in soapUI Pro; the DataSource and DataSource Loop teststeps and how they are meant to be used (see this post as a prequel to DataDriven LoadTesting, which takes things to another level).

What is Data-Driven testing?

Quite simply put; data-driven testing is when you store test data (input, expected output, etc) in some external storage (database, spreadsheet, xml-files, etc) and then use that data iteratively in your tests when running them. For example to test your phone-lookup service, you might have a list of names and expected phone-numbers in a database which you would use to "drive" your test, checking that each name gets the right phone-number back. It's really quite simple.

DataSource teststeps

In soapUI Pro, creating data-driven tests is (supposed to be) really easy; a DataSource TestStep is available for reading test-data into standard soapUI properties from a number of external sources (Excel files, XML properties, JDBC sources, files/directories, etc), these can then be used in following TestSteps (requests, assertions, xpath-queries, scripts, etc,) either via Property-Transfers or standard Property-Expansions. Finally follows a DataSource Loop TestStep which is configured to loop back over the previous TestSteps for each row/record in the DataSource.

What this boils down to is the following setup:

1) DataSource - reads test data into properties from external source
2) TestSteps (any number) - tests service functionality using the DataSource properties made available in (1)
3) DataSource Loop - loops back to (2) for each row in (1)

Enough talk, lets go presto!

1) Create DataSource

Ok, lets make it simple so you get the hang of things; Create an empty soapUI Project and import the publically available CurrencyConverter WSDL available at http://www.webservicex.com/CurrencyConvertor.asmx?wsdl, then add a TestSuite and a TestCase and open its editor:

Now add a DataSource TestStep and select the Grid DataSource from the dropdown in the toolbar, which is a DataSource for managing your test data from inside soapUI without requiring any external storage. You should now have:

Now we need to define which data our DataSource should hold, lets add three properties to our DataSource; From, To and Rate (use the "add" button in the toolbar on top of the properties list to the left);

As you can see, whenever we add a property, the Grid editor to the right adds another corresponding column so now we have three columns in which we can add some test-data. Add the following data:

SEK, DKK, 1.1
USD, EUR, 0.6
CAD, NOK, 0.7

Which in the editor will be:

Ok, now we've set up the test data we are going to feed into our test, lets move on to..

2) Create TestSteps

Now we need to add a Test Request to our TestCase which we will use to test the webservice; press the SOAP Request button in the TestCase editor and select the ConversionRate operation in the CurrencyConverterSoap Interface;

Press OK in all dialogs; a SOAP Request Step will be added to the TestCase and the editor for the request is opened. Switch to the Form editor (if not already there) and from the popup-menu of the two fields;

Select the corresponding properties in the DataSource; soapUI will automatically create the corresponding property-expansion syntax in the Form Fields (here shown in the outline view for legibility);

Now we have set the request to use the data in our DataSource, next step is to also create an assertion for the response to check that we are getting the expected response back; open the "Assertions" tab at the bottom of the Request editor and use the Add button in its toolbar to add an XPath Match assertion;

In the opened XPath Assertion editor, press the "Select XPath" button (top-left) to open the XPath Wizard. Since we don't have a response to select from, soapUI Pro will prompt us to use a default response instead; do so and select the ConversionRateResult element in the popup;

And finally use the "Select Content" button (bottom right) to select the Rate property in the DataSource; the XPath assertion should now be configured as follows;

Almost done now, all we need to do is..

3) Add DataSource Loop

Great! The request now uses the To/From properties in the DataSource and the Assertion uses the Rate. All that is missing now is an DataSource Loop at the end of the TestCase that loops back to the Request for each row in the DataSource; add a DataSource Loop step from the TestCase toolbar, double-click it and configure it to loop back to the Request for each row in our DataSource;

(If we hadn't done this, the request would only have gotten executed once).

All set; lets run the TestCase:

Not surprisingly we get an error from the assertion in the first run of the request; our expected Rate was not that returned by the web-service (since I made the values up..). Double click the Failed TestStep in the TestCase log (which opens the message viewer) and select the response tab to see what was actually returned;

(actually we could see this in the TestCase log as well, I just wanted to show off this feature as well)

Lets put that value into the first row of the Grid DataSource. If we now run the TestCase, we'll get a similar error for the second row and third row (after correcting the second as we did for the first). After correcting the last row the TestCase passes;

Nice! We could basically add as many rows as we wanted and soapUI would happily chew through all of them.

Moving to Excel

Now lets use an Excel spreadsheet instead of our DataGrid; go back to the DataSource and select Excel from the dropdown at the top:

Now the configuration looks as follows;

Create an spreadsheet in any program that can save XLS files and save it as such with the same values as we had (actually you should do this first before changing the DataSource, since you can copy all the values from the Grid into Excel). Also, to make things a little more interesting, insert another row in the middle with the values CNY,USD,4;

Now save and specify the file in soapUI; click the "Run" button in the DataSource window to see that the same values are returned (shown in the log at the bottom of the DataSource editor):

Now when we run the TestCase, it fails at the newly inserted row, but instead of correcting this open the TestCase Options dialog (from the TestCase toolbar) and uncheck the "Abort on Error" option; now when we run the TestCase that step still fails but soapUI will continue running through the other rows:

This is a common scenario when doing DataDriven tests; you don't want the test to stop at the first failing row but instead still process all rows.

Finally, select the "Create Report" button from the toolbar and specify a folder for export (in the 3.0 beta select "JUnit Style HTML Report"), resulting in the following in our browser:

Thats it! You're first Data-Driven test went like a breeze, well done! Now move on by reading about DataDriven Load Testing or just dig more into the different DataSource options, in any case I hope this got you started as expected. And for your reference, here are the two files;

- datadriven-test-soapui-project.xml - soapUI Project file
- datadriven.xls - excel file

Save these to a directory and import the project into soapUI Pro.. enjoy!