In part 3 of this series, lets look at the new reporting functionality in soapUI Pro 3.0 (after all, it is the "Reporting Release"). I'll start by giving a quick overview of the possibilities and will then dig into how your reports can be customized and tailored to specific needs. Check it out!

Reporting Overview

Straight to the point: soapUI Pro 3.0 provides three types of reports;

1. JasperReport based reporting which can be fully customized on both a global and project level
2. DataExport reporting useful for exporting data for post-processing/analysis in external tools
3. HTML-based Test and Coverage reports as were previously available

These are all available at the Project, TestSuite, TestCase and LoadTest levels (except HTML reports which are not available at the LoadTest level), and a corresponding Create Report button is available in all corresponding windows. Pressing it will prompt which type of the above reports to create together with related configuration options, after which the desired report is created and displayed.

The command-line tools have been modified accordingly to allow specification of which report type to create and which format to save as (when applicable).

In the remainder of this post I am going to walk you through the basics of the JasperReport based reporting, read about the others at DataExport Reporting and JUnit-Style Reporting in the User Guide.

JasperReports based Reporting

soapUI Pro 3.0 integrates the popular open source JasperReports reporting engine for generating reports that can be printed and saved in a variety of formats (PDF, HTML, XLS, ODF, RTF, etc). JasperReports uses an XML-based template-language which is extremely powerful but also has a pretty steep learning-curve (IMHO), therefore our goal has been to provide usable reports out-of-the-box, with the possibility to customize them as you wish requiring corresponding know-how of the templating language.

Before we dig into the internals of how all this works, lets start by simply generating a default project report; open the sample project included with soapUI, start its MockService, open the Project window, select the "TestSuites" tab and run all TestSuites with the top-left Run-button. You should see something like the following:

Now press the "Create Report" button in the Toolbar and select the "Project Report" option, you'll see the following:

(The different options displayed are actually controlled by parameters defined in the report template, making it easy to add your own optional content).

Press OK to generate the report; after a bit of churning you should see the following;

Here you can zoom/page around the report and either print or save it in one of the supported file formats via the corresponding toolbar buttons. Saving the report to a PDF results in something like the following: reportsample.pdf

How does this work?

So what actually happened here? Lets dig a bit into the internals of this; open the "Reporting" tab in the Project window, which should be something like this:

As you can see, reporting configuration is divided into 4 parts:

1. Available ("Main") Reports - main report templates for corresponding soapUI Items
2. SubReports - for subreport datasets that are included in a main report
3. Templates - chunks of template-XML that can be reused in your main and subreport templates
4. Parameters - for parameterizing your reports without losing performance

Lets have a look at each of these in order

Main Reports

Each main-report template in soapUI Pro is linked to an item in the soapUI object model; Project, TestSuite, TestCase or LoadTest. When pushing the "Create Report" button for any one of these in their corresponding windows, the list of available reports will include those configured here for the corresponding item (as you can see only one report is available by default for each item).

Selecting a report in the table shows its template XML in the bottom area, together with a Details inspector for showing more detailed information. Global report templates are read-only, they can only be edited in the file-system for now. Project-specific templates on the other hand can be edited directly in the edit-area, and changes will immediately be applied when rendering the corresponding report the next time.

SubReports

Most content of a Main Report comes from related sub-reports. For example, when running a TestSuite, a sub-report containing results for each of the run TestCases is available, which can be included in the report with a corresponding template for formatting.

The SubReports panel looks almost like the one for Main Reports:

Which sub-reports that will actually be included in the report depends on if they have been (sometimes conditionally) included in the main-report templates, and if they actually contain data (otherwise Jasper be default doesn't include them).

Templates

The Templates tab shows the following (initally empty):

Templates are basically "chunks" of XML that you can reuse in any main or SubReport, the actual inclusion mechanism being a standard property-reference in the format ${templateName}. So if you have a common block of XML that you want to use in your custom reports, put it in a template and use that in your reports instead.

Parameters

Parameters are the Jasper-way of parametrizing a report; the Parameters tab is available both at the project level and as a "Report Parameters" tab in the corresponding Model Item editors (TestCase, TestSuite, etc..):

When added here, the corresponding parameter is made available in the report templates with the P${parameterName} syntax. Parameter values themselves can contain property-expansions, thus referring to whatever property that might be of interest.

Property-Expansion in templates

Standard soapUI Property-Expansion also works directly in the template XML, so if you have a TestCase report, you can happily add ${#TestCase#testCasePropertyName} to the report template xml, which will get expanded accordingly when generating the report.

Which is to prefer of these two? It boils down to performance; Jasper Parameters are defined at design-time and their value is specified at generation-time, which allows soapUI to re-use the previously compiled report with different input values. Property-expansion in the templates on the other hand will require recompilation of the report template every time the expanded value changes, which will take substantially longer.

Our first report revisited

Ok, now we know a little more, so here is what actually happened when we chose to generate the default Project Report:

• the "Create Report" button brought up a list of available reports, which included the main "Project Report" seen in the "Available Reports" tab
• When generating the report, a large number of SubReports were included; look in the Project Report template for the (long) list of SubReport elements, the first being

<subreport isUsingCache="true">
   <reportElement positionType="Float" isPrintRepeatedValues="false" x="0" y="1"
       width="535" height="30" isRemoveLineWhenBlank="true">
      <printWhenExpression>$P{IncludeOverview}</printWhenExpression>
     </reportElement>
      ...
    <dataSourceExpression>$P{ReportMetrics}</dataSourceExpression>
    <subreportExpression class="java.lang.String">"subreport:MetricsReport"</subreportExpression>
</subreport>

• which is the inclusion of the "ReportMetrics" subreport using the corresponding "ReportMetrics" datasource.

Still rather confusing I guess, but toy around with this some more, and understanding will come to you!

Customizing Reports

The JasperReports-based Reporting can be customized in many ways:

1. By modifying/extending the global templates or specifying custom versions of these
2. By creating custom templates which can basically contain any desired data from soapUI and/or external sources and using this with the existing templates or on their own
3. By specifying an "onReport" scripts for TestCase/TestSuite/Project/LoadTest which can modify/add report data
4. By overriding any default parameters defined in the existing report templates
5. Any combination of the above..

Options 1 and 2 are a bit more advanced so I'll leave them for a future post, lets start with some easy customizations instead: Changing the logotype and adding custom report metrics.

Changing the logotype

If you look at the main report templates in the editors, you will see that they all contain the following pageHeader:

<pageHeader>
   <band height="51">
      ...
      <imageExpression class="java.lang.String">$P{HeaderLogo}</imageExpression>
      ...
   </band>
</pageHeader>

This is a JasperReport parameter in action ("HeaderLogo"), if you look a bit further up in the template you will see the definition of the parameter:

<parameter name="HeaderLogo" class="java.lang.String" isForPrompting="false">
   <defaultValueExpression>"../../logos/header_logo.jpg"</defaultValueExpression>
</parameter>

So all we need to do is define our own HeaderLogo parameter in soapUI and set its value to our custom logo; open the Parameters Tab in the Reporting panel and add a parameter named "HeaderLogo":

Now when we generate the report, well see the following

Not very Web 2.0, but it does get the point across

Adding custom metrics

As you might recall, the first page of the report contains a "Project Metrics" section, the corresponding sub-report used here is the MetricsReport SubReport template:

Adding custom metrics is as easy as calling the addMetric method on the report object passed to our onReport script; open the Project Overview tab, select the "Report Script" inspector at the bottom, and enter the following:

report.addMetric( "Custom", "user.home", System.getProperty( "user.home", "" ), null)

Now regenerate the report, and voila!, you (should) have the user.home property listed in your project metrics:

ok.. we've only scratched the surface of the reporting-possibilities available, stay tuned for a follow-up post on report customization.. see you!

/Ole
eviware.com