Unit Test Overview

Introduction

Use the StreamBase unit test framework to record target input and output field values for a given EventFlow or StreamSQL application or module, then verify at a later date that the application returns the expected tuples from known inputs. Unit tests are configured using StreamBase Studio, but can be run from the command line or run as part of automated unit test scripts.

The StreamBase unit test framework is based on the Eclipse JUnit Test Framework, and borrows views and functionality from that underlying framework.

Unit Test Lifecycle

The unit test lifecycle for StreamBase applications is as follows:

Develop an application and make sure it is free of typecheck errors.
To help you develop, you can optionally create feed simulation files for your application's input streams and prepare any CSV input data files needed by the feed simulations.
Run the application and confirm that it operates as expected with your feed simulation files, if any are used.
Create a unit test configuration by running the application and recording its interaction with input data. The input data can be from manual input, from your development feed simulation files, or from playback in the Recording view. This procedure creates:
- A test feed simulation file for each input port that records what went in that port during the recording.
- A data validation CSV file for each output port that records what came out that port during the recording.
- A unit test configuration file with sbtest extension to tie all the pieces together into a single test.
Modify the unit test configuration file and the recorded data validation CSV files as required:
- Specify which CSV data values can be excluded from the test. That is, specify values that can differ from the original recording without causing a test failure.
- Specify any CSV data values or other criteria that might differ from the recorded behavior, but must be true for a test to pass.
Run the unit test in Studio and observe pass or fail results.
- If the test fails, go back and modify the test configuration and data validation CSV files. Keep running the test until you get consistent pass results.
Optional. When the test produces consistent pass results in Studio, run the test again with the sbtest command line utility.
Optional. Add the command line test scenario to your build and test automation procedures.

Once a unit test is developed and verified, you can use it later to confirm correct behavior of your application or module, or to confirm there have been no regressions in desired behavior. A developed test can be run in Studio, from the command line interactively, or from test scripts that call the command line test.

Note

To get started learning about unit tests, you can create a unit test configuration by recording your application receiving input from the Manual Input view in Studio instead of from feed simulations. However, to create a production-ready unit test with real-world input stream speed, use feed simulations.

Advanced users can create unit test configurations manually by creating data validation CSV files and creating a unit test file in the Test Editor. StreamBase Systems recommends starting with a unit test recording, and editing the generated test files.

Unit Test Limitations

Be aware of the following limitations of the StreamBase unit test framework:

Test resources must reside in the same directory as the unit test file. Test resources include:
- Feed simulation files for each input stream.
- Any CSV input data files used by those feed simulations.
- The data validation CSV files created by the test recording (or created manually).
The test framework runs in a separate Java VM than the application under test. For applications with large memory requirements, this can lead to memory resource conflicts. While you can adjust the runtime parameters of the VM running your application, there are no adjustments available for the VM running the test.
The unit test framework tests the top-level application or module it is given to test. If that application calls another StreamBase application as a module, the operation of the module is treated normally, as a simple component at the top level with inputs and outputs. If your application is composed of several modules, create separate unit tests for each module and test each module independently.
If the application under test contains an output adapter, the output streams of that adapter cannot be tested. Output adapters send output to file formats or data sources not under the control of the unit test framework.

Your application might be constructed in a way that you allows you to exclude, without breaking the application, an output stream that goes to an adapter. In this case, you can create a unit test that excludes the output adapter. If the output adapter's is more closely integrated into your application, the you will not be able to create a unit test for that application.
Be cautious when recording a test against an application with inputs derived from input adapters. The general recommendation is that any input adapters whose output is not always the same with each application execution be excluded from the test. Do this by removing any streams downstream from the adapter from the test's expected output data definition.
The application under test might have components with predicate settings that change the stream in non-repeatable ways. For example, a component might call the random() function as part of its processing. The tests for such applications are likely to fail with each run. You may be able to exclude the column that contains such fields by using the Ignore Field Character, or you might be able to exclude an input or output stream from the test that avoids the path with the non-repeating predicate setting.

Creating a Unit Test by Recording

Use the following procedure to create a unit test configuration by recording the interaction of a running application with feed simulations:

Run the application as described in Running Applications in Studio.
While the application is running, invoke File → New → StreamBase Test. This brings up the New StreamBase Test dialog.

The Record a test checkbox is checked by default. (This checkbox control is greyed out if you run File → New → StreamBase Test with no application running.)
Select a folder in your workspace, and enter a base name for the test file. Click Finish.
Clicking Finish starts the recording and opens the Recording StreamBase Test dialog. This dialog stays open while the test is recording input and output data.
Supply data to the application using the Manual Input view, the Feed Simulations view, the Recordings view, or from an external source. The Recording dialog continuously reports the number of tuples recorded for use by the unit test.
To stop recording, click Finish in the Recording dialog.
The Test Editor opens, showing the current settings for the test just recorded.
Save the test file and run it, or modify the test settings as described in Using the StreamBase Test Editor.

The test recording saved one data validation CSV file in the project folder for each output port in the application under test. These files are named with a combination of the test name and the port name, with .csv extension. You may need to edit these CSV files to include or exclude certain fields from causing a test failure, as described in Editing Data Validation CSV Files.

Creating a Unit Test Manually

Advanced users can create unit test configurations without a running application. The following procedure creates a new, empty unit test configuration file in which you must specify all components of the test manually.

With no application running, invoke File → New → StreamBase Test. This brings up the New StreamBase Test dialog. (The Record a test checkbox is greyed out in these circumstances.)
Select a folder in your workspace, and enter a base name for the test file. You can optionally use Advanced to link to an existing unit test file instead of creating a new one. Click Finish.
A dialog pops up offering to add the StreamBase Test Support Library to this project's Java Build Path. Click Yes, then OK to perform this action. See Test Support Libray for more information.
The Test Editor opens showing default and empty settings for the new test.
Edit the test configuration, specifying the feed simulation files and data validation CSV files to use, as described in Using the StreamBase Test Editor.
Save the configuration file.
Copy or create one or more data validation CSV files for the test to validate against.

You can also create a unit test configuration by copying an existing unit test file. Use copy and paste in the Package Explorer to select and copy an existing, working unit test file with sbtest extension. Then double-click the copied file in Package Explorer, which opens the Test Editor. Edit the copied test file as described in Using the StreamBase Test Editor.

Editing Data Validation CSV Files

The data validation CSV files used by the unit test framework for output streams have field names on the first row, which must match stream field names in the schema of the target stream. Validation CSV files for input streams do not need to have a header row. Subsequent column values must be convertible from type string to the target StreamBase data type. The only exception is for CSV files representing output stream values where the field value matches the Ignore Field Character defined in the test configuration. If a value in an incoming field matches the Ignore Field Character, then any value is accepted for that field on the output stream, does not cause the test to fail. If a specific value is defined, then that specific value must appear in the output stream tuple when it is expected.

If a test is created by recording a running application, timestamp fields in the recorded output are usually not correct for subsequent runs of the test. Instead, replace the timestamp value in the output CSV file with the Ignore Field Character. This prevents tests from failing because of mismatched timestamps.

StreamBase Test Support Library

To run a unit test in Studio, the StreamBase Test Support Library must be in the Java Build Path of the project that contains the test. This library does not have to be in the path when recording a test, or when editing a test configuration. This library is not required when running a test with the command-line utility, sbtest, because sbtest already includes the library.

When you click Finish in the New StreamBase Test dialog, another dialog offers to add the Test Support library for you under the following circumstances:

No application is running.
An application is running, but you did not check the Record a test now checkbox. In this case, Studio prompts for permission to shut down the currently running application in order to add the library.

If you create a new unit test configuration while an application is running, and you check the Record a test now checkbox, the New StreamBase Test dialog does not offer to add the Test Support Library for you. (Doing so would require stopping the application you are about to record.) In this case, you must add the Test Support Library manually after recording the test.

Follow these steps to add the Test Support Library to a project:

Select the project in the Package Explorer.
Right-click and select Properties.
In the Properties dialog, select Java Build Path.
Click the Libraries tab.
Click Add Library.
Select StreamBase Test Support and click Next.
On the next page, lick Finish.
Click OK.

Adding the Test Support Library automatically adds the StreamBase Client API Library as well.

Running a Test in Studio

Running a test requires a test launch configuration. Studio provides a default test launch configuration that lets you test right away without stopping to configure a launch configuration. You can also create and edit customized test launch configuration the same way as run and debug launch configurations. See Editing Launch Configurations for details.

Your test configuration's Target Application section uses either a Specified Application File or Specified StreamBase URI to locate the application to test. In this case, you can run the test immediately, using the default test launch configuration.
Your test configuration's Target Application section specifies Default. In this case, you must start the application to test before running the test, or you must use an application-specific test launch configuration.

Running a Test with Default Launch Configuration

To run a unit test using Studio's default launch configuration, follow these steps:

Open the EventFlow or StreamSQL file for the application under test.
Open the test configuration file that tests this application. This places both editor sessions side by side.
Run the application by making the application's editor session active, then clicking the Run button. This opens the SB Test/Debug perspective and starts the application.
Now make the Test Editor session active and click the Run button again. The Run button always runs the file in the currently selected editor session. Thus, this time, the test is run, using the default test launch configuration.
The test runs. Remember that a running test takes about as much time as it took to record the test, so give the test time to finish.
When the test finishes, look for results in the JUnit view. A long green bar in this view denotes test success. A red bar is accompanied by information in the Failure Trace pane to help you track down the cause of the test failure.

Running a Test with a Saved Launch Configuration

To run a test with a test-specific launch configuration, follow these steps:

Save a unit test configuration that specifies a Specified Application File or Specified StreamBase URI to locate the application to test.
Open Run → Open Run Dialog.
Select StreamBase Test and click the New Configuration button (or use the Duplicate button to copy an existing configuration).
Studio automatically selects the configuration temporarily named New_configuration.
Give the test launch configuration a name.
Use Browse to locate the unit test file you saved in step 1.
Studio reads the name of the Target Application from the unit test file.
Click Apply, then Run.

Once a launch configuration has run, Studio places its name in the Run → Run History menu. To run this configuration again, select it from the this menu.

When you run a saved launch configuration, Studio starts the application named in the unit test configuration, then runs the test. You do not need to start the application first.

Interpreting the Results of Studio Tests

Test progress and results are shown in the JUnit view. The test begins automatically and takes about as long to complete as the test recording took. On completion, look for a green bar to indicate success or a red bar to indicate test failure.

If the test failed, the Failure Trace pane shows information to help you track down the cause of the failure.

Studio inherits the JUnit view from the Eclipse Java Development Tools feature. Many of the controls in this view apply to Java unit testing, but not to StreamBase unit testing. For instructions on using conrols in the JUnit view, search for "JUnit" in the Java Development User Guide in the Studio help system.

Running a Test from the Command Line

StreamBase provides the utility sbtest to run unit tests at the command line. Use this utility in a terminal window on UNIX or in a StreamBase Command Prompt on Windows. Use steps like the following:

Save a unit test configuration that specifies a Specified Application File or Specified StreamBase URI to locate the application to test.
Navigate to the workspace directory that contains the project with your test file.
Run the test with a command like the following:
```
sbtest test-name.sbtest
```
The test reports a single period to the console to indicates it is running.
On completion, the test returns 0 for success and non-zero for failure. This allows runs of sbtest to be included in test scripts appropriate for the operating system.

On completion, sbtest reports the time taken to execute the test in seconds and milliseconds and either OK for successful completion or There was 1 error followed by the stack trace and a summary count of the total number of tests, failures, and errors.