StreamBase 3.7 Release History

Documentation updates: some pages in the Adapters Guide, although present in the documentation kit, had incomplete navigation links, and were thereby difficult to locate. This was corrected.

The StreamBase Wombat MAMA Input Adapter was extended with the following features. For the full implementation details on these features, see Wombat MAMA Input Adapter in the Adapters Guide.

A new tag for the StreamBase External Adapter for Microsoft Excel: the LOCALFILTER tag performs client-side routing of tuples to spreadsheet cells and should be used instead of the FILTER tag when no real server-side filtering is being done. For more information, see the External Adapter for Microsoft Excel topic in the Adapters Guide.

The following changes were made to the CSV File Reader input adapter:

See the CSV File Reader Input Adapter topic in the Adapters Guide for details about these features.

Previously, the StreamBase-to-StreamBase input adapter did not provide a way to specify port numbers. You can now use adapter properties to specify both upstream and downstream ports, making deployment and maintenance more flexible. See the StreamBase-to-StreamBase Input Adapter topic in the Adapters Guide for details.

This release includes a new embedded input adapter for Opentick data sources. This new adapter is described in the Adapters Guide, and a new sample in streambase-install-dir/sample/adapter/embedded/opentick shows how to use the adapter.

Support was added in the JDBC Operator to optionally output null tuples when a database query returns an empty result set. For 3.7.7, this option is implemented in EventFlow diagrams in StreamBase Studio, but not in StreamSQL or the StreamSQL Converter.

The new feature is seen in the Property view of a Query Table icon that's associated with a JDBC Table icon. In the Query Settings tab of the Property view, there is a new Options line with a checkbox labeled Output null tuple on empty Result Set.

This option only has meaning when the SQL statement field contains a SELECT query. When this checkbox is cleared (the default setting), empty results from the SELECT are discarded. When checked, empty results from the SELECT generate a tuple with the same schema as the output of the query operator, with null values for each field.

Support was added in the JDBC Operator to control behavior when a JDBC connection between the StreamBase Server and the database server is lost. Three new data-source configuration parameters implement this control; in their absence, the default behavior for a connection loss is for the StreamBase Server to display an error message and exit with error status.

The new parameters are:

They are used in your project's sbd.sbconf file as shown in the following example:

<streambase-configuration>
          <data-sources>
          <data-source name="msql-dev1" type="jdbc">
          <uri value="jdbc:mysql://dbserver:3306/qa?user=username&password=secret"/>
          <param name="jdbc-reconnect-regexp" value="Communications link failure"/>
          <param name="jdbc-reconnect-regexp" value="Connection reset"/>
          <param name="jdbc-reconnect-attempts" value="5"/>
          <param name="jdbc-reconnect-sleep" value="250"/> 
          </data-source>
          </data-sources>
          </streambase-configuration>

If a connection is lost between the StreamBase Server and the database server, the next attempted JDBC operation returns an error. There are no standard error codes (or SQLSTATE) for communications failures, so to detect these errors, we must parse the text of the database-specific error messages. Use the jdbc-reconnect-regexp parameter to specify a regular expression to be compared to any received error messages. If one or more specified string is matched in the text of any error message, the StreamBase Server assumes a communications failure has occurred, and attempts to open a new connection to the database server.

You can specify more than one jdbc-reconnect-regexp parameter to specify several error message strings to be considered communications failures. Each specified string is compared in order to the received error message text.

The jdbc-reconnect-attempts parameter controls the number of reconnection attempts a StreamBase Server will attempt. If the connection to the database server is lost, the configuration example above attempts to establish a connection to the databse server five times. If five connection attempts fail, an error message is printed and the StreamBase Server exits with error status. If a connection is reestablished to the databse server, and then the connection is lost some time thereafter, the reconnection count is reset so that five attempts to reconnect will be made again.

A jdbc-reconnect-attempts value of -1 specifies that the server is to continue reconnection attempts without limits. A value of zero disables reconnection attempts.

If the jdbc-reconnect-sleep parameter is specified, reconnection attempts are delayed by the specified number of milliseconds. If no jdbc-reconnect-sleep parameter is used, there will be no delay between reconnection attempts.

Just like the initial connection to the database, reconnection attempts use the jdbc-timeout parameter to limit how long the StreamBase Server waits to establish a connection. The default value for jdbc-timeout is 15000 milliseconds (15 seconds).

Support was added for nested tibrv messages in the StreamBase Adapter for TIBCO Rendevous. This feature is described in the TIBCO Rendezvous External Adapter topic.

Documentation Update. In the StreamBase API Guide, the Creating C++ Clients page describes the requirements for deploying C++ clients on non-StreamBase systems. The following paragraph supplements the information on that page.

Microsoft occasionally issues service packs for Visual Studio .NET. If you build a C++ client with a particular service pack version of Visual Studio, you must install the correspnding version of the Microsoft Visual C++ Redistributable package on your non-StreamBase deployment system. For example, if you build a C++ client with Visual Studio .NET 2005 SP1 (VC 8.0 SP1), you must install the SP1 version of the Redistributable package for VC 8.0.

The Flex adapter serves one or more internal StreamBase streams to Flex client applications for visualization within the Adobe Flash Player.

New support for 64-bit StreamBase on Solaris 10 and Solaris 8. For more information, see the Supported Configurations topic in the Installation Guide.

New support for UTF-16 character data in StreamBase applications. UTF-16 is the 16-bit Unicode Transformation Format. It is a variable-length character encoding for Unicode, capable of encoding the entire Unicode set. The 3.7.1 release adds a new sample, utf16. It includes several short documents that contain non-Latin characters. We chose Hebrew characters, as an example. The StreamBase sample application uses custom java functions that manipulate and search for the Hebrew characters.

For details, see the utf-16_README.html file that is installed in streambase-install-dir/sample/utf-16, or the UTF-16 Sample topic in the Samples Guide.

You can use a source file management product, such as Subversion, to manage the versioning of files in a StreamBase Studio project. As a convenience, starting in the 3.7.1 release, a StreamBase Studio Project view will not display files or directories that start with a period, such as .svn subdirectories, or ones marked as hidden in the file's properties on Windows. More information about this feature will be provided in a technical article on the StreamBase Developer Zone website.

The StreamBase 3.7.0 release introduced the ability to dequeue data from intermediate streams, for debugging purposes. Information about this feature was provided on the StreamBase Developer Zone website. To use the feature, you had to start the StreamBase Server process (sbd) with the -d option. In 3.7.1, you can continue to do that; or you can start the server in its standard (non-debugging) mode, provided that you set the following JVM argument to true:

-Dstreambase.codegen.intermediate-stream-dequeue=true

The default for this JVM argument is false. JVM arguments can be configured in the server's sbd.sbconf.

For details, see Debugging Feature: Dequeuing Data from Intermediate Streams, a new topic in the API Guide.

This 3.7.1 release also fixed a number of product limitations. For details, see the Resolved Limitations section of the Release Notes.

You can define parameters for an application and use them in operator expressions. The parameters can take default values that can be changed in a module reference. This makes module references more powerful, enabling you to reuse modules with different parameter bindings. For more information, see Using Application Modules in the Authoring Guide, and the CREATE PARAMETERS Statement in the StreamSQL Guide.

A new data construct, Materialized Window, has been added for EventFlow applications; for StreamSQL applications, the CREATE MATERIALIZED WINDOW statement is added. A materialized window is a managed view of tuples passing through an input stream. The view can be based on a fixed number of tuples, a time interval, or a field value. It can also be partitioned into multiple windows. You can then directly query the data using one or more (read-only) Query operators. For more information, see Using the Materialized Window Data Construct to View Data in the Authoring Guide, and the CREATE MATERIALIZED WINDOW Statement topic in the StreamSQL Guide.

Query operator enhancements:

For more information, see Using the Query Operator

StreamBase containers are the new building blocks for organizing and connecting multiple StreamBase applications. There is one StreamBase application per container, and each container name serves as a handle - a unique qualifier - for that application's resources. Multiple containers/applications can be run in a single StreamBase Server (sbd) process, and you can share selected resources between applications.

For example, you can connect the output of one StreamBase container/application to the input(s) or output(s) of one or more other StreamBase containers/applications, all running within the same sbd. New container options exist in the sbd.sbconf configuration file, in the API, and on StreamBase commands such as sbadmin addContainer.

Note: In pre-3.7 StreamBase releases, the sbd.sbconf server configuration file used an <application-file> element. It is still supported. However, if you use containers, edit your sbd.sbconf files to use the <application> element that was introduced with the containers feature. It includes elements to specify one or more containers. For example:

<runtime>
          <application file="app.ssql" container="A"/>
          <application file="app.ssql" container="B">
          <container-connection dest="B.InputStream1" source="A.OutputStream1"/>
          <container-connection dest="B.InputStream2" source="A.OutputStream2"/>
          </application>
          </runtime>
        

For details, start in this topic: Using Containers to Organize and Connect Multiple Applications in the Administration Guide.

There are changes to supported configurations and related areas:

StreamBase Feed Simulations are now functionally equivalent whether they are run in StreamBase Studio or by using the sbfeedsim command with a customized .sbfs configuration file. A feed simulation is no longer tied to a particular application, and a feed simulation can send test data to multiple streams. This release also includes enhancements to the user interfaces for the Feed Simulation Editor and the Feed Simulations view. The editor includes a graphical view and a source XML view of each feed simulation's .sbfs configuration file.

The XML elements used in .sbfs files have changed. Pre-3.7 feed simulations can be upgraded automatically in StreamBase 3.7 by saving the file in StreamBase Studio, or by using the sbfeedsim --convert command. The group-by and switch elements from legacy (pre-3.7) .sbfs configurations are not supported by the new Feed Simulator. However, for compatibility purposes, you have the option of using the sbfeedsim-old command with legacy feed simulations. For details, start in Using the Feed Simulation Editor, a topic in the Test/Debug Guide.

This release introduced a new layout for the Eclipse platform system that is used by StreamBase Studio. This change is incompatible with any prior saved StreamBase Studio configuration. If you see an error that StreamBase Studio (3.7.x) cannot start, go to the directory where the existing configuration data resides, and rename that directory (for example, to StreamBaseConfigurationBackup_3.5.7). You should then be able to open StreamBase Studio.

The default locations for the 3.x configuration directories are:

On Windows:         
          C:\Documents and Settings\username\StreamBase Configuration
          
On Linux:
          
          ~/.streambase/sbstudio-configuration

The StreamBase concurrency feature has been extended. In the Properties view for most components, the option to "Run this component in a separate thread" has been moved from the General tab to a new Concurrency tab. In addition, new "Data Parallelism" options can be selected. If appropriate for your application, you can specify that multiple instances of an operator or module should be created. At runtime, each tuple is dispatched to a particular instance of the component. When configured in this way, all tuples with the same value in a field identified as the Key Expression will be sent to the same operator or module instance. Note that there are important caveats to consider. For details, see the StreamBase Execution Order, Concurrency Options, and Data Parallelism topic in the Authoring Guide.

Aggregate expressions are more flexible. Previously, aggregate expressions could only be used in Aggregate operators and could only contain a single aggregate function.

For more information, see these topics: Using the Query Operator to Use or Modify Shared Data, Using the Aggregate Operator to Apply Aggregating Functions, and StreamBase Expression Language and Functions.

A number of StreamSQL statements have been added or updated since the most recent 3.5.x release. Additions include the DECLARE statement for dynamic variables; CREATE PARAMETERS to add parameters to modules; and CREATE MATERIALIZED WINDOW to create a managed view of tuples passing on a stream. Updates include new options (related to the additions and new product features) on APPLY, CREATE INDEX, and SELECT (including the WHERE and HAVING clauses). For details, see the StreamSQL Guide.

The Java API for creating custom embedded adapters has changed. The com.streambase.sb.adapter.Adapter class is replaced by two classes:

com.streambase.sb.adapter.InputAdapter

com.streambase.sb.adapter.OutputAdapter

In any existing embedded adapter source, you must change the class statement from extends Adapter implements InputAdapter to extends InputAdapter, and change extends Adapter implements OutputAdapter to extends OutputAdapter. Then recompile your adapters with this release of StreamBase.

In StreamBase Studio, if you open an application that contains an older embedded adapter's JAR, an error will be reported. Close the IDE, update the class(es), recompile and redeploy the JAR.

For details, see Using the Embedded Adapter API in the API Guide. Also see the Adapters Guide.

Embedded adapters start, pause, resume and shutdown along with the StreamBase application that contains them. It is also possible to suspend and resume embedded adapters independently of their StreamBase application. You can also query the status of embedded adapters. For details, see Using the Embedded Adapter API in the API Guide. In particular, see the section titled "Embedded Adapters and Life Cycle Events."

Similarly, Java operators start, pause, resume and shutdown along with the StreamBase application that contains them. It is also possible to suspend and resume Java operators independently of their StreamBase application. You can also query the status of Java operators. For details, see Using the Java Operator API in the API Guide. In particular, see the section titled "Java Operators and Life Cycle Events."

Each Java Operator or Embedded Adapter changes state along with the server process as a whole. The server waits for each Operator or Adapter to change state before it completes its state change. This release adds the operator-state-change-timeout-ms parameter to the StreamBase Server configuration file, sbd.sbconf. It can be used in the server section of the configuration. The value of operator-state-change-timeout-ms is the amount of time (in milliseconds) the server will wait before timing out the Operator or Adapter. For details, see the StreamBase Server Configuration XML topic in the Reference Guide.

This release introduces some new APIs in both Java and C++. The major change is that the enqueue and dequeue client APIs now have methods that take a StreamProperties object instead of a string stream-name. The new APIs may be faster. The pre-3.7 APIs (using string stream-name) are deprecated. They remain for backwards compatibility. However we recommend that you update your clients.

The StreamBase Client APIs have been extended to support containers.

The StreamProperties (in C++ & Java) class is now much more important because it now has the container name. Here are a few of the important methods on StreamProperties:

There are new APIs for enqueuing and dequeuing using a StreamProperties instead of a stream name. Again, these APIs are the preferred way to enqueue/dequeue to the server as the StreamProperties object contains the container name.

For details, see the StreamBase Java API reference, and StreamBase C++ API reference.

In the Java API, a new DataType factory method has been created, createField(String name, int size). While creating a schema, you can use createField() to create a field of a given DataType and fixed size. See the StreamBase Java API reference for details.

The Palette view has changed in StreamBase Studio, and a new docked Palette is available. If you upgrade, the first time you open StreamBase Studio, you will see a message informing you that the Palette view was not found. You will also notice that no Palette view is displayed at first.

Now, the Palette view is displayed by default only when you open an Application Diagram. In addition, the Palette view now has a special docking function: If an EventFlow application is open and you close the Palette view, a docked Palette appears in the Application Editor. Note that if you upgrade and you previously had the Palette view open, the docked Palette initially appears within each Application Diagram that you open.

You can restore the Palette view at any time by clicking Window → Show View in the StreamBase Studio menu, and finding the Palette view there. The docked Palette is removed and the Palette view is displayed instead. For more information see the help topic, Palette View.

A new view has been added to StreamBase Studio. The Error Log view captures all the warnings and errors logged by plug-ins. The underlying .log file stored in the .metadata subdirectory of the workspace. The Error Log view is available under Window → Show View → Error Log. For details, see the topic, Error Log View.

You can now dequeue tuples from an intermediate stream. That is, from a stream that exists between two connected components, without using an Output Stream. This feature allows you to look at the intermediate processing results from a StreamBase client, before the tuples have been processed to completion in the application. Information about using intermediate streams will be provided in a StreamBase Developer Zone article, and in an update to the StreamBase 3.7.x documentation.

A new command, jsbmonitor, opens a graphical version of the StreamBase Monitor. It contains the same performance data as the character-based sbmonitor display, with the addition of dynamic CPU and memory usage charts. In StreamBase Studio, the graphical StreamBase Monitor is opened when you click Window → Launch Monitor. See the jsbmonitor topic in the Reference Guide for details.

You can configure your StreamBase application to use one of two implementations of the now() function. In the <runtime> section of the server configuration file, sbd.sbconf, we added this default parameter:

<param name="now-implementation" value="system"/>

Details about the now() implementations and configuration options are available in StreamBase Server Configuration XML, a topic in the Reference Guide.

Although not part of the StreamBase kit, we want to make sure you are aware of several resources on the StreamBase Developer Zone. It includes an article that explains how to set up the StreamBase Java Toolkit for Eclipse. This is a Java plug-in to the Eclipse development environment that lets you quickly create templates for StreamBase Java clients, custom embedded adapters, custom functions, and custom Java operators. You can then modify the "// TODO" commented sections in the .java template sources, saving you time as you learn the StreamBase Java APIs.

See Getting Started with the StreamBase Java Toolkit for Eclipse on the Developer Zone. You do not download the StreamBase Java Toolkit for Eclipse from streambase.com. Instead you go into the Eclipse IDE and specify the StreamBase update site URL in a dialog. Details are in the article noted above.

After you set up Eclipse and get the plug-in, there are many related articles on the StreamBase "DevZone" about using the generated templates. See the list of articles in the StreamBase Developer Zone Library.

A number of product limitations are fixed. For more information, see the Resolved Limitations section of the Release Notes.