Release Notes for StreamBase 3.x

This release adds an additional way to control the size of the internal queue used between a Java StreamBaseClient's main thread and the dequeuing thread. Previously, you could control the number of batches of tuples in the queue using the Java property streambase.max-queue-size, which has a default size of 500 batches. However, if a client gets behind in dequeuing, the server starts sending larger batches, which requires more memory usage by the client. To remedy this situation, there is a new Java property in the StreamBaseClient API: streambase.client.max-tuples-in-queue. The default value is 10000 tuples. This property limits the absolute number of tuples in the queue, which promotes better client behavior. This same Java property also affects the StreamBase-to-StreamBase input adapter.

There is also a new Java property for the server, streambase,sbd.max-tuples-in-dequeue-packet. This property limits the number of packets the server includes in any one dequeue packet. The default value is 10000, which matches the default of the new client property described above. Administrators can set both server and client with matching values to make sure client dequeues are not overwhelmed.

In prior releases, under certain unusual circumstances, an update operator could corrupt the primary index of a table, leading to the appearance of duplicate key values, and potential data loss. This error was identified and fixed.

The rare circumstances that triggered the condition were: an update operator was modifying rows in a query table with a primary key of type string, which was downstream from, but in the same parallel region as, a read operator connected to the same table, which returned three or more rows.

Errors in a third party Java library were causing a Null Pointer Exception for users of the StreamBase Reuters Subscribing Adapter accessing certain classes of market data. This release of StreamBase incorporates a fixed version of that library, which resolves the errors.

<param name="jvm-args" value="-Xms64m -Xmx256m -Dstreambase.codegen.intermediate-stream-dequeue = true"/>

In the previous release, setting the switch shown did not work properly, and you could only see intermediate streams in debug mode. The problem was fixed.

=RTD("StreamBase.RTD", , "STREAM:myContainer.All_Transactions", ...)

Note that is usually better NOT to specify container names in Excel cells. That is,

=RTD("StreamBase.RTD", , "STREAM:All_Transactions" , ...)

is generally more maintainable than:

=RTD("StreamBase.RTD",,"STREAM:myContainer.All_Transactions", ...)

In previous releases, copying the operators in the loop would cause the schema of the looping stream to be shared by the operators. As a result, a change to either looping stream would take effect in both. In this release, such a copy now creates a separate schema for the looping stream. The schemas are initially identical, but you can edit each independently of the other.

Instead, the adapter monitors the failed StreamBase server, and when it becomes available, reconnects in the background. The current state of each StreamBase server is available through the new META:SERVER_STATUS tag. The new meta.xls sample spreadsheet shows how this works.

<param name="uri" value="sb://foo:10000"/>

and a spreadsheet cell contains:

=RTD("StreamBase.RTD",,"URIREF:", ...)

then this cell will reference the StreamBase server running at URI sb://foo:10000.

Note that if you want to use an expression that intentionally sets the result to null, you must use one of the data-type-specific null literals, as shown in the following list:

For more information, see Using Nulls in StreamBase Applications in the Authoring Guide.

streambase.codegen.javac-Xmx

Use this flag the same way as the java -Xmx command-line option:

Specify the maximum size, in bytes, of the memory allocation pool. This value must a multiple of 1024. Append the letter k or K to indicate kilobytes, or m or M to indicate megabytes. Examples:

-Xmx83886080
-Xmx81920k
-Xmx80m

If the input field in question were of string type, the StreamBase server could experience a Null Pointer Exception. If the input field were another data type, then the output value could be 0 instead of the expected null data type.

package com.streambase.sb.expr.test; 
  import com.streambase.sb.operator.AggregateWindow;
  /** 
  * Class that implements a custom aggregate function that 
  concatenates strings together. 
  */ 
  public class StringConcatAggregate extends AggregateWindow {
  StringBuffer buf = new StringBuffer();
  public static int getResultLength() {
    return 100;
  } 
  public void init() {
    buf.setLength(0); 
  } 
  public void accumulate(String s) {
    buf.append(s);
  } 
  public String calculate() {
    return buf.toString(); 
  }
  } 

{"strptime('2006-Feb-10 11:01:03', '%Y-%b-%d %H:%M:%S')", SOME_TIMESTAMP}
{"strptime('2006-fEB-10 11:01:03', '%Y-%B-%d %H:%M:%S')", SOME_TIMESTAMP}

Note that month letter codes are limited to three characters: long month names like February are not supported.