29West LBM Publishing Output Adapter

Introduction

The StreamBase 29West LBM Publishing Output Adapter allows a StreamBase application to publish LBM messages to one or more LBM topics. The adapter is embedded in the StreamBase application and has an input port that receives tuples to be published as LBM messages and an output port that emits status tuples. Each tuple enqueued to the adapter's input port causes a single LBM message to be published.

The adapter supports all StreamBase data types, including tuples and lists, though lists cannot contain elements of type list. Tuple fields of type tuple can be used to send nested LBM messages and LBM DECIMAL fields. The schema of the adapter's input port is determined by the output port of the upstream operator.

The adapter is configured through several properties set in the adapter's Properties view in StreamBase Studio. The properties include an optional configuration file to configure the LBM library, the input port field containing the topic to publish to, and the default topic value to use if the topic field is null.

Note

The 29West LBM Publishing Output adapter can consume significant system memory, depending on the data cache options you enable in the LBM configuration file. Memory allocated by the adapter is native memory that is allocated independent of (and competing for memory resources with) the JVM memory consumed by StreamBase Server. See Java VM Memory Settings for more on JVM versus native memory usage.

29West LBM Middleware Dependencies

This adapter requires access to the JAR file that implements the 29West LBM Java API on your system, and any files referenced by that JAR file. The StreamBase installation kit includes version 4.0 of the LBM_4.1.1_jdk1.5.0_12.jar JAR file. If your site's LBM implementation requires a newer version of that file, copy the file to $STREAMBASE_HOME/lib/ext.

The LBM jar file, in turn, relies on various native library files, which are supplied as part of your LBM installation, and are not shipped with StreamBase. Make sure these files are locatable by the LBM JAR file on the PATH environment variable.

If you get an error message whose text refers to can't load lbmj: java.lang.UnsatisfiedLinkError: no lbmj in java.library.path, make sure the directory containing the LBM native libraries is on the path.

The LBM API implementation described in this section is a product of a third party, and its specifications and file names are subject to change by 29West. See your LBM documentation for the latest information.

Adapter Properties

Property	Description
LBM Configuration File	The name of an optional LBM configuration file used to override the default settings used by the LBM library. Configuration files can be shared between one or more LBM subscribing and publishing adapter instances. To support such sharing, configuration files can have named sections, which are referenced by the `LBM Configuration File Section` property below. Named sections are identified by placing a section name inside square brackets within a comment: `#[MySection]` Lines that precede the first section name are considered part of the unnamed (default) section. Lines that follow a section name are considered part of that section. Section names must be unique within and across all referenced configuration files; a runtime error is flagged when a duplicate section name is encountered. Section names can include alphanumeric characters, digits, underscores, dots, and embedded spaces. They must start with an upper- or lower-case letter or an underscore.
LBM Configuration File Section	The name of the LBM configuration file section from which to retrieve the parameters for configuring the LBMContext and LBMSource objects. If unspecified, parameters from the unnamed section are used. For enhanced performance, all LBM adapter instances that specify the same configuration file and section name share an LBMContext object.
Default Topic	Contains the topic value under which to publish a LBM message if no topic field exists in the input schema, or its value is null in a specific input tuple.
Topic Field Name	The name of the input port field containing the topic value under which to publish the LBM message. This field is optional if a Default Topic value is specified, which is used if no topic field exists, or its value is null in a specific input tuple.
Idle LBM Source Timeout	The time, in seconds, to wait before an LBM source for an idle topic is discarded, or zero (the default) to disable the expiration of idle sources. An LBM source is created on demand when publishing to each new topic. When publishing to an unbounded number of topics, this property should be set to a non-zero value to limit the total number of LBM sources — each of which consumes finite resources — maintained by the adapter.
Log Level	Controls the level of verbosity the adapter uses to issue informational traces to the console. This setting is independent of the containing application's overall log level. Available values, in increasing order of verbosity, are: OFF, ERROR, WARN, INFO, DEBUG, TRACE, and ALL.
Enable Pre-defined Messages	If disabled (the default), the LBM publish adapter sends self-describing messages, in which metadata describing the structure of the message is sent in each messsage. The LBM subscribe adapter uses this metadata to parse incoming messages and populate tuples. If enabled, the LBM publish adapter sends pre-defined messages, in which the structure of the message is derived from the schema of the adapter's publish input port. The LBM subscribe adapter uses the schema of its message output port to parse incoming messages and populate tuples.
Schema Matching Policy	If `Strict`, the LBM publish and subscribe adapters must use identical schemas -- ones that contain the same fields in the same order -- to successfully exchange messages. If `BestVersion`, the LBM publish and subscribe adapters must use compatible schemas to successfully exchange messages. Two schemas are compatible if their IDs match and the larger schema can be created simply by adding fields to the end of the smaller one.
Definition ID	An ID that must be configured identically in the LBM publish and subscribe adapters when using BestVersion schema matching.
Major Version	The major version of the message definition when using BestVersion schema matching.
Minor Version	The minor version of the message definition when using BestVersion schema matching.
Publish Type Overrides	A table used to override the default LBM type used to publish specific fields when using self-describing messages. Each table row contains a StreamBase field name and the LBM type used to publish that field. StreamBase fields not present in the override table are published using the default LBM type specified in the table below. See the Publish Type Overrides Table below for the correspondence between StreamBase types, default LBM types, and valid overrides.
Use Fully-Qualified StreamBase Field Names	When enabled, nested StreamBase fields are specified in the Publish Type Overrides table using fully-qualified names. Otherwise, simple field names relative to the enclosing tuple are used. Use fully-qualified names when a top-level and nested field with the same relative name need to be published using different LBM types.

The following table shows, for each StreamBase type, the default LBM type, and the valid override values that can be used to publish the StreamBase field using a non-default LBM type.

Publish Type Overrides Table

StreamBase Type	Default LBM Type	Valid Overrides
blob	BLOB	—
bool	BOOLEAN	—
double	DOUBLE	FLOAT
int	INT32	INT8, INT16, UINT8, UINT16
long	INT64	UINT32, UINT64
string	STRING	UNICODE
timestamp	TIMESTAMP	—
tuple	MESSAGE	DECIMAL
list<blob>	BLOB ARRAY	—
list<bool>	BOOLEAN ARRAY	—
list<double>	DOUBLE ARRAY	FLOAT ARRAY
list<int>	INT32 ARRAY	INT8 ARRAY, INT16 ARRAY, UINT8 ARRAY, UINT16 ARRAY
list<long>	INT64 ARRAY	UINT32 ARRAY, UINT64 ARRAY
list<string>	STRING ARRAY	UNICODE ARRAY
list<tuple>	MESSAGE ARRAY	DECIMAL ARRAY

Using the Adapter in a StreamBase Application

This section discusses how to use the 29West LBM Publishing Output Adapter in a StreamBase application. As shown in the diagram below depicting one of the adapter's sample applications, the publishing adapter has one input and one output port to communicate with the surrounding application.

The 29West LBM Publishing Output Adapter's ports are used as follows:

Publish: Tuples enqueued on this input port cause the adapter to publish LBM messages, one per tuple. The port's schema is derived from the output port of the upstream operator and can contain all StreamBase data types, including lists and nested tuples to provide hierarchy in LBM messages. Tuple fields are mapped to LBM message fields by name. List fields can contain all StreamBase element types, except lists. Fields containing null in the tuple are discarded; no corresponding field is placed in the LBM message.

DECIMAL fields are published using a tuple field of type tuple containing two sub-fields: a long for the mantissa and and an int for the exponent. The two sub-fields may appear in either order in the nested tuple field. If a field to be published as a DECIMAL contains null values in a sub-field, a warning is emitted, the tuple field is discarded, no DECIMAL field is placed in the LBM message. If both sub-fields are null, the tuple field is discarded silently.
PublishStatus: This output port emits status, information, and error tuples. The PublishStatus port has the following schema:
- type, string: Contains one of the following values describing the type of event that occurred:
  - Transport
  - Process Tuple
  - Process LBM Message
  - Subscription
  - Suspend/Resume
- object, string: the name of the object associated with the event, such as the name of the tuple field that couldn't be converted to an LBM field or the name of the adapter that was suspended or resumed.
- action, string: the action associated with the event, such as a conversion error or adapter suspended or resumed.
- message, string: A human-readable message string.

Add an instance of the adapter to a new StreamBase application as follows:

In StreamBase Studio, create a project, and create an EventFlow application file to host the adapter.
Drag the Output Adapter icon from the Operators & Adapters drawer in the Palette view to the canvas. Select the 29West LBM Publishing Output Adapter from the list and click OK.
Connect an input and output stream to the adapter's input and output ports, respectively. Configure the schema of input stream with the fields required in the LBM messages to be published. Include a Topic field of type string if the default value for the Topic Field Name property is to be used.
Select or double-click the adapter icon, and in the Properties view, click Adapter Settings and fill in any desired properties. Either a Topic Field Name or Default Topic must be specified.

Typechecking and Error Handling

The 29West LBM Publishing Output adapter uses typecheck messages to help you configure the adapter in your StreamBase application. In particular, the adapter generates typecheck messages when:

The schema of the input port contains one or more list fields with elements of type list.
An invalid LBM configuration file is specified.
An invalid topic field name within the input port schema is specified.
The topic name within the input port schema is not of type string.
Neither a topic field not a default topic is specified.
The datatypes of one or more fields in the input schema are incompatible with the specified LBM override types.

The adapter generates messages on the status port during runtime under various conditions, including:

The topic field of a publish tuple is null and no default topic has been configured.
An error occurs populating an LBM field from a tuple field.
An error is returned by the LBM library when the adapter attempts to publish an LBM message.
The adapter is suspended or resumed.

Suspend and Resume Behavior

When suspended, the 29West LBM Publishing Output Adapter stops processing input tuples and emitting output tuples.

When resumed, the adapter begins processing input tuples and emitting output tuples once again.