Container Overview

Containers enable a number of useful features. You can:

Dynamically add or remove StreamBase applications from the running StreamBase Server process.
Use containers to run multiple applications in a single server process, without necessarily connecting the applications' resources.
Connect the output of one StreamBase application to the inputs or outputs of one or more other StreamBase applications, all running within the same server.
Establish container connections even for servers running on separate physical servers.
Dynamically modify an existing container in a running application, to add or change a container connection, or to start, stop, or pause enqueuing or dequeuing.
Specify containers and container connections from the command line, or through an sbd.sbconf server configuration file, or with the StreamBaseAdminClient API.

The following is an example of command-line container commands:

sbadmin addContainer c1 newapp.sbapp
sbadmin addContainer c2 myapp-same-schema.sbapp c2.InputStream=c1.OutputStream

This example defines two containers and associates each with an application. The last expression specifies that the output of the c1 container is to be consumed by the input stream of the c2 container. Both containers are running in the same sbd process.

Each container has its own thread of execution. Containers behave much like concurrent (parallel) operators or modules. On a multi-CPU machine, one container does not block the execution of another container.

The following set of conceptual diagrams illustrates the ideas behind containers. The first diagram illustrates a configuration running multiple unrelated StreamBase applications in a single sbd process:

In the next diagram, we added two reader applications (B and C) to a running sbd process that was already hosting application A:

In the next diagram, we added two writer applications (B and C) to a running sbd process that was already hosting application A. Then we added application D to write its output stream to the output stream of application A.

Notes:

In the diagrams above, the StreamBase applications are shown for simplicity as having the same components. This is for illustration only; applications in containers are not required to have the same set of components. However, the streams used between applications must have the same schema.
If you remove a container and application from a running sbd server process, StreamBase Systems recommends that you remove each one in the reverse order that it was added.
You cannot share a Query Table between applications. Each application must use its own Query Table.

Containers Specified in Configuration Files

In the configuration file for a StreamBase Server process, the runtime section provides the <application> element. There can be any number of <application> elements. The application element has two attributes:

file - the application's file name, such as myapp1.ssql or myapp2.sbapp.
container - the container's name

The application element can have any number of sub-elements named container-connection. These sub-elements have dest (destination) and source attributes. For example:

<runtime>
    <application file="app.ssql" container="A"/>
    <application file="app.ssql" container="B">
       <container-connection dest="B.InputStream1" source="A.OutputStream1"/>
       <container-connection dest="B.InputStream2" source="A.OutputStream2"/>
   </application>
</runtime>

Notes:

The discussion above does not list all configuration file elements that affect containers. Generate a skeleton configuration file with sbd –s to see the full set of elements.
In prior releases, the configuration file used an application-file element, which has been deprecated: the element is still supported, but we recommend that you edit your configuration files to use the application element instead.
If you do not define any containers, StreamBase provides a default container, named default.

If you do not specify a container name in a StreamBase command or client, the default container is assumed. For example, the following list command is run against the default container of the server at the default StreamBase URI:
```
sbc list
```
To list the entities for a particular container, specify the container name in the StreamBase URI specified with the -u option. For example:
```
sbc -u sb://localhost:10000/myContainer list
```
You can use the StreamBase URI shortcuts to reduce this command to the following:
```
sbc -u /myContainer list
```

The System Container

When an sbd process starts up (with or without an application), a container named system is always present and running. The system container has the following output streams:

error: Emits a tuple containing each StreamBase Server runtime error, if any.
control: Emits messages from this StreamBase Server in response to certain system-level events. The control stream is discussed in more detail in Using Control Stream Features.
stat: Emits tuples containing StreamBase Monitor statistics about the running server.
connections: Emits a tuple every time a client connects to or disconnects from this server.
subscriptions: Emits a tuple every time a client subscribes to or unsubscribes from any stream on this server.

Entity Notation Syntax and Containers

StreamBase's formal entity notation is designed to avoid naming conflicts when you run multiple applications in the same sbd, when you use modules, and when operators are run in separate threads. StreamBase entity notation can be used to address application entities when using the sbadmin and sbc commands, or while configuring the runtime section of the server configuration file (specifically, in the container-connection element's destination and source attributes). For example, the following command asks for the XML representation of one particular stream:

sbc describe default.OutputStream1

The entity notation uses the following syntax, with a period separator between each element:

containerName.[moduleRefName ...].entityName

containerName

The name of the container hosting the application. Container names are either system, default, or a name you assigned when creating a container. The following example addresses two operators with the same name in separate containers:

container1.ReadOrders
default.ReadOrders

When containerName is omitted, the container named default is assumed. Every StreamBase application has a default container named default, which contains the application unless you explicitly create a non-default container. This means you can usually omit the containerName portion of the address unless you are addressing a non-default container, or if a name conflict would otherwise occur. For example, the following commands are equivalent:

sbc enqueue default.InputStream1
sbc enqueue InputStream1

moduleRefName

A module that is either referenced explicitly in the application, or referenced implicitly by StreamBase. Implicit references occur when a component runs in one separate or multiple parallel threads. In this case, StreamBase creates a module for each running instance. There can be multiple moduleRefName levels in the address syntax.

The following line addresses an operator named SumOrders within a module reference named MWindowModuleRef1:

MWindowModule1Ref1.SumOrders

The next example addresses a group of entities associated with an operator that you have opted to run in a separate thread. In each entity, the prefix references the module that StreamBase creates for the operator's thread.

MyOperator.InputStream1 (input stream)
MyOperator.out:MyOperator_1 (output stream)
MyOperator.schema:RetrieveLastOrders (schema)
MyOperator.MyOperator (operator)

Finally, If you set a component to run in multiple parallel threads, you can address each thread separately. The following addresses two threads for the same operator:

Filter:0.Filter
Filter:1.Filter

entityName

The application component. That is, the operator, stream, schema, and so on.

URI Syntax and Containers

The URI syntax for addressing StreamBase applications allows you to connect to a particular container in a running application. The URI syntax is:

sb://host:port/container-name

For example:

sb://localhost:10000/app2

If you leave off the container-name portion of the URI, the default container is assumed. Thus, the following commands are equivalent:

sbc -u sb://localhost:10000/default
sbc -u sb://localhost:10000

When you specify a container name in the URI, you can omit the container name from any subsequent entity names. For example, the following commands are equivalent:

sbc -u sb://localhost/app2 enqueue InputStream1
sbc -u sb://localhost enqueue app2.InputStream1

If you start sbd without specifying an application name, the sbd process includes a system container, but no default container. In this case, you may see errors when the default container name is assumed in commands. For example, type the following command in a terminal window or a StreamBase Command Prompt:

sbd -p 9999

This starts an instance of the server on port 9999 without an application running. In another window, type the following command:

sbc -p 9999 list

The response is:

container       system
sbc: Container: default does not exist

Multiple URI Syntax

Some commands that accept StreamBase URIs can specify a comma-separated list of URIs to address two or more StreamBase Servers at once. This feature is used primarily to support a cluster of StreamBase Servers that are using high availability features.

See Multiple URI Syntax for HA Scenarios for a discussion of this subject.

Addressing Multiple Containers Independently

If an sbd process has multiple containers, you can enqueue to or dequeue from multiple containers at the same time. For example, you can dequeue with a command like the following:

sbc dequeue foo.OutputStream1 bar.OutputStream1

If you specify a container in the URI, you can still dequeue or enqueue from different containers. For example:

sbc -u sb://localhost/foo dequeue OutputStream1 bar.OutputStream1

OutputStream1 will be dequeued from the container foo, while bar.OutputStream1 will be dequeued from the container bar.

Caution

Because tuples are queued between applications, there is overhead introduced when tuples cross modules. Also note that if the downstream application cannot process tuples as fast as the upstream application can produce them, then the application may become unstable.

Adding Containers

There are several ways to add a container to a running sbd process:

Using the sbadmin command. The format is:

sbadmin addContainer container-name application-name.[ssql | sbapp | sbar]

Using the StreamBaseAdminClient API.
Specifying containers in the sbd.sbconf file, with the <runtime> ... <application> element.

Removing Containers

You can remove containers from a running sbd server process with the StreamBaseAdminClient API or the sbadmin command. For example:

sbadmin removeContainer container-name

When a container is removed:

Enqueuer clients receive an exception:

ContainerManager$NonExistentContainerException
Dequeuer clients will get an EOF if they are dequeuing only from that container. If they are connected to multiple containers, then they should continue to receive tuples from that container.
Containers that depend on a removed container for input or output will continue to function. Their input or output will disappear.

Getting a List of Containers

The sbc list command (and underlying client API) allows you to show all the containers in a running sbd. The following shows example output from sbc list.:

stream ....
schema ...
operator ...
container   foo
container   bar
container   system

The above example shows three containers. Two are defined by the user, and the third is provided by StreamBase: foo, bar, and system. If you specify a container in the URI, you get a list of the entities for that particular container.

Getting More Information About Containers

The sbc describe command shows you the container name and any connections specified when that container was added.

Managing Running Applications in Containers

You can use the sbadmin command to manage multiple container-bound applications running in a single sbd process. The relevant sbadmin commands are:

sbadmin {shutdown | suspend | resume} container-name

You can use sbadmin modifyContainer to:

add a connection between running containers
remove a container connection
start, stop, or pause an enqueue to or dequeue from a container

See sbadmin for details on the modifyContainer command.

Monitoring Container Information

The sbmonitor and sbmanager commands can display statistics from containers. While running sbmonitor, press c to toggle the display of container names in the Operator Name column. The following example shows the top portion of a StreamBase Monitor window:

STREAMBASE MONITOR - T+869 sec - press '?' for help
Memory: 63.3MB total, 3.1MB used, 60.2MB free, 253.2MB max, Deq: 3 Eng: 1

OPERATOR NAME                          IN      OUT     w-us/T   %TIME   SIZE
______________________________________________________________________________
default.skuGT5000Filter                 0        0        N/A    0.00    N/A

See Using StreamBase Monitor and Using StreamBase Manager for more information on the StreamBase monitoring utilities.

Enabling and Disabling Enqueue or Dequeue

Using either the sbadmin addContainer or sbadmin modifyContainer commands, you can dynamically start, stop, and pause enqueuing or dequeuing in containers.

Both commands take the following syntax:

--enqueue=STREAMSTATUS
--dequeue=STREAMSTATUS

The values for STREAMSTATUS can be:

ENABLED. This is the default.
DISABLED. The enqueue or dequeue will actively refuse any enqueue or dequeue requests, and throw an exception.
DROP_TUPLES. The enqueue or dequeue will silently drop tuples.

Client API Support for Containers

The StreamBase Client API for Java, C++, and .NET supports containers. You can write your own applications to connect to and interact with StreamBase Servers and the containers therein.

See API Guide for more information.

Unsupported Features

The following StreamBase features are not supported with containers:

Multiple applications per container
Ad-Hoc queries

Container Overview