Library Articles
The StreamBase Server Container Model
Author: Keith McNeill
Contributor: Dr. John Lifter
StreamBase Systems
26-February-2007
Applicable To: StreamBase 3.7, 5.0
Topics:
Beginning with StreamBase 3.7, a StreamBase server has the ability to host multiple applications. Individual applications may be dynamically added to, and removed from, a running server instance. When you add an application to a running server, it is added within a container. Containers are identified through a unique name and the container name is needed to manage the container and address entities within the container, for example, a stream or operator (see Managing Custom Java Operators and Embedded Adapters).
When adding a container to a server, you can define connections to other containers that are already deployed in the server instance. A connected container uses the output from another container as its input. If multiple applications hosted by the same daemon need to communicate with one another, the data transfer is handled within the daemon process between the containers and does not involve a network invocation. However, there is no requirement that a dependency exist between multiple applications running in the same daemon. You are free to deploy totally unrelated applications into a common daemon process.
There are three legacy approaches to running a StreamBase server:
- Run the server from within StreamBase Studio;
- Use the sbd command line utility, providing the name of an application file as an argument; or
- Use the sbd command line utility, obtaining the name of the application file from a configuration variable.
These three approaches, which are still available, deploy the application into a container named default.
You can also start a StreamBase server with the sbd command line utility without providing the application file. In this situation, no container or application is deployed. You then use the sbadmin command line utility or configuration file entries to create containers and deploy applications.
On startup, all StreamBase server processes create a container named system, which manages the system streams error, control, and stat. You can write applications that obtain data from these management streams.
When writing a client application to a StreamBase application, you need to initialize the client proxy with the URI of the application. The StreamBase command line utilities, such as sbc, also use the StreamBase URI to initialize a proxy against a deployed application. The StreamBase URI now has two formats. The legacy format – sb://host:port – and an extended format that also specifies the container name — sb://host:port/container.
When enqueuing to, or dequeuing from, an application, or addressing one of the custom Java operators or embedded adapters in an application, you may need to include the container name as part of the entity's name.
If you use the legacy format of the URI to initialize your client proxy, the proxy will connect to the default container and entity names referring to components deployed into a different container must be fully qualified (container_name.entity_name). If you use the extended format of the URI, the proxy connects to a specific container and names for entities in the specified container do not need to be fully qualified and can be addressed simply through their entity name.
Regardless of which URI format you use, you can always address an entity through its fully qualified name, which means that a client application can use a common client proxy instance to communicate with all applications deployed in a StreamBase server instance. For example, a client proxy, or command line utility, initialized with the URI sb://host:port/container1, may enqueue to an input stream in the application within container1 using the entity name InputStreamName.
sbc -usb://host:port/container1 enqueue InputStreamName
and may also enqueue to an input stream in the application within container2 using the fully qualified entity name container2.InputStreamName.
sbc -usb://host:port/container1 enqueue container2.InputStreamName
The preceding commands employed the C++ version, sbc, of the client utility. For all uses described in this article, the Java version of the utility, jsbc, could also be used.
If you try to initialize a client proxy with a URI that specifies a container that does not exist, the NonExistentContainerException will be raised. Consequently, attempting to use the legacy URI format to connect to a server that has not created the default container, will lead to a proxy initialization failure.
The sbadmin command is used to create a new container and deploy an application, remove a container, suspend a container, or resume a container. Since sbadmin acts directly on the StreamBase server, and not against entities within containers, you should only use the legacy URI format within the -u command line argument. The following command will create a container named container_name and then deploy the application described in the file application_name. The same command works with both StreamSQL EventFlow and StreamSQL applications.
sbadmin addContainer container_name application_name
Once a container has been created and an application deployed, you can temporarily disable, then re-enable, the container using the suspend and resume command line arguments.
sbadmin suspend container_name
sbadmin resume container_name
And a container may also be permanently disabled or removed.
sbadmin shutdown container_name
sbadmin removeContainer container_name
The preceding commands employed the C++ version, sbadmin, of the administration utility. For all uses described in this article, the Java version of the utility, jsbadmin, could also be used.
You can use the sbc utility with the list command line argument to show all of the streams, schemas, operators, and containers deployed within a container. You must use the -u command line argument to provide the URI for an existing container. For example, if an sbd instance has two containers, A and B, hosting the application files application_1.ssql and application_2.ssql, you need to specify in the URI the name of the container you want to review. For example, running:
sbc -usb://host:port/A list
Returns a list of all containers in the sbd instance (A, B, and system) and the streams, schemas, operators, and embedded adapters within application_1.
Containers deployed in the same sbc instance may be interconnected; an output stream from one container submits tuples to an input stream of another container. The connected streams must have exactly the same tuple field types, sizes, and positions in the schema. The names of the tuple fields are not relevant and may be different.
Container connections are specified in the sbadmin addContainer command or through configuration file entries. For example, the following two sbadmin addContainer commands create two containers with deployed applications and specifies that output from the application deployed into the first container (container A) will become the input to the application deployed in the second container (container B).
sbadmin addContainer A application_1.ssql
sbadmin addContainer B application_2.ssql B.InputStream=A.OutputStream
Suspending or removing a container that is connected to another container has no effect on the remaining container except that its source of input or destination for output will disappear. When the connected application is resumed or redeployed (into a container with the same name), the connections will be re-established.
When containers are connected, tuples are passed between the containers using the same queuing mechanism that is used between parallel modules in a StreamBase application. The upstream container will block only until the tuple is placed into the incoming queue of the downstream container. Then processing will continue in parallel within both containers.
Downstream containers can receive tuples both from their upstream containers and from network clients external to the sbd instance.
While the sbadmin utility can be used to add containers and deploy applications, its usage is tedious and error prone. As an alternative approach, you can include entries within the <runtime> configuration entry so that multiple containers and applications are set up, and interconnections established, during server startup.
Within the <runtime> element, the <application> element identifies a container name and application file. The <runtime> element may have any number of <application> child elements.
Each <application> element has two required attributes, used to identify the application file and specify the container name, and any number of <container-connection> child elements. The <container-conection> child element has two required attributes, which identify an output (source) stream in the upstream container and its corresponding input (destination) stream in the downstream container.
<streambase-configuration>
<runtime>
<application file="./application_1.ssql" container="A"/>
<application file="./application_2.ssql" container="B">
<container-connection source="A.application_1_Output"
dest="B.application_2_Input"/>
</application>
</runtime>
</streambase-configuration>
When starting the StreamBase server daemon, use the -f command line argument to specify the location and name of the configuration file. As sbd initializes, it will create the containers and deploy the applications. When initialization completes, all applications will have been deployed and all interconnections established.
Containers permit the same stream/operator/schema names to be used in the multiple applications hosted in a single sbd instance. However, since streams in different containers may have the same name, the stream name may not necessarily be sufficient to identify a stream. The Stream Properties class encapsulates both the stream and container names and its usage to identify a target stream is preferred over the older approach of simply identifying a stream through its name.
In the StreamBase C++ client API, new overloaded enqueue and dequeue methods, with the following signatures, have been added.
void enqueue (const StreamProperties &streamProperties, Tuple &tuple);
void enqueue (const StreamProperties &streamProperties, TupleList &tuples);
TupleList dequeue (StreamProperties &streamProperties)
throw (StreamBaseException);
In the StreamBase Java client API, new overloaded enqueue methods, with the following signatures, have been added.
public void enqueue (StreamProperties streamProperties, Tuple tuple)
throws StreamBaseException
public void enqueue (StreamProperties streamProperties, Collection tuples)
throws StreamBaseException
The signature of the Java API dequeue method has not changed, but the DequeueResult class now includes a method that will return a StreamProperties object for the stream.
StreamProperties getStreamProperties ()
For both the C++ and Java client APIs, the original enqueue and dequeue methods, which use the stream name to identify the target stream, continue to be supported for backward compatibility.
Support for containers required a significant change to the network protocol used by network clients in communicating with a StreamBase server. Client applications written against earlier releases of StreamBase (pre-v3.7) are incompatible with applications deployed into a StreamBase server with support for containers (v3.7 and later). Since the legacy APIs are still supported, client source code written against earlier releases of StreamBase should not require editing, but these applications must be recompiled using the current JAR or C++ library files. Current versions of the JAR or C++ library files must also be installed on the computers hosting your network client applications.
- Using sbadmin to manage custom Java operators and embedded adapters in deployed StreamBase applications is described in Managing Custom Java Operators and Embedded Adapters.
- New in StreamBase 5.0, the graphical StreamBase Manager component can be used to create, control, and manage containers. See the product documentation for additional information on StreamBase manager
Back to Top ^