Contents
The StreamBase Comstock Input Adapter allows a StreamBase application to retrieve market data from a Comstock Client Site Processor (CSP) using the Comstock Token Format (CTF). The adapter is embedded within the StreamBase application and provides access access to equity, derivative, and market depth information from one or more of approximately 400 exchanges.
The adapter emits market data tuples on one of its two primary output ports. Snapshot tuples, containing a symbol's full image, are sent to the SnapshotOut port, while update tuples, containing non-null data in only those fields that have changed, are sent to the UpdateOut port. The StreamBase application author determines the set of fields present in each market data tuple. StreamBase market data field names must match the corresponding Comstock names, with the exception that dots in Comstock names are replaced with underscores in StreamBase names. The set of available Comstock field names can be retrieved through an information query to the adapter.
The adapter supports both static and dynamic subscriptions. Static (initial) subscriptions are provided to the adapter through a text file, which is processed when the adapter starts. Later, symbols can be subscribed to, or unsubscribed from, by enqueuing tuples to the adapter's dynamic subscription input port.
The adapter provides a third output port to convey information about important events to the StreamBase application. For example, when the state of a connection to the CSP changes, a connection event tuple is emitted from the event port.
The adapter provides an input and output port for querying metadata, including the set of Comstock error codes, the set of Comstock tokens (fields), the subset of tokens selected by the user, the set of Comstock Source IDs (exchanges), and the set of active subscriptions.
Subscriptions can be submitted for a specific symbol on a specific exchange, for all symbols on a specific exchange, or for all symbols on all exchanges. Snapshots of a symbol's current state can be retrieved, as well as depth and derivative information. Certain combinations of subscription options are disallowed. For example, a snapshot may not be requested when subscribing to all symbols on all exchanges. In addition, attempts to request depth and derivative information with a single subscribe request are rejected.
The adapter is configured through a collection of properties set in the adapter's Properties view within StreamBase Studio. Properties specify, among other things, the host name or IP address of the CSP, its port number, and the username and password used to access it.
Note
The Comstock adapter is provided in a separate installation kit, not in the base StreamBase kit. Contact StreamBase Systems if you are interested in the kit.
| Property | Description |
|---|---|
| Host | The host name or IP address of the Comstock Client Site Processor (CSP). |
| Port | The TCP port number of the CSP. |
| Username | The username to log in to the CSP. |
| Password | The password to log in to the CSP. |
| Reconnect Interval | The time, in seconds, to wait between attempts to reconnect to the CSP after the connection to it is lost. |
| Token File | The name of a text file containing the full set of Comstock tokens. If blank, the adapter retrieves token information from the CSP during initialization. Providing a token file streamlines initialization. |
| Error Code File | The name of a text file containing the numeric value and description of each Comstock error code. If blank, the adapter retrieves the error codes from the CSP during initialization. Providing an error code file streamlines initialization. |
| Initial Subscription File | The name of a text file containing subscriptions to be processed during adapter startup. An example initial subscription file is shipped with the Comstock sample. |
| Send Unchanged Fields As Null | Determines the action taken when a field of interest is not present in a market data message. If set (the default), a null value is placed in the corresponding StreamBase field. Otherwise, the value of the field from the previous tuple matching the symbol and exchange is placed in the field. |
| Display Messages Sent | A debugging feature that, if set, displays in the console the contents of each Comstock message sent by the adapter. |
| Display Messages Received | A debugging feature that, if set, displays in the console the contents of each Comstock message received by the adapter. |
| Verbose Message Display | A debugging feature that, if set, shows the raw bytes of each displayed sent or received message. |
| Message Display Filter |
A debugging feature that contains a string used to reduce the set of sent or
received message displayed. Only those messages whose commands match the
filter are displayed. For example, a filter of login|snap causes only sent and received login and snapshot
messages to be displayed.
|
This section demonstrates how to use the Comstock adapter within a StreamBase application. As shown in the diagram below depicting the adapter's sample application, the adapter uses two input ports and three output ports to communicate with the surrounding application.
Note
Although in the sample application the adapter's input and output ports are connected directly to externally-visible input and output streams, in more complex applications these ports will typically be connected to internal StreamBase operators.
The Comstock adapter's ports are used as follows:
-
SubscribeIn: Tuples enqueued on this port cause the adapter to subscribe to, or unsubscribe from, symbols after the adapter has started. Initial subscriptions (those the adapter processes during startup) can later be cancelled using this port. The schema of the SubscribeIn port is derived from the upstream operator or stream and must have the following fields:
-
Subscribe, bool: if true, results in a new subscription. Note that the adapter treats bool fields containing null as false.
-
Snapshot, bool: if true, requests a snapshot. If Subscribe is also true, requests a snapshot followed by a series of market data updates. If both Subscribe and Snapshot are false, interpreted to be an unsubscribe request.
-
Derivatives, bool: if true, requests derivative information, such as options and warrants.
-
Depth, bool: if true, requests market depth information. Depth and Derivatives cannot both be set.
-
Symbol, string(20): specifies the symbol to subscribe to or unsubscribe from. If null or empty, all symbols on the specified exchange are included.
-
SourceID, int: specifies the numeric exchange code such as 558 for NYSE.
-
DerivativesSourceID, int: when requesting derivative information, this optional parameter limits the response to derivatives from the specified exchange.
-
MarketMaker, string(5): when requesting depth information, this optional parameter limits the response to bids and asks from the specific market maker.
-
-
InfoQueryIn: Tuples enqueued on this port request metadata from the adapter. The resulting information is emitted as tuples on the InfoQueryOut port. The InfoQueryIn port has the following fields:
-
Command, string(40): accepts a command specifying the type of metadata being requested. Valid commands include:
-
ListErrorCodes: returns the numeric value and description of all Comstock error codes.
-
ListAvailableTokens: returns the numeric value, name, StreamBase field name, type, and size of all Comstock tokens (fields).
-
ListUserTokens: returns the numeric value, name, StreamBase field name, type, and size of all Comstock tokens currently selected by the user. During the initialization, the adapters requests the CSP to make available all Comstock tokens.
-
ListSourceIDs: returns the numeric code and description of all source IDs (exchanges).
-
ListCurrencies: returns the numeric code and description of all Comstock currencies.
-
ListSubscriptions: returns the current set of pending and active subscriptions.
-
-
Tag, string(40): this value is echoed in each tuple emitted from the information query output port in response to a command, allowing responses to be matched with commands.
-
-
SnapshotOut: Market data snapshot tuples are emitted from this port. The user determines the set of fields present in its schema with the adapter's Edit Schema tab. The set of available Comstock fields is available through the ListUserTokens information query command.
-
UpdateOut: Market data update tuples are emitted from this port. Its schema is derived from the Edit Schema tab and is identical to that of the SnapshotOut port.
-
EventOut: The adapter emits tuples from this port when significant events occur, such as when the state of the connection with the CSP changes. The schema for this port has the following fields:
-
Type, string(50): returns the type of event, such as
Connection,Login, orSubscription. -
Object, string(500): returns an event type-specific value, such as the CSP host and port when a connection event occurs, or the contents of the subscription fields when a subscription failure event occurs.
-
Action, string(50): returns an action associated with the event Type and Object, such as Up or Down for a Connection event, Rejected when a Login failure event occurs, or Failed when a subcription fails.
-
Status, int: returns a Comstock status code, if one is available for the particular event. For example, a subscription request may fail with a Comstock status code of -12 (ERR_AUTH_REQ), indicating the user does not have entitlement for the requested exchange.
-
Info, string(500): Returns a human-readable description of the event.
-
-
InfoQueryOut: This generic adapter output port emits tuples in response to all information query commands. The Comstock sample application uses a filter to split the resulting stream into a set of command-specific streams. The InfoQueryOut port has the following schema:
-
Done, bool: Set to
falsefor all but the last tuple emitted in response to a specific command. The final (marker) tuple has Done set totrueand all other fields set tonull. -
Command, string(40): returns the command value specified in the corresponding information query request tuple.
-
Tag, string(40): returns the tag value specified in the corresponding information query request tuple.
-
Info1-Info5, string(100): returns one or more command-specific values. The description of the InfoQueryIn stream above lists the information returned for each command.
-
Add an instance of the adapter to a new StreamBase application as follows:
-
Within StreamBase Studio, create a project, including a StreamBase EventFlow diagram, which will host the adapter.
-
Import into the project the token, error codes, and initial subscription files, comstock_tokens.txt, error_codes.txt, and initial_subscriptions.txt, which are located in
streambase-install-dir/sample/adapter/embedded/comstock. -
Open the Global Adapters drawer of the Palette view in StreamBase Studio.
-
Drag an instance of the Comstock adapter Global Adapter drawer to the canvas.
-
Connect Input and Output streams to the adapter's two input and three output ports. Configure the schema of the input streams as defined in the descriptions of the SubscribeIn and InfoQueryIn ports above. The adapter informs you of any missing or extraneous fields in the two input ports.
-
Double-click the adapter icon, and in the Properties view, select the Adapter Settings tab. Fill in the host or IP address, TCP port number, username, and password of the Client Site Processor. Alternatively, define these values as operation parameters in a server configuration file and reference them using the ${MyParam} syntax from the adapter settings tab. (The adapter's sample uses this latter technique.) Select the token, error code, and initial subscription files from the respective drop-downs.
-
Select the Edit Schema tab and add the Comstock fields you would like to have appear in the market data output stream. The set of available fields can be retrieved by starting the adapter and sending a ListUserTokens command to the information query input port.
The Comstock adapter uses typecheck messages to help you configure the adapter within your StreamBase application. In particular, the adapter generates typecheck messages if the CSP host, port, username, or password are missing or invalid or when one or more required fields in the two input schemas is missing or is of the wrong type or size.
The adapter generates warning and error messages during runtime under various conditions, including:
-
A field in the market data output port's schema is not recognized as a valid Comstock token.
-
A line in the initial subscription file is invalid.
-
A tuple received on the dynamic subscription port contains incompatible parameters, such as a snapshot request for all symbols on all exchanges, or a request for both the Depth and Derivative information.
-
A tuple received on the information query input port contains an unrecognized Command.
-
A tuple received on the information query input port contains a null Tag value.
-
A field from a market data message is truncated to fit in an output tuple's string field. To avoid overrunning log files, truncation warnings are generated only once per tuple field.
-
An invalid or malformed Comstock message is received from the CSP.
When suspended, the Comstock adapter closes its connection to the CSP.
When resumed, the adapter reopens a connection to the CSP and re-subscribes to the subscriptions that were pending or active at the time of the suspension.