Query Table Overview

Contents

Query Table Introduction
In-Memory or On-Disk Query Tables
Setup For On-Disk Query Tables
On-Disk Query Table Parameters
Null Values

Throughout this topic, remember that with StreamBase Developer Edition, you can only use memory-based Query Tables. You must have an Enterprise Edition license to use disk-based Query Tables.

Query Table Introduction

Query Tables are containers for sharing data and maintaining state within a StreamBase application. They accomplish this by storing tuple values outside of any stream. At runtime, a Query Table can have its values modified by one or more Query operators that write to, read from, delete from, or update their associated Query Table.

Query Tables can be associated with multiple Query operators, but each Query operator can be associated with only one Query Table.

The following figure shows the Query.sbapp sample application from the operator sample shipped with StreamBase, and illustrates four Query operators associated with one Query Table data construct.

In-Memory or On-Disk Query Tables

Query Tables can be declared to reside in-memory, or on-disk. For a memory-resident table, any data stored at runtime is not saved after the hosting StreamBase Server is shut down. For disk-based Query Tables, the data can persist indefinitely. For example, you could store data for a StreamBase application that runs only for several hours a day, and requires data from the previous run during initialization each day. Another possible use of persistent Query Tables tables is to maintain state information for a StreamBase application.

In the following figure, the Table Settings tab of the Properties view for a Query Table data construct is shown with all default values. Notice that the selected Query Table is declared to be in-memory.

Setup For On-Disk Query Tables

In the case of memory-resident Query Tables (and disk-based Materialized Windows), data stored in the table at runtime is not saved after StreamBase Server shuts down. In the case of disk-based Query Tables, the data stored in the tables persists between StreamBase Server sessions.

Note

The use of disk-based Query Tables (or disk-based Materialized Windows) is not available when using a StreamBase trial kit license. The disk-based Query Table option must be enabled in your StreamBase license, and requires the use of StreamBase Enterprise Edition. Contact StreamBase Systems for further information on this licensing option.

To make disk-based Query Table data persist between StreamBase Server sessions, follow these steps:

In the Table Settings tab of the Query Table's Properties view, set the Type property to On disk.
Set the Truncate on startup property to No.
Specify a file system location in which the application is to store its on-disk query tables using one of the following techniques:
- On the sbd command line, use the --datadir option.
- Use the STREAMBASE_DATA environment variable.
- In the StreamBase Server configuration file, define a datadir parameter in the server element. You must specify a closing backslash for Windows to indicate a directory path. For example:
```
<server>
  <param name="datadir" value="C:\Apps\qtstorage\" />
</server>
```
  Use a closing slash for UNIX paths:
```
<server>
  <param name="datadir" value="/home/sbuser/apps/qtstorage/" />
</server>
```
If a data directory is not specified, or if the special value +TEMP+ is defined, StreamBase creates a temporary directory on server startup, which is deleted on server shutdown.

On-Disk Query Table Parameters

StreamBase also provides options for transactional updates of disk-based Query Tables, adding new configuration parameters for the server element of a server configuration file. For example:

<server>
  <param name="datadir" value="C:\myapps\streambase\datadir\" />
  <param name="disk-querytable-transaction_mode" value="1" />
  <param name="disk-querytable-flush-interval-ms" value="5000" />    
</server>

See StreamBase Server Configuration File XML Reference for the XML syntax of server configuration files.

If your application uses disk-based Query Tables, you may be able to improve performance by setting a value for the following parameter for the server element of the configuration file:

<param name="disk-querytable-cache" value="nnn">

The value="nnn" attribute sets the amount of main memory that will be used to cache data from disk-based Query Table operations. The units are in MB of main memory and must be a power of 2. When unspecified, the default value is 1 MB. Use caution when setting this parameter, because too high a value may consume more memory than needed, and could negatively impact other resources that require memory during the execution of the StreamBase application or other applications. As with any resource setting, establish baseline performance metrics and then test the effect of increasing or decreasing values under normal and peak loads.

Null Values

If the key field for a tuple record being written to a Query Table is null, the tuple is stored. In a Query Table with a sorted (btree) index, the null-keyed stored records are evaluated as less (in value) than other non-null records. On a subsequent read (lookup) operation, the null-keyed tuples can be located. For more information, see Using Nulls.