Cassandra Loader Configuration

Configuration Logic

The loader's configuration is set in the config.ini file which includes the loader and session configuration sections. Each section includes configuration parameters, such as queue size, number of threads, etc. which can be updated during the loader's fine-tuning optimization process.

The loader execution modes are:

SINGLE Default. Every query submitted to the loader is executed.
BATCH The loader executes statements in batches, the batch size is set by the BATCH_SIZE property in the config.ini or iifConfig.ini.
TOKEN_AWARE_BATCH The loader executes statements in batches. To improve performance, the statements are grouped into batches by a token. The batch size is set by the BATCH_SIZE property in the config.ini or iifConfig.ini.

Note that writing into the Cassandra DB can be disabled for testing purposes. This is controlled by the IS_NOP loader's configuration parameters.

Work by Priority

The loader works according to the following priority:

  • Batch_process
    • Look for [LU type]batchprocess_loader section.
    • If not exist, user batchprocessloader section.
    • if not exist, use default_loader section.
  • Parser
    • Look for [LU type][parser name]loader section.
    • if not exist, use parser_loader section.
    • If not exist, user default_loader section.
  • iidFinder
    • Look for iidfinderloader section.
    • If not exist, use default_loader section.

The session works according to the following priority:

  • Fabric internal
    • Use the default_session section.
  • Batch_process
    • Look for [LU type]batchprocess_session section.
    • If not exist, use batchprocesssession section.
    • If not exist, use loader_session section.
    • If not exist, use default_session section.
  • Parser
    • Look for [LU type][parser name]session section.
    • If not exist, use parser_session section.
    • If not exist, use loader_session section.
    • If not exist, use default_session section.
  • iidFinder
    • Look for iidfindersession section.
    • If not exist, use loader_session section.
    • If not exist, use default_session section.

Previous

Cassandra Loader Configuration

Configuration Logic

The loader's configuration is set in the config.ini file which includes the loader and session configuration sections. Each section includes configuration parameters, such as queue size, number of threads, etc. which can be updated during the loader's fine-tuning optimization process.

The loader execution modes are:

SINGLE Default. Every query submitted to the loader is executed.
BATCH The loader executes statements in batches, the batch size is set by the BATCH_SIZE property in the config.ini or iifConfig.ini.
TOKEN_AWARE_BATCH The loader executes statements in batches. To improve performance, the statements are grouped into batches by a token. The batch size is set by the BATCH_SIZE property in the config.ini or iifConfig.ini.

Note that writing into the Cassandra DB can be disabled for testing purposes. This is controlled by the IS_NOP loader's configuration parameters.

Work by Priority

The loader works according to the following priority:

  • Batch_process
    • Look for [LU type]batchprocess_loader section.
    • If not exist, user batchprocessloader section.
    • if not exist, use default_loader section.
  • Parser
    • Look for [LU type][parser name]loader section.
    • if not exist, use parser_loader section.
    • If not exist, user default_loader section.
  • iidFinder
    • Look for iidfinderloader section.
    • If not exist, use default_loader section.

The session works according to the following priority:

  • Fabric internal
    • Use the default_session section.
  • Batch_process
    • Look for [LU type]batchprocess_session section.
    • If not exist, use batchprocesssession section.
    • If not exist, use loader_session section.
    • If not exist, use default_session section.
  • Parser
    • Look for [LU type][parser name]session section.
    • If not exist, use parser_session section.
    • If not exist, use loader_session section.
    • If not exist, use default_session section.
  • iidFinder
    • Look for iidfindersession section.
    • If not exist, use loader_session section.
    • If not exist, use default_session section.

Previous