Prokee Module: bws

This module provides the second part of a two part scanning approach.

1. BasicBlockScanner
2. BasicWesternScanner (bws)

Tokens

The following types of tokens are recognized:

• Literals
• Operators
• Keywords
• Other words (separated by operators or change of alphabets)
• Level-2 blocks (as provided by BasicBlockScanner)

Global Settings

Example

[settings]
->useAlphabets="true";
->useDictionary="false";
->dictionary="main_german";
->dictionary="main_english";
->cutat="Operator";
->cutat="Alphabet";
->cutat="Symbol";