public class ProgressLexer extends antlr.CharScanner implements ProgressParserTokenTypes, antlr.TokenStream
ProgressParser
.
Each token has an integer token type by which the parser references and
matches tokens. This is done to make a high performance parser. To make
this work, the lexer must create a token object when it matches a top
level rule, this token object includes the token type and the actual text
found in the original character stream. The lexer's main job is to
lookahead into the character stream, switch into the top level rule that
matches some set of characters and when this is complete, to create a
valid token object and return this to the parser. The lexer thus
implements a token stream interface from the parser's perspective.
Please note that this is a generated file using ANTLR 2.7.4 and a
grammar specified in progress.g
.
The Progress 4GL language is highly context sensitive, particularly in the area of user-defined symbols and language keywords. Examples of this attribute include:
Such constructions cannot be disambiguated by looking ahead on a character basis. Instead, the context depends upon a lookahead at the token level. This means that such items must be handled by the parser rather than the lexer. For this reason, the lexer is only partially able to participate in the process of symbol resolution.
There are many different types of symbols. Language keywords are the
most obvious symbol that are visible in all Progress 4GL source files.
Other types include variables, procedure names, function names, labels,
temp table names, etc. All of these symbols follow the same basic
rules by which they are matched in the lexer, for details on this, see
. As a result, the basic symbol rule in the
lexer matches all of these types and cannot set the token type properly
since it doesn't have enough information to do so at the time it is
matching.
mSYMBOL(boolean)
Symbol resolution is the process by which the proper token type is assigned to a token created from a match to the generic symbol rule. In the lexer, the only symbol resolution that can be done is to match with language keywords. Language keywords are given preference over other symbol types in any case where there is a conflict. This is done because there are significantly more locations in the parser that are matching on language keywords than there are locations where a user defined symbol can be in use. Thus it makes sense to handle the conflicts at the smaller number of locations in the parser, and assume that a keyword match is first a keyword. This has a great simplifying effect on the parser design. The result is selected locations in the parser that override (reassign) the token type based on context.
It seems that the parser could provide feedback to the lexer at certain points in its parsing. By storing shared state with the lexer, the lexer could then assign the token type properly instead of taking the "error on the side of keywords" approach which it takes now. However, due to the manner in which the parser does lookahead, this shared state implementation is not possible. The lookahead can be done at a level in the parser at which the context is not yet known (outside of the rules that actually handle the matching). If the lookahead depth (the "k") is greater than 1, then symbolic tokens (with the default token type set) can be returned before the parser can set the state variables that would properly control the assignment of those same tokens. The decoupling of parser lookahead from the rules that know enough to set the state causes it to be infeasible to implement shared state between the parser and the lexer.
The hundreds (thousands?) of keywords supported by Progress 4GL naturally
drive the use of symbol resolution and token type rewriting rather than
hard coding a rule for every keyword. This is also made necessary by the
need to implement abbreviation support. All of this is handled in the
lexer by the relatively small rule mSYMBOL
. This efficient
approach to keyword processing makes the lexer significantly smaller and
much simpler to implement and maintain that it otherwise might have been
given the keyword-richness of Progress 4GL.
Other key design points:
nextToken
. This
method has a for-loop that uses up to 3 characters of lookahead
to identify which of the top level token rules to call. To do
this, each token rule's left-most match characters are "rolled up"
and tested in this top level rule.
All token types are defined as integer constants in
which is an Interface that
the parser, lexer and other related classes all implement. This allows
all of these classes to directly refer to the token types and share this
common set of definitions. All top-level lexer rules generate a token
of the same name in the ProgressParserTokenTypes
ProgressParserTokenTypes
Interface.
There is a tokens { } section in the parser where artificial tokens are
defined (tokens that are not backed by a lexer rule). These tokens are
also added to the ProgressParserTokenTypes
Interface and can
thus be directly referenced by the lexer and parser.
Two of the most important lexer rules are mSYMBOL(boolean)
and
mNUM_LITERAL(boolean)
. Each of these rules process highly ambiguous
constructs requiring a rule that recognizes multiple token types and
overrides the default type based on the internal matching. Both rules
have to disambiguate the '.' character and the mSYMBOL
rule
also has to handle the '/' character. Changes to these rules must be
handled with the utmost care!
Due to the near total overlap between schema processing and Progress 4GL
source code, this lexer has been updated to support a mode with schema-
specific keywords such that it can lex the standard schema dump-file
(.df format). The only difference is the addition of a small set of
schema-specific keywords that are activated by the method
activateSchemaProcessing()
. Once activated it cannot be removed!
Modifier and Type | Field and Description |
---|---|
static antlr.collections.impl.BitSet |
_tokenSet_0 |
static antlr.collections.impl.BitSet |
_tokenSet_1 |
static antlr.collections.impl.BitSet |
_tokenSet_10 |
static antlr.collections.impl.BitSet |
_tokenSet_2 |
static antlr.collections.impl.BitSet |
_tokenSet_3 |
static antlr.collections.impl.BitSet |
_tokenSet_4 |
static antlr.collections.impl.BitSet |
_tokenSet_5 |
static antlr.collections.impl.BitSet |
_tokenSet_6 |
static antlr.collections.impl.BitSet |
_tokenSet_7 |
static antlr.collections.impl.BitSet |
_tokenSet_8 |
static antlr.collections.impl.BitSet |
_tokenSet_9 |
private boolean |
allowSlashSlash
Allow slash-slash style comments (started in v11, older code can have unquoted filenames
that include this same construct and will break parsing.
|
private boolean |
debug
Debug mode, defaults to
false . |
private boolean |
export
Export mode, defaults to
false . |
private boolean |
hidden
Hidden mode, defaults to
false . |
private static Keyword[] |
ignored
Stores all ignored keyword definitions.
|
private static Keyword[] |
keywords
Stores all keyword definitions in alphabetical order.
|
private boolean |
raw
Raw string processing mode, defaults to
false . |
private boolean |
schema
Schema mode, defaults to
false . |
private SymbolResolver |
sym
Enables language keyword resolution.
|
private boolean |
unixEscapes
Honor backslashes as escape characters.
|
_returnToken, caseSensitive, caseSensitiveLiterals, commitToPath, EOF_CHAR, hashString, inputState, literals, saveConsumedInput, tabsize, text, tokenObjectClass, traceDepth
ABBREVIATED, AGGREGATE, ALTER_TABLE, AMBIGUOUS, ANNOTATION, ANY_LOCK, AREA, ARGUMENTS, AS_BUTTON, ASSIGN, ASSIGNMENT, AST_NODE, AT, ATTR_CHAR, ATTR_CLASS, ATTR_COM_HANDLE, ATTR_DATE, ATTR_DATETIME, ATTR_DATETIME_TZ, ATTR_DEC, ATTR_HANDLE, ATTR_INT, ATTR_INT64, ATTR_LOGICAL, ATTR_LONGCHAR, ATTR_MEMPTR, ATTR_POLY, ATTR_RAW, ATTR_RECID, ATTR_ROWID, BACKSLASH, BEGIN_ATTR, BEGIN_ATTR_METH, BEGIN_BUFFERTYPES, BEGIN_CALLGRAPH, BEGIN_FIELDTYPES, BEGIN_FUNCTYPES, BEGIN_FWD_EXTENSION, BEGIN_INDEX_MATCH_TYPES, BEGIN_LVALUE, BEGIN_METATYPES, BEGIN_METH, BEGIN_MULTIWORD, BEGIN_NATIVE_VARTYPES, BEGIN_OO_METH, BEGIN_RECORDTYPES, BEGIN_RESERVED, BEGIN_SCHEMA, BEGIN_SCHEMAKW, BEGIN_UNRESERVED, BEGIN_VARTYPES, BEGIN_WIDGETS, BEGINS_MATCH, BITWISE_AND, BITWISE_NOT, BITWISE_OR, BITWISE_XOR, BLOCK, BOGUS, BOOL_FALSE, BOOL_TRUE, BROKEN_PREPROC_ARG, BUFFER, BUFFER_SCOPE, CARET, CATEGORY, CHILD_PROCESS, CLASS_DEF, CLASS_EVENT, CLASS_NAME, CLIENT_WHERE, CLIENT_WHERE_SUBST, CLOSE_CURSOR, CLOSE_QUERY, CLOSE_STORED_PROCEDURE, CMT_CLOSE, CMT_OPEN, COLON, COLOR_PHRASE, COLUMN_LIST, COLUMN_REF, COM_INVOCATION, COM_METHOD, COM_OBJECT, COM_PARAMETER, COM_PROPERTY, COMMA, COMMAND_TEXT, COMMAND_TOKENS, COMMENT, CONNECT_OPTIONS, CONNECT_TEXT, CONSTRUCTOR, CONTENT_ARRAY, COPY_FROM_SB, COPY_TO_SB, CREATE_ALIAS, CREATE_BROWSE, CREATE_BUFFER, CREATE_CALL, CREATE_CLIENT_PRINCIPAL, CREATE_DATA_SOURCE, CREATE_DATABASE, CREATE_DATASET, CREATE_GENERIC_OCX, CREATE_INDEX, CREATE_OBJECT, CREATE_QUERY, CREATE_REPORT, CREATE_SAX_ATTRIBUTES, CREATE_SAX_READER, CREATE_SAX_WRITER, CREATE_SERVER, CREATE_SMTP_EMAIL, CREATE_SOAP_HEADER, CREATE_SOAP_HEADER_ENTRYREF, CREATE_SOCKET, CREATE_SRV_SOCKET, CREATE_TABLE, CREATE_TEMP_TABLE, CREATE_TIMER, CREATE_VIEW, CREATE_WIDGET, CREATE_WIDGET_POOL, CREATE_X_DOCUMENT, CREATE_X_NODEREF, CUSTOMER_SPECIFIC, CYCLE_ON_LIMIT, DATA_RELATION, DATA_SET, DATA_SOURCE, DATA_SOURCE_BUFFER, DATA_SOURCE_QUERY, DATABASE, DATE_LITERAL, DATETIME_LITERAL, DATETIME_TZ_LITERAL, DB_EVENT, DB_REF_NON_STATIC, DB_SYMBOL, DDE_ADVISE, DDE_EXECUTE, DDE_GET, DDE_INITIATE, DDE_REQUEST, DDE_SEND, DDE_SERVER, DDE_TERMINATE, DEC_LITERAL, DECLARE_CURSOR, DEFINE_BROWSE, DEFINE_BUFFER, DEFINE_BUTTON, DEFINE_DATA_SOURCE, DEFINE_DATASET, DEFINE_ENUM, DEFINE_EVENT, DEFINE_FIELD, DEFINE_FRAME, DEFINE_IMAGE, DEFINE_MENU, DEFINE_PARAMETER, DEFINE_PROPERTY, DEFINE_PROPERTY_GET, DEFINE_PROPERTY_SET, DEFINE_QUERY, DEFINE_RECTANGLE, DEFINE_STREAM, DEFINE_SUB_MENU, DEFINE_TEMP_TABLE, DEFINE_VARIABLE, DEFINE_WORK_TABLE, DELETE_ALIAS, DELETE_FROM, DELETE_OBJECT, DELETE_PROCEDURE, DELETE_WIDGET, DELETE_WIDGET_POOL, DESCRIPTION, DESTRUCTOR, DIGIT, DISABLE_TRIGGERS, DIVIDE, DLL_CALL_TYPE_LITERAL, DOT, DOTNET_DELEGATE, DROP_INDEX, DROP_TABLE, DROP_VIEW, DSTRING, DUMP_NAME, DYNAMIC, EDITING_BLOCK, EMBEDDED_ASSIGNMENT, EMBEDDED_SQL, EMPTY_TEMP_TABLE, END_ATTR, END_ATTR_METH, END_BUFFERTYPES, END_CALLGRAPH, END_FIELDTYPES, END_FUNCTYPES, END_FWD_EXTENSION, END_INDEX_MATCH_TYPES, END_LVALUE, END_METATYPES, END_METH, END_MULTIWORD, END_NATIVE_VARTYPES, END_OO_METH, END_RECORDTYPES, END_RESERVED, END_SCHEMA, END_SCHEMAKW, END_UNRESERVED, END_VARTYPES, END_WIDGETS, ENUM_DEF, ENUM_VALUE, EOF, EQUALITY_MATCH, EQUALS, EVENT, EVENT_LIST, EVENT_SIGNATURE, EXCLUSIVE_LOCK_LITERAL, EXPANDED_SCOPE, EXPORT_FIELD, EXPRESSION, EXTERNAL_PROCEDURE, FIELD_BIGINT, FIELD_BLOB, FIELD_BYTE, FIELD_CHAR, FIELD_CLASS, FIELD_CLOB, FIELD_COM_HANDLE, FIELD_DATE, FIELD_DATETIME, FIELD_DATETIME_TZ, FIELD_DEC, FIELD_DOUBLE, FIELD_FIXCHAR, FIELD_FLOAT, FIELD_HANDLE, FIELD_INT, FIELD_INT64, FIELD_LOGICAL, FIELD_RAW, FIELD_RECID, FIELD_ROWID, FIELD_SHORT, FIELD_TIME, FIELD_TIMESTAMP, FILE_RESOURCE, FILENAME, FILEROOT, FILTER_SPEC, FORM_ITEM, FORMAT_PHRASE, FRAME_ALLOC, FRAME_ELEMENT, FRAME_LOCK, FRAME_PHRASE, FRAME_SCOPE, FRAME_UNLOCK, FREE_REFERENCE, FROZEN, FUNC_CALL_TYPE_LITERAL, FUNC_CHAR, FUNC_CLASS, FUNC_COM_HANDLE, FUNC_DATE, FUNC_DATETIME, FUNC_DATETIME_TZ, FUNC_DEC, FUNC_HANDLE, FUNC_INT, FUNC_INT64, FUNC_LOGICAL, FUNC_LONGCHAR, FUNC_MEMPTR, FUNC_POLY, FUNC_RAW, FUNC_RECID, FUNC_ROWID, FUNCTION, FUNCTION_CALL, GET_ATTR_CTYPE_LITERAL, GROUP_BY, GT, GTE, HANDLE_TO, HEX_DIGIT, HEX_LITERAL, HIDDEN, IMPORT_FIELD, IN_WIDGET_POOL, INCLUDE_FILE, INCLUDES, INCREMENT, INDEX, INDEX_FIELD, INLINE_VAR_DEF, INLINE_VAR_DEF_ARRAY, INNER_BLOCK, INNER_JOIN, INPUT_CLEAR, INPUT_CLOSE, INPUT_FROM, INPUT_OUTPUT_CLOSE, INPUT_OUTPUT_THRU, INPUT_THRU, INSERT_INTO, INT_PROC, INTERFACE_DEF, INTERNAL_PROCEDURE, IO_OPTIONS, IPC_RESOURCE, IS_NOT_NULL, IS_NULL, JUNK, KEY_FIELD, KEY_FUNCTION, KW_3D, KW_ABBV, KW_ABORT, KW_ABS, KW_ABSTRACT, KW_ACC_CHG, KW_ACC_RCHG, KW_ACCEL, KW_ACCUM, KW_ACT_FORM, KW_ACT_WIN, KW_ACTIVATE, KW_ACTIVE, KW_ACTOR, KW_ADD, KW_ADD_1ST, KW_ADD_BCCA, KW_ADD_BUF, KW_ADD_C_C, KW_ADD_C_F, KW_ADD_C_N, KW_ADD_CC_A, KW_ADD_EVTP, KW_ADD_F_F, KW_ADD_F_N, KW_ADD_HENT, KW_ADD_IDXF, KW_ADD_IMG, KW_ADD_INVL, KW_ADD_L_C, KW_ADD_L_N, KW_ADD_LAST, KW_ADD_LIKF, KW_ADD_LIKI, KW_ADD_N_N, KW_ADD_NEWF, KW_ADD_NEWI, KW_ADD_PREL, KW_ADD_REL, KW_ADD_SLOC, KW_ADD_SRCB, KW_ADD_SUP, KW_ADD_TAB, KW_ADD_TO_A, KW_ADM_DATA, KW_ADM_TO, KW_ADVISE, KW_AFT_BUFF, KW_AFT_FILL, KW_AFT_R_F, KW_AFT_ROID, KW_AFT_TBL, KW_ALERT_BX, KW_ALIAS, KW_ALIGN, KW_ALL, KW_ALLW_C_S, KW_ALLW_REP, KW_ALT_KEY, KW_ALTER, KW_ALW_ON_T, KW_AMBIG, KW_ANALYZ, KW_AND, KW_ANSI_ONL, KW_ANY, KW_ANY_KEY, KW_ANY_PRT, KW_ANYWHERE, KW_APP_INFO, KW_APP_PW, KW_APP_UID, KW_APPEND, KW_APPEND_C, KW_APPEND_L, KW_APPL, KW_APPL_A_B, KW_APPL_CBK, KW_APPL_CID, KW_APPLY, KW_AREA, KW_ARR_MSG, KW_AS, KW_AS_THRD, KW_ASC, KW_ASCEND, KW_ASCII_2H, KW_ASK_OVER, KW_ASSEMBLY, KW_ASSIGN, KW_AST_ACTI, KW_ASYNC, KW_ASYNC_RC, KW_ASYNC_RH, KW_AT, KW_AT_W_BRW, KW_ATT_DSRC, KW_ATT_FILE, KW_ATT_PLST, KW_ATT_URL, KW_ATTR, KW_ATTR_NAM, KW_AUD_CTRL, KW_AUD_ENAB, KW_AUD_EV_C, KW_AUD_POL, KW_AUTHEN_F, KW_AUTHORZN, KW_AUTO_COM, KW_AUTO_D_X, KW_AUTO_DEL, KW_AUTO_END, KW_AUTO_GO, KW_AUTO_IND, KW_AUTO_RES, KW_AUTO_RET, KW_AUTO_SYN, KW_AUTO_VAL, KW_AUTO_ZAP, KW_AUTOMATC, KW_AVAIL, KW_AVERAGE, KW_AVG, KW_AVL_FMTS, KW_AVL_MSGS, KW_B_ENDIAN, KW_B4_BUFF, KW_B4_FILL, KW_B4_HIDE, KW_B4_R_F, KW_B4_ROWID, KW_B4_TABLE, KW_BACK_TAB, KW_BACKGRND, KW_BACKSP, KW_BACKWARD, KW_BAS_LOGG, KW_BASE_ADE, KW_BASE_KEY, KW_BASE64_D, KW_BASE64_E, KW_BATCH_MO, KW_BATCH_SZ, KW_BEG_EV_G, KW_BEGINS, KW_BELL, KW_BETWEEN, KW_BGCOLOR, KW_BGCOLRGB, KW_BIGINT, KW_BINARY, KW_BIND, KW_BIND_WH, KW_BL_BTNFN, KW_BL_CGR_K, KW_BL_CURGR, KW_BL_G_KEY, KW_BL_GADD, KW_BL_GCLR, KW_BL_GIFP, KW_BL_GIKEY, KW_BL_IADD, KW_BL_ITMFN, KW_BLANK, KW_BLK_IT_D, KW_BLK_LVL, KW_BLOB, KW_BLOCK, KW_BORD_B_C, KW_BORD_B_P, KW_BORD_L_C, KW_BORD_L_P, KW_BORD_R_C, KW_BORD_R_P, KW_BORD_T_C, KW_BORD_T_P, KW_BOTH, KW_BOTTOM, KW_BOTTOM_C, KW_BOX, KW_BOX_SEL, KW_BREAK, KW_BREAK_L, KW_BROWSE, KW_BTN_LIST, KW_BTOS, KW_BUF_COMP, KW_BUF_COPY, KW_BUF_CREA, KW_BUF_DEL, KW_BUF_FLD, KW_BUF_GRPI, KW_BUF_GRPN, KW_BUF_HNDL, KW_BUF_NAME, KW_BUF_PARI, KW_BUF_REL, KW_BUF_TENI, KW_BUF_TENN, KW_BUF_VAL, KW_BUF_VLID, KW_BUFFER, KW_BUFFER_C, KW_BUFFER_L, KW_BUFPOOL, KW_BUFRPOOL, KW_BUILD_TR, KW_BUTTON, KW_BY, KW_BY_PTR, KW_BY_REF, KW_BY_VALUE, KW_BY_VAR_P, KW_BYTE, KW_BYTES_R, KW_BYTES_W, KW_C_NOD_D, KW_C_T_V_N, KW_CACHE, KW_CACHE_SZ, KW_CALBGCLR, KW_CALCUFMT, KW_CALENDAR, KW_CALFGCLR, KW_CALFMTST, KW_CALL, KW_CALL_NAM, KW_CALL_TYP, KW_CALLBACK, KW_CALTITBG, KW_CALTITFG, KW_CALTRLFG, KW_CALUPDWN, KW_CALVALUE, KW_CAN_CREA, KW_CAN_DEL, KW_CAN_DO, KW_CAN_DUMP, KW_CAN_FIND, KW_CAN_LOAD, KW_CAN_QRY, KW_CAN_READ, KW_CAN_SET, KW_CAN_WRT, KW_CANC_BRK, KW_CANCEL_B, KW_CANCEL_P, KW_CANCEL_R, KW_CANCELLD, KW_CAPFNT, KW_CAPFNTSZ, KW_CAPS, KW_CARE_PNT, KW_CASE, KW_CASE_SEN, KW_CAST, KW_CATCH, KW_CATEGORY, KW_CBIT, KW_CDECL, KW_CEASE, KW_CENTER, KW_CFG_NAME, KW_CHAINED, KW_CHAR, KW_CHAR_LEN, KW_CHARSET, KW_CHART, KW_CHECK, KW_CHECK_AM, KW_CHECK_BO, KW_CHECKED, KW_CHLD_BUF, KW_CHLD_NUM, KW_CHOICES, KW_CHOOSE, KW_CHR, KW_CLASS, KW_CLEA_TAB, KW_CLEA_WIN, KW_CLEAR, KW_CLIP, KW_CLNT_C_I, KW_CLNT_DIS, KW_CLNT_PRL, KW_CLNT_TTY, KW_CLNT_TYP, KW_CLNT_WS, KW_CLOB, KW_CLOB_COL, KW_CLOB_CP, KW_CLOB_TYP, KW_CLONE_ND, KW_CLOSE, KW_CLOSE_AP, KW_CLOSE_LG, KW_CLR_ALL, KW_CLR_AP_C, KW_CLR_ATTL, KW_CLR_BCCL, KW_CLR_CC_L, KW_CLR_EMBL, KW_CLR_LOG, KW_CLR_NIML, KW_CLR_NODS, KW_CLR_S_AR, KW_CLR_SEL, KW_CLR_TABS, KW_CLR_TO_L, KW_CLS_TYPE, KW_CNCL_R_A, KW_CNTRL_BX, KW_CNTRL_FR, KW_CNTRL_NM, KW_CODE, KW_CODEBASE, KW_COL, KW_COL_BGC, KW_COL_CP, KW_COL_DC, KW_COL_FGC, KW_COL_FONT, KW_COL_L_SA, KW_COL_LAB, KW_COL_MOV, KW_COL_NODE, KW_COL_OF, KW_COL_PFC, KW_COL_R_O, KW_COL_RES, KW_COL_SCR, KW_COL_SRT, KW_COLL_A_E, KW_COLL_ALL, KW_COLLATE, KW_COLON, KW_COLON_AL, KW_COLOR, KW_COLR_TAB, KW_COLUMNS, KW_COM_DATA, KW_COM_HNDL, KW_COM_SELF, KW_COMBO_BX, KW_COMMAND, KW_COMPARE, KW_COMPARES, KW_COMPILE, KW_COMPILER, KW_COMPLETE, KW_CONN, KW_CONN_ED, KW_CONNTYPE, KW_CONSTRUC, KW_CONTAINS, KW_CONTENTS, KW_CONTROL, KW_CONTROLS, KW_CONVERT, KW_COOKIE_D, KW_COPY, KW_COUNT, KW_COUNT_OF, KW_COVERAGE, KW_CP, KW_CP_CVT, KW_CPCASE, KW_CPCOLL, KW_CPINTERN, KW_CPLOG, KW_CPPRINT, KW_CPRCODEI, KW_CPRCODEO, KW_CPSTREAM, KW_CPTERM, KW_CPY_DSET, KW_CPY_LOB, KW_CPY_SATR, KW_CPY_TTBL, KW_CR_LK_SQ, KW_CRC, KW_CRC_VAL, KW_CRE_IMAG, KW_CREAT_LK, KW_CREAT_ND, KW_CREAT_NN, KW_CREAT_OA, KW_CREAT_RL, KW_CREAT_SN, KW_CREAT_TF, KW_CREATE, KW_CRT_COL, KW_CRT_MIMG, KW_CTOS, KW_CTX, KW_CTX_H, KW_CTX_H_F, KW_CTX_H_ID, KW_CTX_PATH, KW_CTX_POP, KW_CUR_CHAR, KW_CUR_CHG, KW_CUR_COL, KW_CUR_DATE, KW_CUR_DOWN, KW_CUR_ENV, KW_CUR_ITER, KW_CUR_LANG, KW_CUR_LEFT, KW_CUR_LINE, KW_CUR_OFF, KW_CUR_QRY, KW_CUR_R_M, KW_CUR_RES, KW_CUR_RGHT, KW_CUR_RQI, KW_CUR_RSI, KW_CUR_UP, KW_CUR_VAL, KW_CUR_WIN, KW_CURR_TAB, KW_CURRENCY, KW_CURRENT, KW_CURSOR, KW_CUT, KW_CVT_2OFF, KW_CVT_3D_C, KW_CVT_DT, KW_CYCLE, KW_DATA_BND, KW_DATA_E_R, KW_DATA_REL, KW_DATA_SCM, KW_DATA_SM, KW_DATA_SRC, KW_DATA_SRI, KW_DATABASE, KW_DATASET, KW_DATASRV, KW_DATATYPE, KW_DATE, KW_DATE_FMT, KW_DATE_SEP, KW_DATE_TZ, KW_DATETIME, KW_DAY, KW_DB_CTXT, KW_DB_LIST, KW_DB_REF, KW_DB_REM_H, KW_DBCOLL, KW_DBCP, KW_DBG_ALRT, KW_DBG_LST, KW_DBNAME, KW_DBPARAM, KW_DBREST, KW_DBTASKID, KW_DBTYPE, KW_DBVERS, KW_DCLK_EXP, KW_DCOLOR, KW_DD_OTREE, KW_DDE, KW_DDE_ERR, KW_DDE_ID, KW_DDE_ITEM, KW_DDE_NAME, KW_DDE_NOTI, KW_DDE_TOPI, KW_DEBLANK, KW_DEBUG, KW_DEBUGGER, KW_DEC, KW_DEC_SEP, KW_DECIMALS, KW_DECL_NSP, KW_DECLARE, KW_DECRYPT, KW_DEF_ACTN, KW_DEF_BUFH, KW_DEF_COMM, KW_DEF_EXTN, KW_DEF_NOXL, KW_DEF_POP, KW_DEF_STR, KW_DEF_UEVM, KW_DEF_VAL, KW_DEF_WIN, KW_DEFAULT, KW_DEFINE, KW_DEFLT_BN, KW_DEL_C_R, KW_DEL_CHAR, KW_DEL_COL, KW_DEL_COOK, KW_DEL_E_L, KW_DEL_FLD, KW_DEL_H_EN, KW_DEL_LINE, KW_DEL_NODE, KW_DEL_PROC, KW_DEL_R_L, KW_DEL_S_C, KW_DEL_S_R, KW_DEL_S_RS, KW_DEL_WORD, KW_DELEGATE, KW_DELETE, KW_DELIMIT, KW_DESCEND, KW_DESCR, KW_DESEL_FR, KW_DESEL_R, KW_DESEL_SR, KW_DESELCTN, KW_DESTRUCT, KW_DET_DSRC, KW_DIALOG, KW_DICT, KW_DIR, KW_DIRECTRY, KW_DIS_A_ZA, KW_DIS_CEDT, KW_DIS_D_TR, KW_DIS_L_TR, KW_DIS_REDR, KW_DIS_STRP, KW_DISABL_C, KW_DISABLE, KW_DISABLED, KW_DISCONN, KW_DISP, KW_DISP_MSG, KW_DISP_TYP, KW_DISP_TZ, KW_DISPATCH, KW_DISTINCT, KW_DLL_C_T, KW_DMP_NAME, KW_DO, KW_DOMAIN_D, KW_DOMAIN_N, KW_DOMAIN_T, KW_DOS, KW_DOUBLE, KW_DOWN, KW_DRAG_EN, KW_DRAG_NOD, KW_DRAG_OVR, KW_DRAGDROP, KW_DROP, KW_DROP_DWN, KW_DROP_FN, KW_DROP_LST, KW_DROP_TAR, KW_DSET_HND, KW_DSL_MGR, KW_DUMP, KW_DUMP_LGN, KW_DYN_CAST, KW_DYN_CURV, KW_DYN_ENUM, KW_DYN_FUNC, KW_DYN_INVK, KW_DYN_NEW, KW_DYN_NEXV, KW_DYN_PROP, KW_DYNAMIC, KW_E_PROC_C, KW_EACH, KW_ECHO, KW_EDGE_C, KW_EDGE_P, KW_EDIT_B_T, KW_EDIT_C_P, KW_EDIT_C_U, KW_EDIT_CLR, KW_EDIT_CPY, KW_EDIT_CUT, KW_EDIT_PAS, KW_EDIT_TAB, KW_EDIT_UND, KW_EDITING, KW_EDITOR, KW_EH_FL, KW_EH_SR, KW_ELSE, KW_EMB_FILE, KW_EMB_URL, KW_EMPTY, KW_EMPTY_DS, KW_EMPTY_SN, KW_EMPTY_TT, KW_ENABLE, KW_ENABLE_C, KW_ENABLE_E, KW_ENABLED, KW_ENC_AMK, KW_ENC_SALT, KW_ENCODE, KW_ENCODING, KW_ENCRYPT, KW_END, KW_END_B_SN, KW_END_DOC, KW_END_ELEM, KW_END_ERR, KW_END_EV_G, KW_END_F_D, KW_END_MOV, KW_END_RESZ, KW_END_RRES, KW_END_SEAR, KW_END_USER, KW_ENDKEY, KW_ENS_N_V, KW_ENT_EX_L, KW_ENT_TLST, KW_ENTER_MB, KW_ENTERED, KW_ENTRY, KW_ENUM, KW_EQ, KW_ERR_CODE, KW_ERR_COL, KW_ERR_OBJD, KW_ERR_ROW, KW_ERR_S_T, KW_ERR_STAT, KW_ERR_STR, KW_ERROR, KW_ESCAPE, KW_ETIME, KW_EVENTS, KW_EVT_ACTI, KW_EVT_GRID, KW_EVT_PROC, KW_EVT_TYPE, KW_EXC_LOCK, KW_EXCEPT, KW_EXCL_ID, KW_EXCL_WEB, KW_EXEC_LOG, KW_EXECUTE, KW_EXISTS, KW_EXIT, KW_EXITCODE, KW_EXP, KW_EXP_N_IC, KW_EXP_NODE, KW_EXP_ON_E, KW_EXP_SCLK, KW_EXPA_ALL, KW_EXPAND, KW_EXPANDBL, KW_EXPLICIT, KW_EXPORT, KW_EXPORT_P, KW_EXTENDED, KW_EXTENT, KW_EXTERN, KW_F_KEY_H, KW_FETCH, KW_FETCH_SR, KW_FGCOLOR, KW_FGCOLRGB, KW_FIELD, KW_FIL_C_D, KW_FIL_C_T, KW_FIL_INFO, KW_FIL_M_D, KW_FIL_M_T, KW_FIL_NAME, KW_FIL_OFF, KW_FIL_SIZE, KW_FIL_TYPE, KW_FILE, KW_FILL, KW_FILL_IN, KW_FILL_MOD, KW_FILL_WST, KW_FILLED, KW_FILTERS, KW_FINAL, KW_FINALLY, KW_FIND, KW_FIND_1ST, KW_FIND_BR, KW_FIND_CS, KW_FIND_CUR, KW_FIND_FD, KW_FIND_GLO, KW_FIND_LST, KW_FIND_NO, KW_FIND_NOD, KW_FIND_NXT, KW_FIND_PO, KW_FIND_PRV, KW_FIND_SEL, KW_FIND_UNI, KW_FIND_WA, KW_FINDER, KW_FIR_DSET, KW_FIRST, KW_FIRST_AR, KW_FIRST_BU, KW_FIRST_CH, KW_FIRST_CO, KW_FIRST_DS, KW_FIRST_FM, KW_FIRST_N, KW_FIRST_OB, KW_FIRST_OF, KW_FIRST_PR, KW_FIRST_QR, KW_FIRST_SO, KW_FIRST_SR, KW_FIRST_SS, KW_FIRST_TI, KW_FIT_LCOL, KW_FIX_C_L, KW_FIX_CP, KW_FIXCHAR, KW_FIXD_ONL, KW_FLAGS, KW_FLAT_BUT, KW_FLD_TRG, KW_FLOAT, KW_FLTR_MAP, KW_FMT_DT, KW_FMT_SA, KW_FNT_BOLD, KW_FNT_ITAL, KW_FNT_NAME, KW_FNT_SIZE, KW_FNT_UNDL, KW_FOC_ASEL, KW_FOC_NKEY, KW_FOC_NODE, KW_FOCUS, KW_FOCUS_R, KW_FOCUS_RS, KW_FONT, KW_FONT_TAB, KW_FOR, KW_FORCE_F, KW_FORE, KW_FORM, KW_FORM_INP, KW_FORM_LIN, KW_FORMAT, KW_FORMATTE, KW_FORWARD, KW_FR_COL, KW_FR_DB, KW_FR_DOWN, KW_FR_FIELD, KW_FR_FILE, KW_FR_INDEX, KW_FR_LINE, KW_FR_NAME, KW_FR_ROW, KW_FR_SPACE, KW_FR_VAL, KW_FR_X, KW_FR_Y, KW_FRAGMENT, KW_FRAME, KW_FREQ, KW_FROM, KW_FROM_CHR, KW_FROM_CUR, KW_FROM_PIX, KW_FROZEN, KW_FS_NKEY, KW_FS_NODE, KW_FULL_H_C, KW_FULL_H_P, KW_FULL_W_C, KW_FULL_W_P, KW_FULLPATH, KW_FUNC_C_T, KW_FUNCT, KW_FV_NODE, KW_FWD_ONLY, KW_G_C_BGCO, KW_G_C_FGCO, KW_G_COL_P, KW_G_COL_W, KW_GEN_MD5, KW_GEN_PBEK, KW_GEN_PBES, KW_GEN_RNDK, KW_GEN_UUID, KW_GET, KW_GET_1ST, KW_GET_A_CT, KW_GET_A_N, KW_GET_ATTL, KW_GET_ATTR, KW_GET_B_A, KW_GET_B_OR, KW_GET_BCCL, KW_GET_BDAT, KW_GET_BITS, KW_GET_BLUE, KW_GET_BR_C, KW_GET_BUFH, KW_GET_BYTE, KW_GET_BYTS, KW_GET_C_S, KW_GET_CBPC, KW_GET_CBPN, KW_GET_CC_L, KW_GET_CFG, KW_GET_CFGV, KW_GET_CGI, KW_GET_CGIL, KW_GET_CGIV, KW_GET_CGL, KW_GET_CGLV, KW_GET_CHG, KW_GET_CHLD, KW_GET_CLL, KW_GET_CLNT, KW_GET_CODP, KW_GET_COLL, KW_GET_COOK, KW_GET_CREL, KW_GET_CUR, KW_GET_D_E, KW_GET_D_F, KW_GET_DBCL, KW_GET_DBL, KW_GET_DIR, KW_GET_DS_B, KW_GET_DYN, KW_GET_EMBL, KW_GET_ERRC, KW_GET_ERRR, KW_GET_FCN, KW_GET_FILE, KW_GET_FLD, KW_GET_FLT, KW_GET_FNAM, KW_GET_FOFF, KW_GET_GRN, KW_GET_HD_E, KW_GET_I64, KW_GET_IBNN, KW_GET_IBQN, KW_GET_ITER, KW_GET_K_V, KW_GET_L_V, KW_GET_LAST, KW_GET_LIC, KW_GET_LNBI, KW_GET_LONG, KW_GET_MOP, KW_GET_MSG, KW_GET_MSGG, KW_GET_MSGS, KW_GET_N_AT, KW_GET_N_BG, KW_GET_N_FG, KW_GET_N_HC, KW_GET_N_N, KW_GET_NEXT, KW_GET_NLEV, KW_GET_NODE, KW_GET_NSN, KW_GET_NTXT, KW_GET_NUM, KW_GET_NVV, KW_GET_PAR, KW_GET_PARN, KW_GET_PREV, KW_GET_PROP, KW_GET_PRT, KW_GET_PSN, KW_GET_PTR, KW_GET_QNBI, KW_GET_R_R, KW_GET_RED, KW_GET_REL, KW_GET_RGB, KW_GET_S_O, KW_GET_SELW, KW_GET_SER, KW_GET_SHRT, KW_GET_SIG, KW_GET_SIMG, KW_GET_SNC, KW_GET_SRCB, KW_GET_STR, KW_GET_SZ, KW_GET_T_S, KW_GET_TBI, KW_GET_TBNN, KW_GET_TBQN, KW_GET_THCH, KW_GET_THPX, KW_GET_TI, KW_GET_TNOD, KW_GET_TO_L, KW_GET_TOPB, KW_GET_TWCH, KW_GET_TWPX, KW_GET_U_F, KW_GET_UBI, KW_GET_UL, KW_GET_USHT, KW_GET_VAL, KW_GET_VBI, KW_GET_VBNN, KW_GET_VBQN, KW_GET_WAIT, KW_GET_WKDR, KW_GETATTRI, KW_GETCLASS, KW_GLOBAL, KW_GO, KW_GO_ON, KW_GO_PEND, KW_GOTO, KW_GRANT, KW_GRAPHIC, KW_GRD_F_H, KW_GRD_F_V, KW_GRD_SNAP, KW_GRD_UHC, KW_GRD_UHP, KW_GRD_UWC, KW_GRD_UWP, KW_GRD_VIS, KW_GROUP, KW_GROUP_BX, KW_GT, KW_GTE, KW_GUID, KW_GW, KW_H_SCRL_P, KW_HANDLE, KW_HANDLER, KW_HAS_LOBS, KW_HAS_REC, KW_HAVING, KW_HEADER, KW_HEIGHT_C, KW_HEIGHT_P, KW_HELP, KW_HELP_SA, KW_HELP_TOP, KW_HEX_DECD, KW_HEX_ENCD, KW_HID_FLD, KW_HID_FLDL, KW_HIDDEN, KW_HIDE, KW_HINT, KW_HIT_TEST, KW_HIT_TFWD, KW_HOME, KW_HONOR_PK, KW_HONOR_RK, KW_HORIZ, KW_HOST_B_O, KW_HTML_BWS, KW_HTML_CHS, KW_HTML_ENC, KW_HTML_EOL, KW_HTML_EOP, KW_HTML_FRB, KW_HTML_FRE, KW_HTML_H_B, KW_HTML_H_E, KW_HTML_T_B, KW_HTML_T_E, KW_HTMLERR, KW_HWND, KW_HYPERLNK, KW_ICFPARM, KW_ICON, KW_IDEPHWND, KW_IDEWNTYP, KW_IDX_FLD, KW_IDX_HINT, KW_IDX_INFO, KW_IDX_REPO, KW_IF, KW_IGN_CMOD, KW_IL_BGCOL, KW_IL_HEIGH, KW_IL_IMG, KW_IL_LIST, KW_IL_MASK, KW_IL_OVER, KW_IL_UMASK, KW_IL_WIDTH, KW_IMAGE, KW_IMG_DOWN, KW_IMG_INS, KW_IMG_LIST, KW_IMG_ONLY, KW_IMG_SZ, KW_IMG_SZ_C, KW_IMG_SZ_P, KW_IMG_UP, KW_IMM_DISP, KW_IMP_NODE, KW_IMP_PRNC, KW_IMPLEMTS, KW_IMPORT, KW_IN, KW_IN_HNDL, KW_IN_OUT, KW_INACTIVE, KW_INC_EX_I, KW_INCR, KW_INDENT, KW_INDEX, KW_INDICAT, KW_INFO, KW_INH_BGC, KW_INH_FGC, KW_INHERITS, KW_INIT, KW_INIT_C_P, KW_INIT_D_T, KW_INIT_DIR, KW_INIT_FLT, KW_INIT_SA, KW_INIT_SES, KW_INITIATE, KW_INNER, KW_INNER_C, KW_INNER_L, KW_INPT_VAL, KW_INPUT, KW_INS_ATTR, KW_INS_B4, KW_INS_BTAB, KW_INS_COL, KW_INS_FILE, KW_INS_FLD, KW_INS_FLDD, KW_INS_FLDL, KW_INS_MODE, KW_INS_ROW, KW_INS_STR, KW_INS_TAB, KW_INSERT, KW_INST_PRC, KW_INT, KW_INT_ENT, KW_INT64, KW_INTERFAC, KW_INTERVAL, KW_INTO, KW_INVOKE, KW_IS, KW_IS_ATTR, KW_IS_C_VIS, KW_IS_CLASS, KW_IS_COLCP, KW_IS_CP_FX, KW_IS_DB_MT, KW_IS_JSON, KW_IS_LEAD, KW_IS_M_TEN, KW_IS_MSEL, KW_IS_N_EXP, KW_IS_OPEN, KW_IS_P_SET, KW_IS_PART, KW_IS_R_SEL, KW_IS_SEL, KW_IS_XML, KW_ISO_DATE, KW_ITEM, KW_ITEM_ROW, KW_ITER_CHG, KW_IUNKNOWN, KW_JAVA, KW_JOIN, KW_JOIN_BY, KW_KBLABEL, KW_KEEP_CON, KW_KEEP_MSG, KW_KEEP_SEC, KW_KEEP_TAB, KW_KEEP_ZOR, KW_KEY, KW_KEYCODE, KW_KEYFUNC, KW_KEYLAB, KW_KEYS, KW_KPAD_CLE, KW_KPAD_HOT, KW_KPAD_QRY, KW_KW, KW_KW_ALL, KW_L_ENDIAN, KW_L_TRIM, KW_LAB_BGC, KW_LAB_DC, KW_LAB_FGC, KW_LAB_FONT, KW_LAB_H_C, KW_LAB_PFC, KW_LABEL, KW_LABEL_SA, KW_LABELS, KW_LANDSCAP, KW_LANGUAGE, KW_LARGE, KW_LAST, KW_LAST_AR, KW_LAST_BAT, KW_LAST_CH, KW_LAST_EVT, KW_LAST_FRM, KW_LAST_OBJ, KW_LAST_OF, KW_LAST_PRC, KW_LAST_SOC, KW_LAST_SRV, KW_LAST_SS, KW_LAST_TI, KW_LASTKEY, KW_LC, KW_LCD_REFR, KW_LCD_SWIN, KW_LCD_WSTR, KW_LCHR_2NV, KW_LDBNAME, KW_LEAVE, KW_LEFT, KW_LEFT_AL, KW_LEFT_END, KW_LEFT_MC, KW_LEFT_MD, KW_LEFT_MDC, KW_LEFT_MU, KW_LENGTH, KW_LG_2_SM, KW_LIB, KW_LIB_C_C, KW_LIKE, KW_LIKE_SEQ, KW_LINE, KW_LINE_CNT, KW_LIST_ITM, KW_LIST_PNM, KW_LISTING, KW_LISTINGS, KW_LIT_QSTN, KW_LOAD, KW_LOAD_DMN, KW_LOAD_I_D, KW_LOAD_I_I, KW_LOAD_I_U, KW_LOAD_ICO, KW_LOAD_IMG, KW_LOAD_M_P, KW_LOAD_PIC, KW_LOAD_S_I, KW_LOADCTRL, KW_LOB_AREA, KW_LOB_BYTE, KW_LOB_DIR, KW_LOB_SIZE, KW_LOC_C_N, KW_LOC_HOST, KW_LOC_L_N, KW_LOC_NAME, KW_LOC_P_ID, KW_LOC_PORT, KW_LOC_S_ID, KW_LOC_TYPE, KW_LOC_V_I, KW_LOCK_REG, KW_LOCKED, KW_LOG, KW_LOG_A_EV, KW_LOG_E_TS, KW_LOG_EN_T, KW_LOG_HOST, KW_LOG_ID, KW_LOG_MGR, KW_LOG_STAT, KW_LOG_THRS, KW_LOGF_NAM, KW_LOGG_LEV, KW_LOGICAL, KW_LOGOUT, KW_LONG, KW_LONGCHAR, KW_LOOKAHD, KW_LOOKUP, KW_LST_EVNT, KW_LST_PAIR, KW_LST_QRY, KW_LST_SET, KW_LST_WID, KW_LT, KW_LTE, KW_M_D_I_P, KW_M_U_I_P, KW_MACH_CLS, KW_MAINMENU, KW_MAN_HIGH, KW_MAND, KW_MAP, KW_MARG_EX, KW_MARG_H_C, KW_MARG_H_P, KW_MARG_W_C, KW_MARG_W_P, KW_MARK_NEW, KW_MARK_RS, KW_MATCHES, KW_MAX, KW_MAX_BTN, KW_MAX_CHAR, KW_MAX_D_G, KW_MAX_H_C, KW_MAX_H_P, KW_MAX_HT, KW_MAX_LVL, KW_MAX_ROWS, KW_MAX_SZ, KW_MAX_VAL, KW_MAX_W_C, KW_MAX_W_P, KW_MAX_WID, KW_MAXIMIZE, KW_MD5_DIG, KW_MD5_VAL, KW_MDL_MC, KW_MDL_MD, KW_MDL_MDC, KW_MDL_MU, KW_MEMBER, KW_MEMPTR, KW_MENU, KW_MENU_BAR, KW_MENU_DRP, KW_MENU_ITM, KW_MENU_KEY, KW_MENU_MOU, KW_MERGE_BF, KW_MERGE_CH, KW_MERGE_RC, KW_MESSAGES, KW_METHOD, KW_MF_DFR, KW_MF_DI, KW_MF_DIP, KW_MF_DL, KW_MF_DR, KW_MF_DT, KW_MF_DTAX, KW_MF_DTAXL, KW_MF_DTAXR, KW_MF_DTB, KW_MF_DTBL, KW_MF_DTBR, KW_MF_DTSX, KW_MF_GF, KW_MF_GFH, KW_MF_GFSD, KW_MF_GFSR, KW_MF_GPH, KW_MF_GPN, KW_MF_GPW, KW_MF_GTW, KW_MF_GXY, KW_MF_GZF, KW_MF_INIT, KW_MF_IR, KW_MF_MPDF, KW_MF_P2MU, KW_MF_RP, KW_MF_SF, KW_MF_SFC, KW_MF_SFH, KW_MF_SI, KW_MF_SIA, KW_MF_SLA, KW_MF_SLC, KW_MF_SLM, KW_MF_SLS, KW_MF_SNP, KW_MF_SNTL, KW_MF_SPF, KW_MF_SPN, KW_MF_SPNP, KW_MF_SPNT, KW_MF_SPO, KW_MF_SR, KW_MF_STA, KW_MF_STC, KW_MF_STS, KW_MF_SXY, KW_MF_SZF, KW_MIN, KW_MIN_BTN, KW_MIN_CWCH, KW_MIN_CWPX, KW_MIN_H_C, KW_MIN_H_P, KW_MIN_SCHM, KW_MIN_SZ, KW_MIN_VAL, KW_MIN_W_C, KW_MIN_W_P, KW_MNEMON, KW_MOD, KW_MODIFIED, KW_MODIFIRS, KW_MONTH, KW_MOU_MC, KW_MOU_MD, KW_MOU_MDC, KW_MOU_MU, KW_MOU_MVC, KW_MOU_MVD, KW_MOU_MVDC, KW_MOU_MVU, KW_MOU_PTR, KW_MOU_SC, KW_MOU_SD, KW_MOU_SDC, KW_MOU_SU, KW_MOU_XC, KW_MOU_XD, KW_MOU_XDC, KW_MOU_XU, KW_MOUSE, KW_MOV_2_B, KW_MOV_2_T, KW_MOV_2EOF, KW_MOV_A_T, KW_MOV_B_T, KW_MOV_COL, KW_MOVABLE, KW_MOVE, KW_MPE, KW_MPTR_2NV, KW_MS_CLICK, KW_MS_DBCLK, KW_MS_DOWN, KW_MS_MOVE, KW_MS_UP, KW_MSG, KW_MSG_AFNT, KW_MSG_AMSG, KW_MSG_AREA, KW_MSG_DIG, KW_MSG_LINE, KW_MSICON, KW_MSPNTNUM, KW_MTIME, KW_MULT_CMP, KW_MULT_KEY, KW_MULTI_IN, KW_MULTIPLE, KW_MUST_EXI, KW_MUST_UND, KW_N_AEDIT, KW_N_BEDIT, KW_N_CLICK, KW_N_COLED, KW_N_COLING, KW_N_EXPED, KW_N_EXPING, KW_N_HEIGHT, KW_N_KEY_ID, KW_NAME, KW_NAMESP_P, KW_NAMESP_U, KW_NATIVE, KW_NE, KW_NEEDS_AP, KW_NEEDS_PR, KW_NESTED, KW_NEW, KW_NEW_INST, KW_NEW_LBL, KW_NEW_LINE, KW_NEW_ROW, KW_NEXT, KW_NEXT_COL, KW_NEXT_ERR, KW_NEXT_FR, KW_NEXT_PMT, KW_NEXT_RID, KW_NEXT_SIB, KW_NEXT_TAB, KW_NEXT_VAL, KW_NEXTWORD, KW_NNMSP_SL, KW_NO_APPLY, KW_NO_ARMSG, KW_NO_ASSGN, KW_NO_ATTR, KW_NO_ATTRL, KW_NO_AUTOV, KW_NO_BIND, KW_NO_BOX, KW_NO_COLS, KW_NO_CONS, KW_NO_CUR_V, KW_NO_CV_3D, KW_NO_CVT, KW_NO_DEBUG, KW_NO_DRAG, KW_NO_ECHO, KW_NO_EH_FL, KW_NO_EH_SR, KW_NO_EM_SP, KW_NO_EMBED, KW_NO_ERROR, KW_NO_FILL, KW_NO_FOCUS, KW_NO_HELP, KW_NO_HIDE, KW_NO_IDX_H, KW_NO_INHBG, KW_NO_INHFG, KW_NO_JOIN, KW_NO_LABEL, KW_NO_LOBS, KW_NO_LOCK, KW_NO_LOOKA, KW_NO_MAP, KW_NO_MSG, KW_NO_OVRRD, KW_NO_PAUSE, KW_NO_PRE, KW_NO_RET_V, KW_NO_ROW_M, KW_NO_SCH_M, KW_NO_SCR_V, KW_NO_SEP_C, KW_NO_SEPS, KW_NO_TAB_S, KW_NO_UNDL, KW_NO_UNDO, KW_NO_VALID, KW_NO_WAIT, KW_NO_WRAP, KW_NODE_BLD, KW_NODE_CHE, KW_NODE_CNT, KW_NODE_EXP, KW_NODE_ICO, KW_NODE_ID, KW_NODE_IDX, KW_NODE_KEY, KW_NODE_PAR, KW_NODE_TXT, KW_NODE_V2M, KW_NODE_VAL, KW_NODES, KW_NODV_2LC, KW_NON_SER, KW_NONE, KW_NORMAL, KW_NORMALZE, KW_NOT, KW_NOT_ACTV, KW_NOT_CS, KW_NOW, KW_NULL, KW_NULLALWD, KW_NUM_2RTN, KW_NUM_ALIA, KW_NUM_BUFF, KW_NUM_BUTT, KW_NUM_CH_R, KW_NUM_CHLN, KW_NUM_COL, KW_NUM_COPY, KW_NUM_D_P, KW_NUM_DBS, KW_NUM_DROP, KW_NUM_ENT, KW_NUM_FLD, KW_NUM_FMT, KW_NUM_FMTS, KW_NUM_HD_E, KW_NUM_ITER, KW_NUM_ITMS, KW_NUM_LK_C, KW_NUM_LNS, KW_NUM_LOGF, KW_NUM_MSG, KW_NUM_PARM, KW_NUM_PNTS, KW_NUM_REF, KW_NUM_REL, KW_NUM_REPL, KW_NUM_RES, KW_NUM_SEP, KW_NUM_SR, KW_NUM_SRCB, KW_NUM_SW, KW_NUM_TABS, KW_NUM_TOPB, KW_NUM_V_C, KW_NUMERIC, KW_OBJECT, KW_OCT_LEN, KW_OCX_MBTN, KW_OCX_MSHT, KW_OCX_MSX, KW_OCX_MSY, KW_OF, KW_OFF, KW_OFF_END, KW_OFF_HOME, KW_OK, KW_OK_CAN, KW_OLD, KW_OLE_AEFF, KW_OLE_CDRG, KW_OLE_DATF, KW_OLE_DD, KW_OLE_DEFC, KW_OLE_DGMD, KW_OLE_DGOV, KW_OLE_DRAG, KW_OLE_DRMD, KW_OLE_EFFE, KW_OLE_GIVF, KW_OLE_INVL, KW_OLE_NAML, KW_OLE_SETD, KW_OLE_STAT, KW_OLE_STDG, KW_OLE_X, KW_OLE_Y, KW_ON, KW_ON_FR_B, KW_ONLY, KW_OPEN, KW_OPEN_L_A, KW_OPEN_URL, KW_OPENMIME, KW_OPENPAGE, KW_OPENPOPU, KW_OPSYS, KW_OPTION, KW_OPTIONS, KW_OPTIONSF, KW_OR, KW_ORD_JOIN, KW_ORDER, KW_ORDINAL, KW_ORG_HAND, KW_ORG_ROID, KW_OS_APPND, KW_OS_CMD, KW_OS_COPY, KW_OS_DEL, KW_OS_DIR, KW_OS_DRV, KW_OS_ERR, KW_OS_G_ENV, KW_OS_MKDIR, KW_OS_REN, KW_OS_UID, KW_OS2, KW_OTHER, KW_OUT_HDR, KW_OUT_HH, KW_OUT_JOIN, KW_OUT_MSGS, KW_OUTER, KW_OUTPUT, KW_OUTPUTCT, KW_OVERLAY, KW_OVERRIDE, KW_OWN_DOC, KW_OWNER, KW_P2J_RC, KW_PAGE, KW_PAGE_B, KW_PAGE_DWN, KW_PAGE_LFT, KW_PAGE_NUM, KW_PAGE_RT, KW_PAGE_SZ, KW_PAGE_T, KW_PAGE_UP, KW_PAGE_WID, KW_PAGED, KW_PAR_BUFF, KW_PAR_FLDA, KW_PAR_FLDB, KW_PAR_IFLD, KW_PAR_IREL, KW_PAR_REL, KW_PAR_W_C, KW_PARENT, KW_PARM, KW_PARSE_ST, KW_PART_KEY, KW_PASCAL, KW_PASSWD_F, KW_PASTE, KW_PATHNAME, KW_PAUSE, KW_PB_APPEA, KW_PB_BRSTY, KW_PB_ENABL, KW_PB_MAX, KW_PB_MIN, KW_PB_ORIEN, KW_PB_SCROL, KW_PB_VALUE, KW_PBE_H_AL, KW_PBE_KEYR, KW_PDBNAME, KW_PERF, KW_PERSIST, KW_PFCOLOR, KW_PICK, KW_PICK_ARE, KW_PICK_BTH, KW_PIX_COL, KW_PIX_ROW, KW_PIXELS, KW_PK_PRIV, KW_PK_PROT, KW_POLY, KW_POP_MENU, KW_POP_ONLY, KW_PORTRAIT, KW_POS, KW_PRE_LBL, KW_PRECISN, KW_PREF_DS, KW_PREP_STR, KW_PREPARED, KW_PREPROC, KW_PRESEL, KW_PREV, KW_PREV_COL, KW_PREV_FR, KW_PREV_SIB, KW_PREV_T_I, KW_PREVWORD, KW_PRIM_P_P, KW_PRIMARY, KW_PRINT, KW_PRINTER, KW_PRIV_DAT, KW_PRIVATE, KW_PRIVILEG, KW_PRMT_FOR, KW_PRO_ARCH, KW_PROC, KW_PROC_C_T, KW_PROC_COM, KW_PROC_HND, KW_PROC_ST, KW_PROC_T_B, KW_PROC_TXT, KW_PROC_W_R, KW_PROCESS, KW_PROCNAME, KW_PROCTYPE, KW_PRODATAS, KW_PROFILER, KW_PROFILNG, KW_PROG_BAR, KW_PROG_SRC, KW_PROGNAME, KW_PROGRESS, KW_PROMPT, KW_PROMSGS, KW_PROPATH, KW_PROPERTY, KW_PROTECTD, KW_PROVER, KW_PROX_PWD, KW_PROX_UID, KW_PROXY, KW_PRS_C_D, KW_PRS_PROC, KW_PRT_C_H, KW_PRT_HDC, KW_PRT_NAME, KW_PRT_PORT, KW_PRT_SET, KW_PSC, KW_PUB_EVTS, KW_PUB_ID, KW_PUBLIC, KW_PUBLISH, KW_PUT, KW_PUT_BITS, KW_PUT_BYTE, KW_PUT_BYTS, KW_PUT_DBL, KW_PUT_FLT, KW_PUT_I64, KW_PUT_K_V, KW_PUT_LONG, KW_PUT_SHT, KW_PUT_STR, KW_PUT_UL, KW_PUT_USHT, KW_QRY_CLOS, KW_QRY_OFF, KW_QRY_OPEN, KW_QRY_PREP, KW_QRY_TUNE, KW_QUAL_UID, KW_QUE_MSG, KW_QUERY, KW_QUEST, KW_QUIT, KW_QUOTER, KW_R_INDEX, KW_R_TRIM, KW_RADIO_B, KW_RADIO_S, KW_RANDOM, KW_RAW, KW_RAW_TRAN, KW_RCOD_INF, KW_READ, KW_READ_AVL, KW_READ_E_N, KW_READ_FIL, KW_READ_JSN, KW_READ_ONL, KW_READ_RES, KW_READ_XML, KW_READ_XSC, KW_READKEY, KW_REAL, KW_REC_LEN, KW_RECALL, KW_RECID, KW_RECT, KW_RECURSE, KW_REF_ONLY, KW_REFR_A_P, KW_REFRABLE, KW_REFRESH, KW_REG_DMN, KW_REJ_CHGS, KW_REJ_RCHG, KW_REJECTED, KW_REL_FLDS, KW_RELEASE, KW_RELS_ACT, KW_REM_ATTR, KW_REM_CHLD, KW_REM_EVTP, KW_REM_HOST, KW_REM_NOCO, KW_REM_NODE, KW_REM_PORT, KW_REM_SUP, KW_REMOTE, KW_REP_CHLD, KW_REP_STXT, KW_REPEAT, KW_REPL_CRE, KW_REPL_DEL, KW_REPL_WRI, KW_REPLACE, KW_REPORT, KW_REPORTS, KW_REPOS, KW_REPOS_2I, KW_REPOS_2R, KW_REPOS_B, KW_REPOS_F, KW_REPOS_M, KW_REQ_INFO, KW_REQUEST, KW_RES_BASE, KW_RESET, KW_RESIZABL, KW_RESIZE, KW_RESORT, KW_REST_RID, KW_REST_ROW, KW_RESULT, KW_RESUME_D, KW_RET_2SD, KW_RET_INS, KW_RET_SHAP, KW_RET_VAL, KW_RET_VDT, KW_RET_VLT, KW_RETAIN, KW_RETRY, KW_RETRY_C, KW_RETURN, KW_RETURNS, KW_REV_FROM, KW_REVERT, KW_REVOKE, KW_RFRSH_UI, KW_RGB_VAL, KW_RIGHT, KW_RIGHT_AL, KW_ROLE, KW_ROLES, KW_ROUND, KW_ROUNDED, KW_ROUTINEL, KW_ROW, KW_ROW_CRT, KW_ROW_CRTD, KW_ROW_DEL, KW_ROW_DELD, KW_ROW_DISP, KW_ROW_ENTR, KW_ROW_H_C, KW_ROW_H_P, KW_ROW_LEAV, KW_ROW_MARK, KW_ROW_MODD, KW_ROW_OF, KW_ROW_RESZ, KW_ROW_STAT, KW_ROW_UMOD, KW_ROW_UPD, KW_ROWID, KW_RPT_ADDC, KW_RPT_CAP, KW_RPT_CSV, KW_RPT_DOCX, KW_RPT_DSGN, KW_RPT_GSRC, KW_RPT_GSRR, KW_RPT_HTML, KW_RPT_ICSV, KW_RPT_PDF, KW_RPT_RTF, KW_RPT_SRC, KW_RPT_XLS, KW_RPT_XLSX, KW_RSP_INFO, KW_RT_END, KW_RT_MC, KW_RT_MD, KW_RT_MDC, KW_RT_MU, KW_RTOPSYS, KW_RULE, KW_RULE_ROW, KW_RULE_Y, KW_RUN, KW_RUN_PROC, KW_RUN_W_O, KW_S_C_BGCO, KW_S_C_FGCO, KW_S_COL_C, KW_S_COL_W, KW_SAVE, KW_SAVE_AS, KW_SAVE_FIL, KW_SAVE_RCH, KW_SAVE_WST, KW_SAX_ATTR, KW_SAX_COMP, KW_SAX_PARE, KW_SAX_PARF, KW_SAX_PARN, KW_SAX_PARS, KW_SAX_READ, KW_SAX_RUNN, KW_SAX_UNIN, KW_SAX_WBEG, KW_SAX_WCOM, KW_SAX_WCON, KW_SAX_WELM, KW_SAX_WERR, KW_SAX_WIDL, KW_SAX_WRIT, KW_SAX_WTAG, KW_SCH_CHG, KW_SCH_LOC, KW_SCH_MARS, KW_SCH_PATH, KW_SCHEMA, KW_SCR_2CR, KW_SCR_2ITM, KW_SCR_2SR, KW_SCR_DELT, KW_SCR_LEFT, KW_SCR_MODE, KW_SCR_NODC, KW_SCR_NOT, KW_SCR_NTOP, KW_SCR_OFFS, KW_SCR_RPOS, KW_SCR_RT, KW_SCR_VERT, KW_SCR_WLNS, KW_SCREEN, KW_SCRN_IO, KW_SCRN_LNS, KW_SCRN_VAL, KW_SCROLL, KW_SCROLL_H, KW_SCROLL_V, KW_SCROLLBA, KW_SCROLLBL, KW_SCROLLIN, KW_SDBNAME, KW_SEAL, KW_SEAL_TST, KW_SEAR_SLF, KW_SEAR_TRG, KW_SEARCH, KW_SECTION, KW_SECUR_P, KW_SEEK, KW_SEL_ALL, KW_SEL_END, KW_SEL_FOCR, KW_SEL_LST, KW_SEL_NEXT, KW_SEL_NID, KW_SEL_NKEY, KW_SEL_NODE, KW_SEL_PREV, KW_SEL_ROW, KW_SEL_SFVN, KW_SEL_STRT, KW_SEL_TXT, KW_SELECT, KW_SELECTBL, KW_SELECTED, KW_SELECTN, KW_SELF, KW_SEND, KW_SEND_SQL, KW_SENSITIV, KW_SEP_CONN, KW_SEP_FGC, KW_SEPS, KW_SEQUENCE, KW_SERIALAB, KW_SERIALZH, KW_SERIALZN, KW_SERIALZR, KW_SERVER, KW_SESS_END, KW_SESSION, KW_SESSN_ID, KW_SET, KW_SET_A_CT, KW_SET_A_L, KW_SET_A_N, KW_SET_ACTR, KW_SET_ACTX, KW_SET_ATTR, KW_SET_B_OR, KW_SET_BLUE, KW_SET_BRK, KW_SET_BUF, KW_SET_C_IC, KW_SET_C_P, KW_SET_C_S, KW_SET_C_VI, KW_SET_CB_P, KW_SET_CBAC, KW_SET_CLNT, KW_SET_COMM, KW_SET_CONT, KW_SET_COOK, KW_SET_CPTM, KW_SET_DBCL, KW_SET_DYN, KW_SET_EVMO, KW_SET_F_D, KW_SET_GRN, KW_SET_IMGF, KW_SET_IMGH, KW_SET_IMGW, KW_SET_ISRC, KW_SET_JMOD, KW_SET_JSTX, KW_SET_JSTY, KW_SET_LSTK, KW_SET_M_UN, KW_SET_MSEL, KW_SET_N_BG, KW_SET_N_F, KW_SET_N_FG, KW_SET_N_HC, KW_SET_NODE, KW_SET_NTXT, KW_SET_ODST, KW_SET_OPT, KW_SET_PARM, KW_SET_PENW, KW_SET_PROP, KW_SET_PTR, KW_SET_R_P, KW_SET_RED, KW_SET_RGB, KW_SET_ROLL, KW_SET_RPOS, KW_SET_RRP, KW_SET_S_AR, KW_SET_S_O, KW_SET_SEL, KW_SET_SERD, KW_SET_SNC, KW_SET_SWIN, KW_SET_SZ, KW_SET_T_S, KW_SET_TBME, KW_SET_TN_I, KW_SET_TSTA, KW_SET_U_F, KW_SET_W_S, KW_SET_WAIT, KW_SET_WKDR, KW_SETTINGS, KW_SETUSER, KW_SH_LOCK, KW_SHA1_DIG, KW_SHARED, KW_SHORT, KW_SHOW_BUT, KW_SHOW_HDR, KW_SHOW_ITB, KW_SHOW_ST, KW_SHR_D_F, KW_SIDE_L, KW_SIDE_L_H, KW_SIGNATUR, KW_SILENT, KW_SIMPLE, KW_SING_RUN, KW_SINGLE, KW_SINGLTON, KW_SIZE, KW_SIZE_C, KW_SIZE_P, KW_SKIP, KW_SKIP_D_R, KW_SLIDER, KW_SMAL_ICO, KW_SMAL_TTL, KW_SMALLINT, KW_SMTP_EML, KW_SMTP_PW, KW_SMTP_VAL, KW_SMTPFROM, KW_SMTPHOST, KW_SMTPHTML, KW_SMTPPORT, KW_SMTPREPL, KW_SMTPSUBJ, KW_SMTPTEXT, KW_SMTPUSER, KW_SOAP_F, KW_SOAP_F_A, KW_SOAP_F_C, KW_SOAP_F_D, KW_SOAP_F_N, KW_SOAP_F_R, KW_SOAP_F_S, KW_SOAP_FMH, KW_SOAP_FSC, KW_SOAP_HDR, KW_SOAP_HER, KW_SOAP_VER, KW_SOCKET, KW_SOME, KW_SOR_C_C, KW_SOR_COLS, KW_SORT, KW_SORT_ASC, KW_SORT_MAP, KW_SORT_NUM, KW_SOURCE, KW_SPACE, KW_SPRSHEET, KW_SQL, KW_SQL_WID, KW_SQRT, KW_SRC_PROC, KW_SRV_C_B, KW_SRV_C_BR, KW_SRV_C_C, KW_SRV_C_I, KW_SRV_OP_M, KW_SRV_SOCK, KW_SSL_SRVN, KW_ST_2_FIT, KW_STANDALN, KW_START, KW_START_BS, KW_START_DC, KW_START_EL, KW_START_MV, KW_START_RR, KW_START_RS, KW_START_SC, KW_STARTING, KW_STAT_A_F, KW_STAT_DET, KW_STATIC, KW_STATUS, KW_STATUS_A, KW_STDCALL, KW_STOP, KW_STOP_AFT, KW_STOP_D, KW_STOP_PRS, KW_STOPPED, KW_STORPROC, KW_STR_VAL, KW_STR_XREF, KW_STREAM, KW_STRIC_ER, KW_STRICT, KW_STRING, KW_STRM_HND, KW_STRM_IO, KW_STUP_PAR, KW_SUB_AVG, KW_SUB_CNT, KW_SUB_M_H, KW_SUB_MAX, KW_SUB_MENU, KW_SUB_MIN, KW_SUB_TOT, KW_SUBSCRIB, KW_SUBSTIT, KW_SUBSTR, KW_SUBTYPE, KW_SUM, KW_SUP_NS_P, KW_SUP_PROC, KW_SUP_WARN, KW_SUPER, KW_SUPW_LST, KW_SYM_EN_A, KW_SYM_EN_I, KW_SYM_EN_K, KW_SYM_SUPP, KW_SYNCHRON, KW_SYS_A_B, KW_SYS_DLG, KW_SYS_HELP, KW_SYS_ID, KW_TAB, KW_TAB_CRCL, KW_TAB_HAND, KW_TAB_IDX, KW_TAB_LIST, KW_TAB_ML, KW_TAB_MOD, KW_TAB_NUM, KW_TAB_POS, KW_TAB_SCAN, KW_TAB_SHOW, KW_TAB_STOP, KW_TAB_TRG, KW_TABLE, KW_TABS, KW_TABSET, KW_TAG, KW_TAR_PROC, KW_TARGET, KW_TEMP_DIR, KW_TEMP_TAB, KW_TEN_ID, KW_TENNAME, KW_TERM, KW_TERM_HK, KW_TERMINAT, KW_TEXT, KW_TEXTEDIT, KW_THEN, KW_THIS_OBJ, KW_THIS_PRC, KW_THO_SEP, KW_THR_SAFE, KW_THROUGH, KW_THROW, KW_TIC_MARK, KW_TIME, KW_TIME_SRC, KW_TIMER, KW_TIMESTMP, KW_TIMEZONE, KW_TITL_BGC, KW_TITL_DC, KW_TITL_FGC, KW_TITL_FON, KW_TITLE, KW_TN_TOID, KW_TNOD_VAL, KW_TO, KW_TO_ROWID, KW_TODAY, KW_TOGGL_BX, KW_TOOLTIP, KW_TOOLTIPS, KW_TOP, KW_TOP_COL, KW_TOP_NAVQ, KW_TOP_ONLY, KW_TOPIC, KW_TOTAL, KW_TRAC_CHG, KW_TRACING, KW_TRAILING, KW_TRAN_I_P, KW_TRAN_MOD, KW_TRANS, KW_TRANSPAR, KW_TRC_FILT, KW_TREE_R_H, KW_TREELIST, KW_TREEVIEW, KW_TRIG_COL, KW_TRIG_NOD, KW_TRIGGER, KW_TRIGGERS, KW_TRIM, KW_TRUNC, KW_TT_PREP, KW_TTCP, KW_TXT_CURS, KW_TXT_SEG, KW_TXT_SEL, KW_TYPE, KW_TYPE_OF, KW_U_CTRL, KW_U_MSG, KW_U_PCTRL, KW_U_SERIAL, KW_UNBOX, KW_UNBUF, KW_UNDERLIN, KW_UNDO, KW_UNDO_T_S, KW_UNFORMAT, KW_UNION, KW_UNIQ_ID, KW_UNIQ_KEY, KW_UNIQ_MAT, KW_UNIQUE, KW_UNIX, KW_UNIX_END, KW_UNL_HID, KW_UNLOAD, KW_UNS_BYTE, KW_UNS_LONG, KW_UNS_SHRT, KW_UNSUBSCR, KW_UP, KW_UPD_ATTR, KW_UPDATE, KW_UPLOAD, KW_URL, KW_URL_DECO, KW_URL_ENCO, KW_URL_FLD, KW_URL_FLDL, KW_URL_FMT, KW_URL_PW, KW_URL_UID, KW_USE, KW_USE_DCT, KW_USE_FIL, KW_USE_IDX, KW_USE_REV, KW_USE_TXT, KW_USE_UND, KW_USE_WIDP, KW_USER_ID, KW_USERID, KW_USING, KW_USR_DATA, KW_UTC_OFF, KW_V_N_CNT, KW_V_SCRL_P, KW_V6DISP, KW_V6FRAME, KW_VAL_CHG, KW_VAL_ENAB, KW_VAL_EVT, KW_VAL_EXPR, KW_VAL_HND, KW_VAL_MSG, KW_VAL_OBJ, KW_VAL_SEAL, KW_VAL_XML, KW_VALEXP, KW_VALIDATE, KW_VALMG_SA, KW_VALMSG, KW_VALUE, KW_VALUES, KW_VAR, KW_VERBOSE, KW_VERSION, KW_VERT, KW_VIEW, KW_VIEW_AS, KW_VIEW_FCR, KW_VIRT_HC, KW_VIRT_HP, KW_VIRT_WC, KW_VIRT_WP, KW_VISIBLE, KW_VMS, KW_VOID, KW_WA_H_P, KW_WA_W_P, KW_WA_X, KW_WA_Y, KW_WAIT, KW_WAIT_FOR, KW_WARN, KW_WC_AD_AP, KW_WEB, KW_WEB_CTX, KW_WEB_OUT, KW_WEEK, KW_WHEN, KW_WHERE, KW_WHERE_ST, KW_WHILE, KW_WID_ENT, KW_WID_HAND, KW_WID_ID, KW_WID_LEAV, KW_WID_POOL, KW_WIDGET, KW_WIDTH, KW_WIDTH_C, KW_WIDTH_P, KW_WIN_CLOS, KW_WIN_DMIN, KW_WIN_MAX, KW_WIN_MIN, KW_WIN_NAME, KW_WIN_NORM, KW_WIN_RESI, KW_WIN_REST, KW_WIN_STAT, KW_WIN_SYS, KW_WINDOW, KW_WITH, KW_WORD_IDX, KW_WORD_WRP, KW_WORK_TAB, KW_WR_CDATA, KW_WR_CHARS, KW_WR_CMNT, KW_WR_D_ELM, KW_WR_DATA, KW_WR_E_ELM, KW_WR_ENT_R, KW_WR_EXDTD, KW_WR_FRAGM, KW_WR_JSON, KW_WR_MSG, KW_WR_PRINS, KW_WR_STAT, KW_WR_XML, KW_WR_XMLSC, KW_WRITE, KW_X, KW_X_DOC, KW_X_NODE, KW_X_OF, KW_XCODE, KW_XML_DTYP, KW_XML_E_EL, KW_XML_NNAM, KW_XML_NTYP, KW_XML_S_ER, KW_XML_SCHP, KW_XML_SNSP, KW_XOR, KW_XPR2PDF, KW_XREF, KW_XREF_XML, KW_Y, KW_Y_OF, KW_YEAR, KW_YEAR_OFF, KW_YES_NO, KW_YES_NO_C, LABEL, LABEL_DEF, LANGUAGES, LBRACKET, LEFT_OUTER_JOIN, LETTER, LIBRARY_PROCEDURE, LIBRARY_REF, LPARENS, LT, LTE, MANDATORY, MANY_TO_ONE, METH_CHAR, METH_CLASS, METH_COM_HANDLE, METH_DATE, METH_DATETIME, METH_DATETIME_TZ, METH_DEC, METH_HANDLE, METH_INT, METH_INT64, METH_LOGICAL, METH_LONGCHAR, METH_MEMPTR, METH_POLY, METH_RAW, METH_RECID, METH_ROWID, METH_VOID, METHOD_DEF, METHOD_INVOCATION, MINUS, MISSING, MORE, MULTIPLY, NATIVE_API, NATIVE_PROCESS, NATIVE_PROGRAM_RESOURCE, NETWORK_CONNECTION, NO_LOCK_LITERAL, NO_MATCH, NO_REFERENCE, NO_TRANSACTION, NO_WAIT_LITERAL, NOT_BETWEEN, NOT_ENTERED, NOT_EQ, NOT_EXISTS, NOT_IN, NOT_LIKE, NOT_NULL, NULL_TREE_LOOKAHEAD, NUM_LITERAL, OBJECT_INVOCATION, OCX_CONTROL, ONE_TO_MANY, ONE_TO_ONE, OO_METH_CHAR, OO_METH_CLASS, OO_METH_COM_HANDLE, OO_METH_DATE, OO_METH_DATETIME, OO_METH_DATETIME_TZ, OO_METH_DEC, OO_METH_HANDLE, OO_METH_INT, OO_METH_INT64, OO_METH_LOGICAL, OO_METH_LONGCHAR, OO_METH_MEMPTR, OO_METH_POLY, OO_METH_RAW, OO_METH_RECID, OO_METH_ROWID, OO_METH_VOID, OPEN_CURSOR, OPEN_QUERY, OPTION, ORDER, ORDER_BY, OUTPUT_CLOSE, OUTPUT_THRU, OUTPUT_TO, PARAMETER, PARENT_CHILD_RELATION, PARENT_ID_RELATION, PLUS, PORT_TYPE, POSITION, PROC_CALL_TYPE_LITERAL, PROCEDURE, PROCEDURE_FILE, PROCESS_EVENTS, PROPERTIES, PROPERTY_INVOCATION, PUT_CURSOR, PUT_FIELD, PUT_SCREEN, QUERY, QUERY_SUBST, RANGE_MATCH, RAW_STRING, RBRACKET, READ_AVAILABLE_LITERAL, READ_EXACT_NUM_LITERAL, RECORD_PHRASE, RELEASE_EXTERNAL, RELEASE_OBJECT, RIGHT_OUTER_JOIN, RPARENS, RSTRING, RUN_FILENAME, RUN_FILENAME_ON_SERVER, RUN_INT_PROC, RUN_INT_PROC_ON_SERVER, RUN_LIBRARY_REF, RUN_LIBRARY_REF_ON_SERVER, RUN_PORT_TYPE_ON_SERVER, RUN_PORT_TYPE_VALUE_ON_SERVER, RUN_STORED_PROCEDURE, RUN_SUPER, RUN_VALUE, RUN_VALUE_ON_SERVER, RUNTIME_BROWSE_OPTIONS, RUNTIME_FRAME_OPTIONS, SAVE_CACHE, SCHEMA_FILE, SEARCH_SELF_LITERAL, SEARCH_TARGET_LITERAL, SELF_REFERENCE, SEQUENCE, SET_ATTR_CTYPE_LITERAL, SHARE_LOCK_LITERAL, SHARED_LIBRARY, SLASH_SLASH, SORT_MATCH, SQL_WIDTH, SSTRING, STATEMENT, STR_OPTIONS, STREAM, STRING, STRONG_REFERENCE, SUB_TRANSACTION, SUPPORT_LVL_FULL, SUPPORT_LVL_FULL_WITH_LIMITATIONS, SUPPORT_LVL_NONE, SUPPORT_LVL_PARTIAL, SYM_CHAR, SYMBOL, SYS_HANDLE, SYSTEM_DIALOG_COLOR, SYSTEM_DIALOG_FONT, SYSTEM_DIALOG_GET_DIR, SYSTEM_DIALOG_GET_FILE, SYSTEM_DIALOG_PRINTER_SETUP, TABLE, TEMP_TABLE, TILDE, TRAILER, TRANSACTION, TRANSACTION_DISTINCT, TRANSACTION_MODE_AUTO, TRIGGER_BLOCK, TRIGGER_PHRASE, TRIGGER_PROCEDURE, UN_MINUS, UN_PLUS, UNKNOWN_RESOURCE, UNKNOWN_TOKEN, UNKNOWN_VAL, UNQUOTED_TEXT, UPDATE_SQL, VALEXP, VALID_SYM_CHAR, VALIDATION, VALMG_SA, VALMSG, VAR_BYTE, VAR_CHAR, VAR_CLASS, VAR_COM_HANDLE, VAR_DATE, VAR_DATETIME, VAR_DATETIME_TZ, VAR_DEC, VAR_DOUBLE, VAR_FLOAT, VAR_HANDLE, VAR_INT, VAR_INT64, VAR_LOGICAL, VAR_LONG, VAR_LONGCHAR, VAR_MEMPTR, VAR_POLY, VAR_RAW, VAR_RECID, VAR_ROWID, VAR_SHORT, VAR_ULONG, VAR_USHORT, WEAK_REFERENCE, WEB_PROC, WHEN_LIST, WHERE_CURRENT_OF, WID_BROWSE, WID_BROWSE_COL, WID_BUTTON, WID_COMBO, WID_DIALOG, WID_EDITOR, WID_FILL_IN, WID_FRAME, WID_IMAGE, WID_LITERAL, WID_MENU, WID_MENU_ITM, WID_RADIO, WID_RECT, WID_SEL_LST, WID_SLIDER, WID_SUB_MENU, WID_TEXT, WID_TOGGLE, WID_WINDOW, WIDGET_LIST, WIDGET_POOL, WITH_CHECK_OPTION, WITH_GRANT_OPTION, WORD_MATCH, WORK_TABLE, WS, XSTRING
Constructor and Description |
---|
ProgressLexer(antlr.InputBuffer ib) |
ProgressLexer(java.io.InputStream in) |
ProgressLexer(java.io.InputStream in,
SymbolResolver sr)
Constructs a lexer using stream as input and associating a
SymbolResolver with the instance. |
ProgressLexer(java.io.InputStream in,
SymbolResolver sr,
boolean kludge)
Constructs a lexer using stream as input and associating a
SymbolResolver with the instance. |
ProgressLexer(antlr.LexerSharedInputState state) |
ProgressLexer(java.io.Reader in) |
ProgressLexer(java.io.Reader in,
SymbolResolver sr)
Constructs a lexer using a reader as input and associating a
SymbolResolver with the instance. |
ProgressLexer(java.io.Reader in,
SymbolResolver sr,
boolean kludge)
Constructs a lexer using a reader as input and associating a
SymbolResolver with the instance. |
Modifier and Type | Method and Description |
---|---|
void |
activateExportMode()
Activates export-specific processing in the lexer.
|
void |
activateHiddenProcessing()
Activates hidden stream processing in the lexer.
|
void |
activateSchemaProcessing()
Activates schema-specific processing in the lexer.
|
void |
close()
Allow this instance to cleanup any resources that were allocated.
|
static Keyword[] |
getIgnoredKeywords()
Get the keyword ignore list.
|
static Keyword[] |
getKeywords()
Get the keyword list.
|
void |
ignoreKeywords(java.lang.String[] names)
Remove each of the given keywords from the keyword dictionary.
|
private void |
initializeKeywordDictionary()
Process the keyword array to add each
Keyword into the
keyword dictionary. |
private void |
initializeSchemaKeywords()
Creates a
Keyword array and initializes that array to
a the list of all Progress schema-specific keywords. |
boolean |
isDebug()
Reports on the state of debug in the lexer.
|
boolean |
isExport()
Reports on whether export-specific processing is activated in the lexer.
|
boolean |
isHidden()
Reports on whether hidden stream processing is activated in the lexer.
|
boolean |
isRaw()
Reports on whether the lexer is currently in raw string processing
mode.
|
boolean |
isSchema()
Reports on whether schema-specific processing is activated in the lexer.
|
boolean |
isUnixEscapes()
Reports on whether the lexer is currently honoring the backslash as
an escape character.
|
static void |
main(java.lang.String[] args)
Provides a command line interface for an end user to drive and/or test
the ProgressLexer class.
|
void |
mAT(boolean _createToken)
Matches the '@' character.
|
void |
mCARET(boolean _createToken)
Matches the '^' character.
|
protected void |
mCMT_CLOSE(boolean _createToken)
Matches the close comment sequence of characters:
*/ |
protected void |
mCMT_OPEN(boolean _createToken)
Matches the open comment sequence of characters:
/* |
void |
mCOLON(boolean _createToken)
Matches the ':' character.
|
void |
mCOMMA(boolean _createToken)
Matches the ',' character.
|
void |
mCOMMENT(boolean _createToken)
Matches an entire comment, including any nested comments.
|
void |
mDB_REF_NON_STATIC(boolean _createToken)
Matches the '::' non-static database reference "operator".
|
protected void |
mDIGIT(boolean _createToken)
Matches any numeric digit:
0 - 9 . |
protected void |
mDSTRING(boolean _createToken)
Matches an opening double quote, arbitrary contents and an ending double
quote.
|
void |
mEQUALS(boolean _createToken)
Matches the '=' character.
|
void |
mGT(boolean _createToken)
Matches the '>' character.
|
void |
mGTE(boolean _createToken)
Matches the '>=' character sequence.
|
protected void |
mHEX_DIGIT(boolean _createToken)
Matches any single character that can match a hexadecimal digit.
|
protected void |
mJUNK(boolean _createToken)
Matches non-visible ASCII codes that should be silently ignored.
|
private static long[] |
mk_tokenSet_0() |
private static long[] |
mk_tokenSet_1() |
private static long[] |
mk_tokenSet_10() |
private static long[] |
mk_tokenSet_2() |
private static long[] |
mk_tokenSet_3() |
private static long[] |
mk_tokenSet_4() |
private static long[] |
mk_tokenSet_5() |
private static long[] |
mk_tokenSet_6() |
private static long[] |
mk_tokenSet_7() |
private static long[] |
mk_tokenSet_8() |
private static long[] |
mk_tokenSet_9() |
void |
mLBRACKET(boolean _createToken)
Matches the '[' character.
|
protected void |
mLETTER(boolean _createToken)
Matches any alphabetic character:
a - z . |
void |
mLPARENS(boolean _createToken)
Matches the '(' character.
|
void |
mLT(boolean _createToken)
Matches the '<' character.
|
void |
mLTE(boolean _createToken)
Matches the '<=' character sequence.
|
void |
mMULTIPLY(boolean _createToken)
Matches the '*' character.
|
void |
mNOT_EQ(boolean _createToken)
Matches the '<>' character sequence.
|
void |
mNUM_LITERAL(boolean _createToken)
Matches all forms of valid integer literals, decimal literals and due to
ambiguity, the period character (
DOT token), all date
literals, the minus sign (MINUS token), the plus sign
(PLUS ) and all FILENAME tokens that begin with
'.' or '..' must be matched here as well. |
void |
mRBRACKET(boolean _createToken)
Matches the ']' character.
|
void |
mRPARENS(boolean _createToken)
Matches the ')' character.
|
protected void |
mRSTRING(boolean _createToken)
This rule is only invoked when in raw processing mode.
|
protected void |
mSLASH_SLASH(boolean _createToken)
Matches the start of a new-style single-line Progress 4GL comment that will extend to the end
of the current line.
|
protected void |
mSSTRING(boolean _createToken)
Matches an opening single quote, arbitrary contents and an ending single
quote.
|
protected void |
mSTR_OPTIONS(boolean _createToken)
Matches the options that can be appended to a Progress string literal.
|
void |
mSTRING(boolean _createToken)
Matches various string types depending upon the processing mode:
in raw processing mode, matches a double-quoted string
encoded string which represents encoded binary (i.e., raw) data,
setting the token type to
RAW_STRING ; otherwise
in export processing mode, matches a double-quoted string
which represents a Progress string as it would appear at runtime,
except that embedded double quotes are doubled up; otherwise
matches a single or double quoted string followed by any (optional)
string options. |
protected void |
mSYM_CHAR(boolean _createToken)
Matches all characters that can be made part of a valid user-defined
symbol (except alphabetic and numeric characters):
# $ % & - _ |
void |
mSYMBOL(boolean _createToken)
Match any valid symbol name in the Progress 4GL language or the standalone
DIVIDE operator '/'. |
void |
mTILDE(boolean _createToken)
Matches any tilde character '~' (that is not in a string or comment) and sets the token type
to
skip which drops such characters from the token stream. |
void |
mUNKNOWN_TOKEN(boolean _createToken)
Matches other unexpected characters and generates a token rather than throwing an exception
(this rule must be changed when matched characters are moved to another top level rule).
|
void |
mUNKNOWN_VAL(boolean _createToken)
Matches the unknown value '?'.
|
protected void |
mVALID_SYM_CHAR(boolean _createToken)
Matches any single character that can appear in the 2nd or later position
in a symbol (matches any
LETTER, DIGIT or SYM_CHAR ). |
void |
mWS(boolean _createToken)
Matches any amount of whitespace in a program and sets the token type to
"skip".
|
protected void |
mXSTRING(boolean _createToken)
This rule is only invoked when in export processing mode.
|
antlr.Token |
nextToken() |
void |
readCfg()
Initialize configuration values.
|
void |
setDebug(boolean dbg)
Sets the state of debug in the lexer.
|
void |
setRaw(boolean raw)
Turns on/off raw string processing mode.
|
void |
setUnixEscapes(boolean unixEscapes)
Sets whether the lexer is currently honoring the backslash as
an escape character.
|
append, append, commit, consume, consumeUntil, consumeUntil, getCaseSensitive, getCaseSensitiveLiterals, getColumn, getCommitToPath, getFilename, getInputBuffer, getInputState, getLine, getTabSize, getText, getTokenObject, LA, makeToken, mark, match, match, match, matchNot, matchRange, newline, panic, panic, reportError, reportError, reportWarning, resetText, rewind, setCaseSensitive, setColumn, setCommitToPath, setFilename, setInputState, setLine, setTabSize, setText, setTokenObjectClass, tab, testLiteralsTable, testLiteralsTable, toLower, traceIn, traceIndent, traceOut, uponEOF
private static Keyword[] keywords
private static Keyword[] ignored
private boolean debug
false
.private boolean schema
false
.private boolean export
false
.private boolean hidden
false
.private boolean raw
false
.private boolean unixEscapes
private boolean allowSlashSlash
private SymbolResolver sym
public static final antlr.collections.impl.BitSet _tokenSet_0
public static final antlr.collections.impl.BitSet _tokenSet_1
public static final antlr.collections.impl.BitSet _tokenSet_2
public static final antlr.collections.impl.BitSet _tokenSet_3
public static final antlr.collections.impl.BitSet _tokenSet_4
public static final antlr.collections.impl.BitSet _tokenSet_5
public static final antlr.collections.impl.BitSet _tokenSet_6
public static final antlr.collections.impl.BitSet _tokenSet_7
public static final antlr.collections.impl.BitSet _tokenSet_8
public static final antlr.collections.impl.BitSet _tokenSet_9
public static final antlr.collections.impl.BitSet _tokenSet_10
public ProgressLexer(java.io.Reader in, SymbolResolver sr)
SymbolResolver
with the instance. The reader will be
wrapped in a DotKludgeReader
instance.in
- The reader representing the Progress 4GL source code.sr
- Provides keyword dictionary storage and lookup for symbol
resolution in the lexer.public ProgressLexer(java.io.Reader in, SymbolResolver sr, boolean kludge)
SymbolResolver
with the instance.in
- The reader representing the Progress 4GL source code.sr
- Provides keyword dictionary storage and lookup for symbol
resolution in the lexer.kludge
- true
to force the reader to be wrapped in a
DotKludgeReader
instance.public ProgressLexer(java.io.InputStream in, SymbolResolver sr)
SymbolResolver
with the instance. The stream will be
wrapped in a DotKludgeStream
instance.in
- The stream representing the Progress 4GL source code.sr
- Provides keyword dictionary storage and lookup for symbol
resolution in the lexer.public ProgressLexer(java.io.InputStream in, SymbolResolver sr, boolean kludge)
SymbolResolver
with the instance.in
- The stream representing the Progress 4GL source code.sr
- Provides keyword dictionary storage and lookup for symbol
resolution in the lexer.kludge
- true
to force the stream to be wrapped in a
DotKludgeStream
instance.public ProgressLexer(java.io.InputStream in)
public ProgressLexer(java.io.Reader in)
public ProgressLexer(antlr.InputBuffer ib)
public ProgressLexer(antlr.LexerSharedInputState state)
public void readCfg()
public static Keyword[] getKeywords()
public static Keyword[] getIgnoredKeywords()
public void ignoreKeywords(java.lang.String[] names)
names
- Keywords to remove.public boolean isDebug()
public void setDebug(boolean dbg)
dbg
- Debug state.public boolean isSchema()
public void activateSchemaProcessing()
public boolean isExport()
public void activateExportMode()
public boolean isHidden()
public void activateHiddenProcessing()
TokenStreamHiddenTokenFilter
and disables
the lexer's normal 'skip' processing of certain token types.
This is a 'one way' state change and cannot be reversed!public void setRaw(boolean raw)
raw
- Raw processing mode: true
to turn on,
false
to turn off.public boolean isRaw()
true
if in raw string mode, else
false
.public void setUnixEscapes(boolean unixEscapes)
unixEscapes
- true
if in UNIX escape mode, else
false
.public boolean isUnixEscapes()
true
if in UNIX escape mode, else
false
.public void close()
private void initializeKeywordDictionary()
Keyword
into the
keyword dictionary. This method is called during construction so that
all lexer processing can have keyword lookup facilities available
from the first token. The keyword array is a static member initialized
during class load.private void initializeSchemaKeywords()
Keyword
array and initializes that array to
a the list of all Progress schema-specific keywords. The
initialization is done in-line and the source file is organized in
an easy to maintain column format. A loop then processes this array
and adds each Keyword
into the keyword dictionary. This
method is called during construction so that all lexer processing can
have keyword lookup facilities available from the first token.
The following values are needed for each Keyword
object:
false
)
See
for more details.
Keyword
The core idea here is to populate a facility where the symbol text (in
the mSYMBOL method) can be submitted to the keyword dictionary for a
lookup. If a match is found, then the token type of the token is set
based on the token type field in the Keyword
object
found in the dictionary. This returned token type will override the
default token type of SYMBOL
. Since all of these keywords
are not encoded in the matching rules of the grammar itself, there are
no actual token definitions in the lexer that match these keywords.
For this reason, we take advantage of the feature in the parser in
which we can create (reserve) a unique integer value and associate it
with a specific artificial token name (e.g. KW_DEFINE). These
artificial token types are used as the values when constructing
Keyword
objects.
When adding Keyword
objects to this array, make sure to
add the associated artificial token type to the tokens { } section
of the parser.
public static void main(java.lang.String[] args)
Syntax:
java ProgressLexer <Progress_4GL_source_file>Where:
args
- List of command line arguments.public antlr.Token nextToken() throws antlr.TokenStreamException
nextToken
in interface antlr.TokenStream
antlr.TokenStreamException
public final void mWS(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
Spaces, tabs and carriage returns and line feeds (newlines) are all matched. Although the preprocessor generally removes carriage returns, if the input file was to include a ~015 (escaped carriage return), it would be placed in the output file. For safety, we must leave the carriage return processing in this rule to ensure that any such matches can be consumed as whitespace. Normally such escaped carriage returns would only be found inside strings, but in complex preprocessor replacements, it is very possible that a coding error could leave such characters outside a string in some cases. Progress 4GL is tolerant of such escaped carriage returns and thus our whitespace rule must also handle this condition.
Non-visible ASCII codes decimal 127 and below are matched here as well.
This is a top level lexer rule which means that there is an associated
WS
token.
If the activateHiddenProcessing()
is called, then tokens generated
using this rule will not be skipped, but instead will flow through to
the caller.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mJUNK(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mTILDE(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
skip
which drops such characters from the token stream. This is consistent
with how Progress handles such characters. This also will match a '~' followed by either
quote char. This can be seen in 4GL code (but it usually is only valid as part of a shell
command line). If this construct is matched the token type is UNKNOWN_TOKEN
.
This is a top level lexer rule which means that there is an associated TILDE
token, however these will never be seen by the parser.
If the activateHiddenProcessing()
is called, then tokens generated using this rule
will not be skipped, but instead will flow through to the caller.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mCOMMENT(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
The lexer's internal line counter is updated whenever a newline character is encountered.
Of great importance is the fact that the closure subrule for the contents of a comment must not operate on a greedy basis. Being greedy means that the loop will consume all possible matches to its rule without regard to the end of loop condition. This is the default way that the lexer works, but since this loop can match any character (we use the wildcard character because any character can be embedded in a comment), a greedy loop would not ever be able to detect its end condition. Instead we turn off greedy mode and this makes the loop watch for the matching comment close sequence of characters. Note that nested comments are possible by recursively referencing this same rule inside the comment contents subrule.
Whitespace is maintained inside the resulting comment text and the lexer's line counter is properly incremented upon encountering a newline.
This is a top level lexer rule which means that there is an associated
COMMENT
token.
If the activateHiddenProcessing()
is called, then tokens generated
using this rule will not be skipped, but instead will flow through to
the caller.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mCMT_OPEN(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
/*
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mCMT_CLOSE(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
*/
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mSTRING(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
RAW_STRING
; otherwise
This is a top level lexer rule which means that there is an associated
STRING
token.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mRSTRING(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
Matches an opening double quote, arbitrary contents and an ending double quote. Opening and closing double quotes are discarded.
Currently, no assumptions are made about the contents of the string, except that it will not contain an instance of a double quote, since this is the end delimiter. All other data is accepted exactly as is. Currently, no decoding is performed.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mXSTRING(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
Matches an opening double quote, arbitrary contents and an ending double quote. Two contiguous double quote characters are accepted as contents (they do not terminate the string), but they are collapsed into a single quote character. Opening and closing double quotes are discarded.
Any newlines inside the string are identified and the lexer's internal newline counter is properly maintained.
Tabs and spaces and all non-printing characters are maintained inside strings (including embedded binary control characters).
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mSSTRING(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
Any newlines inside the string are identified and the lexer's internal newline counter is properly maintained. Note that all such newlines are maintained in the output string because these are the escaped chars that have been left behind by the preprocessor. If such characters are not escaped, then the preprocessor removes such chars (carriage returns and line feeds). This is how Progress 4GL handles these characters. This means that in a raw source file that has not been preprocessed, a string literal can be split across any number of lines and the Progress preprocessor will put the string back together, ignoring the carriage returns and newlines.
Tilde and the backslash can BOTH be Progress escape characters and as such, they need special attention. Backslash is only honored if UNIX escapes mode is set on. In particular, this rule must separately match 4 or 8 constructs as part of a string depending on which escape sequences are honored.
~' '~' (probably a bug in the 4GL) \' '\' (probably a bug in the 4GL) ~~ ~\ \\ \~ ~ \
The first 4 are a way of embedding a quote in a string. The next 4 are important because if this rule were to encounter '~~', '\\', '~\' or '\~' (which are valid strings) the rule would consume the first escape and then encounter an escaped quote which would not terminate the string properly, leading to a non-ending string. So this rule matches on any escaped escape char to eliminate this situation. Finally, a single ~ or \ that is not followed by a " or a duplicate escape char is matched as a single character. This is required (one may think that the closure rule should handle this case) because when ANTLR sees a ~ or \ in the leftmost position of the alternatives, it DOES NOT include it in the list of 'everything that is not a closing quote'. Likewise \n is not included. Of course, these tilde constructions must be placed in a specific order and if so, the ambiguity warnings that ANTLR reports can be disabled.
The greedy option does not need to be disabled here as the closure rule termination is built into the subrule itself: it accepts anything that isn't an unescaped single quote character (see above).
Tabs and spaces are maintained inside strings.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mDSTRING(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
Any newlines inside the string are identified and the lexer's internal newline counter is properly maintained. Note that all such newlines are maintained in the output string because these are the escaped chars that have been left behind by the preprocessor. If such characters are not escaped, then the preprocessor removes such chars (carriage returns and line feeds). This is how Progress 4GL handles these characters. This means that in a raw source file that has not been preprocessed, a string literal can be split across any number of lines and the Progress preprocessor will put the string back together, ignoring the carriage returns and newlines.
Tilde and the backslash can BOTH be Progress escape characters and as such, they need special attention. Backslash is only honored if UNIX escapes mode is set on. In particular, this rule must separately match 4 or 8 constructs as part of a string depending on which escape sequences are honored.
~" "~" (probably a bug in the 4GL) \" "\" (probably a bug in the 4GL) ~~ ~\ \\ \~ ~ \
The first 4 are a way of embedding a quote in a string. The next 4 are important because if this rule were to encounter '~~', '\\', '~\' or '\~' (which are valid strings) the rule would consume the first escape and then encounter an escaped quote which would not terminate the string properly, leading to a non-ending string. So this rule matches on any escaped escape char to eliminate this situation. Finally, a double ~ or \ that is not followed by a " or a duplicate escape char is matched as a double character. This is required (one may think that the closure rule should handle this case) because when ANTLR sees a ~ or \ in the leftmost position of the alternatives, it DOES NOT include it in the list of 'everything that is not a closing quote'. Likewise \n is not included. Of course, these tilde constructions must be placed in a specific order and if so, the ambiguity warnings that ANTLR reports can be disabled.
The greedy option does not need to be disabled here as the closure rule termination is built into the subrule itself: it accepts anything that isn't an unescaped double quote character (see above).
Tabs and spaces are maintained inside strings.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mSTR_OPTIONS(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
justification 'r' | 'l' | 'c' | 't' translatable 'u' | 'x' size integer literal (one or more contiguous digits)
The use of 'X' is an undocumented feature of the 4GL and it can only appear if 'U' is not present. The meaning of this is not yet known.
These options can come in any order and at least one of them must appear. This causes some problems since there is no simple syntax to specify such conditions in the grammar. Whitespace cannot be placed between any of the options.
In addition, just as the lookahead for the mDEC_LITERAL
rule
could incorrectly 'pull' processing down into that rule as a false match,
so too can this happen here. For this reason, the possible alternatives
are specified and then the empty alternative is used to detect a false
match and the lexer's stream input is manually reset to backup and restart
processing at the top-level.
None of this would be possible if comments or whitespace were allowed in string options. They are not.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mSYMBOL(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
DIVIDE
operator '/'. This includes the following:
SKIP
pathname<<membername>>
All symbols must start with an alphabetic character. Starting in the
second character and continuing to a maximum of 32 characters (including
the initial character), can be any alphabetic and numeric characters or
one of the special symbol characters ( # $ % & - _
).
Symbols are matched case-insensitively.
An exception to the previous rule that a symbol must start with an alphabetic character is for special meta-data entries in the Progress schema dictionary, whose table and field names start with an underscore. For this reason, such entries are allowed more generally such that the rule as implemented is: the first letter of every symbol must be an alphabetic character or an underscore.
Another exception to the previous rule that a symbol must start with an alphabetic character is for a filename. Filenames may also start with the '/' or '\' characters (in which case they are absolute filenames). The second and following characters in filenames can include '/', '\' or '.' in addition to all the special symbol characters listed above. Filenames may also include a "naked" embedded double quote or single quote character so long as there is no intervening whitespace. These embedded quotes cannot be the leading character but can appear deeper in and they are not considered the start of a string. These can be referenced in RUN statements and as procedure names (in an internal definition or as an external filename).
In addition, DOS and Windows filenames can start with a drive letter
followed by a ':' as in 'c:'. This construct is matched as a symbol
followed by a colon. This is needed to allow single character variable
names + attribute or method calls to properly work in the parser. The
difficulty with putting the whole thing together again into a single
filename is left to the ProgressParser.filename(int[])
rule.
Note that there are many characters that are not allowed in filenames
at this time. This is an artificial limitation designed to keep the
current lexer simple and compatible with the known/common cases of
filenames while not introducing too many potential ambiguities.
Examples of characters that are not matched include: '~', '!', '@', '^',
'+', '='. The general rule: if the character is not a valid symbol char,
'/', '\', '.' or '..' then it is not matched. This can be changed if
necessary, but the decision was made to defer such handling until the
need arises. When a filename is detected the token type is set to
FILENAME
. Both relative and absolute filenames are matched
by this rule (except for the drive letter component as noted above).
Since the '/' character is matched in this rule, an inherent ambiguity
was created with any top level DIVIDE
rule that matches '/'.
For this reason, the ambiguity was resolved by matching on a standalone
'/' in this rule and setting the resulting token type to
DIVIDE
. The top level DIVIDE
rule was
removed.
The '\' is a special case in Progress. This character can be inserted
anywhere and it is simply ignored. This feature is supported in this
rule as well, but matching a standalone backslash and setting it's type
to SKIP
. Note that a backslash that is part of a larger
filename, is NOT skipped.
To avoid lexer non-determinism, any filename that starts with './', '../',
'.\' or '..\' is actually matched at the top level by
mNUM_LITERAL(boolean)
. This is needed because the DOT
in the
first position conflicts. So instead, once mNUM_LITERAL
matches one of these strings, it includes this rule optionally. This
allows all normal filename processing to occur as needed but doesn't
require it since the previous string can legitimately stand alone.
R-code library references are matched with the format
pathname<<membername>>
. This allows any filename
to be specified as a prefix and then the library member name followed
as a suffix, but contained in the unambiguous double angle brackets.
When this construction is matched, the token type is set to
LIBRARY_REF
.
A special case exists for partially or fully qualified database symbols.
Such symbols are comprised of 2 or 3 regular symbols, each separated
by a '.' and no whitespace. This method recognizes such optional
constructions and sets the token type to DB_SYMBOL
. In
order to make this extra check work, the optional section (following the
first or main symbol definition) is differentiated using a semantic
predicate. This only allows the database symbol matching if the following
conditions hold:
DOT
character is found after the symbol without
any intervening whitespace
DOT
is an alphabetic
character (it can't be numeric or a special symbol because each
part of the qualified database symbol name must follow the
standard rules (see above) for symbol naming
There is a second optional section nested inside the first one, which handles the condition where there is a fully qualified field name. The resulting possible constructions are:
user-defined symbol or unqualified db/table/field name --> sym fully qualified table or partially qualified field name --> sym.sym fully qualified field name --> sym.sym.sym
Any unqualified database, table or field names will be lexed as a standard symbol (which is the default token type). But in any case where a simple symbol is followed by a '.' AND an alphabetic character with no intervening whitespace, this is known to be a qualified database name and it cannot be a generic symbol or language keyword. If there is ever a testcase found that does not follow this rule, this method will break, badly!
It is also important to note that any relative filename that does not
include any path separator ('/' or '\') and which does not have any more
than 2 '.' characters will be matched as a DB_SYMBOL
due to
inherent ambiguity in the Progress decision to use the '.' as the
database qualifier. This means that the parser must be aware that some
filenames can appear as database symbols.
Most importantly, once a symbol has been found, a keyword lookup occurs
and the token type is overridden from the default (SYMBOL
)
to the artificial token type associated with the keyword (see
initializeKeywordDictionary
). The parser may subsequently
override this token type yet again, but this is the extent of the lexer's
symbol resolution. Keyword token types take precedence over the
default token type SYMBOL
, however only SYMBOL
types are overridden since there are no circumstances in which there are
language keywords that can match database symbols, filenames or library
references. All
DIVIDE, DB_SYMBOL, FILENAME and LIBRARY_REF
token types
bypass the keyword lookup and override.
See the overview for a discussion of how and why the symbol resolution
in this method is limited to keywords and the
SYMBOL, DIVIDE, DB_SYMBOL, FILENAME and LIBRARY_REF
token
types.
This is a top level lexer rule which means that there is an associated
SYMBOL
token. The
DIVIDE, DB_SYMBOL, FILENAME and LIBRARY_REF
tokens are
artificially created in the parser so that they can be manually set
here as an override.
Standalone backslashes are set to skip
by default. However,
if the activateHiddenProcessing()
is called, then tokens generated
using this rule will not be skipped, but instead will flow through to
the caller.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
Keyword
,
KeywordDictionary
,
SymbolResolver
protected final void mVALID_SYM_CHAR(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
LETTER, DIGIT or SYM_CHAR
). This is
simply a helper rule to make references easier.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mSLASH_SLASH(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mLETTER(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
a - z
.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mCOLON(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
COLON
token type.
The hidden token stream can be used to determine whether the colon was followed by whitespace or not. This is needed in the parser to differentiate between labels (which must always be followed by whitespace) and attributes/methods where the colon cannot be followed by whitespace.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mDB_REF_NON_STATIC(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
DB_REF_NON_STATIC
token type.
The hidden token stream can be used to determine whether the colon was followed by whitespace or not. This is needed in the parser to differentiate between labels (which must always be followed by whitespace) and attributes/methods where the colon cannot be followed by whitespace.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mCOMMA(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
COMMA
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mEQUALS(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
EQUALS
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mNOT_EQ(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
NOT_EQ
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mGT(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
GT
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mLT(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
LT
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mGTE(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
GTE
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mLTE(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
LTE
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mLPARENS(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
LPARENS
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mRPARENS(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
RPARENS
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mLBRACKET(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
LBRACKET
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mRBRACKET(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
RBRACKET
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mMULTIPLY(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
MULTIPLY
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mAT(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
AT
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mCARET(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
CARET
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mUNKNOWN_VAL(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
UNKNOWN_VAL
token type.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mNUM_LITERAL(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
DOT
token), all date
literals, the minus sign (MINUS
token), the plus sign
(PLUS
) and all FILENAME
tokens that begin with
'.' or '..' must be matched here as well.
This is a very special method that encapsulates and resolves all ambiguity caused by the different usage of '.' as a decimal point and also as a statement separator. Likewise it handles the ambiguity presented by the traditional minus/plus operators and the prefixing/postfixing of a negative or positive sign on a numeric or decimal literal. Finally, it also handles the ambiguity associated with the use of '-' as a separator in a date literal.
Scenarios that are matched:
DOT
token type.
FILENAME
token
type (the following mSYMBOL(boolean)
is optional) :
mSYMBOL
mSYMBOL
mSYMBOL
mSYMBOL
MINUS
token type
(if this is not followed by digits or a decimal point and digits).
NUM_LITERAL
.
DEC_LITERAL
.
DEC_LITERAL
token type.
PLUS
token type
(if this is not followed by digits or a decimal point and digits).
NUM_LITERAL
. The '+' is discarded as syntactic sugar.
DEC_LITERAL
. The '+' is discarded as
syntactic sugar.
DEC_LITERAL
token type. The '+' is discarded as
syntactic sugar.
NUM_LITERAL
token type. There can be an optional
leading negative/positive sign (the cases noted above). If no
leading sign is present, then an optional postfixed sign (negative
or positive) can be present.
NUM_LITERAL
token type. There can be an optional leading negative/positive
sign (the cases noted above).
DEC_LITERAL
token type. There can be an
optional leading negative/positive sign (the cases noted above).
If no leading sign is present, then an optional postfixed sign
(negative or positive) can be present.
DEC_LITERAL
token type.
There can be an optional leading negative/positive sign (the cases
noted above). If no leading sign is present, then an optional
postfixed sign (negative or positive) can be present.
DATE_LITERAL
. If there are no following
digits after the separator, then it is not matched as a date,
instead it is a integer literal followed by an operator. The
second separator for the year portion is optional, as is the year
itself and there is an optional negative sign for BC dates. The
second separator can also be any of the '-', '/' or '.' characters
and the first and second separators can be any combination of
these characters (the two separators don't have to be the same
character in a single date literal).
The key to this rule is that there cannot be any intervening whitespace
between digits and decimal points or between digits and the date separators
'-' or '/'. Likewise, there cannot be any intervening whitespace between
a leading or trailing '-' or '+' and the numeric or decimal literal it
precedes or follows respectively. In such a case the '-' is lexed as a
MINUS
token, the '+' is lexed as a PLUS
token
and the parser must use context to determine if it is a unary or binary
operator.
By handling the case of '.' (by itself) here instead of in a (more intuitive) dedicated top level rule, the ambiguity between decimal literals and the standalone dot can be properly managed. This is also true of the '-' processing which would be easier and more intuitive in a top level rule, but which needs to be here to resolve ambiguity.
In the case of a decimal with no digits on the right side of the decimal
point, this is always interpreted as an integer literal AND a subsequent
2nd token of a DOT
. This oddity cannot be easily resolved
BUT interestingly enough this is exactly how Progress itself
handles this situation AND this is a perfectly valid match since it
essentially is an integer.
Date literals are generally matched in mm/dd/yyyy or mm-dd-yyyy formats, although the '-' and '/' separators can even be mixed in the same literal (mm-dd/yyyy or mm/dd-yyyy). Both months and days can be specified in 1 or 2 digits (actually this rule matches any number of digits for months). Either separator character can be used, although in Progress one cannot use the '-' between the mm and dd if the constant is being used in an expression. This lexer has no such limitation. In Progress the month must be an integer between 1 and 12 but this limit is not implemented here. In Progress the day must be between 1 and the number of days in that month, though this limit is not implemented here. The second separator and the year digits are optional. There can be an optional negative sign preceding any year digits and one can specify a dd and yyyy separator without specifying any year digits at all. Any number of year digits are valid, though in Progress the year 0 is treated specially, as follows:
So y, yy, yyy, yyyy, yyyyy... and -y, -yy, -yyy, -yyyy... all are valid year specifications, except for the specific year zero issue noted above. These limitations on how the year digits work are not implemented here. Over all, this rule supports a superset of the possible date literal formats in Progress and no checking of the values is done.
WARNING: at this time there is no support for a quirk of Progress date literals in which constructs like 1/1--- are valid dates equivalent to 01/01/00.
This is a top level rule. The token type defaults to
NUM_LITERAL
and is overridden by specific actions depending
on the scenario matched.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mDIGIT(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
0 - 9
.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mHEX_DIGIT(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
LETTER
and DIGIT
rules. This is simply a helper rule to make
references easier.antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
public final void mUNKNOWN_TOKEN(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
'`' '!' '#' '$' '%' '&' '{' '}' '|' ';' '' through '' (all UNICODE characters starting in the extended ASCII range and above)
Anything that is not matched here will still trigger an exception. Note that ANTLR's filter
feature seems to be a better approach to resolve this problem (of wanting to generate an
UNKNOWN_TOKEN
rather than throw an exception), however it can only handle the
consumption of such characters. The filter rule that is called when there is no top level
match cannot generate tokens because of the way that nextToken
is generated by
ANTLR. Such filters are not allowed to generate a token so this less-optimal approach of
specifying the special characters in advance is the result.
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
protected final void mSYM_CHAR(boolean _createToken) throws antlr.RecognitionException, antlr.CharStreamException, antlr.TokenStreamException
# $ % & - _
antlr.RecognitionException
antlr.CharStreamException
antlr.TokenStreamException
private static final long[] mk_tokenSet_0()
private static final long[] mk_tokenSet_1()
private static final long[] mk_tokenSet_2()
private static final long[] mk_tokenSet_3()
private static final long[] mk_tokenSet_4()
private static final long[] mk_tokenSet_5()
private static final long[] mk_tokenSet_6()
private static final long[] mk_tokenSet_7()
private static final long[] mk_tokenSet_8()
private static final long[] mk_tokenSet_9()
private static final long[] mk_tokenSet_10()