com.goldencode.p2j.preproc

Class BracesLexer

java.lang.Object

antlr.CharScanner

com.goldencode.p2j.preproc.BracesLexer

All Implemented Interfaces:

antlr.TokenStream, PreprocTokenTypes

public class BracesLexer
extends antlr.CharScanner
implements PreprocTokenTypes, antlr.TokenStream

Tokenizes the input stream of characters from the portion of Progress source file between braces and returns tokens to the caller according to the needs of preprocessor.

There is only a single context for this lexer as no comments are honored inside the braces and strings are treated in a special way: only double quoted strings are recognized, they can be adjacent to text and even unclosed.

See Also:

BracesParser

Field Summary

Fields

Modifier and Type	Field and Description
static antlr.collections.impl.BitSet	_tokenSet_0
static antlr.collections.impl.BitSet	_tokenSet_1
static antlr.collections.impl.BitSet	_tokenSet_2
private Environment	env keeps the reference to the shared environment
private boolean	isUnixEscapes Flag to denote if the backslash is honored as an escape sequence.

Fields inherited from class antlr.CharScanner

_returnToken, caseSensitive, caseSensitiveLiterals, commitToPath, EOF_CHAR, hashString, inputState, literals, saveConsumedInput, tabsize, text, tokenObjectClass, traceDepth

Fields inherited from interface com.goldencode.p2j.preproc.PreprocTokenTypes

AELSE, AELSEIF, AENDIF, AGLOBAL, AIF, ALPHA, AMESSAGE, AMPER, APOST, ARESUME, ASCOPED, ASTMT, ASTRING, ASUSPEND, ATHEN, AUNDEFINE, BREAK_CODE_CHUNKS, CODE, COMM_CLOSE, COMM_OPEN, COMMENT, DIGIT, DIGITS, EOF, EQUALS, LBRACE, NL, NULL_TREE_LOOKAHEAD, PPNAME, QSTRING, QUOTE, RBRACE, SLASH_SLASH, SPECIAL, STAR, STAR_COMMENT, STRING, TAB, WS, XAPOST, XQUOTE

Constructor Summary

Constructors

Constructor and Description
BracesLexer(Environment env) Constructor.
BracesLexer(antlr.InputBuffer ib)
BracesLexer(java.io.InputStream in)
BracesLexer(antlr.LexerSharedInputState state)
BracesLexer(java.io.Reader in)

Method Summary

Modifier and Type	Method and Description
void	mAMPER(boolean _createToken) Matches a single ampersand character.
void	mCODE(boolean _createToken) Matches a preprocessor input that has no other interpretation.
protected void	mDIGIT(boolean _createToken) Matches a single decimal digit on input.
void	mDIGITS(boolean _createToken) Matches any number of successive decimal digits on input.
void	mEQUALS(boolean _createToken) Matches a single equal character.
private static long[]	mk_tokenSet_0()
private static long[]	mk_tokenSet_1()
private static long[]	mk_tokenSet_2()
void	mQUOTE(boolean _createToken) Matches a single quote (not apostrophe) character.
void	mSTAR(boolean _createToken) Matches a single asterisk character.
void	mWS(boolean _createToken) Matches any amount of whitespace on input.
antlr.Token	nextToken()

Methods inherited from class antlr.CharScanner

append, append, commit, consume, consumeUntil, consumeUntil, getCaseSensitive, getCaseSensitiveLiterals, getColumn, getCommitToPath, getFilename, getInputBuffer, getInputState, getLine, getTabSize, getText, getTokenObject, LA, makeToken, mark, match, match, match, matchNot, matchRange, newline, panic, panic, reportError, reportError, reportWarning, resetText, rewind, setCaseSensitive, setColumn, setCommitToPath, setFilename, setInputState, setLine, setTabSize, setText, setTokenObjectClass, tab, testLiteralsTable, testLiteralsTable, toLower, traceIn, traceIndent, traceOut, uponEOF

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

env

private Environment env

keeps the reference to the shared environment

isUnixEscapes

private boolean isUnixEscapes

Flag to denote if the backslash is honored as an escape sequence.

_tokenSet_0

public static final antlr.collections.impl.BitSet _tokenSet_0

_tokenSet_1

public static final antlr.collections.impl.BitSet _tokenSet_1

_tokenSet_2

public static final antlr.collections.impl.BitSet _tokenSet_2

Constructor Detail

BracesLexer

public BracesLexer(Environment env)

Constructor. Creates a lexer attached to the input stream taken from the environment. Saves the environment for future needs.

Parameters:

env - Shared preprocessor environment.

BracesLexer

public BracesLexer(java.io.InputStream in)

BracesLexer

public BracesLexer(java.io.Reader in)

BracesLexer

public BracesLexer(antlr.InputBuffer ib)

BracesLexer

public BracesLexer(antlr.LexerSharedInputState state)

Method Detail

nextToken

public antlr.Token nextToken()
                      throws antlr.TokenStreamException

Specified by:

nextToken in interface antlr.TokenStream

Throws:

antlr.TokenStreamException

mWS

public final void mWS(boolean _createToken)
               throws antlr.RecognitionException,
                      antlr.CharStreamException,
                      antlr.TokenStreamException

Matches any amount of whitespace on input.

Spaces, tabs and newlines are all matched. This is a top level lexer rule which means that there is an associated WS token.

Throws:

antlr.RecognitionException

antlr.CharStreamException

antlr.TokenStreamException

mEQUALS

public final void mEQUALS(boolean _createToken)
                   throws antlr.RecognitionException,
                          antlr.CharStreamException,
                          antlr.TokenStreamException

Matches a single equal character.

This is a top level lexer rule which means that there is an associated EQUALS token.

Throws:

antlr.RecognitionException

antlr.CharStreamException

antlr.TokenStreamException

mSTAR

public final void mSTAR(boolean _createToken)
                 throws antlr.RecognitionException,
                        antlr.CharStreamException,
                        antlr.TokenStreamException

Matches a single asterisk character.

This is a top level lexer rule which means that there is an associated STAR token.

Throws:

antlr.RecognitionException

antlr.CharStreamException

antlr.TokenStreamException

mAMPER

public final void mAMPER(boolean _createToken)
                  throws antlr.RecognitionException,
                         antlr.CharStreamException,
                         antlr.TokenStreamException

Matches a single ampersand character.

This is a top level lexer rule which means that there is an associated AMPER token.

Throws:

antlr.RecognitionException

antlr.CharStreamException

antlr.TokenStreamException

mQUOTE

public final void mQUOTE(boolean _createToken)
                  throws antlr.RecognitionException,
                         antlr.CharStreamException,
                         antlr.TokenStreamException

Matches a single quote (not apostrophe) character.

This is a top level lexer rule which means that there is an associated QUOTE token.

Throws:

antlr.RecognitionException

antlr.CharStreamException

antlr.TokenStreamException

mDIGITS

public final void mDIGITS(boolean _createToken)
                   throws antlr.RecognitionException,
                          antlr.CharStreamException,
                          antlr.TokenStreamException

Matches any number of successive decimal digits on input.

This is a top level lexer rule which means that there is an associated DIGITS token.

Throws:

antlr.RecognitionException

antlr.CharStreamException

antlr.TokenStreamException

mDIGIT

protected final void mDIGIT(boolean _createToken)
                     throws antlr.RecognitionException,
                            antlr.CharStreamException,
                            antlr.TokenStreamException

Matches a single decimal digit on input.

Throws:

antlr.RecognitionException

antlr.CharStreamException

antlr.TokenStreamException

mCODE

public final void mCODE(boolean _createToken)
                 throws antlr.RecognitionException,
                        antlr.CharStreamException,
                        antlr.TokenStreamException

Matches a preprocessor input that has no other interpretation. Due to the fact that escaped '}' characters must be ignored here, this rule must include matching for the RBRACE token. Note that the ClearStream implementation returns the escape character instead of "eating" it when it is parsing braces. The escape character is either tilde or backslash (but backslash is only honored if UNIX escapes are honored) and must be detected here to differentiate between text and the RBRACE. If an escaped right brace is matched, note that the escape character is dropped unless we are parsing inside a string literal (further up the stack).

This is a top level lexer rule which means that there is an associated CODE token.

Throws:

antlr.RecognitionException

antlr.CharStreamException

antlr.TokenStreamException

mk_tokenSet_0

private static final long[] mk_tokenSet_0()

mk_tokenSet_1

private static final long[] mk_tokenSet_1()

mk_tokenSet_2

private static final long[] mk_tokenSet_2()