ClearStream.java
/*
** Module : ClearStream.java
** Abstract : Implements an intelligent input stream for the Preprocessor
** which handles alternative codings, various escape sequences
** with tildes and the {} constructs. The reason it takes a
** stream implementation is that all of the above may happen
** in the middle of token creation. To complete the token,
** the input stream must handle those transparently and produce
** a 'clear' input.
**
** Copyright (c) 2004-2025, Golden Code Development Corporation.
**
** -#- -I- --Date-- --JPRM-- ----------------------------Description-----------------------------
** 001 NVS 20041112 @18759 Created. A highly dynamic nature of the pre-
** processing with rescanning, which is the case
** with Progress, makes it a problematic task
** for the ANTLR tool. This class hides most of
** the dynamics and presents just another input
** stream that ANTLR can easily handle.
** 002 NVS 20041220 @19031 Original newlines in strings have to be ign-
** ored, if -keeptildes option is turned off
** 003 NVS 20041222 @19049 By Progress specification, include files are
** followed by an extra space character. The
** problem is at the top level. The main entry
** point creates a virtual include file from the
** command line. This include does not have to
** be followed by a space. mread() method is
** modified to check if the EOF is for the
** virtual (top level) include and if so to
** bypass extra space.
** 004 NVS 20041222 @19065 Reworked #002 to account for escaped newlines
** inString signal from the lexer to the
** ClearStream is always late dute to the look-
** ahead (k=2) in the lexer. Conversion of new-
** lines inside strings is reworked to use
** escape characters inserted into the stream.
** CR is used to mark all converted CRs and NLs.
** Original CRs are always removed from the
** stream. Original NLs are always removed from
** strings. Late signalling makes it possible
** for the escapes that immediately follow the
** string like in "..."~n to produce "..." CR NL
** although this normally shouldn't happen. The
** lexer checks to see if the lookahead buffer
** LA(1) is CR after matching the closing chara-
** cter of the string and consumes it fixing the
** problem.
** 005 NVS 20050207 @19630 ClearStream creates another level of Environ-
** ment when braces are encountered. This new
** support requires that the arguments Map mem-
** ber is copied from the second level Environ-
** ment object to the first level when the par-
** sing is done.
** 006 NVS 20050210 @19755 Fixed bug in reading "~{...~}" which used to
** leave ClearStream in an unbalanced state of
** watchEOF due to wrong handling of "~}". Also
** deleted adjustLineColumn method (obsolete)
** and enhanced debugging by adding dumpState
** method and some output around {include}.
** 007 NVS 20050211 @19782 Fixed a bug in preprocessor related to the
** top level include file handling. The name of
** the virtual file in the FileScope constructor
** call has changed to "/". This signifies a
** name than NEVER can be a real filename.
** ClearStream checks the name to see if the top
** level scope is for this virtual name and then
** bypasses appending an extra space character,
** which is otherwise required.
** 008 NVS 20050215 @19802 Added support to a newly discovered Progress
** "feature": the combination of '\' and NL is
** interpreted as yet another line continuation.
** It works inside and outside of strings, in
** comments and in braces. The combination of
** '~', '\' and NL, however, leaves everything
** inplace.
** 009 NVS 20050421 @20832 The original logic that tries to keep the
** preprocessor output as close as possible to
** that of the Progress, now has changed to meet
** the needs of code conversion. The contents of
** strings with regular escapes are now preser-
** ved. On output, those escapes appear without
** substitution, making the output string the
** exact equivalent of input from the code con-
** version point of view.
** 010 NVS 20050509 @21133 In Progress, tildes and backslashes may be
** used interchangeably. Fixed the algorithm to
** handle them equally and to preserve the ori-
** ginal characters while in strings.
** 011 NVS 20050511 @21149 ClearStream now inserts markers into the
** input stream. Markers are special sequences
** that provide some useful information from the
** input stream (braces as references and inclu-
** ded files) synchronously to the output stream
** where they are filtered out. This technique
** preserves the relative position of events
** taking place on input, on output.
** 012 NVS 20050516 @21213 Fixing marker handling bug. Markers now inc-
** lude a trailing space. See ##21210-21212 for
** details.
** 013 NVS 20050518 @21236 Simplified the marker handling.
** 014 NVS 20050519 @21239 Include file hint gets the full filename.
** 015 GES 20070403 @32709 Provide control over honoring backslash as
** an escape sequence. Progress only does this
** on UNIX, not on Windows.
** 016 GES 20071204 @36192 Refactored to improve readability. Added
** extra marker deferral mechanism for braces
** expansion. Allow the braces parser to handle
** the matching of open and closing braces
** instead of trying to simulate it using state
** that is incomplete and/or inaccurate. This
** last change also led to a much cleaner design
** for braces processing.
** 017 GES 20080308 @37645 Switched to using character readers instead
** of byte streams. This handles I18N needs.
** 018 GES 20081010 @40333 Switched base class to ReversibleReader
** which allows a more flexible method of
** pushback. This allowed a flaw with missing
** braces pushback to be fixed.
** 019 GES 20100225 @44691 ~E was improperly translated to 0x11 when it was
** supposed to be 0x1B which is ESC.
** 020 GES 20110628 Minor signature change and some comment cleanup.
** 021 GES 20110901 Added warning message destination support.
** 022 CA 20140313 The IncludeHint needs to use the project-relative name, not the
** absolute name.
** 023 ECF 20150715 Replace StringBuffer with StringBuilder.
** 024 EVL 20160223 Javadoc fixes to make compatible with Oracle Java 8 for Solaris 10.
** 025 CA 20180731 Save the symbols in the associated .pphints, for each nested include
** reference.
** 026 CA 20190529 Do not expand the escaped chars if they are in comments.
** 027 GES 20190930 When outside of a string, pass any escape chars through when they
** precede a single or double quote char. This is how the 4GL does
** it. These constructs are only valid in some cases like filenames
** or command line text (e.g. UNIX stmt).
** RFB 20191001 027 change needed to validate that we are *not* inside braces before
** passing an escaped quote.
** 028 GES 20200625 Argument markers and the end of include file markers can be out of balance
** with the start of include file markers. This caused an abend. The conditions
** are now matched so the results are correct and the abend is avoided.
** 029 GES 20220321 Implemented a quirk in defines where ~" is returned as " (the tilde gets
** silently consumed).
** CA 20220418 Each top-file has a local preprocessor variable scope, beside the global scope.
** TJD 20220429 The last two variable scope levels need to be merged up before EOF,
** as other components rely on it
** TJD 20220504 Java 11 compatibility minor changes
** CA 20221129 Do not process unix escapes found in comments.
** 030 GBB 20230512 Logging methods replaced by CentralLogger/ConversionStatus.
** 031 AL2 20240222 Added support for unicode characters (~uXXXX).
** 032 CA 20241028 Attempt to fix trailing '~' outside of preprocessor or strings - they are dropped
** which is not how OE behaves (it preserves them).
** 033 AOG 20250318 Incremented 'clearCount' instead of resetting it to 1 when 'unread()' is invoked.
*/
/*
** This program is free software: you can redistribute it and/or modify
** it under the terms of the GNU Affero General Public License as
** published by the Free Software Foundation, either version 3 of the
** License, or (at your option) any later version.
**
** This program is distributed in the hope that it will be useful,
** but WITHOUT ANY WARRANTY; without even the implied warranty of
** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
** GNU Affero General Public License for more details.
**
** You may find a copy of the GNU Affero GPL version 3 at the following
** location: https://www.gnu.org/licenses/agpl-3.0.en.html
**
** Additional terms under GNU Affero GPL version 3 section 7:
**
** Under Section 7 of the GNU Affero GPL version 3, the following additional
** terms apply to the works covered under the License. These additional terms
** are non-permissive additional terms allowed under Section 7 of the GNU
** Affero GPL version 3 and may not be removed by you.
**
** 0. Attribution Requirement.
**
** You must preserve all legal notices or author attributions in the covered
** work or Appropriate Legal Notices displayed by works containing the covered
** work. You may not remove from the covered work any author or developer
** credit already included within the covered work.
**
** 1. No License To Use Trademarks.
**
** This license does not grant any license or rights to use the trademarks
** Golden Code, FWD, any Golden Code or FWD logo, or any other trademarks
** of Golden Code Development Corporation. You are not authorized to use the
** name Golden Code, FWD, or the names of any author or contributor, for
** publicity purposes without written authorization.
**
** 2. No Misrepresentation of Affiliation.
**
** You may not represent yourself as Golden Code Development Corporation or FWD.
**
** You may not represent yourself for publicity purposes as associated with
** Golden Code Development Corporation, FWD, or any author or contributor to
** the covered work, without written authorization.
**
** 3. No Misrepresentation of Source or Origin.
**
** You may not represent the covered work as solely your work. All modified
** versions of the covered work must be marked in a reasonable way to make it
** clear that the modified work is not originating from Golden Code Development
** Corporation or FWD. All modified versions must contain the notices of
** attribution required in this license.
*/
package com.goldencode.p2j.preproc;
import com.goldencode.p2j.convert.*;
import com.goldencode.p2j.util.*;
import com.goldencode.p2j.io.*;
import javax.xml.parsers.ParserConfigurationException;
import java.io.*;
import java.util.*;
import java.util.logging.*;
/**
* Implements an intelligent input stream for the Preprocessor
* which handles alternative codings, various escape sequences
* with tildes and the {} constructs internally. Thus, no dynamic
* input is ever seen in the clients of this class, like the ANTLR
* generated lexers.
* <p>
* The reason it takes a stream implementation is that all of the above may
* happen in the middle of token creation by the lexer. To complete the
* token, the input stream must handle those transformations transparently
* and produce a 'clear' input.
* <p>
* This class extends the PushbackInputStream. The latter provides
* functionality needed for input stream mutations and rescanning of
* substitutions.
* <p>
* This is what this class does to the underlying input stream:
* <ul>
* <li>transforms all variations of new line separation into a single
* predictable NL character '\n', so the lexers and parsers don't have
* to account for every possible valid line separation sequence;
* <li>counts lines and updates line counts for the lexers;
* <li>marks converted escapes within strings for CR and NL with another
* CR, which helps the lexer differentiate them later from the original
* ones;
* <li>translates the so called alternative codings into their regular
* characters; these are ';&', ';≤', ';≥', ';*', ';\', ';(', ';%', ';)'
* and ';?' meaning '@', '[', ']', '^', '\', '{', '|', '}' and '~'
* respectively;
* <li>arranges for the parsing of all text in braces {} which may result
* in inclusion of another text (substitutions) or another file;
* <li>rescans substitutions so they are handled in the same way as
* the original input;
* <li>swithes input streams for the proper include file processing;
* <li>translates escape sequences including line continuations and
* alternative coding escapes;
* </ul>
* <p>
* Include file processing involves creation of a new scope in the
* preprocessor dictionary. When EOF is signalled for the included input
* file, its scope is removed from the dictionary and input is resumed
* from the including file. Input files form a stack that is
* represented by stackable scopes in the dictionary.
* <p>
* @author NVS
*/
public class ClearStream
extends ReversibleReader
{
/** Logger */
private static final ConversionStatus LOG = ConversionStatus.get(ClearStream.class);
/** Reference to the environment object */
private Environment env = null;
/** Reference to the MultiReader object */
private MultiReader mlt = null;
/**
* Count of bytes currently on top of the stream that have to be read
* without translation. This happens as the result of the translation of
* sequences like '~{' or '~;?' etc.
*/
private int clearCount = 0;
/**
* Configurable marker character. Markers are inserted into the input
* stream to indicate boundaries of included files.
*/
private char marker = '\001';
/** Nesting count of how many brace parsers are in use. */
private int inBraces = 0;
/** <code>true</code> if current input is within a Progress comment. */
private boolean inComment = false;
/** <code>true</code> if current input is within a Progress string. */
private boolean inString = false;
/** <code>true</code> if tildes have to be retained on output. */
private boolean keepTildes = false;
/** <code>true</code> if backslash is honored as an escape character. */
private boolean unixEscapes = true;
/**
* Constructor.
*
* @param mlt
* The input source.
* @param opts
* Preprocessor options to honor.
*/
public ClearStream(MultiReader mlt, Options opts)
{
super(mlt);
this.mlt = mlt;
this.keepTildes = opts.isKeepTildes();
this.marker = opts.getMarker();
this.unixEscapes = opts.isUnixEscapes();
}
/**
* Tells if the character sequence represented by parameters a and b is
* a valid alternative coding.
*
* @param a
* first character from a pair
* @param b
* second character from a pair
*
* @return <code>true</code> if the pair represents a valid alternative
* coding
*/
public static boolean isAlternativeCoding(int a, int b)
{
if (a != ';')
return false;
switch (b)
{
case '&':
case '<':
case '>':
case '*':
case '\'':
case '(':
case '%':
case ')':
case '?':
break;
default:
return false;
}
return true;
}
/**
* Translates a valid alternative coding into a character.
*
* @param b
* second, significant character from a pair
*
* @return translated character
*/
public static int getAlternativeCoding(int b)
{
int retByte = 0;
switch (b)
{
case '&':
retByte = '@';
break;
case '<':
retByte = '[';
break;
case '>':
retByte = ']';
break;
case '*':
retByte = '^';
break;
case '\'':
retByte = '\'';
break;
case '(':
retByte = '{';
break;
case '%':
retByte = '|';
break;
case ')':
retByte = '}';
break;
case '?':
retByte = '~';
break;
default:
;
}
return retByte;
}
/**
* Sets the reference to environment where the braces grammar parser
* and lexer were created.
*
* @param env
* Environment object where the braces grammar parser and lexer
* were created
*/
public void setEnvironment(Environment env)
{
this.env = env;
}
/**
* Sets the inComment state flag.
*
* @param inComment
* new value of the flag
*/
public void setInComment(boolean inComment)
{
this.inComment = inComment;
}
/**
* Sets the inString state flag.
*
* @param inString
* new value of the flag
*/
public void setInString(boolean inString)
{
this.inString = inString;
}
/**
* Detects if the processing is inside a string literal.
*
* @return <code>true</code> if the current processing is inside a
* string literal.
*/
public boolean isInString()
{
return inString;
}
/**
* Gets the input stream.
*
* @return MultiReader object this ClearStream objected is connected to
*/
public MultiReader getStream()
{
return mlt;
}
/**
* Returns the logically next byte from the stream after all due handling
* of escape sequences, alternative coding and braces.
* <p>
* From this perspective, the input is made of comments, strings and
* regular text.
* <p>
* Alternative codings are translated in comments and text
* (not in strings).
* <p>
* Braces are translated in strings and text (not in comments).
* <p>
* Escapes are handled as follows:
* <ul>
* <li>line continuations are translated everywhere; these may be grouped
* and require rescanning;
* <li>regular escapes are translated everywhere; these are:
* ~~, ~", ~', ~\, ~nnn, ~t, ~r, ~n, ~E, ~b, ~f and ~{;</li>
* <li>another form of regular escapes is:
* \~, \", \', \\, \nnn, \t, \r, \n, \E, \b, \f and \{;</li>
* <li>escapes for alternative codings ~;... and \;... are translated in
* comments and text (not in strings).</li>
* </ul>
* <p>
* Newlines are normalized. This means all variations of "\r", "\n", "\r\n"
* that are allowed on input, are replaced with single "\n". To help the
* preprocessor handle newlines within strings, this method has to provide
* enough information to the lexer about all newlines inserted during the
* ~n, \n, ~nnn or \nnn substitutions (the lexer deletes only the original
* newlines and has to keep all the escapes).
* <p>
* As due to the normalization, all CRs are deleted, the CR is used as an
* other "escape" character between this method and the lexer. If, after
* a substitution, a CR or NL character is created, one more CR is inserted
* into the stream just in front of it. This first CR triggers special
* CR or NL processing inside the strings in the lexer. The first CR is
* removed and the second character that follows it (CR or LF) is kept.
* <p>
* Multiple tildes and backslashes need special consideration, because the
* rightmost ~ or \ in the group is converted first and that conversion
* affects the meaning of the previous tilde or backslash. They all are
* accumulated in a temporary array until another character (or sequence
* in case of ~nnn, \nnn, ~;... and \;...) is read, then converted as a
* group and pushed back into the stream with the proper clearCount set.
* The first character of the resulting group is returned.
* <p>This method also
* inserts a special marker into the input stream designating the start of
* the included file. This marker is made of:
* <ul>
* <li>opening marker character;
* <li>'i' character;
* <li>sequential number of marker object in the list;
* <li>closing marker character.
* </ul>
*
* @return byte read or -1 to signal EOF.
* @throws IOException
* if a character matching the marker character is found on input.
*/
public int read()
throws IOException
{
while (true)
{
// read a byte from the stream
int nextChar = mread();
// check to see if the byte must be returned immediately
if (clearCount > 0)
{
clearCount--;
return nextChar;
}
// safety check
if (nextChar == marker)
throw new IOException("marker found on input");
// only certain characters need further translation
if ((nextChar == '\\' && (inComment || !unixEscapes)) ||
(nextChar != '{' &&
nextChar != ';' &&
nextChar != '~' &&
nextChar != '\\' &&
nextChar != '\r' &&
nextChar != '\n'))
return nextChar;
// the byte MAY need translation. check further
// local flags
boolean notInComments = !inComment;
boolean notInStrings = !inString;
boolean inDefine = env.isInDefine();
// normalize new lines
if (nextChar == '\r')
{
int pastNextChar = mread();
if (pastNextChar != '\n')
{
super.unread(pastNextChar);
}
nextChar = '\n';
}
if (nextChar == '\n')
{
return nextChar;
}
// process braces
if (nextChar == '{')
{
if (notInComments)
{
// instantiate the braces parser, call it and handle any
// nesting and hints issues
processBraces();
// reread the modified stream
continue;
}
else
{
return nextChar;
}
}
// process alternative codings
if (nextChar == ';')
{
if (notInStrings)
{
int pastNextChar = mread();
if (isAlternativeCoding(nextChar, pastNextChar))
{
super.unread(getAlternativeCoding(pastNextChar));
// reread the modified stream
continue;
}
else
{
super.unread(pastNextChar);
return nextChar;
}
}
else
return nextChar;
}
// process tilde/backslash escapes
// step 0. preparations
boolean translated = false;
Stack leaders = new Stack();
int n = 0;
int i;
// step 1. read all leaders and the byte that follows them
while (nextChar == '~' || (nextChar == '\\' && unixEscapes))
{
leaders.push(Character.valueOf((char)nextChar));
n++;
nextChar = mread();
}
// step 2. process new line escapes; CR or CRLF or LF are all valid
switch (nextChar)
{
case '\r':
int pastNextChar = mread();
if (pastNextChar != '\n')
super.unread(pastNextChar);
// fall through
case '\n':
// push back excess leaders
for (i = 0; i < n - 1; i++)
super.unread(((Character)leaders.pop()).charValue());
// this is a preproc line continuation (just put the lines back
// together by dropping the newline and reading the next
// character from the stream)
if (inBraces == 0 && !env.isInDefine() && !inString)
{
// dropping the tilde is not correct, but I don't have a better fix at this time.
return leaders.size() == 1 ? nextChar : mread();
}
// do this only if we are inside braces
continue;
default:;
}
// step 3. process single character escapes
// this is the case of ~~ so we push back the excessive data
if (n > 1)
{
super.unread(nextChar);
for (i = 0; i < n - 2; i++)
{
super.unread(((Character)leaders.pop()).charValue());
}
nextChar = ((Character)leaders.pop()).charValue();
if (inString || keepTildes || inBraces > 0)
{
super.unread(nextChar);
nextChar = ((Character)leaders.pop()).charValue();
clearCount++;
}
return nextChar;
}
// the 4GL has a quirk where it eats the escape char in front of a double quote char IF and ONLY IF
// processing a global or scoped define; otherwise it will potentially pass that turd through down
// below (see passThru usage); this doesn't happen for escaped single quote chars
if (inDefine && nextChar == '"')
{
// ~" in a &scoped-define or &global-define is returned as "
return nextChar;
}
boolean passThru = false;
// from now on, there is one leader on the stack
char leader = ((Character)leaders.pop()).charValue();
if (notInStrings && notInComments)
{
translated = true;
switch (nextChar)
{
case '"':
case '\'':
// the 4GL passes through the escape char and following quote char even
// when outside of a string or comment; this usually will cause a compile
// error in the 4GL but there are special cases (e.g. shell command line
// content like for the UNIX stmt) where it can be compiled in the 4GL
if (inBraces == 0)
{
passThru = true;
}
break;
case '{':
case ';':
// no change here
break;
case 't':
nextChar = 0x09;
break;
case 'r':
nextChar = 0x0D;
break;
case 'n':
nextChar = 0x0A;
break;
case 'E':
nextChar = 0x1B;
break;
case 'b':
nextChar = 0x08;
break;
case 'f':
nextChar = 0x0C;
break;
case 'u':
passThru = true;
break;
default:
translated = false;
}
// step 4. process octals
if (!translated)
{
if (nextChar >= '0' && nextChar <= '9')
{
// the Progress PP doesn't check bytes 2 and 3
int byte2 = mread();
int byte3 = mread();
nextChar = 64 * (nextChar - '0') + 8 * (byte2 - '0')
+ (byte3 - '0');
if (nextChar == marker)
throw new IOException("octal escape matches marker");
}
}
} // not in strings
// step 5. push the results back onto the stream
if ((inString || inComment) && (nextChar == 0x0D || nextChar == 0x0A))
{
// inside strings, insert CR escape in front of CR or NL
super.unread(nextChar);
clearCount++;
if (keepTildes)
{
super.unread(0x0D);
clearCount++;
return leader;
}
else
return 0x0D;
}
// during braces parsing, the leading (escape character) must be
// returned to allow the closing right brace to be differentiated
// from an escaped right brace (which is just text)
boolean ignoreRBrace = (nextChar == '}' && inBraces > 0);
if (inString || inComment || keepTildes || ignoreRBrace || passThru)
{
super.unread(nextChar);
clearCount++;
return leader;
}
else
return nextChar;
}
}
/**
* Reads up to len bytes of data from this input stream into an array
* of bytes. Overrides the method of the base class.
* This is a safety net as this method is not expected to be called.
* @param b
* the buffer into which the data is read.
* @param off
* the start offset in array <code>b</code> at which
* the data is written.
* @param len
* the maximum number of bytes to read.
* @return the total number of bytes read into the buffer, or -1 if
* there is no more data because the end of the stream
* has been reached.
* @throws IOException
*/
public int read(byte[] b, int off, int len)
throws IOException
{
throw new IOException(
"Method read(byte[] b, int off, int len) not implemented.");
}
/**
* Dumps the internal state variables.
*/
public void dumpState()
{
if (env == null)
return;
StringBuilder indent = new StringBuilder();
for (int i = 0; i < env.getSym().size(); i++)
indent.append(" ");
env.eprint(indent + "ClearStream state: " +
" clearCount = " + clearCount +
"; inComment = " + (inComment ? "yes" : "no") +
"; inString = " + (inString ? "yes" : "no") +
"; keepTildes = " + (keepTildes ? "yes" : "no"));
}
/**
* Pushes a marker into the input stream.
*
* @param prefix
* prefix of the marker.
* @param index
* marker's index.
* @throws IOException
* when the pushback buffer is full
*/
public void pushMarker(String prefix, int index)
throws IOException
{
String idx = new String(prefix + Integer.toString(index));
env.pushBack(" ");
env.pushBack(marker);
env.pushBack(idx);
env.pushBack(marker);
clearCount += 3 + idx.length();
}
/**
* Invoke the braces parser to consume and handle any include file or
* reference expansions. A braces lexer and parser will be instantiated.
* This includes providing a complete nested environment and handling
* any hints that are required.
*/
private void processBraces()
throws IOException
{
// create a new instance of Environment exclusively for
// handling this level of {} in the braces grammar
Environment envb = null;
try
{
envb = new Environment(env,
this,
null,
env.getErr(),
env.getWarn(),
env.getSym(),
env.getOpt());
}
catch (ParserConfigurationException xc)
{
throw new IOException("ParserConfigurationException " + xc, xc);
}
envb.setFile(false);
BracesLexer brsLexer = new BracesLexer(envb);
envb.setLex(brsLexer);
brsLexer.setLine(env.getLex().getLine());
brsLexer.setColumn(env.getLex().getColumn() + 1);
BracesParser brsParser = new BracesParser(envb, brsLexer);
envb.setPar(brsParser);
envb.setHints(env.getHints());
ScopedSymbolDictionary sym = env.getSym();
int scopesBefore = sym.size();
boolean included = false;
// switch to the child Environment
env = envb;
try
{
// increment nesting count
inBraces++;
// the method call below replaces {} with the evaluation
brsParser.braces();
// decrement nesting count
inBraces--;
if (sym.size() > scopesBefore)
{
included = true;
}
}
catch (Exception e)
{
LOG.log(Level.SEVERE, "Braces parsing failed!", e);
throw new IOException("Braces parsing failed!", e);
}
// switch back to the parent Environment
envb = env;
env = envb.getParent();
Hints hintsObj = env.getHints();
if (included)
{
String filename = ((FileScope)sym.getScope()).getFile().
getName();
String fullname = ((FileScope)sym.getScope()).getFile().
getPath();
env.getLex().setFilename(filename);
env.getLex().setLine(1);
env.getLex().setColumn(1);
env.getPar().setFilename(filename);
// create an include-type hint
if (hintsObj != null)
{
List hints = hintsObj.getHintsList();
int index = hints.size();
hints.add(new IncludeHint(fullname));
if (env.isPush())
{
pushMarker("i", index);
}
else
{
env.queueMarker("i", index);
}
}
}
else
{
if (hintsObj != null)
{
String bufMarks = envb.getMarkers();
if (env.isPush())
{
env.pushBack(bufMarks);
clearCount += bufMarks.length();
}
else
{
env.queueMarker(bufMarks);
}
}
}
env.setMap(envb.getMap());
}
/**
* Reads from the stream and switches to the previous stacked stream
* at EOF.
* Inserts a special marker into the input stream designating the end of
* the included file. This marker consists of two consequtive marker
* characters.
*
* @return byte read or -1 if no more stacked files.
* @throws IOException
*/
private int mread()
throws IOException
{
int c = super.read();
char ch = (char) c;
// check for EOF on the current input stream
if (c != -1)
return c;
// access the current scope in the symbol dictionary
ScopedSymbolDictionary sym = env.getSym();
FileScope fscur = (FileScope)sym.getScope();
int savedLine = fscur.getSavedLine();
int savedColumn = fscur.getSavedColumn();
// debug output
if (env.getOpt().isDebug())
{
for (int i = 0; i < sym.size() - 1; i++)
env.getErr().print("--- ");
env.getErr().println("exiting scope --- " + fscur);
((ClearStream)env.getIns()).dumpState();
}
// honor any deferred pushback
String deferred = fscur.getDeferredPushback();
if (deferred != null && deferred.length() > 0)
{
env.pushBack(deferred);
}
// pop the current scope off and return EOF if no more stacked scopes
// there will always be a global scope and a local scope for the current file
if (sym.size() > 2)
{
env.addConstantDefinitions();
sym.deleteScope();
}
else
{
// The last two variable scoped need to be merged as other components rely on it
sym.deleteScope(true);
return -1; // report the final EOF
}
// access the new current scope in the symbol dictionary
FileScope fspre = (FileScope)sym.getScope();
// switch input streams
fscur.close();
mlt.switchTo(fspre.getStream());
// set up reporting
env.getLex().setFilename(fspre.getFileName());
env.getLex().setLine(savedLine);
env.getLex().setColumn(savedColumn);
env.getPar().setFilename(fspre.getFileName());
Hints hintsObj = env.getHints();
if (hintsObj != null)
{
// produce arguments and end of include hints
if (env.isPush())
{
env.pushBack(" ");
env.pushBack(marker);
env.pushBack(marker);
clearCount += 3;
insertArgumentsHints(env, fscur); // LIFO
}
else
{
insertArgumentsHints(env, fscur); // FIFO
char[] eoi = {marker, marker, ' '};
env.queueMarker(new String(eoi));
}
}
// return one extra space as per Progress Reference Manual.
// this is to be done for all real includes; if the top file is
// a virtual include for the command line, it is bypassed. Otherwise,
// one extra space without newline would be output.
if (sym.size() == 1 && fspre.getFileName().equals("/"))
{
return super.read();
}
else
{
clearCount++;
return ' ';
}
}
/**
* Creates include file arguments hints and inserts the corresponding
* markers into the input stream.
* <p>
* The include file argument hints report the name, type and value of every
* argument along with the flag indicating whether the argument was ever
* referenced by the include file.
*
* @throws IOException
* if pushback buffer is too small
*/
private void insertArgumentsHints(Environment env, FileScope fs)
throws IOException
{
Hints hintsObj = env.getHints();
if (hintsObj == null)
return;
List hints = hintsObj.getHintsList();
boolean allPos = fs.isAllPosUsed();
boolean allNamed = fs.isAllNamedUsed();
// iterate over all arguments
Map args = fs.getArguments();
Iterator mi = args.entrySet().iterator();
Map.Entry me = null;
String key = null;
String val = null;
boolean used = false;
int pos = 0;
while (mi.hasNext())
{
pos++;
me = (Map.Entry)mi.next();
key = (String)me.getKey();
val = (String)me.getValue();
// figure out used status
used = fs.isArgUsed(key);
if (key.startsWith("{"))
used |= allPos;
else
used |= allNamed;
// create the hint
ArgumentHint ah = new ArgumentHint(key, pos, val, used);
int index = hints.size();
hints.add(ah);
// insert the marker
if (env.isPush())
{
pushMarker("a", index);
}
else
{
env.queueMarker("a", index);
}
}
}
}