FWD » Core Development » Conversion Tools

During preprocessing, the following:

// outside of comments and strings there is a quirk of the "alternative codings" 
// - the ;) alternative coding construct will be converted by the preproc to the } character
// - this occurs in comments and in regular code
// - no conversion occurs inside strings (either '' or "" style)
// - the quirk is that any ;) in code that immediately follows a string literal (either '' or "" style) will NOT con    vert but will emit as a single ) char
// - no whitespace or other chars can be between the string literal and the ;), the quirk only happens if it is dire    ctly following
// - this quirk does not manifest in comments
// - since alternative codings don't convert in strings, this quirk also doesn't appear in strings

// outside of comments and strings there is a quirk of the "alternative codings" 

// - the } alternative coding construct will be converted by the preproc to the } character

// - this occurs in comments and in regular code

// - no conversion occurs inside strings (either '' or "" style)

// - the quirk is that any } in code that immediately follows a string literal (either '' or "" style) will NOT convert but will emit as a single ) char

// - no whitespace or other chars can be between the string literal and the }, the quirk only happens if it is directly following

// - this quirk does not manifest in comments

// - since alternative codings don't convert in strings, this quirk also doesn't appear in strings

// outside of comments and strings there is a quirk of the "alternative codings" 
// - the } alternative coding construct will be converted by the preproc to the } character
// - this occurs in comments and in regular code
// - no conversion occurs inside strings (either '' or "" style)
// - the quirk is that any } in code that immediately follows a string literal (either '' or "" style) will NOT convert but will emit as a single ) char
// - no whitespace or other chars can be between the string literal and the }, the quirk only happens if it is directly following
// - this quirk does not manifest in comments
// - since alternative codings don't convert in strings, this quirk also doesn't appear in strings

Created branch 9886a with revision 15851.
The added changes fix the issue that when preprocessing with FWD a file, if a // (double slash) comment is present, a NL is not appended after the message.

Code Review Task Branch 9886a Revisions 15850 and 15851

1. Could the problem here be something related to the Windows platform? On Windows, the platform newline is the CR (carriage return '\r' a.k.a. 0x0D) + LF (line feed '\n' a.k.a. 0x0A) where Linux just has the LF. What do the binary bytes look like at the end of the // lines?

2. If we go ahead with these changes:

• Please change the Environment.get|setNewLine() to Environment.is|setNeedsNewLine(). getNewLine() doesn't explain the purpose clearly enough.
• text.g needs a history entry.

Greg Shah wrote:

Code Review Task Branch 9886a Revisions 15850 and 15851

1. Could the problem here be something related to the Windows platform? On Windows, the platform newline is the CR (carriage return '\r' a.k.a. 0x0D) + LF (line feed '\n' a.k.a. 0x0A) where Linux just has the LF. What do the binary bytes look like at the end of the // lines?

• in OE with /* */ comments we don't have two /r/n, but with // we do. The following processes as it is.
  

  /* outside of comments and strings there is a quirk of the "alternative codings" */
  message "test".
  /* - the ;) alternative coding construct will be converted by the preproc to the } character */
  

• if we have // instead of /* */, the output is:
  

  // outside of comments and strings there is a quirk of the "alternative codings" 
  
  message "test".
  // - the ;) alternative coding construct will be converted by the preproc to the } character
  
  

• in OE we have \r\n when we have a NL; Meaning when we have a // comment, we have two /r/n after it.
• in FWD whether we use Preprocessor on windows or unix we have ONLY \n, due to the following I think
  

           // normalize new lines
           if (nextChar == '\r')
           {
              int pastNextChar = mread();
              if (pastNextChar != '\n')
              {
                 super.unread(pastNextChar);
              }
              nextChar = '\n';
           }
           if (nextChar == '\n')
           {
              return nextChar;
           }
  

I updated the branch, new rev 15852.

Code Review Task Branch 9886a Revision 15852

I'm good with the changes.

in FWD whether we use Preprocessor on windows or unix we have ONLY \n, due to the following I think

Please register a bug for this. I think we can defer that work, but eventually we may need to match the line endings to the platform. The question is whether we need to do that to match the legacy platform that was used or to the runtime platform on which we execute.

I have rebased and tested this with a customer application. The "deploy" step passed.

I have rebased and tested this with a customer application. The "deploy" step passed.

Was this an application that normally runs on Windows?

Please also check conversion of ChUI regression testing.

• flag is not initialized at the beggining of the SLASH_SLASH
• in ClearStream we look at the flag and check if it's followed by a WS or EOF, if so we unread it and return a \n
• if we are at the EOF and have the flag set on true, we have to return \n

All changes have been moved to 6859b.

• comments

And the other tests have this checking for SLASH_SLASH creating a NL, but comments is dedicated to SLASH_SLASH and STAR_COMMENT.