FWD » Core Development » Conversion Tools

-keeptildes exists to emulate the Progress Tilde Bug — OE leaves tildes in the saved listing that it has already removed from the stream it compiles. That emulation is correct for escapes written directly in the source. It is wrong for content that arrived by macro expansion: OE resolves the escape during expansion and the listing carries only the resulting byte, with no tilde text to preserve. FWD keeps the text and appends the byte in both cases.

                            OE listing            FWD -keeptildes
define_tilde_t_tab          c = "A<TAB> B".       c = "A~t <TAB>B".
define_octal_011_tab        c = "A<TAB> B".       c = "A~011 <TAB>B".
define_tilde_n_terminates   c = "A".              c = "A~n".
define_double_tilde_n       c = "AAA.~nBBB.".     c = "AAA.~~nBBB.".

The .preproc (no-keeptildes) output for all of these is byte-identical to the OE baseline, which confirms the defect is confined to the keeptildes path and that converted output is unaffected.

The wiki's Known Behavioral Deviations list has carried this as "No test asserts this: which behavior is correct is undecided, since -keeptildes exists to emulate an OE bug in the first place." It is now decided by measurement — OE has no tilde bug to emulate for expanded content — and it is now asserted by tests.

1. Testcases

Currently failing on this and nothing else:
define_double_tilde_n, define_tilde_n_terminates, define_octal_012_terminates, define_tilde_ctrl_escapes, keeptildes_escaped_braces_in_define, and the pre-existing tildes_in_preproc_defines.

Probably also printable_ascii_characters (def frame a~041. in OE versus def frame a~041!. in FWD — the same "text plus byte" shape); worth confirming when this is fixed.

2. Do not resolve this by changing what the tests compare

Comparing this class against .preproc instead of .keeptildes.preproc makes them pass, and it is tempting because the justification sounds principled. It is the wrong move: it would leave the defect in place and remove the only tests that detect it. Measured across the whole plan, 95 tests pass against .keeptildes.preproc and 15 pass only against it, so the keeptildes comparison is right for the suite; the five that pass only against .preproc are precisely the ones this issue is about.

I've created task branch 11835a.

Testing was completed on branch 11747a. Applying the fix successfully resolved the 6 specific tests mentioned in the initial note.

• With the fix: 110 PASSED, 26 FAILED
• Without the fix: 104 PASSED, 32 FAILED

The defect was at define-scan time. ClearStream.read() consulted keepTildes unconditionally, baking the escape text directly into the stored symbol value so every later expansion incorrectly reproduced it.

I resolved it by routing the eight keepTildes evaluations in read() through a new helper method that explicitly ignores define values:

return keepTildes && env != null && !env.isInDefine();

I'll review it here, but the change probably should be in 11747a where we have other preprocessor changes.

I'll keep that in mind. Please let me know if you would like me to port the revision once it gets the green light. (reviewed and properly tested, if the preprocessor suite is not enough to validate this).

Code Review Task Branch 11835a Revision 16746

I'm good with the change. Please apply the change to 11747a.