FWD » Core Development » User Interface

The first proposal of the Windows console native implementation for you to review. A lot of comments still left - be cleaned up later. But the general way is here.

Also I did separation for common code in terminal.c file to have only platform specific code in terminal_linux(win).c files. The Linux part works OK in my small local testing. Not ready to test in devsrv01 for runtime because the changes in Linux code are still possible.

Continue working.

The next drop for review. This is the first workable version. The main issue I struggled with is character coding to display. The testing shows the Windows doe not have good support for Unicode in console so looks like we have to be limited to use OEM coding in this case. This makes the code dependent on the codepage used on client machine(or temporary installs the 850 cp for console session because not all codepages can properly display box-drawing characters and other pseudo-graphics).

Another point here is the color/attributes usage in console. If we need the color or intensity change - we need to pack character key to display and color/attribute in a single int value. I guess it is better to store attribute/color in a upped word if the int, while character value - in the lower half of int.

The code can be tested in any frame oriented testcase(one example has been uploaded here too) with one note - the current key reader does not properly handle F4 - it uses getch() temporary because ReadConsole() usage is under investigation.

Continue working...

The next drop for review. The changes:
1. Implemented key reader code. The java KeyProcessor.java class has also been modified respectively.

2. Attribute handling base implementation. Now support includes NORMAL, REVERSE and UNDERLINE attributes. One important note here. The native 4GL code in Windows console does not support underlined characters due to console implementation restrictions. So in Windows the underlined chars are the same as reversed.

Another note - the attribute constants array has been changed to be able to store individual attributes as binary flags in upper 16 bits of the integer char. The lower 16 bits are used to store the char itself(for the case if we decide to work with Unicode sometimes in the future).

The documented Windows flags
COMMON_LVB_UNDERSCORE and COMMON_LVB_REVERSE_VIDEO
does not work with SBCS. From official MSDN forum: these attributes are working with DBCS for Japanese, Korean and China codepages which is certainly not our case. So the reverse attribute is implemented as the simulation by combining other color attributes.

These currently supported 3 attributes works the same way as in native 4GL under Windows.

3. The new testcases have been added.

Continue working on more attribute investigations, testcases and others.

The second drop for today. The NORMAL text attribute has been changed for color displays according to the Progress documentation. Also the new exit function has been added to restore Windows console to the state before starting the P2J application.

Looks like we need the testcases for every widget like combo-box, selection-list, radio-set ... to verify the CHUI driver correct work.

Another point I'm worry about is error handling in our windows console module. What if something become wrong and Win32 console API calls will return FALSE. What is our strategy here? Ignoring? For example if cursor positioning fails - the screen will possibly become corrupted and further input/output action will produce wrong lay out even if they will work without errors. Or in the case of any error we have to terminate application? Or we need to have error handler to try to fix the issue and continue?

Looks like writing the console testcases I've found two P2J AlertBox issues. Consider the following code:

...
message color skip
   "This is the color attribute text. Depending on the " skip(0)
   "current console OS color settings it can be colored" skip(0)
   "or not. The buttons below should have one letter   " skip(0)
   "underligned and button <No> should have reverse    " skip(0)
   "color attribute meaning the current selection. Hit " skip(0)
   "arrow keys to change the current selection, ENTER  " skip(0)
   "to select curent button.                           " skip(0)
    view-as alert-box warning buttons yes-no-cancel update res as log.
...

and

...
message skip
   "This is the normal attribute text. Depending on the" skip(0)
   "current console OS color settings it can be colored" skip(0)
   "or not. The buttons below should have one letter   " skip(0)
   "underligned and button <No> should have reverse    " skip(0)
   "color attribute meaning the current selection. Hit " skip(0)
   "arrow keys to change the current selection, ENTER  " skip(0)
   "to select curent button.                           " skip(0)
    view-as alert-box warning buttons yes-no-cancel update res as log.
...

The first one has potentially wrong 4GL clause (standalone color phrase) but it is considered as correct in 4GL and produces the alert box below(like following skip has been ignored). To see: color-4gl-linux.jpg and color-4gl-windows.jpg
The P2J converts this as:

...
            messageBox(new Object[]
            {
               "This is the color attribute text. Depending on the ",
               new SkipEntity(0),
               "current console OS color settings it can be colored",
               new SkipEntity(0),
               "or not. The buttons below should have one letter   ",
               new SkipEntity(0),
               "underligned and button <No> should have reverse    ",
               new SkipEntity(0),
               "color attribute meaning the current selection. Hit ",
               new SkipEntity(0),
               "arrow keys to change the current selection, ENTER  ",
               new SkipEntity(0),
               "to select curent button.                           ",
               new SkipEntity(0)
            }, false, new AccessorWrapper(res), ALERT_WARNING, BTN_YES_NO_CANCEL, null, new ColorSpec("skip"));
...

Note the "color skip" also ignore the "skip" word considering the "skip" as color name. This is wrong and the resulting alert box becomes in wrong color. This is the first issue. We can not detect this in regression testing because the text attributes are not comparing. The both Linux and Windows P2J implementations show the same results: color-p2j-linux.jpg and color-p2j-windows.jpg. The correct color to be used in this case is ColorSpec("MESSAGES").

The second 4GL code above converts OK and has the correct color attribute in result. Compare normal-4gl-linux.jpg and normal-4gl-windows.jpg with normal-p2j-linux.jpg and normal-p2j-windows.jpg. However here we see the second issue: in P2J text start coordinate for all lines except last on is different(one more extra " " char inserted before line start). And the alert--box width become different. The converted code is:

...
            messageBox(new Object[]
            {
               new SkipEntity(),
               "This is the normal attribute text. Depending on the",
               new SkipEntity(0),
               "current console OS color settings it can be colored",
               new SkipEntity(0),
               "or not. The buttons below should have one letter   ",
               new SkipEntity(0),
               "underligned and button <No> should have reverse    ",
               new SkipEntity(0),
               "color attribute meaning the current selection. Hit ",
               new SkipEntity(0),
               "arrow keys to change the current selection, ENTER  ",
               new SkipEntity(0),
               "to select curent button.                           ",
               new SkipEntity(0)
            }, false, new AccessorWrapper(res), ALERT_WARNING, BTN_YES_NO_CANCEL, null);
...

It is happening in some rare conditions so in our tests we just can not screens to verify this mismatch.

Note this behavior is not depending on conversion OS, so the problem is somewhere in common code, probably AlertBox class and conversion rule that handles the "color skip" phrase.

4GL Original for Linux and Windows:

           ┌─────────────────────── Warning ───────────────────────┐
           │ This is the color attribute text. Depending on the    │
           │ current console OS color settings it can be colored   │
           │ or not. The buttons below should have one letter      │
           │ underligned and button <No> should have reverse       │
           │ color attribute meaning the current selection. Hit    │
           │ arrow keys to change the current selection, ENTER     │
           │ to select curent button.                              │
           │ ───────────────────────────────────────────────────── │
           │                  <Yes> <No> <Cancel>                  │
           └───────────────────────────────────────────────────────┘
           ┌─────────────────────── Warning ───────────────────────┐
           │                                                       │
           │ This is the normal attribute text. Depending on the   │
           │ current console OS color settings it can be colored   │
           │ or not. The buttons below should have one letter      │
           │ underligned and button <No> should have reverse       │
           │ color attribute meaning the current selection. Hit    │
           │ arrow keys to change the current selection, ENTER     │
           │ to select curent button.                              │
           │ ───────────────────────────────────────────────────── │
           │                  <Yes> <No> <Cancel>                  │
           └───────────────────────────────────────────────────────┘

P2J Linux and Windows:

           ┌─────────────────────── Warning ───────────────────────┐
           │ This is the color attribute text. Depending on the    │
           │ current console OS color settings it can be colored   │
           │ or not. The buttons below should have one letter      │
           │ underligned and button <No> should have reverse       │
           │ color attribute meaning the current selection. Hit    │
           │ arrow keys to change the current selection, ENTER     │
           │ to select curent button.                              │
           │ ───────────────────────────────────────────────────── │
           │                  <Yes> <No> <Cancel>                  │
           └───────────────────────────────────────────────────────┘
          ┌─────────────────────── Warning ────────────────────────┐
          │                                                        │
          │  This is the normal attribute text. Depending on the   │
          │  current console OS color settings it can be colored   │
          │  or not. The buttons below should have one letter      │
          │  underligned and button <No> should have reverse       │
          │  color attribute meaning the current selection. Hit    │
          │  arrow keys to change the current selection, ENTER     │
          │ to select curent button.                               │
          │ ────────────────────────────────────────────────────── │
          │                  <Yes> <No> <Cancel>                   │
          └────────────────────────────────────────────────────────┘

I've just to note this, probably fixing the issues are the separate task we need to create.

More investigation results from testcases written:

1. Toggle-Box widget is not implemented in our runtime.
2. Slider widget is not implemented(need more investigation if we have even conversion support).
3. Radio-button has a bit different look as in 4GL form the attributes point of view. The focused button character between braces ( ), X or " " should have blinked attribute in Linux and reverse one in Windows.
4. As you can see from opened combo-box screenshots and one new console_test4-4gl-windows.jpg we need to introduce usage of the 4 new characters into OutputManager and related logic:
- the known '>' current selection character in selection-list should be replaced with dynamically calculated char depending on underlying platform because in Windows we have the small 1(Superscript One may be) digit instead.
- the small up and down arrows character should be used in combo-box in Windows (0x18,0x19,0x1A,Ox1B in IBM850 charset).
- the checkerboard character should be used in Windows to draw the scrollbars in vertical selection list. May be we need another two for horizontal srcollbars(0x1A,Ox1B for left and right).
5. Also I have noted the scrollbar in selection list does not reflect the current position of the scrollable window. I mean the thumb char is not moving when scrolling. In linux it is not detected in regression tests because the scrollbar is completely consist of ' ' chars with different attributes. This is the issue we certainly need to fix for windows code because there the scrollbar has the character other that ' '.

Continue working.

This drop is a result of checking different widgets layout on Windows 4GL system. I have modified in addition OutputManager class to add 6 new special characters, 4 for scrollbars edge "buttons", in CHUI we need 2 of them for vertical and 2 - for horizontal. All these chars are just ' ' in Linux. Two other chars are selection char('>' for Linux and 0xFB for Windows) to be used in selection-list widget and checkerboard character(' ' for Linux and 0xB0 for Windows) to draw the scrollbar body.

One additional finding: The 4GL CHUI for Windows does not show the horizontal scroll bars in selection-list even it is required and explicitly declared. You can see this on console_test7.p screenshots attached. The Window 4GL interface has the horizontal scrollbar for selection-list.

The other change - the class KeyProcessor.java was removed from update because the Windows -> NCURSES key transformations are now performed inside terminal_win.c module according to previously found issue.

To demonstrate the changes is working I've made some fast fixes to different widgets layout artifacts(which has not many time to debug). The layout fixes are NOT included in this drop. For the test console_test7.p you can see the screenshots taken from original Windows 4GL system and one from P2J implementation.

The today drop includes error processing code approach for terminal_win.c module for you to review. The gathering info will have module info, function name where error happen and the Windows integer error code. May be this is too much for user to know such details, what do you think?

Also I have included the screenshot for P2J selection list with vertical scrollbar. We need to fix Java code for scroll pane implementation class.

Next I'm going to document all remaining widget layout issues found while testing console module if you do not mind before resuming work on CHUI client implementation for Windows. The rest points there are suspending,resuming,restarting and refreshing code.

The drop includes the get/set terminal type implementation. The real 4gL Windows shows the terminal type is constant for given console and returns "CO80". Moreover it is not possible to change it to either "BW80" or "MONO"(testcase console_test10.p). According to Progress doc the TERMINAL function results depends on current display capability.

The implemented approach uses GetDeviceCaps() to get info about color resolution for parameters BITSPIXEL and NUMCOLORS. If bit per pixel > 8 - we certainly have "CO80" display. If NUMCOLORS < 16 - this probably means we have monochrome display. In case of palette based color table 16 or 256 it is "BW80"(or may be "CO80" in certain cases). So probably we need the better way to detect. Moreover the current implementation requires the module to link with gdi32 library which probably we need to avoid. The simplest way is to consider the console is always "CO80".

The testcases update is also uploaded.

Continue to work on suspend/resume implementation...

The drop includes code checked for suspend/resume compatibility. The only usage foe these functions I have found is the running external OS command in standalone session. The example is simple Progress code:
...
os-command "dir".
...
In addition to modification fro terminal*.c code we need to make some changes in other modules:

1. Native process_win.c module. The function pseudoTerminalLaunch():
- The code when we use SW_HIDE option for stating process should be executed when we have any redirected output/input. Otherwise the new console session will be not visible.
- We need to set up the window title for newly created console. Because we use CMD.EXE to start process or command - we can just pass "cmd.exe" string. Otherwise we will have the title inherited from process-starter.
- The most important and possibly need discussion implementation is change the key for our CMD.EXE launcher from /c to /k in the case when we have not silent mode and have no any redirected input/output. Without this modification the behaviour will differ from one we have on real 4GL Windows system. In real 4GL the started session should remain running until the user closes it explicitly.

2. Java class com/goldencode/p2j/util/ProcessDaemon.java:
- Running the external process in NOT silent mode different in Linux and Windows. In Windows there is no pause with key waiting for this case. So we need to check the OS behind before prompting for additional pause.

As to recent change for process_win.c code I agree the fix is looks like a hack and probably we need more general solution in common ProcessDaemon.java class when preparing the arguments for process to be started. For example we can change prepareCommandLine() method or even respective launch(String[]. boolean, boolean). What do you thing for this point?

Another point to discuss - currently we have the code for clear screen in process_win.c module. Is it reasonable to move this function to console related module(terminal_win.c)?

The today drop includes:
1. Minor changing to clean up from old comments in terminal_win.c module and adding clear screen code on exiting the console client.
2. process_win.c also has the addition to detect the full path to the cmd.exe starter to reflect the native 4GL console layout for started application.

So this is the candidate for first release of the console module implementation. Now I need to re-test for possible regressions in the process_win.c with previously tested input/output/input-output-through samples.

The today drop includes small addition for restoring the cursor status that was before executing the console application.

The other note is there are no regressions found in the input/output/input-output-through samples. So we do not brake the process execution Windows module.

However there is one more difference found between our P2J implementation for Windows and original. The 4GL in Windows unlike 4GL in Linux does not show the cursor while PAUSE statement. So P2J implementation in Windows is different. To fix this I can offer to modify ThinClient pause() method:

...
   public int pause(int seconds, String text, boolean noMessage)
   {
      // EVL***<<
      boolean cursorRestore = PlatformHelper.isUnderWindowsFamily();
      boolean cursorOn = false;
      if (cursorRestore)
      {
         cursorOn = tk.setCursorStatus(false);
      }
      // EVL***>>
      // reset need pause condition
      setNeedPause(false);
...
      finally
      {
         // erase status message
         if (!noMessage)
         {
            status(null);

            // hide messages
            Window.getDefaultWindow().setMessageText(null, null);
         }

         pauseCleanup();
         // EVL***<<
         if (cursorRestore)
         {
            tk.setCursorStatus(cursorOn);
         }
         // EVL***>>
      }
...

This can fix the issue. What do you think about this approach(not yet included in this drop)?

0. Merged with the recent P2J code base.
1. Reworked process_win.c to separate path calculation function.
2. Added ThinClient.java modification to hide the cursor unconditionally under Windows.
3. Reworked terminal_win.c to use Unicode range on console output.
4. generateException() has been changed.
5. clearScreen() is now in terminal_win module.
6. The console title is the same as in 4GL in Windows.

Continue refactoring according your feedback. There is only one TODO point left:
The selection symbol 0x00B9 displays differently in native 4GL(upper 1) and in my systems(? - means unknown I guess).

The today drop includes:
1. Improving readability of the terminal_win.c code.
2. Refactoring the initConsole() function.
3. The terminal type detection code has been moved into separate function.
4. The terminal selection approach has been explained in comments.

For the selection char('>' in Linux) I have performed investigation in Windows and found the Unicode symbol used is also 0x00B9. So we can use this code and leave the mapping and rendering up to current OS used to run P2J. The real representation can be different depending on the current locale settings.