FWD » Core Development » User Interface

1. COLOR-TABLE
2. support for reading configuration to setup the base colors
3. COLOR stmt forms that don't already have support
4. proper color support at the driver level (CHUI is partial but doesn't support real colors, GUI must support full colors from the beginning)

Based on the work in #1801 and #1811, the basic color implementation for ChUI (all 3 drivers) is fully functional. The only thing for ChUI support that is not yet implemented is the ability to implement custom color overrides (encoded escape sequences for the NCURSES terminal and RGB values for Swing/AJAX drivers). This is the equivalent of item 2 in note 1 above.

GUI support is not implemented at all.

Please finish the ChUI support and add GUI support.

I fixed the issue by using OE 10.2b instead OE 11.3, thanks Constantin.

Code Review 0814a

1. Line 463 of language_statements.rules has a reference to KW_FONT.

2. The progress.g change is not correct. As far as I know, RGB-VALUE is a 4GL built-in function. It is not a handle-based method. The function return value is already in the file on line 5714. Don't be fooled that the KW_RGB_VAL is not on that line. The function keyword resolution works different than the method/attribute lookup.

3. Since a ColorTableEntry[] is transferred to the client, it is important to implement Externalizable instead of Serializable. We found that that approach significantly improves performance.

4. Do we need to push each individual call (e.g. SET-RED-VALUE() and SET-BLUE-VALUE()...) down to the client? I would prefer storing the state on the server and then using state sync to push the changes to the client the next time control transfers to the client side. Otherwise it will be a very chatty/slow approach.

5. ClientExports is missing a history entry.

The valuse are stored on Windows registry and are used by Windows GUI widgets.
I think that our GUI implementation should be a cross platform solution so I stored the system colors inside a XML file (system-colors.xml)

Exactly right. I don't want to implement any direct integration with the EnvironmentDaemon or calls to native APIs. We will read the values from stored configuration.

My only question is how to handle the "save" functionality.

However I found some conversion bugs which are not caused by my changes. A simple test case like:
...
is converted with compile errors. Another simple test like:
...
is not converted at all!

Yes, we don't support the handle type in frames.

These are very good findings.

8. I could not do tests using dos-hex-attribute on Windows ChUI (see 2-b) However we could implements also this option.

I also tried some Windows CHUI tests with these:

• using escaped octal characters ~044 and "~044"
• using non-string inputs 24, x24 and 0x24
• added a FORM statement before the COLOR statement and explicitly using WITH frame to make sure all statements were referencing the same frame

None of these work. I am guessing this was something that only worked on real DOS (and Windows 3.1 but never on Windows 32-bit). Progress did have an MS-DOS client at one time.

Don't worry about implementing this right now, since we can't find a way to make it work on Windows. If we encounter a customer using MS-DOS or can find a way to make it work on Windows, then we will fix it later.

Please document this limitation in the Javadoc for the associated classes, so we don't forget this fact. Also: create a new Redmine task to implement that feature. Have it reference the notes in this task for details.

Question from MAG:

ColorPalette should holds multiple entries depending on platform Linux or Windows. ColorPalette is implemented as a Java enumeration which is not a clever idea as we need to have different entries depending on platform. At this moment ColorPalette contains only 8 colors which correspond to the values defined in curses.h. For Windows OS we need 16 colors. Fortunately the existing colors are also needed for Windows.

We have 2 solutions to fix this issue:

1. Add new entries to enum and use the order as color index. The first 8 colors should remains on the same position to preserve the same ordinal number.
2. Redesign the ColorPalette as a Java class instead of an enumeration and do the appropriate changes to related classes.

What's in your opinion the best solution 1 or 2?

Enums have the advantage of being very concise in syntax and being easy to read/understand/use. The down side of using an enum is the implicit nature of the indexing and limitations on how values can be structured.

If you were to redesign ColorPalette as a Java class, what approach do you suggest? I don't want to burden the class with lots of getters/setters.

I've created #2379 to capture the work item for adding color support to the native Windows console (ChUI) client. We won't work on that as part of this task.

Code Review 0821a

I like the result. Some minor feedback:

1. All data members should be at the top of the class, above any constructors or static methods or static initializers. That means the two private members would be located just after the private static data members.

2. The ColorPalette constructor is missing a javadoc entry for its new parameter.

3. As much as possible, we enable the converted application to be executed on any client platform regardless of the original client platform. Part of doing this means that we need to make the P2J runtime client platform act like the original platform acted, so that the application behaves and appears like it always did (from the user's perspective). This means that we should be able to run an app that used to run in Windows console (ChUI) in P2J on Linux and vice versa. The code in ColorSpec uses PlatformHelper to detect the current runtime client. The problem here is that I think we need to detect what the original client was. That is probably a setting we need in the directory. Based on that we would "show" the original system's color map, no matter what the runtime client type is.

Code Review 0822a

The changes are fine.

the code inside ColorSpec class is running only on the server side, never on the client side.

Good point.

If we need a more flexible configuration we have to design one based on server registry.

Exactly.

Add a static method to util/EnvironmentOps.java called getLegacyPlatform(). The idea of this is different than the purpose of EnvironmentOps.getOperatingSystem(). The getOperatingSystem() is used to back the OPSYS built-in in Progress, which is often used to select different paths at runtime. The most common use is to execute different shell commands based on the actual runtime system type. So this is not something we want to map to the original system's platform type.

But this new getLegacyPlatform() will provide a mechanism to know what behavior should be simulated, when we have the ability to make all client systems act like the legacy platform. The ColorSpec is a perfect example.

This code should lookup the platform as a string in the directory.

String platform = Utils.getDirectoryNodeString(null, "legacyPlatform", runtimeClientPlatform);

The default value if one is not specified (runtimeClientPlatform in the example above) should be read from the getOperatingSystem(). The calculated platform should be saved in the context-local work area so that subsequent calls the getLegacyPlatform() can use the cached value.

In conclusion I think that we have to follow the same behaviour and implements a mechanism to save COLOR-TABLE somewhere on the server side

in different files for each P2J user.

I agree with the general approach, however I think we would not use separate files. We would just store the changes into the server's directory, in some area associated with the user's account.

However, please note that we probably do not need this feature right now. My review of the reports for the current GUI code we are working with is inconclusive (because there are some cases where PUT-KEY-VALUE is being called with runtime values), BUT from looking at the code it does not seem likely that these are COLOR usages. I will check with the customer to confirm. In the meantime, please create a new task for this feature. Make sure to:

• Reference the relevant notes from this task.
• Add this task as a "Related Task" to the new one.

If we decide to load and store custom environments also on server side like default environment we could move the code for COLOR-TABLE management (API) on server side in ColorTable class.
The system colors should be kept on client side in ColorManager class because the system colors are used solely when drawing GUI widgets.

Code Review 0822b

1. Minor javadoc changes to getLegacyPlatform():

specify a legacy platform regarding the real platform

should be changed to:

specify the legacy platform on which the original application code ran, which may be different than the converted application's real platform

and this:

    * The value is read from server directory. If no value is set for legacyPlatform
    * The value of legacyPlatform MUST be UNIX or WIN32.
    * the returned value is the same as returned by getOperatingSystem call.

probably should be this:

    * The value is read from server directory. If no value is set for legacyPlatform
    * then the returned value is the same as returned by the getOperatingSystem call.
    * If set in the directory, the value of legacyPlatform MUST be UNIX or WIN32.

2. The getOperatingSystem().value should never be used. Instead, please use character.toStringMessage().

Code Review 0825a

I'm fine with the changes.

5. In P2J the COLOR-TABLE management is done on each environment on the client side. That's why all server API for COLOR-TABLE are delegated to client.
Code inside ColorTable run on server side but the API calls are delegated to client side on ColorManager class.
Here we have to discuss from where client environments are loaded and to decide where the color tables are managed on server side or on client side.

Please look at #1794. Constantin had to deal with a very similar set of problems.

We want to store well-known configuration on the server side. In this case, we want the data in the directory, like the per-user fonts can be stored.

I also wonder if his font management code for the directory can be re-purposed for use with colors too? Optimally, we would have a generalized solution (common code for both fonts and color configuration).

6. Another issue which I found in P2J relating to environment loading. An instance of EnvironmentDaemon is used to load environments. > EnvironmentDaemon is designed to work on Windows OS only using the Registry class which is a JNI interface for Windows registry. It works also with INI files which could be a cross platform solution.
Even more the initialization of EnvironmentDaemon inside ThinClient.initializePost(...) line 2587 looks like:

Yes, this is a bug. See note 136 of #2252.

As you note, there is no reason we can't use stanza-based INI files on non-Windows platforms. In fact, our whole approach is to try to make the 4GL applications as portable as possible in such cases. Please check with Constantin to see if he has already done anything on this, if not, then you can fix it.

If we decide to load and store custom environments also on server side like default environment we could move the code for COLOR-TABLE management (API) on server side in ColorTable class.

As far as I understand things, the client can't change the COLOR-TABLE, it just uses whatever has been loaded/modified. If so, then we can keep things clean (no need to make many calls to the client) by managing the COLOR-TABLE on the server and then just synching the state to the client the next time control flow shifts.

The system colors should be kept on client side in ColorManager class because the system colors are used solely when drawing GUI widgets.

This makes sense.

I found a conversion problem and cannot fix it. Both COLOR-TABLE and FONT-TBALE have a NUM-ENTRIES attribute.
When are used as wrapped resource the conversion is done with error.I have the following P4 code:

ht = COLOR-TABLE:HANDLE.

ix = ht:NUM-ENTRIES.
ht:NUM-ENTRIES = ix + 1.

ht:SET-DYNAMIC(ix, TRUE).

ht.assign(ColorTable.asHandle());
ix.assign(ht.unwrapFontTable().getNumEntries());
ht.unwrapFontTable().setNumEntries(plus(ix, 1));
ht.unwrapColorTable().setDynamic(ix, true);

               <!-- FONT-TABLE -->
               <rule>methodText.length() == 0 and
                     (htype == prog.kw_font_tab or ref.type == prog.var_handle)
    ……………. 

               <!-- COLOR-TABLE -->
               <rule>methodText.length() == 0 and
                     (htype == prog.kw_colr_tab or ref.type == prog.var_handle)
    ………………  

    <rule>methodText.length() == 0 and
                     (htype == prog.kw_font_tab or ref.type == prog.var_handle)

        <rule>methodText.length() == 0 and
                     (htype == prog.kw_colr_tab or ref.type == prog.var_handle)

Marius Gligor wrote:

I found a conversion problem and cannot fix it. Both COLOR-TABLE and FONT-TBALE have a NUM-ENTRIES attribute.

The problem here is that NUM-ENTRIES is common attribute for both the COLOR-TABLE and FONT-TABLE resources. When handle vars are used, there is no way of determining at conversion time the type of resource referenced by that handle. To solve this, we extract the converted API into its own interface, and add unwrap APIs for that interface. See the unwrapInstantiatingProcedure and the INSTANTIATING-PROCEDURE attribute for an example.

Constantin Asofiei wrote:

Marius Gligor wrote:

I found a conversion problem and cannot fix it. Both COLOR-TABLE and FONT-TBALE have a NUM-ENTRIES attribute.

The problem here is that NUM-ENTRIES is common attribute for both the COLOR-TABLE and FONT-TABLE resources. When handle vars are used, there is no way of determining at conversion time the type of resource referenced by that handle. To solve this, we extract the converted API into its own interface, and add unwrap APIs for that interface. See the unwrapInstantiatingProcedure and the INSTANTIATING-PROCEDURE attribute for an example.

I've fixed the conversion issue. I created a new interface NumEntries for NUM-ENTRIES getter and setters.
FontTableResource and ColorTableResource interfaces extends NumEntries.
I added handle.unwrapNumEntries and I did the appropriate rules changes for COLOR-TABLE and FONT-TABLE.
The conversion is now OK and the converted code is compiled without errors. Also runtime tests for conversion procedure works fine.
Thanks.

Greg Shah wrote:

As you note, there is no reason we can't use stanza-based INI files on non-Windows platforms. In fact, our whole approach is to try to make the 4GL applications as portable as possible in such cases. Please check with Constantin to see if he has already done anything on this, if not, then you can fix it.

I haven't worked on this. Also, note that this problem I think was encountered by Ovidiu, too: some configuration was being read from the .ini file, which I think was set at the appserver agent configuration. So, we need to ensure that .ini files can be specified for normal users, batch processes and appserver agents.

More, in case of custom auth plugins, I think we need to push a default/per-server .ini file (we might be able to limit this to only default-window/font/color-table config, but I think is best to allow access to the entire .ini file).

we need to ensure that .ini files can be specified for normal users, batch processes and appserver agents

Yes.

I think is best to allow access to the entire .ini file

Agreed.

Code Review 0828a

The code is fine. I like the direction it is moving.

Because system colors are not part of 4GL environment I think that system color table should be stored on server directory only globally (per server not per user).

Actually, since legacy users will often have had the app installed on their local windows system, they may have had custom system colors assigned. And there may be disabled people for which assessibility is important but whose changes would not be shared with everyone. I think it is best to keep the same structure of per-account/group with the defaults being allowed at the server level.

Also the table should be read only which means that could not be changed from Java code.

Yes.

If the current COLOR-TABLE is stored on server side each time we convert a FGCOLOR or BGCOLOR index into a RGB color we have to do a call on server because the draw is done on client side.

We can avoid that by using our state sync approach.

You said something related to a synch between server and client. How we can do that? Could you give an example to understand how it works?

Please see:

ui/ClientState
ui/ServerState
ui/LogicalTerminal (both getChanges() and applyChanges())
ui/chui/ThinClient (both getChanges() and applyChanges())
net/StateSynchronizer
net/Protocol (see registerSynchronizer(), attachChanges() and applyChanges())

The basic idea is to "piggyback" the transfer of state changes on the next message sent to the other side. By timing the apply process on the target side such that it is before any possible use of that state, it appears to the code like the state is always synchronized with the version on the other side.

From where INI files are loaded from server side or from client side?

For standard application INI files, I'd like to do this from the server. This is different from how Progress works, but if we can solve that problem then it is a nice improvement for our customers.

We should be able to read a default version from the application jar file. It will be tricky in the case where a custom version needs to be read. Also, the current conversion/runtime doesn't support this server-side idea.

Constantin: what do you think?

We could use system colors solely as default colors when draw GUI widgets by replacing the colors defined in GuiOptions with corresponding system color.

Yes, this makes sense. We will have to capture a good set of basic (Windows 7?) defaults. These can be hard coded into our Java source and then the customer should be able to override.

1. The P2J implementation for LOAD, USE and UNLOAD statements is conforming with OpenEdge manual.
It works only if client is running on Windows OS and the API calls are delegated to client side (see EnvironmentDaemon line 60).
I think that we have do some changes here in order to accommodate our needs for GUI drivers perhaps on another Redmine task.

2. Another issue related to environments. Environments are application specific.
All users which are authorised to run an application which use custom environments must have INI files having the name of environments.
The question here is: An application environment (colors and fonts) should be the same for all users?

I think that we have do some changes here in order to accommodate our needs for GUI drivers perhaps on another Redmine task.

I'm fine with that approach so long as we determine the requirements now.

The question here is: An application environment (colors and fonts) should be the same for all users?

I think we will need per-user level. The reason is the same as for the system colors, current installations are often on Windows desktop systems which 1 system per user. Each user can have different app environments today. Even in a Windows Terminal Server environment, I think the registry is split for users so different users have different content.

Code Review 0903a

The changes look good.

1. WindowConfig is missing a history entry.

2. The call from the client to the server's LogicalTerminal.getColor() will be expensive. Should we pull down all colors for the referenced environment in one call instead of potentially making multiple calls to obtain all the colors?

Code Review 0903b

The changes look good. I like the more general approach to reading color tables from the server.

What is the list of remaining work?

I have no other works to do on this task.
Nevertheless they are other issues to implements like LOAD, USE, UNLOAD, PUT-KEY-VALUE which are common to both color and font tables.
I think that should be covered on other Redmine tasks.

I have no other works to do on this task.

OK, please get this regression tested (both conversion and runtime).

Nevertheless they are other issues to implements like LOAD, USE, UNLOAD, PUT-KEY-VALUE which are common to both color and font tables.
I think that should be covered on other Redmine tasks.

Please create a new task (make sure to document the details of the work needed in the task history and not in the description field). Make it related to this task and also to #1649. Assign it to yourself. You can work that next.

Code Review 0904b

I am OK with the changes and I do think they are safe enough to avoid further testing.

I have a minor concern that we are taking the overhead of String.format() in every iteration of the main for loop in ColorTable.makeColorTable(), even though it is an unlikely case that the log message is needed. The only change I'd like to see is to make a private logging method that hides both the String.format() and the call to LOG.warning(). This can be called from all 3 locations, so we don't duplicate code.

It also may make sense to put a different bit of text in each of the 3 places, to explain the reason for the failure:

• system color name is invalid
• no system color is defined for the given (valid) system color name
• improperly formatted color string was provided on input

Code Review 0904c

It is good. Go ahead and check it in/distribute it.