FWD » Core Development » User Interface

Regarding the implementation on P2J I read your specifications.
Here is what I understood so far:

1. The browser client will open a URL that is serviced by Jetty in the P2J server process. I'll call this the "server jetty". The URL will be something like https://p2jserverhost:jettyport/chui/ (for the AJAX ChUI) or https://p2jserverhost:jettyport/gui/ (for the AJAX GUI).

Q: The "server jetty" is in fact the WebServer or another instance, a new design?

2. Dynamically start a new P2J client process (we already do something very similar in starting "appserver instances")
This will be done by a special launcher process that is there to create child processes under the control of the P2J server
The new client will be started in "browser mode" (ChUI or GUI) and the client code will be the standard P2J java client
That P2J client will start up and will make an initial connection to the server's secure port using the P2J protocol (this is the "client jetty")

Q: The "client jetty" is started similar to ServerDriver#launchAppServer?

3. The P2J client will notify the P2J server about the URL for which the client jetty is listening, this will include a custom/dynamic port number
The server jetty will respond to the original browser request with a page that redirects the browser to the new client jetty URL
The browser will connect to the client jetty, which will send down a page that is the proper AJAX client (ChUI or GUI)
That AJAX client will establish a secure websocket connection to the client jetty

Q: The P2J client is a new process allocated per connection having an embedded jetty server which should deliver
Web Sockets Services and the main page?

Q: The port number should be generated by an algorithm randomly or in sequence?

Q: The jetty protocol is wss over https?

Q: The SSL context is the same as the "server jetty" or if in the directory is registered a SSL context
(keystore and credentials) for this purpose use this one as a first option?

Q: The "server jetty" is in fact the WebServer or another instance, a new design?

I don't see any important advantage to have another instance. My plan was to just have a different URL (than the admin console) but to extend the same WebServer.

If you have any reasons to do something different, let me know.

Q: The "client jetty" is started similar to ServerDriver#launchAppServer?

Not exactly. That code is related, but I am referring to the server-side code in AppServerLauncher.java. That is the code that really does the launching work. Specifically, look at builder() and launchWorker().

But the development for client launching part will be done in another task than #2200.

Q: The P2J client is a new process allocated per connection having an embedded jetty server which should deliver Web Sockets Services and the main page?

Exactly.

Q: The port number should be generated by an algorithm randomly or in sequence?

I don't want to allocate it at all. Let the operating system allocate it dynamically. We can read the allocated port number from the resulting socket instance and send it back to the server as part of the complete URL.

Q: The jetty protocol is wss over https?

Let's not call it the "jetty protocol", but rather the "web client protocol".

Yes, it is wss from an https session.

Q: The SSL context is the same as the "server jetty" or if in the directory is registered a SSL context (keystore and credentials) for this purpose use this one as a first option?

Yes, exactly.

Initial Feedback

I realize that this is a work in progress and an early version. Thank you for posting it and the useful description of the approach. I will refrain from a full code review because this is still early and will change a great deal.

I like what you are doing in general. There is some very good stuff here. I have the following questions/comments:

1. WsServer has very little to do with WebSockets. Other than the WsHandler inner class and the addWsHandler() method, the code is otherwise quite generic. In fact, it seems to me that it is very, very similar to our current com/goldencode/p2j/main/WebServer. I prefer to maximize common code here, since the server jetty and client jetty will both have similar configuration requirements and will need to support similar usage patterns.

a. I would like to enhance or extend our current WebServer to be used as both the server jetty and client jetty. This eliminates having a 2nd implementation that is so similar to the code we are already maintaining. To the degree that we need to provide more control over the startup/configuration/shutdown of the WebServer class to make it more flexible, we should do so. It also makes it easier to add WebSockets support to the server jetty, which is something will need in the future as we shift our admin console to AJAX.

b. I'd like to see the WsHandler code moved out of that class and put in its own class. The registration code can be hidden there too (to register itself with the WebServer instance). In addition, please call it WebSocketsHandler because the "ws" abbreviation can mean too many different things.

2. I don't understand the need for the WsClient interface. We can always create a helper class for providing common code for the startup/shutdown of the client jetty. It doesn't need to be in the inheritance hierarchy of the client driver. I prefer that approach because there are less layers of abstraction.

3. If I understand the code properly, all the "static" HTML can be loaded from our p2j.jar, right? That is exactly how we want this to work, since the code will only change when we have a revision of P2J. That avoids maintaining separate files in the filesystem which is very good.

4. I like the handleReplacements() support you have provided. It is useful and will be needed in the future. However, for the specific use case in your test code, I think you can use the javascript location object to get the host, port and so forth.

Feedback

This is very good. Some thoughts:

1. Change the "ws" package to "web". This will be the place for generic web server classes.

2. As you get a real client implementation going, you will not need the test package anymore. You can keep it for now, for development purposes, but the final update should not have it.

3. Make sure you are reviewing the ClientDriver, com/goldencode/p2j/ui/chui/ThinClient (in regards to how it integrates with the "drivers" and also how it does much of the startup processing) and the ChUI drivers (com/goldencode/p2j/ui/client/chui/driver/*). These will help you design the changes for how the ChUI web client should be implemented.

4. In regards to coding standards, we now have a 98 character line limit instead of 78 (docs aren't up to date yet). Also, please make sure that all methods and members are javadoc'd. There will be more details on this later, when the code is nearing completion.

Keep going. It is good work.

Today I did a lot of tests regarding the ClientDriver and ThinClient.
I'm able to starts and debug from my IDE both instances, the server and the client.
Here is what I understood from this tests. Please let me know if something are missing.

1. The ClientDriver is a remote client started from a command line which run on separate process having
only one thread, the main thread.

The command line arguments define the ScreenDriver type and parameters.
Also the server host and port are specified as command line parameters.
It is possible to provide the same parameters using a configuration file like client.xml

net:server:host=localhost
net:server:insecure_port=3333

client:driver:type=swing_chui_frame
client:chui:rows=24
client:chui:columns=80
client:chui:background=0x000000
client:chui:foreground=0xFFA500
client:chui:selection=0x0000FF
client:chui:fontname=monospaced
client:chui:fontsize=12

2. From the command line arguments a BootstrapConfig object is constructed used to create, initialize and start
a generic client ThinClient (CoreClient#start).
It's possible to create a remote standalone client running on a separate process like ClientDriver do
or a client running within the server process.

3. Next the UI services are initialized: ThinClient#initializePrep(config, driver, single);
An OutputManager instance wrapping a ScreenDriver object is created.
The OutputManager is used to draw on the screen via an OutputPrimitives interface (ScreenDriver#getPrimitives)
Currently p2j has 2 screen drivers:

- ConsoleDriver - or chui native based on NCURSES package.
- SwingChuiDriver - a Java SWING based client.

I tested both screen drivers and I found to work within ClientDriver.

Also a Java applet ChuiApplet client can be used embedded in an html page loaded by a web browser.

https://localhost:7443/chui

According to my tests this client only draw the main window and a cursor on the top left corner.
It seems that this client is not fully implemented.

4. When the ThinClient instance is created the ClientExports and SessionExports (for static methods) interfaces are exported remotely.
(The SessionExports interface seems to be not used remotely. I didn't found any remote import for this interface).
The registered (exported) ClientExports interface is imported by the LogicalTerminal instance an used to do RMI calls to ThinClient.

5. Next the client is authenticated and a security context for this thread is created.

6. Next finish initializing the thin client: ThinClient.initializePost(session, context)
Imports ServerExports and RemoteErrorData interfaces exported by the remote StandardServer used for RMI calls to server.

7. Finally the MainEntry interface exported (registered) on the StandardServer is imported, a
ClientParameters is constructed having the converted 4gl application
command line parameters and the remote application is started.
In my case Ask#execute is called which is the Java converted version of the 4gl Ask.p

// call the application
running = main.standardEntry(params);

8. The client code waits here until the user will terminate the running application.

9. When the application is terminated by the user, some clean-up statements are executed
and finally the client exits: System.exit(0);

As a conclusion of my tests, I have a question:

I think that we have to design a new ScreenDriver having an embedded server
used to communicate with the JS code which run inside the HTML page on the web
browser via web sockets.
In this case what kind of UI components are used inside the HTML page?

a - HTML specific tags?
b - JS based?
c - java SWING applet?
d - others?

Before I review the code, I will provide some comments and answers to note 10.

The ClientDriver is a remote client started from a command line which run on separate process having only one thread, the main thread.

Yes, ClientDriver is a separate Java process that runs in its own JVM, outside of the P2J server.

Yes, it is remote in the sense that it communicates with the P2J server using our own proprietary remote object protocol.

No, it does not only have 1 thread. There is a type-ahead (key reading) thread that supports our requirement to provide asynchronous interrupts for CTRL-C. And the network socket that connects the client to the P2J server has 2 threads (1 reader and 1 writer). The main thread is in fact the primary thread for doing real work. It is enslaved to the "conversation mode" that we implement between the client and server. See the Conversation class in the net package.

FYI, there is a special-purpose mode in which the ServerDriver can be run as both client and server in one JVM. This is not something we use often and no customers use it today.

The command line arguments define the ScreenDriver type and parameters.
Also the server host and port are specified as command line parameters.
It is possible to provide the same parameters using a configuration file like client.xml

Yes, mostly right. I would note that the bootstrap config can be given as a file and any values in it can be overridden on the command line. The idea is to use that for values that are needed for early configuration of the client (or server, since it can be used there too). Generally, we try to push as much configuration into the P2J server's directory.xml (often just referred to as the "directory") as possible. This maximizes central management of the configuration and can make deployments easier.

For programs that use the P2J classes to make their own connection to a P2J server, the bootstrap configuration can be completely built and controlled under program control.

According to my tests this client only draw the main window and a cursor on the top left corner.

That may be a regression or perhaps it is some applet problem on your system/browser. When I put that applet in, it was working without issues. However, if you were trying it with MAJIC, it would not work today because MAJIC is very dependent upon executing child processes (and other direct filesystem access). That is not possible in the applet approach.

The SessionExports interface seems to be not used remotely. I didn't found any remote import for this interface

No, it is in use. Please see com/goldencode/p2j/util/SessionUtils.java. It is used to "push" some 4GL session-level configuration down to the client so that to the extent that the UI can render or otherwise need to use this configuration, it is in synch with the state on the server.

The registered (exported) ClientExports interface is imported by the LogicalTerminal instance an used to do RMI calls to ThinClient.

Almost. I would not call our approach RMI, which is a very specific Java implementation. However, the idea is the same. We have our own "remote object" protocol (sometimes called the "DAP"/distributed application protocol, but mostly just called "the protocol"). This is the equivalent of Java's RMI, but it has better security and it is easier to implement as a developer. It also doesn't need a name registry process to be running.

But otherwise, your description is correct.

Also note that we instantiate some other classes (e.g. FileSystemDaemon, StreamDaemon, ProcessDaemon, MemoryDaemon...) which also register client-side services that can be called from the server. These are the daemons that allow the client-side process launching, file system access and so forth.

The remote object protocol is designed such that either side of the link can export an API. This peer-based or birdirectional RPC is actively used.

Next the client is authenticated and a security context for this thread is created.

Yes. Note that the context is on the server. On the client, there is only one "session" and the security manager is intentionally a null thing. Since all the actual converted code is running on the server, in a session that has a specific context associated with it, this is OK.

The client code waits here until the user will terminate the running application.

Not exactly. The main thread will be unblocked to execute remote API calls through our exported objects (like ClientExports). And with validation expressions and UI triggers, called client-side code can recursively call back to the server, which can then call back down to the client and so forth. In other words, there can be a complex stack of operations where the control flow of execution for the program will shift back and forth between the client and server, possibly on a deeply nested basis. Ultimately, these nested calls will eventually return and pop off the stack until the program exits, returning here for the main thread to exit (or to restart depending on the "stop disposition").

I think that we have to design a new ScreenDriver having an embedded server used to communicate with the JS code which run inside the HTML page on the web browser via web sockets.

Yes.

a - HTML specific tags?

No.

b - JS based?

Yes. But this is not part of this task. The short answer is that we will use the Canvas interfaces to directly draw our own "Window" implementation and everything that is inside of it. There will not be any actual browser components or plugins used.

c - java SWING applet?

No.

d - others?

No.

Unfortunately, I have just realized that all of these discussions have been going on in the wrong redmine task. Please note my original comment in note 1:

This task is just for time tracking and project management. All task history should be placed in the parent task #1811.

I've been just using the email link to reply back to your posts, but we need to move all this to #1811.