FWD » Core Development » Base Language

The 4GL event model is inherently "client" oriented. It's primary usage is built around the user-interface and the implementation of triggers for a call-back UI model that was needed for GUI support in the v7 timeframe. This is the WAIT-FOR/PROCESS EVENTS/APPLY event loop implementation.

As other event processing has been added for non-UI purposes, it seems that the same event model was just extended. This is possible in the 4GL because the system is inherently designed such that the entire application runs on the client side and just accesses data from a remote DB server. The (relatively) more recent additions server-side processing (appserver and WebSpeed) has not changed this model, but in such cases the UI portions are essentially null or unused, so there is no conflict.

The problem for us is that we have split the UI off into its own remote thin-client/presentation engine, where we handle all the UI event processing. This includes all the keyboard (low level key labels, key presses and also including the high level key functions like "GO"), mouse, high level events (e.g. LEAVE) and direct manipulation (i.e. drag and drop). These are naturally handled in our current approach.

But all of the real application processing, including all of the business logic runs on the P2J application server. The following types of event processing would normally process on the server in P2J.

1. sockets: we should consider whether this processing is really is a client-side activity like file system access, perhaps only the "client socket" stuff qualifies, but I am not sure

2. database triggers: I am hopeful this one can be split off cleanly, if (as I suspect) the database trigger events are executed independent of a WAIT-FOR

3. appserver (e.g. PROCEDURE COMPLETE)

4. procedure processing (e.g. ON "close" OF THIS-PROCEDURE): note that this includes all the different forms of procedure handles AND it also seems to be similar to the processing of CURRENT-WINDOW/ACTIVE-WINDOW/DEFAULT-WINDOW (e.g. ON "close" OF CURRENT-WINDOW)

5. "user" or "developer" events (e.g. "U1")

6. DDE events (not needed for milestone 7)

7. ProDataSet events (not needed for milestone 7)

I believe the "named events" (PUBLISH/SUBSCRIBE) event model is separate and can be safely implemented on the server. I suspect the database triggers (even those implemented using the ON statement) are also separate, but testing will have to be done to determine if this is correct and any dependencies that may exist if it is not correct.

The objective of this task is to determine the exact requirements for the above issues, design a proper solution and implement it. The items for DDE and ProDataSets will be ignored for now.

when using the aforementioned events, the <handle> in the WAIT-FOR ... OF <handle>. statement needs to be in a state so that it can receive events (i.e. SENSITIVE)

1. What types of handle can be used there?
2. Can you write "traditional" 4GL triggers for all of these event types?
3. What default processing is there for these events?

the complex approach is to rewrite the event processing/listening loops in ThinClient to allow these new event types. But, considering that some events originate on the client side (like socket events) and other on the server side (like procedure-complete and user events), we will need to rewrite the event management so that events can be pushed in the event queue on the P2J client by the P2J server (or by the client too, in case of socket events).

I think we have to do this "the right way". Otherwise we will be spending time on another short term solution that is throw-away. We always get burned when we take short cuts. Please provide more thoughts on the kind of changes you plan to make.

In general, please do consider that we prefer to move away from the use of the AWT-like events for managing the UI. The 4GL UI is completely synchronous, but we have an asynchronous event model as the basis. This has caused a great deal of difficulty in trying to get the UI to work the way the 4GL works. In particular, enter/leave, focus management, validation, drawing and trigger processing are probably much more complicated than needed.

1. What types of handle can be used there?

• active-window
• current-window
• frame
• widgets
• procedure
• socket and server-socket (only after they are connected, i.e. the SENSITIVE flag is set to true).

WAIT-FOR does not accept PSEUDO-WIDGETS (like SESSION, FILE-INFO, etc) and widgets which do not have the SENSITIVE attribute (the exception to this rule is the PROCEDURE handle, which can be used, although it doesn't accept the SENSITIVE attribute).

2. Can you write "traditional" 4GL triggers for all of these event types?

Yes. This works:

on procedure-complete of h do:
   message "procedure-complete" self:handle.
end.
on read-response of h do:
   message "read-response" self:handle.
end.
on connect of h do:
   message "connect" self:handle.
end.

3. What default processing is there for these events?

1. procedure-complete - this event is raised when the async request is completed; if the EVENT-PROCEDURE procname clause is used with the RUN ASYNC statement, then procname is invoked. Else, nothing is invoked.
2. read-response - raised when data is received on the socket or the socket disconnects; if the set-read-response-procedure was used to specify a procedure, that procedure is invoked.
3. connect - raised when a server-socket has received a new connection; if the set-connect-procedure was used to specify a procedure, that procedure is invoked.

User-defined events looks like are implemented like key-events; U1-U10 have the 551-560 key codes and for the CLOSE event the docs state that is mapped to the F8/ESC-Z/CTRL-ALT-Z keys, but I couldn't duplicate it.

Please post an update on your research findings and your planned approach.

1. when key is pressed, an event is raised for it, but the target widget to which it applies is not computed until an event-processing statement is executed
2. when key is pressed, an event is not raised for it; instead, the keys are buffered (as P2J does now), and events are raised only when an event-processing statement is executed

1. TC.waitForWorker picks keys supplied by the TypeAhead buffer and posts events to the EventManager
2. TC.processEventsWorker only picks events supplied by the EventManager

1. TC.waitForWorker should not only listen for keys, but for "server-side event" posting too.
2. TC.processEventsWorker should be fixed so that it can access the keys supplied by TypeAhead.

1. send the event name (CONNECT, PROCEDURE-COMPLETE, READ-RESPONSE), the event ID and the resource ID to the client side; this will be used to create a ServerEvent instance, and post it to the EventManager, but will not process it.
2. on server side, any "server-side event" will be registered on a map; when the client-side processes a ServerEvent instance, it will call back to the server to process it. The server side will identify the event based on the provided ID.
3. if any triggers are registered for the event, invoke them too (some behavior in 4GL is undetermined, as i.e. input variables, context can't be determined, but we can at least inform the user that trigger invocation is not possible, if one is registered).

The resource ID is needed to check for exit conditions in the WAIT-FOR loop. The event loop will end when:
- a pair of (event-name, resource id) matches the event list used by the WAIT-FOR loop
- all resources/widgets referenced by the event list are no longer in SENSITIVE state.
This takes us to creating the event list for a WAIT-FOR event of handle or event2 of handle ... statement. Here, I think the EventList needs to differentiate between a resource ID and a widget ID. We can use an EventDefinition subclass or we can extend EventDefinition to hold the resource IDs too. For resource case, whenever is needed to know if the resource is still SENSITIVE, we can either maintain the resource's SENSITIVE state on the client-side too (via a push approach, as we do know with the ScreenDefinition), or we can use a pull approach (i.e. call a server export to check the resource state).

1. Server-Socket handle
   Is incorrect to think that a server socket will start listening for connections only when the event processing statement is called, as the event processing statement does only event processing. Thus, once the server-socket handle is in SENSITIVE state (via ENABLE-CONNECTIONS), the server socket is listening for incoming connections; when an incoming connection is received, an event is posted and will be processed only when the event processing statement is executed. (As a server-socket implementation note, this means the server socket will need to listen for events in a separate thread, and this will start right after ENABLE-CONNECTIONS succeeds).
2. Socket handle
   When CONNECT is called, this will only send a connect request to the socket; it will not wait for the CONNECT event on the server socket to finish. Instead, it will make the resource SENSITIVE and it will start listening for incoming data on the socket (in a separate thread); when data is received, a READ-RESPONSE event is generated and posted to the client-side. The thread listening for incoming data is suspended until the client-side processes this event (and the data will eventually be read from the socket). When the event is finished processing, the data-listening thread is notified and will start listening again for data. This suspension I think is needed because, if the server keeps sending data to the client while the client is in the READ-RESPONSE procedure, no new READ-RESPONSE events are generated (i.e. nested READ-RESPONSE events are not possible, for the same client socket).

For the appserver's PROCEDURE-COMPLETE event, the event posting will be similar to the socket approach. What will be different is how the PROCEDURE-COMPLETE events will be generated. For this, we need to keep in mind that truly-async requests are possible only in State-free mode, as in State-managed mode all "async" requests are queued as FIFO. In State-managed mode, there will be enough a single thread (per appserver resource) to receive and queue the "async" requests, execute them and generate the PROCEDURE-COMPLETE events. For State-free mode, as this is the only truly async mode, I think there is no way but to execute the async request in its own thread and each thread will be responsible of posting PROCEDURE-COMPLETE event.

1. make the changes to P2J event processing as part of the async requests changes (the ASYNCHRONOUS EVENT-PROCEDURE clause for the RUN statement). Idea is, the async-related changes look pretty simple and should not take too long.
2. add support for the async handle (in a different update only if I determine it to be more complex than I think of it at this time)
3. add the server-socket and socket handle support

Note that the U1..U10 user events and the CLOSE event work properly in P2J (they are named events which are supported).

The findings are interesting. Your approach makes sense. Well done. Please continue as planned.

I am in the middle of the code review. So far I have finished the ui and net package code. I have these initial questions:

1. Do key events always get processed in preference to server events in Progress? This seems to be the way that the TypeAhead.getKeystroke() method is implemented. Also, that code needs to be documented to explain that if honorServerEvents is true and there is no key in the typeahead buffer, then this will return null to allow the caller to process the server event.

2. Is it important for the new EventManager server-event methods to synchronize on wa instead of wa.serverEvents? This seems more granular and just as safe. Likewise, the other methods would synchronize on the contained wa.queue instance instead of wa.

3. What are "exit conditions" in terms of the EventList or EventDefinition? This is mentioned several times, but it is not clear what this means from just reading the code.

More questions:

4. In regard to your assumption in note 9:

From my testing, I think is reasonbly to think a developer will not use the server-side events in user-defined triggers or APPLY statement; this is because in all cases the SELF handle is set to a specific resource and also the target procedure may have parameters. With this in mind, I didn't duplicate this behavior, as in most cases in 4GL env it can end in abnormal ways (as the SELF handle/parameters are not available) or with error messages that the procedure context is not available.

I don't have a problem with this. However, this needs to be documented somewhere in the code as a potential deviation from the 4GL. If there are ways of successfully using this (probably unintended) "feature" of Progress, we will eventually encounter 4GL code that does so. I would like future readers of our implementation to know that this was deliberately bypassed.

5. Does the issue in 4 above lead directly to the choice to keep the resource IDs separate from widget IDs? The resulting event processing implementation is more complex because of this split. I wonder what the negative implications are this decision AND if this is something different from how Progress does it.

6. What is the status of these (your status reports suggested you solved these problems, but the history here does not document it):

STOP and QUIT attribute implementation (they track when the remote call ends via a STOP or QUIT condition)

CANCEL-REQUESTS and CANCEL-REQUESTS-AFTER (still couldn't duplicate it in 4GL env); I added support as described in the docs, and what is currently missing is sending the QUIT condition to the remote side

OK, I have completed my code review (to the 0827a level). Overall: it is really good! The code is now also insanely complicated. I am still having difficulty keeping track of which parts run in which places (non-P2J Java client, P2J server, appserver agent, P2J interactive client...). Wow.

Last questions:

6. Is it true that the 4GL cancels the "cancel timer" thread (in our code this is in ServerImpl) when the timeout is negative? If so, add javadoc to that effect.

7. Is there only 1 possible "current" timer for cancelling requests in the 4GL?

8. Is the use of IllegalArgumentException in ControlFlowOps.invoke*AsyncWithMode() there because converted 4GL code should never trigger this error?

9. Please provide more details behind the TODOs in ProxyProcedureWrapper.delete().

ca_upd20130829a.zip has passed regression testing.

Code Review 0829a

Everything looks good.

1. Please add javadoc to explain the details in note 14 answers 1, 3, 5, 6 and 8.
2. Then commit the change and distribute it.