FWD » Core Development » Base Language

Implement the following:

- appserver dispatcher/thread pool
- handle context issues
- startup/shutdown procedures (state aware, stateless, state free)
- connect/disconnect procedures (state reset, state aware)
- activate/deactivate procedures (state free)
- Server Handle Object
- methods: SESSION:EXPORT
- attributes: SESSION:SERVER-OPERATING-MODE, SESSION:REMOTE
- language stmts: CREATE SERVER, DELETE PROCEDURE/OBJECT (already working, but make sure it works for deleting the server handle object), RUN PERSISTENT ON SERVER, RUN IN <remote_server_handle>
- Server Handle Object methods: CONNECT, CONNECTED, DISCONNECT
- user-defined functions: FUNCTION IN <remote_server_handle>

Simple 4GL client code example:

def var appsrv as handle.

... more logic and var defs ...

CREATE SERVER appsrv.
appsrv:CONNECT("-H my-hostname -AppService service_name", ?, ?, ?).

IF VALID-HANDLE(appsrv) AND appsrv:CONNECTED() THEN
DO:
   RUN remote_procedure_name.p ON appsrv (INPUT parm1, INPUT parm2, OUTPUT my-var).

   ... more logic here if needed ...

   appsrv:DISCONNECT().
   DELETE OBJECT appsrv.
END.
ELSE
   MESSAGE "uh oh".

Lower priority features:

Server Handle Object methods:
CANCEL-REQUESTS
CANCEL-REQUESTS-AFTER

Server Handle Object attributes:

ASYNC-REQUEST-COUNT
CLIENT-CONNECTION-ID
FIRST-ASYNC-REQUEST
FIRST-PROCEDURE
HANDLE
INSTANTIATING-PROCEDURE
LAST-ASYNC-REQUEST
LAST-PROCEDURE
NAME
NEXT-SIBLING
PREV-SIBLING
PRIVATE-DATA
REQUEST-INFO
RESPONSE-INFO
RESTART-ROWID
SSL-SERVER-NAME
SUBTYPE
TYPE

For Server handle the folllowing attributes have types that are not yet implemented

REQUEST-INFO -> Progress.Lang.OERequestInfo class
RESPONSE-INFO -> Progress.Lang.OERequestInfo class

NAME
PREV-SIBLING
NEXT-SIBLING
PRIVATE-DATA
SUBTYPE
TYPE

Regarding the Server handle example , I could not get a CONNECT to work and get a functional test (tried on the default port but the AppServer probably needs to be started or it's working on a different port than the default)

What would you advise on REQUEST-INFO and RESPONSE-INFO attributes?

Don't implement them at this time.

 I could not get a CONNECT to work and get a functional test (tried on the default port but the AppServer probably needs to be started or it's working on a different port than the default)

Yes, unless you have a server configured and started, this would not work. It is not needed for our case. Write the testcases for server and client sides as best you can. Make sure they can compile in Progress. You don't have to use all possible features, just focus on the features that are high priority. On the server side, create a few functions and a few procedures and try using a range of parameter types and return types, including returning a temp-table. That should be sufficient for our needs.

1. RUN PERSISTENT [SET handle] ... ON server can obtain a handle to a remote procedure (which later can be used using RUN ... IN handle statement). For normal procedure handles obtained using RUN PERSISTENT, all state is preserved until the procedure is deleted. Is this the same for remote procedures? Does the remote side keep the procedure in memory until the requester deletes it?
2. when I added varargs to the ControlFlowOps APIs, I didn't thought things will get complicated when we will add other clause support. Idea is, we will need to add the following APIs to cover all RUN ... ON ... cases:
• ControlFlowOps.invokeRemotePersistent[Set][WithMode](..., serverHandle, transactionDistinct, [procargs]) - to cover the RUN ... [PERSISTENT [SET]] ON serv-handle [TRANSACTION-DISTINCT]. We can make the TRANSACTION-DISTNCT emit true when is set and false when is missing, thus the transactionDistinct parameter will specified always.
• ControlFlowOps.invokeRemote[Persistent[Set]]Async[WithMode](..., serverHandle, transactionDistinct, asyncRequestHandle, eventProcedure, procedureContext, [procargs]) - to cover the RUN ... [PERSISTENT [SET]] ON server-handle [TRANSACTION-DISTINCT] ASYNCHRONOUS [SET async-request-handle] [EVENT-PROCEDURE event-internal-procedure [IN procedure-context]]. Similar: the async-request-handle will convert to null if the SET clause doesn't exist, event-internal-procedure and procedure-context will convert to null if EVENT-PROCEDURE clause doesn't exist, and procedure-context will default to THIS-PROCEDURE if EVENT-PROCEDURE exists but IN procedure-context doesn't exist.
• ControlFlowOps.invokeRemoteAsync(..., asyncRequestHandle, eventProcedure, procedureContext, [procargs]) to cover the RUN internal-procedure ... ASYNC ... statement with ASYNC support (parameters emitted same as for RUN ... ON ... ASYNC).
• ControlFlowOps.invokeRemoteInAsync(..., asyncRequestHandle, eventProcedure, procedureContext, [procargs]) to cover the RUN internal-procedure ... IN proc-handle ASYNC statement with ASYNC support (parameters emitted same as for RUN ... ON ... ASYNC).

I like your approach with the ControlFlowOps APIs.

Is this the same for remote procedures? Does the remote side keep the procedure in memory until the requester deletes it?

Yes, as I understand it. From the Progress docs:

When an ABL client application executes a remote persistent procedure, two persistent
procedure handles are created: one within the client application session and another
separate handle within the AppServer session where the persistent procedure is
created. OpenEdge internally maintains a mapping between the two handles. The
handle within the client application is a proxy persistent procedure handle, and its
PROXY attribute and PERSISTENT attribute are set to TRUE. The corresponding handle
within the AppServer agent is a remote persistent procedure handle, and its REMOTE
attribute and PERSISTENT attribute are set to TRUE.

Unlike the values of persistent procedure handles and the THIS-PROCEDURE handle that
reference the same local procedure context, the proxy persistent procedure handle and
the remote persistent procedure handle are truly separate handles. For example,
setting the PRIVATE-DATA attribute on a remote persistent procedure handle has no
effect on the PRIVATE-DATA attribute of the corresponding proxy persistent procedure
handle in the client application.

Please see the "Understanding remote procedure handles" section of the "Developing Appserver Applications" book for more details.

A note about emitting null for missing clauses: we can't map null with a 4GL token, so instead the solution is to create an EXPRESSION node with its text set to "null" and add code to expression.rules to convert such nodes (under very specific cases) to java.NULL_LITERAL.

Good idea.

• SSL-SERVER-NAME (converted to SslServerAttribute interface) - make sure you don't break socket's support for this attribute
• BaseSession/Session - the appserver-related attributes have nothing to do with our proprietary communication protocol... they need their own classes/interfaces.
• progress.g - we need to check if the RUN's ASYNC and ON SERVER (sub)clauses can be in random order (as our changes hard-coded the order of these clauses)
• make sure all interfaces are properly registered with the main interface describing the resource (i.e. check SslServerAttribute and SubTypeAttribute)

Tomorrow will see how the inner-workings of runtime can be designed.

A note about delete: currently, each deletable resource must be registered in HandleOps.delete and call its dedicated delete method, based on resource. I wonder if you it would not be better if we would add a "delete" API to the Deletable interface, which will be implemented by all resources (including the system handles). This way, HandleOps will just do a "referent instanceof Deletable" and if ies, call that method.

Good idea. Please go with the Deletable approach.

Also: please add support for DELETE WIDGET, which is the equivalent of DELETE OBJECT/DELETE PROCEDURE. In other words, DELETE WIDGET can be used on non-widget handles and it works. The only difference here is that DELETE WIDGET can take a list of handles, instead of just 1 handle. This usage is present in the current project.

OK, I will work on this. I will not push this change to all resources yet, only to what we are currently working.

Also, do we need to implement the procedure-handle:SERVER attribute ? This seems related to appserver handles. More, the following works:

def var h as handle.
h = this-procedure.
def var h2 as handle.

procedure proc0.
  message "msg1".
  pause.
end.

procedure event-proc.
  message "msg2".
end.

message h:server. /* this is unknown */
run proc0 in h asynchronous set h2 event-procedure "event-proc" in h.

message h2:type. /* this is "ASYNC-REQUEST" */
message "msg3".

Interesting finding. The permutations never seem to end. This also has implications about our runtime implementation. The GUI project and server projects will be running in the same JVM. The GUI project extensively uses appserver calls to execute logic and get data from the server project. If we short-circuit the server_object_handle:CONNECT to provide a handle that accesses a same-JVM instance, then it seems like the actual execution of logic is essentially the same as your example.

Yes, we should add procedure handle support for attributes SERVER, REMOTE, PROXY and TRANSACTION.

PS: I suggest to test and release this together with the #1620 changes, as we will be able to integrate the Deletable interface with other changes on queue (like SOAP).

The delete widget changes look good. We will keep it separate from the rest of the appserver update and instead include it with the #1620 work (and 2 other updates) that will go into testing next.

The RUN statement's clauses can be "switched" to some extent and produce valid code in 4GL, and the conclusion is the parser needs to emit this rooting:

RUN ... ON
-> SET /* for the ASYNC clause */
-> PERSISTENT
  --> SET
-> ON
  --> TRANSACTION DISTINCT
-> ASYNC
  --> EVENT-PROCEDURE
    ---> IN handle

RUN ... IN
-> IN
-> SET /* for the ASYNC clause */
-> ASYNC
  --> EVENT-PROCEDURE
    ---> IN handle

In fixup phase, the ASTs will fixed so that the rooting will be the one in the official syntax for the RUN statement.

PS: there is one syntax which can't be fixed that easy. Is related to RUN ... IN, as:

run "foo" asynchronous event-procedure "bar" in h1 in h2.

in_procedure_context_clause   
   :
      KW_IN^ lvalue
   ;

• SET clause can be used without the ASYNCHRONOUS (only ON SERVER is required):
  

  run "foo" set h1 on server h2.
  

• SET can be present multiple times:
  

  run "foo" set h1 set h2 on server h3 set h4 set h5.
  

All this is valid 4GL code, which I don't know how to interpret - should I assume if SET is present without ASYNC, then ASYNC is "by default" added ? And with the multiple occurrences, which one will be set - first, last, all ?

Some thoughts:

The RUN statement's clauses can be "switched" to some extent and produce valid code in 4GL, and the conclusion is the parser needs to emit this rooting

Yes, that is why the run_stmt rule matches these in a ()* loop.

In fixup phase, the ASTs will fixed so that the rooting will be the one in the official syntax for the RUN statement.

I assume that you don't need to change the looping of the parser. Instead you are referring to using fixups to force a well known order for the tree?

where h2 is the target for "foo" and h1 is the target for "bar". The parser fails for these cases, as the IN h2 is glued with the h1 IN h2, treating the IN keyword as if it was for a widget in frame expression.

Yes, this is a problem area that deserves some attention. I have tried various "quick and dirty" fixes for this. The core problem is that we have to match with an lvalue that is a handle. But when we reuse the lvalue rule for this case, we match to a wider range of possibilities than the 4GL can accept. This leads to improper trees as a result.

In may cases, I have already found that a more limited subset of possible matches is OK. In these cases I have coded the usage to the "handle" rule. This is still using an lvalue but we do place some limits on it. This rule is meant to only be able to match a handle type BUT IT IS FLAWED. It uses lvalue and then just uses a semantic predicate to fail the parse if the type of the result is not one of a list of handle types. So it is just a consistency check on what was matched and not a real enforcement of what can be matched. In fact, the check is incomplete anyway. There are times when it probably shouldn't accept some matches (like a non-writable attribute when an assignable handle is needed). But at least we know that these places are safe to match just a handle, and if we fix the handle rule, then these should work fine. The following rules all are already coded to use the "handle" rule:

create_query_stmt
create_dataset_stmt
create_datasource_stmt
create_buffer_stmt
create_temp_table_stmt
create_widget_stmt
delete_widget_stmt
frame_handle_clause
in_window_handle_clause
create_browse_stmt
create_soap_header_stmt
create_soap_header_entryref_stmt
create_server_stmt
create_socket_stmt
create_client_principal_stmt
create_xml_stmt
create_sax_stmt
create_call_stmt
from_handle_clause
proc_handle_clause

I think we would like to move to a "handle" rule that is as limited as is accepted in the 4GL (and no more). Then we would want to replace the usage of lvalue/expr in the above IN handle cases AND we would naturally get better limits in all the current places that use the handle rule.

The next steps

1. Determine the limits of what can be matched from the following "IN handle" cases:
in_super_or_in_handle
in_window_clause
in_procedure_clause
in_procedure_context_clause

In each of these cases we either match an lvalue or an expr instead of a more selective choice. Can this be a more selective match? Is it possible to match to anything other than a single/simple handle reference? For example, can we match an subscripted array reference (I guess so): my-handle-var³

Can we match anything like a widget qualifier in these places like "FRAME my-frame" (probably not). Can we match a func_handle, meth_handle, attr_handle...?

The more that is possible in these IN handle cases, the bigger the problem.

2. Review the use of the allowInProc variable which is one of the hacks that I used to allow the parser to match. Besides being overly complex and something we don't want in general, the resulting tree is wrong when this case is required.

After you have results from 1 and 2 above, let's discuss them.

All this is valid 4GL code, which I don't know how to interpret - should I assume if SET is present without ASYNC, then ASYNC is "by default" added ? And with the multiple occurrences, which one will be set - first, last, all ?

Yuck. Here is more 4GL stupidity. I suspect that the 4GL ignores most of these, but we will have to write testcases to determine the limits.

Guy: I have added you as a watcher because we can really use your help in working up an appserver configuration that we can use for simple testcases. Can you point us to the configuration files in use on WINDEV01? And can you offer any suggestions on how to simplify and use these to run simple testcases? THANKS!!!

I assume that you don't need to change the looping of the parser. Instead you are referring to using fixups to force a well known order for the tree?

The parser needed some changes (TRANSACTION DISTINCT needed to be handled by the on_server_clause, plus some other minor fixes). And yes, fixups are used to force a well known order for the tree. Another interesting fact is that TRANSACTION keyword can be used without the DISTINCT keyword, in a ON SERVER clause (thus the current approach of converting TRANSACTION DISTINCT to a KW_TRANS_DS token will not be correct).

Yuck. Here is more 4GL stupidity. I suspect that the 4GL ignores most of these, but we will have to write testcases to determine the limits.

Guy: I have added you as a watcher because we can really use your help in working up an appserver configuration that we can use for simple testcases. Can you point us to the configuration files in use on WINDEV01? And can you offer any suggestions on how to simplify and use these to run simple testcases? THANKS!!!

(thus the current approach of converting TRANSACTION DISTINCT to a KW_TRANS_DS token will not be correct).

You can do something like this:

trans_distinct_clause   
   :
      t:KW_TRANS^ { #t.setType(TRANSACTION_DISTINCT); } (KW_DISTINCT!)?
   ;

This makes the downstream matching very easy. It doesn't "impersonate" a keyword (KW_TRANS_DS suggests this is a keyword and it is not). And we handle the undocumented optional KW_DISTINCT. BTW, please note (in the javadoc for that rule) that it is an "undocumented feature" in 4GL that the KW_DISTINCT is optional.

1. Determine the limits of what can be matched from the following "IN handle" cases:

For all cases, the handle can be any expression returning a handle: function call, dynamic-function call, variable, fields (do we have handle fields in the project?), subscripted variable, attributes/methods returning a handle.

2. Review the use of the allowInProc variable which is one of the hacks that I used to allow the parser to match. Besides being overly complex and something we don't want in general, the resulting tree is wrong when this case is required.

• dynamic_function_func - for DYNAMIC-FUNCTION(... IN handle)
• subscribe_stmt - for SUBSCRIBE ... IN publisher-handle
• unsubscribe_stmt - for UNSUBSCRIBE ... IN publisher-handle

In all cases, the handle can be any expression returning a handle.
---
An important note is that handles can not be used directly in frames (they must be wrapped using the INTEGER function), thus we can can not have widgets of type handle. Also, for all the CREATE ... handle and SET handle cases (all cases which need to save a resource in a handle), in 4GL is mandatory to provide a variable, field or an instance member of a class, of type handle. This excludes writeable attributes. So, we have only a limited set of cases when lvalue's are required. For all other cases, when an expression returning a handle is required (this can be even the IF ternary operator), we can limit the expr rule to ignore widgets to some extent, as following is valid:

def var i as int.
form i with frame f1.

function foo1 returns int in (if i:screen-value = "0" then this-procedure else super-procedure). 
function foo2 returns int in (if i:screen-value in frame f1 = "0" then this-procedure else super-procedure). 

do we have handle fields in the project?

Yes. I think they may only be supported in temp-tables in the 4GL. We have complete support as far as I recall.

Based on your findings, I think we can probably "get away with" an approach that does a similar thing to what we are doing with allowInProc. Basically: all "IN handle" (expression) usage and the handle rule itself would both be "protected" from downstream inappropriate matches by setting a flag and unsetting it after the matching is complete. Then any features that need to be turned off would have a semantic predicate added that only matches if the flag is off.

Greg: as a side-note, at this time, how much of the runtime do you want to be stubbed out ? Do you want us to get into the high-level design of the appserver's dispatcher/thread pool, startup/shutdown/connect/disconnect procedures, states or is enough to have stubs for attributes/methods + RUN statement ?

Do you want us to get into the high-level design of the appserver's dispatcher/thread pool, startup/shutdown/connect/disconnect procedures, states

This part is not needed right now.

or is enough to have stubs for attributes/methods + RUN statement ?

Yes.

I think the final solution is to get rid of the allowInProc quirk and call the widget_qualifier rule (in lvalue) only under these conditions:

LA(2) != KW_WID_POOL && LA(2) != KW_WINDOW && (LA(2) == KW_FRAME || LA(2) == KW_MENU || LA(2) == KW_BROWSE

I don't have a problem with your approach. I think it may work, but I do suspect there will be lookahead issues. You will have to remove the expr option from that rule. I don't think that will solve all the lookahead issues, but it will reduce them AND we would not need it any longer. Of course, in these allowInProc locations, you will have to ensure that they can always match the IN clause.

But still there may be lookahead issues. When we use semantic predicates to avoid matching something, the parser includes the Java code directly into the lookahead testing in the match processing for that alternative. BUT the way the parser gets generated, the lookahead tests for all alternatives in a given rule are aggregated and used in the locations that call that rule for determining if the rule needs to be called or not. BUT, our semantic predicate will not be used in those calling locations. SO: it may require that the semantic predicate be explicitly placed in other calling locations to avoid improperly calling lvalue when the generated lookahead would normally match the widget_qualifier code. If you get ambiguity warnings, that may be the solution.

Be sure to carefully test the possible conflicting syntax (e.g. make sure you test cases like "var IN proc_handle").

Let me know how it goes. Do you think we might have an update ready tonight?

Greg Shah wrote:

You will have to remove the expr option from that rule.

Yes, expr from widget_qualifier is no longer needed.

Be sure to carefully test the possible conflicting syntax (e.g. make sure you test cases like "var IN proc_handle").

I've duped all cases which use constant as proc/function name to use character variable, and it seems to work.

Do you think we might have an update ready tonight?

I need to finish the review + Costin needs to add some little more javadocs (again, there are methods which don't accept longchar for character or dec for integer), but we should have something today.

Code Review Feedback:

I'm in the middle of the code review, but there is at least 1 major issue. The subscribe_stmt and unsubscribe_stmt can no longer match on a character expression after the optional TO keyword. If I am reading this right, this code is no longer possible:

SUBSCRIBE PROCEDURE my-proc TO "some-event-name" IN some-handle.
SUBSCRIBE PROCEDURE my-proc TO "some-event-name".

Am I missing something?

More Feedback:

1. handle.java is missing the "--JPRM--" in the header.

2. Is HandleChain.setType() always going to ignore non-null values? That seems wrong. See the "type == null" here:

   public void setType(String type)
   {
      if (type == null)
      {
         this.type = type;
      }
   }

3. Much of the tree changes done in fixups/control_flow.rules should really be done during annotations. The parts that sort things are OK (it just is making a more regular order). But the places where we are inserting null expressions and boolean true/false, we are completely changing the code. It is no longer valid 4GL. We always leave the trees as valid 4GL until we get into annotations. This allows us to run reports and do other things (like someday antiparsing .ast files back to 4GL source code). Please move those parts to a step in annotations.

I'm in the middle of the code review, but there is at least 1 major issue. The subscribe_stmt and unsubscribe_stmt can no longer match on a character expression after the optional TO keyword.

You are correct, it was wrong to cut "expr" off from there.

1. handle.java is missing the "--JPRM--" in the header.

OK, fixed.

2. Is HandleChain.setType() always going to ignore non-null values? That seems wrong. See the "type == null" here:

OK, fixed.

3. Much of the tree changes done in fixups/control_flow.rules should really be done during annotations. The parts that sort things are OK (it just is making a more regular order). But the places where we are inserting null expressions and boolean true/false, we are completely changing the code. It is no longer valid 4GL. We always leave the trees as valid 4GL until we get into annotations. This allows us to run reports and do other things (like someday antiparsing .ast files back to 4GL source code). Please move those parts to a step in annotations.

OK, so the idea the front phase should leave the ASTs as unchaged as possible. I'll fix this too and re-post update.

Please let me know if you've seen anything else incorrect.

OK, so the idea the front phase should leave the ASTs as unchaged as possible.

Yes, that's right.

Please let me know if you've seen anything else incorrect.

Everything else looks OK.

I have reviewed the code, the only problem is that the unsubscribe_stmt still cannot match on the event name after the optional TO. The expr needs to be added back there as well.

Otherwise it is good to go. Please go ahead and run a conversion test on Majic. I think there is enough change to warrant a separate check. Don't use staging as Eric is about to put some changes from Vadim in there. When Vadim's changes pass (tonight I hope), the appserver update is next. Tomorrow, you will have to merge up and then we will apply to staging, reconvert and do runtime testing there.

I don't see the Asynchronous request object handle mentioned in this task, but please let me know if this is necessary for this task.

We don't need to handle the Asynchronous request handle object for milestone 4. We will probably go ahead and implement it for milestone 7, unless it is too much work.

The changes passed regression testing. Please check them into bzr and distribute the update.

Committed to bzr revision 10179.

[javac] .../server/common/Apitest1.java:134: error: no suitable method found for connect(String,character,character,character)

Regarding this problem:

It seems that AppServer handle can be called with a connect() signature different from Socket and WebServices.

The connect() method of AppServer can have the following parameters connect(connect_options, user, password, server_info).
where only connect_options is mandatory and the rest are optional and if not given they are considered ?.

The result is we can have connect with 1, 2, 3 and 4 parameters.

As a solution I've created an interface called ConnectableServer which extends connected, and emmited unwrap to ConnectableServer when we have the Connect statement with more than 1 parameter.

How would you like the connect with 2, 3 and 4 parameters to be handled? make all the combination possible of String and character or have the parameters wrapped as character and only create versions with character?

LE: GES removed a customer-specific package path from this entry.

You can force things to be wrapped in that case where there is more than 1 parameter.

Code Feedback:

1. The javadoc and formatting of ConnectableServer should be this:

/**
 * Provides access to the following handler methods.
 * <table>
 * <tr>
 *    <th>attribute or method</th>
 *    <th>API</th>
 * </tr>
 * <tr>
 *    <td>CONNECT()</td>
 *    <td>{@link #connect(character, character, character, character)},
 *        {@link #connect(character, character, character)},
 *        {@link #connect(character, character,)}
 *    </td>
 * </tr>
 * </table>
 */
public interface ConnectableServer
extends Connectable

instead of this:

/**
 * Provides access to the following handler methods           <----------  HERE
 * handler methods.                                           <----------  HERE
 * <table>
 * <tr>
 *    <th>attribute or method</th>
 *    <th>API</th>
 * </tr>
 * <tr>
 *    <td>CONNECT()</td>
 *    <td>{@link #connect(character, character, character, character)},
 *        {@link #connect(character, character, character)},
 *        {@link #connect(character, character,)}
 *        {@link #connect(character)}                       <--------- HERE
 *    </td>
 * </tr>
 * </table>
 */
public interface ConnectableServer extends Connectable      <--------- HERE

2. Merge methods_attributes.rules to the latest bzr version.

Upload the changed version I will get it conversion tested.

Conversion testing passed. Check it in and distribute it.

Checked in to bzr as revision 10310.

The Progress Explorer tool on windev01 is not working. I want to use to test how agents can be configured and how clients can connect to it.

Some thoughts in regard to the design:

• Each appserver agent will be a separate P2J client process that connects to the P2J server and calls an appserver-specific entry point.
• On the P2J server side, each of the above noted appserver agent processes will have a session context and the conversation thread would be put into a thread pool, waiting for requests to process.
• The appserver broker facility does not really exist in the same way on P2J since it doesn't need to connect the appserver client with the appserver agent. Instead, any appserver client code would be re-written to connect to the P2J server and call into a standard appserver invocation entry point. This would handle any session setup (e.g. connect procedures, security context switches) and then the code would post a request for the appserver agent processes that are waiting in the thread pool (the first point above).
• The appserver agent processes would automatically be set into batch mode.

See #2124 and #1787 for related thoughts.

1. connection to a AppServer can be done either by specifying the Broker's port directly, or by specifying the NameSpace port and the AppServer's service name. You mention that "any appserver client code would be re-written to connect to the P2J server and call into a standard appserver invocation entry point". Does this mean that any AppServer client will send requests to the same socket? If so, then this means that each request will have to use the service name, as if the NameSpace was used. And we can't map the port for direct connection to a single service name, as the same AppServer can have multiple service names (unless we make this a limitation). More, I think we can let the runtime disambiguate which AppServer is targeted, based on the received CONNECT parameters.
2. for each AppServer, the initial, minum and maximum number of agents can be configured. Also, I noticed that OpenEdge allows agents to be added/removed at runtime, for a certain AppServer. So, if a P2J Client can represent only one AppServer Agent, how will the P2J Server be able to start new Agents?
3. in session-free operating mode, the AppServer client can establish multiple connections, to i.e. send async requests. I don't see any problems here, the AppServer Client will just have to establish that number of connections and each time RUN statements are invoked, it will choose an unbound connection.
4. at some point, we will need to decide which settings can be configured/managed using the Admin Console.
5. each AppServer has its own working dir and its own propath. These configurations (together with other settings, like operating mode) can be on the P2J client associated with the AppServer, but I think will be best to keep them in the directory, as it will allow us easier future management via Admin Console.
6. how will the P2J Client which acts as an AppServer Agent authenticate with the P2J Server? Certificate, user/pass?
7. async requests events work in a similar way as socket CONNECT event does. When the async request is complete, the PROCEDURE-COMPLETE event is queued, and will be picked up by the next UI blocking statement or WAIT-FOR, or PROCESS EVENTS statement. The good part is that, in session-managed mode, all async requests are treated by the same Agent (they are queued and results returned in the same order as they requests were sent). Only in session-free mode async requests are trully async, and the response can be received in a different order as the requests were sent, as the requests can be treated by any unbound Agent.
8. activate/deactivate procedures are executed depending on the agent's bound/unbound state. This state can be explicilty set, using the SESSION:SERVER-CONNECTION-BOUND-REQUEST attribute. Not sure yet, but I think we might need to integrate some basic support for the agent's bound/unbound state.

Does this mean that any AppServer client will send requests to the same socket? If so, then this means that each request will have to use the service name, as if the NameSpace was used. And we can't map the port for direct connection to a single service name, as the same AppServer can have multiple service names (unless we make this a limitation).

My idea was the following:

1. Java code that was written to call classes in the Progress 4GL appserver jar file (the "Progress Java Open Client") would be re-written by hand to use the P2J net package and remote object protocol. They would call a new appserver invocation entry point that would be exported as part of the code you are writing.

2. For 4GL code that accesses the appserver, I would want to automatically convert it to equivalent calls in the converted environment (without any manual re-writing of code). I would hope that these invocations could also use the same appserver invocation entry point as for number 1 above. However, there are some interesting cases to consider:

• If the client really does need to connect over a network to a remote P2J server, then we would need to enable/allow that.
• The common case will be were the converted 4GL clients and the converted 4GL server code runs in the same P2J server. In that case, the calls should not need to go over the network, but should be direct inside the same JVM.

The invocation entry point can have a method signature (or there can be multiple entry points) to provide all the variability/options that exist in the 4GL. But I see no reason to actually directly expose the appserver agents via specific ports or service names at the operating system/socket level. Hopefully, we can virtualize the same behavior inside our server. Perhaps we can map specific P2J appserver clients to specific "ports" or "service names" that are then selected via the appserver invocation entry point.

So, if a P2J Client can represent only one AppServer Agent, how will the P2J Server be able to start new Agents?

I'm open to suggestions. I want to make it as easy as possible to setup and manage. I was planning some work in #2124 to improve this, but you may need to figure this out as part of your work.

each AppServer has its own working dir and its own propath. These configurations (together with other settings, like operating mode) can be on the P2J client associated with the AppServer, but I think will be best to keep them in the directory, as it will allow us easier future management via Admin Console.

Agreed.

And we should be able to easily customize it from the command line when launching an appserver client.

how will the P2J Client which acts as an AppServer Agent authenticate with the P2J Server? Certificate, user/pass?

The certificate method should be possible so that a more secure environment can be built. BUT it seems to me that all process account environments will need to be allowed without certificates too (as they are in the 4GL). The idea here is that we cannot force all customers to improve their security, especially since that improved security comes at a cost: lots of extra configuration and key management. I don't think the customer will want to do all that config/mgmt as part of deploying their servers.

So we will have to allow the client to come in and specify the account under which it is processing, with an optional password.

1. In 4GL, each AppServer has a Broker which can start Agents. Unless we hard-code the number of available agents (i.e. we don't allow creation of new Agents and create the maximum number of Agents for an Appserver upfront), then we need a very limited broker implementation on the P2J Client, which:
   - when starting a P2J AppServer, the P2J Broker will start the initial number of P2J Agents
   - when the P2J Server determines that a new Agent is needed (as existing ones are busy), it calls back to the P2J Broker to attempt to start a new P2J Agent
   - the only question here is: can more than one P2J Client work in the same JVM at this time? If yes, great, but else, we will need to make this possible.
   Note that this approach will also allow us to create/delete P2J Agents, in the Admin Console, while the P2J Server is running.
2. we don't change the way we made the CONNECT method convert. Our runtime will still work with the CONNECT method's signature, but we can add some more info via hints. The hints will map a certain AppService/host/{port|service name} to a P2J Server (note that when CONNECT is called, AppServer name is needed only if connecting via a NameServer, so the mapping will be a little more complex). Actually, thinking about it, we can hard code this mapping in the directory, as it will be easier to maintain over time. No matter how we keep this mapping, we will use it to decide if the AppServer call is a networked call or is a call which can be handled in the same P2J Server.
3. if we don't want to modify any existing 4GL code by hand, then we will need to manage all the connect-related properties for an AppServer, in the directory. The legacy AppServer's settings (like NameServer, host, port, service names) will be used to add the Agent to the correct pool and also build a reverse mapping to the Agent Pool and the P2J Broker; this mappings will be used to handle CONNECT requests (i.e. determine which P2J Server implements the AppServer).
4. when a CONNECT method is called, the P2J runtime will determine which server to invoke; after this, it will use the dedicated APIs and the disambiguated P2J server (based on the mappings on the previous point) to send the CONNECT request. This will mean that the AppServer data will need to be managed both on the caller and the target P2J Server (and note that the mapping can be different, as the 4GL client code which connects to the AppServer can use different host names than the 4GL Server defining the AppServer). After the P2J server receives a CONNECT request, it will determine which agent pool to use based on its mappings. If the pool is empty, it will invoke the associated P2J Broker to create a new Agent (and act accordingly if this fails or succeeds).
5. I can't find any info in the documentation related to what the 4GL Broker does, when a newly created 4GL Agent finishes its job; from my testing, it looks like it doesn't 'kill' it, but it will allow to continue serving other requests.
6. when a CONNECT request is received, if no Agent is available and no other Agent can be created by the Broker, 4GL puts the request in a queue. The documentation doesn't mention a timeout for Stateless mode (i.e. it will queue it indefinetly). For State-free, it will queue it but there will also be a timeout (which again is not mentioned by the docs). I need to finish this testing to determine the real-life behavior.
7. in State-free mode, when the 4GL Client calls CONNECT method, the documentation states that it can reserve a number of Agents upfront (I guess this is usable for async calls). This means that in this case, the P2J Server defining the AppServer will need to reserve and create a pool of Agents for this specific connection; also, 4GL allows this pool to increase to a determined maximum size; each time requests are received via this connection, it will always use an agent in this specific pool.

when the P2J Server determines that a new Agent is needed (as existing ones are busy), it calls back to the P2J Broker to attempt to start a new P2J Agent

1. My primary concern here is in getting the environment and pseudo-terminal/terminal setup correct for the "agents". In other words, when agents are started by a parent process, the operating system can implement limits on what those child processes can do. These limits may break features that we rely upon. We also may have a difficult time implementing any features such as starting an agent with a specific operating-system user-account. Even the launch of a child process with a specific working directory may be difficult since the Java process launching facilities are very weak.

2. The general architecture for such a "agent starter" would be to have it be a special kind of P2J client that calls into a special entry point on the server and waits for instructions on how/what to start. We might even just create a queue of start requests and it will process in order and then wait for new requests when the queue is empty.

3. We will have to determine if there are specific parent process/child process relationships in place in the 4GL between the broker and the agents.

the only question here is: can more than one P2J Client work in the same JVM at this time? If yes, great, but else, we will need to make this possible.

This is not possible in P2J today.

I don't think it will be possible in the future, because there are some client features which simply cannot be virtualized. We must plan that every agent will have its own process (and thus its own JVM, environment, working directory, operating system user account...).

Actually, thinking about it, we can hard code this mapping in the directory, as it will be easier to maintain over time.

Yes, drive everything from the directory.

if we don't want to modify any existing 4GL code by hand, then we will need to manage all the connect-related properties for an AppServer, in the directory

Yes.

The customer reports that they only use stateless mode. We should probably discuss this with them and determine exactly the features they need (and the configurations they actually use). I would like to put support in for all features, but we had better know if we can defer any of the work so that we can make future planning decisions appropriately. Please discuss this with Guy Mills as needed.

1. My primary concern here is in getting the environment and pseudo-terminal/terminal setup correct for the "agents". In other words, when agents are started by a parent process, the operating system can implement limits on what those child processes can do. These limits may break features that we rely upon.

Unlike batch processes, I can't find where the Agents are writing or reading data, when the I/O streams are not redirected. Launching child processes is allowed from Agents, but if the child process is something which is using standard I/O, then it will launch, but no I/O will be possible (launching i.e. notepad.exe shows the process in Task Manager, but the actual notepad.exe window is not shown; the Agent does wait for the UNIX command to finish, which is possible only by manually terminating the notepad.exe process, as no access to it is available). Also, Agents don't seem to be created as sub-processes, they look like are managed in the same _proapsv.exe process, as I see only one _proapsv.exe instance in Task Manager, for my AppServer; this would explain why the Agents have no standard I/O available.

We also may have a difficult time implementing any features such as starting an agent with a specific operating-system user-account.

4GL doesn't allow a different OS user-account for the Agent; this setting is at the Broker, and it is inherited by all the agents.

the only question here is: can more than one P2J Client work in the same JVM at this time? If yes, great, but else, we will need to make this possible.

I don't think it will be possible in the future, because there are some client features which simply cannot be virtualized. We must plan that every agent will have its own process (and thus its own JVM, environment, working directory, operating system user account...).

If we make the P2J Agent a child process, I think Java's ProcessBuilder class may be of help, as it allows setting explicit environment and working directory for the sub-process. As the Agent inherits the Broker's OS user account, I see no problems related to this when starting the P2J Agent as sub-process.

The customer reports that they only use stateless mode.

At this time, I'm not so worried about the Agent's operating modes - I think these can be coded easily, once we have the AppServer architecture in place. But, I'll work with them to see what settings they use for their AppServers.

Good. That all sounds promising.

When connecting to an app server, how will the app server client (on P2J server S1) connect to the P2J server running the app server (on P2J server S2)? I we use the user/process authenticated on S1, then the user/process will have to be configured on both S1 and S2. But, as we need data about a certain appserver on both S1 and S2, I suggest to use the appserver's account when establishing the server-to-server virtual connection (as the authenticated P2J user on S1 has no meaning on the remote side, S2).

Agreed.

I am interested in learning more about what you have found about the requirements for the appserver agents. This includes startup, environment, management and so forth. When you have these details and ideas on implementation (ProcessBuilder details...), please share them here.

The design of the appserver implementation in P2J is split in these parts:
I. appserver configuration and management
II. launching and administration of the appserver(s)
III. connecting to the appserver residing on P2J Server 2 (S2), from P2J Server 1 (S1) (via 4GL code).
IV. connecting to the appserver residing on P2J Server 2 (S2) from Java code.

I. appserver configuration and management¶

1. To manage the configured appservers which can be launched by a P2J Server, the directory.xml will have all appservers defined under the server/standard/appservers node, as in:
   

           <node class="container" name="appservers">
               <node class="appserver" name="ca_srv">
                  <!-- broker settings -->
                  <node-attribute name="operating_mode" value="stateless"/>
                  <node-attribute name="port_number" value="2222"/>
                  <node-attribute name="name_server" value="NS"/>
                  <node-attribute name="service_names" values="ca_srv"/>
                  <node-attribute name="service_names" values="bla"/>
                  <node-attribute name="max_clients" value="512"/>
                  <node-attribute name="priority" value="0"/>
                  <node-attribute name="registration_retry" value="30"/>
                  <node-attribute name="startup_timeout" value="3"/>
                  <node-attribute name="request_timeout" value="15"/>
                  <node-attribute name="auto_trim_timeout" value="1800"/>
                  <!-- agent settings -->
                  <node-attribute name="propath" value="."/>
                  <node-attribute name="initial_agents" value="5"/>
                  <node-attribute name="min_agents" value="1"/>
                  <node-attribute name="max_agents" value="10"/>
                  <node-attribute name="activate" value="activate.p"/>
                  <node-attribute name="deactivate" value="deactivate.p"/>
                  <node-attribute name="connect" value="connect.p"/>
                  <node-attribute name="disconnect" value="disconnect.p"/>
                  <node-attribute name="startup" value="startup.p"/>
                  <node-attribute name="shutdown" value="shutdown.p"/>
                  <node-attribute name="startup_parameter" value="bla"/>
               </node>
          </node>
   

   
   This appserver definition contains all the broker configuration and the agent configuration. Note that some attributes will never be used by P2J, like port_number, and we can either leave them as legacy info or can eliminate them.
   Each P2J account which has the appserver behavior will have this attribute:
   

      <node-attribute name="appserver" value="ca_srv"/>
   

   
   When connecting to the P2J Server whith an account marked with the appserver attribute, that client will be part of the appserver (manager or agent).
2. The second part of managing appservers is related to the requester side, when a P2J client needs to connect to a remote (or local) appserver. For this, the server/default/app_services will contain mappings of 4GL-style connect options to a P2J server running that appserver:
   

           <node class="container" name="app_services">
             <node class="container" name="ca_srv">
               <!-- the appserver is running on this P2J Server, with this subjectId -->
               <node class="integer" name="p2j_port">
                  <node-attribute name="value" value="2222"/>
               </node>
               <node class="string" name="p2j_host">
                  <node-attribute name="value" value="p2jhost"/>
               </node>
               <node class="string" name="p2j_account">
                  <node-attribute name="value" value="p2jaccount"/>
               </node>
   
               <!-- route any of the following to the specified p2j server-->
               <node class="strings" name="aliases">
                  <node-attribute name="values" value="alias"/>
               </node>
               <node class="strings" name="nameservers">
                  <node-attribute name="values" value="ns_host"/>
               </node>
               <node class="strings" name="hosts">
                  <node-attribute name="values" value="host"/>
               </node>
               <node class="integers" name="ports">
                  <node-attribute name="values" value="5444"/>
               </node>
               <node class="strings" name="services">
                  <node-attribute name="values" value="service"/>
               </node>
   
               <!-- this application is default for the following nameservers -->
               <node class="strings" name="defaults">
                  <node-attribute name="values" value="ns_host_or_ip"/>
               </node>
             </node>
           </node>
   

   
   The idea here is that, when a server:CONNECT call is encountered and this call is for an appserver, it will use the specified mappings to determine which P2J server runs that appserver:
• aliases - the aliases for this app server
• as we can connect in different ways to a certain appserver (by adjusting CONNECT statement's options), the following need to be populated not with the settings where the appserver is defined, but with the options used throught the legacy code to connect to this appserver:
  - nameservers - list of nameservers when "-H" option is specified and not connecting in direct mode
  - hosts - list of "-H" hosts, when "-DirectConnect" is specified
  - ports - list of ports when "-S" targets a port
  - services - list of services when "-S" targets a service
• in 4GL, is possible to connect to an appserver by specifying only the nameserver; in this case, 4GL looks for the default service for that NS. In P2J, the "defaults" list will specify, for an appserver, the nameservers for which it is the default (using the nameserver's ip or host, as used in CONNECT's options); runtime will take care of not allowing a NS to have more than one default appserver.
• the p2j_account attribute is the account which will be used to connect to the remote P2J server; this account is expected to be configured to run the proper appserver.
  The rules to search are, using the CONNECT statement's appserver-related options:
  - if "-S" option is not specified, default port to 5162;
  - if "-H" option is not specified, default to locahost;
  - if "-DirectConnect" is not specified and "-AppService" is not specified, it means the "-H" host is the host of a nameserver; so, it searches the default for that nameserver host
  - go through all defined app_services and return the first one which satisfies all following conditions:
  

  1. "hosts" contains the "-H" host
  2. if port is specified, "ports" contains the "-S" port
  3. if service is specified, "services" contains the "-S" service
  4. "-DirectConnect" is specified OR "-AppService" is specified and "aliases" contains the specified service
  

II. launching and administration of the appserver(s)¶

• no agent management will be needed on the P2J Client side.
• the P2J agents or "manger" will not open any ports. Sending commands to these agents will be done using a special exported network server, AppServerEntry, which is called by a remote P2J Server, and has APIs to, i.e:
  - establish a connection
  - disconnect
  - invoke procedures/functions
• the P2J Clients which act as agents will be started using a ProcessBuilder instance, which will inherit from the manager's P2J Client:
  - environmentt
  - the working dir
  - the library path
  - the Java classpath
  - any JVM options (note that the debug-related options will be dropped, unless we make it use different debug ports)
  - the bootstrap configuration, which will be passed using categ:group:key=value, from this client's BootstrapConfig instance.

• terminate
• invokePersistent (for RUN ... ON SERVER ... PERSISTENT cases)
• invoke: for RUN ... ON SERVER (without PERSISTENT), RUN ... IN handle, DYNAMIC-FUNCTION( ... IN handle), FUNCTION ... IN handle cases - i.e. all RUN and function call cases when the target procedure handle is a remote procedure or ON SERVER clause is used, without the PERSISTENT clause.
• note that the async request invocation is not fully-designed yet. I will work on the implementation only if I can come up with something simple, to manage it (the server project does not use RUN ... ASYNC calls).
  4. the agent management is done depending on the configured operating mode.

III. connecting to the appserver residing on P2J Server 2 (S2), from P2J Server 1 (S1) (via 4GL code).¶

1. determine if the target is an appserver or webservice; only appserver case will be explained here
2. determine which P2J Server runs the appserver (based on the configured app_services and the CONNECT statement's options)
3. S1 will establish a server-to-server connection (if not already established) and will use the configured p2j_account to authenticate on S2
4. once a connection is established, S1 will obtain a proxy to the AppServerEntry instance exported by S2. This will also take care to obtain a local instance, if S2 is actually S1 (i.e. the appserver is ran by the same P2J Server).
5. S1 calls AppServerEntry.connect to establish the connection (by sending the "connect" command to the appserver's manager), and saves the returned connection ID. On S2 side, it will reserve agents (or not) depending on the operating mode.
   The RUN ... [PERSISTENT [proc-handle]] ON SERVER server-handle case:
• the statement will use the server-handle's AppServerEntry instance to execute the invoke
• if PERSISTENT clause is used:
1. S2 will save the procedure in the agent's procedure list (and the agent becomes bound)
2. S2 will send to S1 the ID for the PersistentProcedure resource instance created by this call (the ID is obtained via handle.toString(), i.e. is the resource's hash code).
3. S1 will create a RemoteProcedureWrapper instance where the ServerImpl instance and the code sent by S1 will be saved.
4. RemoteProcedureWrapper will send all allowed procedure attribute/method calls to the specified server. Note that only a few attributes/methods are allowed for remote procedures (this good as it simplifies some things).
5. if an explicit handle is used, that handle will have as resource the RemoteProcedureWrapper instance.
   When IN remote-proc-handle is used, P2J will send the call to the ServerImpl saved at the RemoteProcedureWrapper, and targeting the resource with the specified ID
   Some thoughts about async requests:
6. only State-free allows trully async requests
7. in this case, 4GL allows us to establish multiple connections upfront, when CONNECT statement is called. But, this does not reserve agents in any away, this are just communication paths. In P2J case, we can:
• obtain a specific number of virtual sessions from S1 to S2 and pool them
• when executing an async request, it will create a separate thread which will extract a session from the pool (or create one, if possible), execute the request, and will generate the event for this async request, after the response is received
• on the remote side S1, all will work the same in terms of agent management.

IV. connecting to the appserver residing on P2J Server 2 (S2) from Java code.¶

   public Session connectVirtual(InetSocketAddress address,
                                 String            certAlias,
                                 SessionListener   notify,
                                 String            account)

1. connect to the remote side:
   

   InetSocketAddress address = ...; // the address of the remote P2J server
   Session session = sessMgr.connectVirtual(address, null, null, p2jAccount);
   

2. obtain the AppServerEntry proxy:
   

   boolean local = sessMgr.isLocalNode(address);
   AppServerEntry appServer = null;
   if (local)
   {
     appServer = (AppServerEntry) RemoteObject.obtainLocalInstance(AppServerEntry.class, true);
   }
   else
   {
     appServer = (AppServerEntry) RemoteObject.obtainNetworkInstance(AppServerEntry.class, session);
   }
   

3. establish the appserver connection:
   

   boolean sessionFree = false; // false indicating that this is a Session-free mode, true to indicate a Session-managed mode
   character connectionId = appServer.connect(sessionFree, user, pwd, serverInfo);
   if (connectionId.isUnknown())
   {
      // show/manage connection error
      ...
   }
   

4. invoke stuff on the remote side, using the AppServerEntry proxy. Note that this needs to be invoked 4GL-style (at least to handle raised conditions):
   

   appServer.invoke(connectionId, name, ..., modes, args);
   

1. In 4GL, if the PROPATH function is used at runtime and to set the propath to i.e. ., then all RUN statements will be able to execute external programs only from current folder. Currently, I see that only SEARCH function uses this managed propath and SourceNameMapper uses only the hard-coded propath set at file-system/propath in the directory and returned by EnvironmentOps.getLegacySearchPath.
2. please confirm this: when invoking methods via a network server, the passed arguments are immutable, right? I mean, no parameters changes (assuming the parameter is a mutable object, like the P2J wrappers) made on the remote side are sent to the requester side.

• requester side sends the table's definition (plus the data, in case of INPUT/INPUT-OUTPUT mode).
• remote side creates the table and populates it (if INPUT/INPUT-OUTPUT mode).
• remote side invokes target.
• on finish, if OUTPUT/INPUT-OUTPUT mode, remote side serializes the table again (data should be enough this time).
• on finish, requester side receives the table's data (if OUTPUT/INPUT-OUTPUT mode) and will replace all existing data.

I think for the TEMP-TABLE parameters we will need the dynamic temp-table support, because this way we can let the remote call flow naturally once the temp-table is in place.

I have reviewed the design in detail. It looks very good.

Here are my questions/comments:

1. Is there any need for this information (e.g. the legacy port number)?

This appserver definition contains all the broker configuration and the agent configuration. Note that some attributes will never be used by P2J, like port_number, and we can either leave them as legacy info or can eliminate them.

If we don't have to use it in any way and there is no way for the value to be queried/set by 4GL code, then I don't see why we would want it to be stored in the directory.

2. What is the purpose of p2j_port and p2j_host in the app_services section?

3. If I understand the following correctly, you mean that the first P2J client to start which is in appserver mode will have its server-side thread/session defined as the appserver manager. I assume this is similar to the Progress appserver broker?

on the P2J Server side, when establishing the first P2J client connection for a specific appserver, this first connection will act as the appserver manager. This client will listen for commands to administer the agents (i.e. start/stop agents, automatically trim agents, etc).

4. All the management will be done by the appserver manager's server-side thread inside the server itself (without needing anything to be processed on the client except when it must start a new agent)?

5. I assume that the appserver manager itself will not process invocation requests? I guess it is just there for management only and it stays around for the life of the P2J server.

6. What happens if the P2J client that is the appserver manager abends or is accidentally killed without the P2J server also being killed/shutdown?

7. I wonder if it makes sense to take your design ("first P2J client is the appserver manager") a step further?

What if we have a Java launcher that implements the ProcessBuilder facility. That launcher can do the following:

• Launch the P2J Server itself (as the first thing it does).
• Launch any appserver agents as needed.
• Launch batch processes as needed. We could enhance the directory to define how/when to launch batch processes.

In cases where we don't need appserver or batch clients, then we might not have this extra process but instead just launch the P2J server the way we do today. I guess this launcher approach could be done as a separate "Driver" program or it could be done as an option to the ServerDriver.

Advantages: it unifies the architecture for all of this and simplifies the server startup since only one process needs to be launched.

The key question in my mind is whether there is anything related to the environment, working directory etc... that would be needed for the P2J Server process or for batch processes, that we would not be able to duplicate with the ProcessBuilder approach.

8. In regard to the explicit Java connection, why are we using a boolean flag instead of some enum with 4 possible session types?

false indicating that this is a Session-free mode, true to indicate a Session-managed mode

9. I would like to hide some of the complexity of connecting to appservers from Java code. I also don't understand why we would need to use connectVirtual() when we are talking directly to the P2J server we want.

• Call connectDirect() to create a new ses sion with the P2J server hosting the appserver code we want to access OR call connectVirtual() if the Java code is running in P2J server S1 and we need to access P2J server S2.
• Call a helper function to obtain the AppServerEntry object AND to establish the connection (via connect()). This helper would hide the determination of local/remote for the session. I would also like the AppServerEntry object to hide the connection ID stuff if this is reasonable to do. In other words, if the AppServerEntry object is specific to the connection you are using, then the caller of invoke() would not have to pass the connection ID.

10. Yes, we need to fix the SourceNameMapper to use the effective propath. Eugenie is also working on a fix for propath assignment (see #2130).

SourceNameMapper uses only the hard-coded propath set at file-system/propath in the directory and returned by EnvironmentOps.getLegacySearchPath.

11. Yes, parameters passed via RemoteObject proxies are immutable. So we must invent something to implement:

• OUTPUT parms
• INPUT-OUTPUT parms
• temp-table/result set processing

I will post some additional information about temp-table serialization shortly.

-------- Original Message --------
Subject:     Progress information of transfer of temp-tables...
Date:     Wed, 13 Mar 2013 16:44:07 +0000
From:     Guy Mills

I’ve found some documentation on Progress Java Open Client, I’m not sure if that’s what we’re using, but it sounds about right (someone correct me if I’m wrong though). 

http://documentation.progress.com/output/OpenEdge102b/pdfs/dvjav/dvjav.pdf

Page 4-9 explains further about how data is passed across – from my understanding, it does agree with what Mark Edwards said regarding passing data across.

Regards,
Guy

1. Is there any need for this information (e.g. the legacy port number)?

OK, I'll leave out of the directory any appserver-configuration info which can't be seen by 4GL code and is not used by P2J.

2. What is the purpose of p2j_port and p2j_host in the app_services section?

The idea is to identify the P2J server which runs the appserver, based on the CONNECT statement's options.

3. If I understand the following correctly, you mean that the first P2J client to start which is in appserver mode will have its server-side thread/session defined as the appserver manager. I assume this is similar to the Progress appserver broker?

Yes, to some extent. It will be responsible of managing the agents (start/trim/end from admin console), but it will not have any responsibility in actually assigning agents to the incoming connections/requests. I have a different data structure (a pool for each appserver) which will be responsible for this, and the access to it can be done from any thread which is authenticated as that appserver.

4. All the management will be done by the appserver manager's server-side thread inside the server itself (without needing anything to be processed on the client except when it must start a new agent)?

Correct. I couldn't find any gain in letting the P2J client manage appserver stuff.

5. I assume that the appserver manager itself will not process invocation requests? I guess it is just there for management only and it stays around for the life of the P2J server.

Correct.

6. What happens if the P2J client that is the appserver manager abends or is accidentally killed without the P2J server also being killed/shutdown?

At this time, it is coded to fully terminate the entire appserver (i.e. send the terminate command to all the agents).

7. I wonder if it makes sense to take your design ("first P2J client is the appserver manager") a step further?

What if we have a Java launcher that implements the ProcessBuilder facility. That launcher can do the following:

• Launch the P2J Server itself (as the first thing it does).
• Launch any appserver agents as needed.
• Launch batch processes as needed. We could enhance the directory to define how/when to launch batch processes.

A P2J Server should be able to start more than one appserver. And, note that agents can be started dynamically (explicitly or implicitly), so this means that this Launcher needs to be more complex than just start appservers upfront.

In cases where we don't need appserver or batch clients, then we might not have this extra process but instead just launch the P2J server the way we do today. I guess this launcher approach could be done as a separate "Driver" program or it could be done as an option to the ServerDriver.

Considering that we need access to it from within the P2J Server, I would implement it in the ServerDriver. I think it would be a good idea to mark batch processes/appservers as "auto-start" (or maybe even simulate cron jobs, for end of day/month/etc batch processes). If we go this path, we will need to hard-code in the directory additional info to start it (like the bootstrap configuration for the client), but I think it will be cleaner and easier to manage if each appserver/batch processes will be configured to know the startup script and the working dir where it will started. Else, we will need to get all the info which we now have in .sh and .xml files into the directory.

Advantages: it unifies the architecture for all of this and simplifies the server startup since only one process needs to be launched.

I agree with this, will be much cleaner to start a P2J server with appservers/batch processes.

The key question in my mind is whether there is anything related to the environment, working directory etc... that would be needed for the P2J Server process or for batch processes, that we would not be able to duplicate with the ProcessBuilder approach.

Note that each appserver can have its own working dir, so we will still need to launch processes in separate directories; and from my testing, the ProcessBuilder is OK in setting the environment and the working dir for the new process.

8. In regard to the explicit Java connection, why are we using a boolean flag instead of some enum with 4 possible session types?

false indicating that this is a Session-free mode, true to indicate a Session-managed mode

In 4GL code, when CONNECT statement is used, it can state only one of the two: Session-free or Session-managed. The appserver client can't see which one of the (currently 3) Session-managed modes the appserver is running.

9. I would like to hide some of the complexity of connecting to appservers from Java code.

Yes, I'm thinking of that.

I also don't understand why we would need to use connectVirtual() when we are talking directly to the P2J server we want.

Correct, when connecting from outside P2J server, connectDirect can be used.

• Call connectDirect() to create a new ses sion with the P2J server hosting the appserver code we want to access OR call connectVirtual() if the Java code is running in P2J server S1 and we need to access P2J server S2.
• Call a helper function to obtain the AppServerEntry object AND to establish the connection (via connect()). This helper would hide the determination of local/remote for the session. I would also like the AppServerEntry object to hide the connection ID stuff if this is reasonable to do. In other words, if the AppServerEntry object is specific to the connection you are using, then the caller of invoke() would not have to pass the connection ID.

At this time, the AppServerEntry object is a network server exported by the P2J Server running the appserver; so, it can't be connection specific. And I don't think is possible to export connection-specific network servers. So, the only approach remaining is to hide this in helper methods/classes: when CONNECT is called, return an instance which allows access to invoke APIs and also hides internally the connection ID and the AppServerEntry.

10. Yes, we need to fix the SourceNameMapper to use the effective propath. Eugenie is also working on a fix for propath assignment (see #2130).

OK. Will Eugenie work on the SourceNameMapper part too? IMO all it needs to be done is to add a context-local class where the propath is used, and check that propath when the class is searched.

11. Yes, parameters passed via RemoteObject proxies are immutable. So we must invent something to implement:

• OUTPUT parms
• INPUT-OUTPUT parms
• temp-table/result set processing

My idea was to let the AppServerEntry.invoke* APIs return a InvocationResult object which will hold the returned value, updated OUTPUT/INPUT-OUTPUT params and any other needed info (like the new temp-table data). And build a helper class which hides the AppServerEntry call plus the post-invocation parameter processing.

Some info about agent's context management: I've tested all modes, by creating a global shared var in the connect procedure (or activate for state-free) and looks like only State-reset mode actually resets the agent's context. To implement this, I think we need to take a snapshot of the agent's context right after the connect procedure was called, to save the state of all context-local data. This is somehow difficult, as we need to clone all context-local data, and the current implementation (were we either use WorkArea-like classes or anonymous classes as storage) doesn't support cloning.

10. Yes, we need to fix the SourceNameMapper to use the effective propath. Eugenie is also working on a fix for propath assignment (see #2130).

OK. Will Eugenie work on the SourceNameMapper part too? IMO all it needs to be done is to add a context-local class where the propath is used, and check that propath when the class is searched.

I will have Eugenie add the SourceNameMapper propath changes in. Can you suggest some simple testcases that he can use to test this out?

I will have Eugenie add the SourceNameMapper propath changes in. Can you suggest some simple testcases that he can use to test this out?

• foo/foo.p
  

  message "foo".
  

• bar/bar.p
  

  message "bar".
  

• test1.p
  

  propath = "foo". /* this appends foo to the system propath */
  propath = "bar". /* this appends bar to the system propath. "foo" is no longer included. */
  message propath.
  run bar.p. /* shows "bar" */
  run foo.p. /* foo.p can't be found */
  

• test2.p
  

  propath = "foo,bar". /* this appends foo and bar to the system propath */
  message propath.
  run bar.p. /* shows "bar" */
  run foo.p. /* shows "foo" */
  

Actually, I see that the PROPATH assignment is implemented in EnvironmentOps.set/getSearchPath, and is saved in the EnvironmentOps.WorkArea.propath, for each context.

11. Yes, parameters passed via RemoteObject proxies are immutable. So we must invent something to implement:
...
temp-table/result set processing

1. when called from inside P2J code, the result-set set will be used to read and send (for INPUT case) or to update (for OUTPUT case) the associated temp-table.
2. when called from outside P2J code, there will be:
• APIs to access the result-set associated with a certain parameter, after the call has finished
• APIs/interfaces to build a result-set associated with an INPUT/OUTPUT/INPUT-OUTPUT table parameter

• receive result-sets and will recreate the temp-table from its metadata
• create and return the result-set for OUTPUT/INPUT-OUTPUT cases

Yes, I think your approach with result-sets is correct. Pure Java code (like BPM) does not want to be dealing with legacy temp-tables directly AND we don't want to replicate the persistence layer outside of the P2J server. This keeps it simple. I believe this is also how Progress does it in the Open Java Client. Have you taken a look at the "<app_name> Comms" example to confirm?

LE: GES removed the customer-specific app name from this entry.

I believe this is also how Progress does it in the Open Java Client.

Yes, this is similar to how the progress documents the handling of temp-table params with the Open Java Client.

Have you taken a look at the "<app_name> Comms" example to confirm?

Yes, but I couldn't analyze it fully, as I'm limitted by the shell editors.

LE: GES removed the customer-specific app name from this entry.

Understood. Good.

I'm trying to make the agents unaware of UI code (to behave as if all output is redirected to /dev/null), but the "headless" mode is not usable (and I don't think it was intended for what I'm trying to achieve). Idea is, the appservers are not aware of a terminal and do not read data from it (i.e. UPDATE, pause before hide message, etc are silently no-op'ed), but if we redirect the unnamed input/output stream to something else, then data will be read/written to it. So, we can't deactivate the entire UI-related code, but just silently ignore it.

What hurts us most is the wait-for-key blocking. The fact that it writes data to the terminal is not that problematic, but key-waiting cases need to be no-op, if the unnamed input is not redirected.

I believe this is the equivalent of batch mode (see #1787). This is expected. I was planning to work on it after I get done with the native library support (which has a long way to go).

The key question: what is the minimum you can do to get through this task?

The key question: what is the minimum you can do to get through this task?

This does not hurt at all the appserver infrastructure, and by starting the agents in swing mode I have access to their input (so I can "unfreeze" it if an agent waits for space or other input).

OK, then just do the following:

1. As part of the appserver agent startup, set the session:batch-mode to true.
2. Make do with the swing client for testing purposes.

I'll complete the batch mode support later.

Something else about the appserver agents: in cases when a direct connection is established from Java code and this connection authenticates using the subject ID associated with the appserver, this needs to act as a normal P2J connection, and not as an agent. So, my solution was to force agent startup only if its bootstrap config has the net:client:appserver set to true (i.e. to inform the P2J server that this P2J client is part of the appserver). In the other case, when this setting is not used and a subject ID (configured as an appserver) is used to authenticate from java code via a direct connection, then this can be used to communicate with the appserver, and acts as a channel for receiveing request from the remote side and dispatching them to the agents.

1. The implementation iteratively looks up the directory node under: /server/<serverID>/runtime/<account_or_group>/batchMode
2. If no user/process or group nodes are present, then this is checked: /server/<serverID>/runtime/default/batchMode
3. If no /server/<serverID>/runtime node exists, this is checked (it is the global default area for all servers): /server/default/runtime/<account_or_group>/batchMode
4. Finally, if no user/process or group nodes are present in the global default area, then this is checked: /server/default/runtime/default/batchMode
5. If no value is found via this lookup, then the default value of false will be returned.

So I don't need to set anything explicitly for the agent, as this is a directory configuration.

then this can be used to communicate with the appserver, and acts as a channel for receiveing request from the remote side and dispatching them to the agents.

I don't understand exactly what you mean by this part. Perhaps you can provide more detail/an example?

And about session:batch-mode: I see that this setting is configured in the directory
...
So I don't need to set anything explicitly for the agent, as this is a directory configuration.

I prefer to set batch-mode automatically for appserver agents. The reason is simple: appserver agents should always be in batch mode. I don't want to rely upon someone remembering to configure the directory properly, in order to get the proper behavior.

As a general rule, I want to minimize the configuration needed to set all this up. In other words, I want sensible defaults (that cover the most common cases) and the minimum configuration needed to customize the facilities for the non-common cases.

I don't understand exactly what you mean by this part. Perhaps you can provide more detail/an example?

1. from Java code, one can authenticate on S1 with any P2J account and the API which creates the appserver connection will receive the explicit appserver name too. This way we don't expose credentials for appserver accounts outside of the P2J server running the appserver.
2. from P2J server S2, the virtual session to S1 will authenticate using the configured account (in S2's app_services section) for the target appserver. This helps us not to duplicate accounts accross servers.

As a general rule, I want to minimize the configuration needed to set all this up. In other words, I want sensible defaults (that cover the most common cases) and the minimum configuration needed to customize the facilities for the non-common cases.

OK.

For OUTPUT and INPUT-OUTPUT parameters from Java code, I was thinking to use the P2J data wrappers + the string representation of the argument's modes. Idea is, Java-type references/values are sent from Java code, and the runtime internals will wrap these into P2J wrappers, which will be sent to the remote side. If this is not OK, we need to build special wrappers for OUTPUT and INPUT-OUTPUT values, to be used only by appserver calls.
More, about the representation of 4GL types which don't map easily to Java types (like memptr and handle) - do we need to build support for these, from Java code?

Let's go ahead and use the P2J wrapper types all the way out to the custom Java code. This way, we can duplicate the full range of results that the 4GL implements (including things like unknown value).

Yes, we can use a string based mode list. That is fine.

Since our types are mutable, we can update the passed in values as defined in the mode string, based on the response.

Plan to support the same types that the 4GL supports. For example: do they support handle from Java code to/from an appserver? I don't see how the handle can be transferred across processes in a meaningful way, BUT if the 4GL does it, then we will support it. For example, if they allow a handle to be returned back and it can then be passed over to the other side and used properly, then we may have to use the STRING/WIDGET-HANDLE approach to serialization/de-serialization.

I do know that MEMPTR is very important (it is used in the app).

Of course, we will need to have "special" versions of MEMPTR (definitely needed) and HANDLE (if needed). For example, the special Java client version of MEMPTR can be backed by a byte[] since real memory operations are not going to be done on that. We can't just use RAW since that has a size limit.

Plan to support the same types that the 4GL supports. For example: do they support handle from Java code to/from an appserver?

The docs state that handle values from 4GL are converted to some com.progress.open4gl.Handle instance.

I don't see how the handle can be transferred across processes in a meaningful way, BUT if the 4GL does it, then we will support it. For example, if they allow a handle to be returned back and it can then be passed over to the other side and used properly, then we may have to use the STRING/WIDGET-HANDLE approach to serialization/de-serialization.

You are right, any handle can be passed back to the remote side, but it has meaning only if it really was created on the remote side; and looks like if a handle is received from the remote side, the requester can use it, but on the requester side the resource to which will end up being mapped is somehow random (I guess it uses the received ID to determine a local resource).

the requester can use it, but on the requester side the resource to which will end up being mapped is somehow random (I guess it uses the received ID to determine a local resource).

I don't fully understand what you mean here. I assume that "requester side" is the hand-coded Java that is using the Open Java Client to connect to the appserver. And I assume that "remote side" is the 4GL code running in the appserver agent.

Is there any scenario in which the returned handle (which was created on the appserver agent and returned back to the requester via something like an output parameter) can actually be used on the requester side? I assume it can be passed back to the remote side in another appserver call AND that when doing this the result is that the called code on the remote side will be referencing the same handle as was originally returned.

But on the requester side it can't really be used in Java, right? I mean there are no 4GL resources to access on that side, so what can a handle actually reference in Java?

Greg Shah wrote:

the requester can use it, but on the requester side the resource to which will end up being mapped is somehow random (I guess it uses the received ID to determine a local resource).

I don't fully understand what you mean here. I assume that "requester side" is the hand-coded Java that is using the Open Java Client to connect to the appserver. And I assume that "remote side" is the 4GL code running in the appserver agent.

Is there any scenario in which the returned handle (which was created on the appserver agent and returned back to the requester via something like an output parameter) can actually be used on the requester side? I assume it can be passed back to the remote side in another appserver call AND that when doing this the result is that the called code on the remote side will be referencing the same handle as was originally returned.

But on the requester side it can't really be used in Java, right? I mean there are no 4GL resources to access on that side, so what can a handle actually reference in Java?

By requester side I meant the 4GL code which initiated the remote request. You are right, in Java code a handle has no meaning and can't be used. Is just a container for a resource ID, and you are correct on its usage.

Some later edits:
4. the AppServerFactory class was renamed to ServerFactory, as a SERVER object handle can connect to either an appserver or webservice. Thus, using the AppServerFactory name can be misleading.
5. async requests were not implemented. This can be split in these parts:
a. the processing of the PROCEDURE-COMPLETE event is the same as the READ-RESPONSE event for sockets. Thus, I think the infrastructure for these kind of events can be part of the socket runtime task, as this has higher priority than the asyn requests.
b. trully async requests are available only in State-free mode. For this mode to support async requests, it can act like this:
- after the initial appserver connection is established, a number of P2J sessions (each with its own AppServerHelper instance) are created and pooled. All these instances will use the same connection ID to communicate with the remote side.
- when a RUN ASYNC request is encountered, an AppServerHelper instance is retrieved from the pool and used to send the request to the appserver.
c. async requests for Session-managed modes are not really async, they are just queued and the same channel is used for communication. All requests are processed in the order they arrive, in a FIFO order.
6. thinking about connection IDs, they are pretty "insecure" at this time. I will add a security mechanism so that an authenticated P2J session (on the remote side) can accept only known connection IDs, when appserver requests are received.

Code Review (0617b)

Wow! This is very, very good work. And it is massive too. I took me most of the day to review it. Some thoughts and questions:

1. The file Protocol.java has no changes compared to the one in bzr.

2. The approach to serialization in TableResultSet will be problematic when the contained data includes conflicting strings (e.g. "END_DATA"). Perhaps it would make sense to serialize explicit sizing information first and then read only the exact amount that is specified?

3. I think that the TableResultSet property support might be cleaner with a PropertyDefinition class that would contain the property name, type, extent... all in 1 small object.

4. Do I understand correctly that the TableResultSet and TempTableResultSet are used on both the P2J server and on the requesting side (which can be some other Java application connecting to a P2J server)? If so, then what kind of dependencies are we adding to such applications? For example, it seems like we may be requiring them to use our Temporary class and Hibernate? That may pull in quite a heavy set of requirements on their side and I am uneasy with the implications.

5. Does the AppServerEntry connect() parameters need to be character type instead of String? If the connect procedures can check unknown value and so forth on those, then it may be necessary.

6. In HandleChain.delete() there is this code:

      // if not chained, exit
      if (!(hasPrevSibling() && hasNextSibling()))
      {
         return;
      }

Shouldn't it be if (!(hasPrevSibling() || hasNextSibling()))? It seems that the first or last resource in the chain would fail this test even though they are chained.

7. Does AgentPool.nextId() need protection to ensure that ID used is not already in use? This generally could only happen in very long running instances, but I guess it may be possible.

8. The Agent.invokeScoped() seems to be incomplete. Its javadoc states that it should return an AppServerInvocationResult instance but it is coded to return a BDT. Is this expected?

9. Does Agent.invokeFailure() need to walk the chain of causes or is the NumberedException always the topmost cause?

10. In Agent.invoke() there is this code: if ("Disconnect".equals(kind)). Is this always supposed to be case-sensitive?

I am fine with the resulting code. Thoughts:

4. Do I understand correctly that the TableResultSet and TempTableResultSet are used on both the P2J server and on the requesting side (which can be some other Java application connecting to a P2J server)? If so, then what kind of dependencies are we adding to such applications? For example, it seems like we may be requiring them to use our Temporary class and Hibernate? That may pull in quite a heavy set of requirements on their side and I am uneasy with the implications.

TempTableResultSet should be used only by the P2J server; in AppServerHelper.preProcessParameters, I've made sure that the remote side will always receive a TableResultSet, as the requester can initially send a sub-class of TableResultSet, which is not available to the remote side. Also, TableResultSet can be extended by the requester in any way it wants, there are no constraints to it.

Please add some javadoc to the class to explain these important points.

Also, don't we need to provide a basic subclass of TableResultSet for use on the remote side? For example, if we try to use TableResultSet directly, features such as hasMoreProperties(), nextProperty(), hasMoreRows(), nextRow() will all be "broken".

Finally, I think Progress actually returns something that is compatible with a JDBC ResultSet. That suggests that we probably should do the same to ensure that when apps switch over to our approach, all the same features are there and the change is minimal.

hasPrevSibling (and hasNextSibling) will return true only if the resource supports the PREV-SIBLING (and the NEXT-SIBLING) attribute. The code in HandleChain.delete was added with only the double-linked case in mind, I need to double-check if the cases when only one attribute is used can be supported by the existing code or I need to rewrite something.

Perhaps these methods should be renamed hasPrevSiblingAttr()/hasNextSiblingAttr()? If not, then please put in a comment explaining that point.

9. Does Agent.invokeFailure() need to walk the chain of causes or is the NumberedException always the topmost cause?

For that specific case, the NumberedException is the topmost cause.

Please put in a comment to this affect.

10. In Agent.invoke() there is this code: if ("Disconnect".equals(kind)). Is this always supposed to be case-sensitive?

The "kind" parameter will always receive one of the "Startup", "Shutdown", "Connect" or "Disconnect" values; they are hard-coded.

Please put in a comment to this affect.

Also, don't we need to provide a basic subclass of TableResultSet for use on the remote side? For example, if we try to use TableResultSet directly, features such as hasMoreProperties(), nextProperty(), hasMoreRows(), nextRow() will all be "broken".

At first, I wanted to make TableResultSet abstract, and let the hasMoreProperties(), nextProperty(), hasMoreRows(), nextRow() methods be abstract and force the user to implement them. But, this didn't solve the fact that the remote side needed to know the sub-class too. To let TableResultSet be abstract, I will need to wrap it in another class, which will be known to the remote side; the solution would be to extract the Externalizable behavior from TableResultSet into another, distinct, class and make TableResultSet abstract; any TableResultSet will then be wrapped in this new class, to be able to sent it via the network.

Finally, I think Progress actually returns something that is compatible with a JDBC ResultSet. That suggests that we probably should do the same to ensure that when apps switch over to our approach, all the same features are there and the change is minimal.

We can allow java.sql.ResultSet implementations to be passed in place of temp-table parameters, but this poses some problems:
1. before sending the request to the remote side, we will need an adapter to access the java.sql.ResultSet instance so that the TableResultSet can use it, and the remote side will receive a known class.
2. for OUTPUT or INPUT-OUTPUT mode, passing a java.sql.ResultSet instance I don't think can work (in terms of the requester reading the data from the remote side). From the little examples in the dvjava.pdf file, for OUTPUT case they have a ResultSetHolder class, and I can't find an example for INPUT-OUTPUT mode.

hasPrevSibling (and hasNextSibling) will return true only if the resource supports the PREV-SIBLING (and the NEXT-SIBLING) attribute. The code in HandleChain.delete was added with only the double-linked case in mind, I need to double-check if the cases when only one attribute is used can be supported by the existing code or I need to rewrite something.

Perhaps these methods should be renamed hasPrevSiblingAttr()/hasNextSiblingAttr()? If not, then please put in a comment explaining that point.

The javadoc for these methods already states that they are related to attributes. Did you mean something else?

hasPrevSibling (and hasNextSibling) will return true only if the resource supports the PREV-SIBLING (and the NEXT-SIBLING) attribute. The code in HandleChain.delete was added with only the double-linked case in mind, I need to double-check if the cases when only one attribute is used can be supported by the existing code or I need to rewrite something.

Perhaps these methods should be renamed hasPrevSiblingAttr()/hasNextSiblingAttr()? If not, then please put in a comment explaining that point.

The javadoc for these methods already states that they are related to attributes. Did you mean something else?

I was referring to where they are used since the behavior is not obvious by the name.

1. in memptr.java file, we directly use a MemoryDaemon instance (this means the java code will need to use the p2jlib.so, to access the java code) or we can use a LowLevelBuffer implementation which uses the Java heap (the buffer will be a byte[] array). Note that from Java code, the context-local variables can't be used (as there is no SecurityManager instsance), so we will need to hide this either in a static field (and use this field instead of context var, when the field is set) or in a sub-class.
2. as the requester can be on a different machine than the appserver, we need to send the actual data to the remote side (and back in case of OUTPUT/INPUT-OUTPUT mode). I see that memptr.read/writeExternal only send the memory address, and not the data. Can you tell me of a case where the memptr can be sent via network, in a converted code? Because, memptr's can't be used directly in the 4GL's UI/stream statements, and I can't see the reason why read/writeExternal sends only a memory address.

Thoughts on memptr:

I don't want to make any changes to memptr.java itself. As far as I understand it, the Open Java Client memptr implementation is not a "real" memptr like our memptr.java class. If that is correct, then we only need to implement a minimal class backed with a byte[]. I did not see the javadoc for those classes on the web and the PDF doesn't list the interfaces for each data type (or I couldn't find it). I guess I am saying that we could potentially use a class that is not a BDT since it is just for the transfer over the network. Please let me know if you find that we must support more features. I will ask the the customer staff if they have javadoc for those classes.

On the server side, anything sent would be instantiated as a real memptr. When a memptr is sent back we would just extract the byte[], wrap it in our container class (MemoryBuffer?) and serialize it to the client.

Can you tell me of a case where the memptr can be sent via network, in a converted code?

I can't think of one.

I can't see the reason why read/writeExternal sends only a memory address.

The reason is that I didn't think about this much before implementing. :)

For tables used in OUTPUT and INPUT-OUTPUT mode, do we need to allow the user to access the new data via a java.sql.ResultSet instance? If yes, I suggest to implement it via a dynamic proxy, as the interface defines almost 200 methods and not all of them are needed. And something else about using java.sql.ResultSet - to access the metadata, the only way is for the user to send an implementation of ResultSetMetadata; and in this case, the user should not specify the SQL column type.

When the user specifies property types, I think we need to build some maps to easily identify the Hibernate UserType instance, if the user specifies a P2J wrapper type name or Java type name, for the PropertyDefinition.type field or in the implementation of java.sql.ResultSetMetadata.getColumnType. Currently, my idea for PropertyDefinition.type was to be the name of a subclass of P2J's AbstractWrapperUserType (which extends UserType), and always force the user to specify this Hibernate-compatible type. But in the java.sql.ResultSet case, the user might be confused and send a SQL column type, which is dialect specific and might not be known to P2J - thus I think we need to pose some constraints on what kind of types the user can specify. And I think it is a good idea to change PropertyDefinition.type from String to Class, or at least check if the specified class name can be resolved.

Constantin Asofiei wrote:

... And something else about using java.sql.ResultSet - to access the metadata, the only way is for the user to send an implementation of ResultSetMetadata; and in this case, the user should not specify the SQL column type.

I'm a bit confused...AFAIK, to get a ResultSetMetadata object, you just ask for it using ResultSet.getMetadata(). Where does the user providing the ResultSetMetadata implementation come into play?

BTW, any news on memptr about the "is it safe to store it on the Java heap" issue?

I thought my comments in note 100 addressed this.

The basic points I was trying to make:

1. Please confirm that the Progress "Memptr" class included with the Open Java Client does NOT provide real memptr support.
2. Do not make changes to memptr, raw or BinaryData (assuming "real" memptr support is not needed).
3. Provide the same features as Progress on the Open Java Client, using a simple class named MemoryBuffer that allows a byte[] to be transferred to the server (where it arrives as a memptr) and received back from the server (where it is created from a memptr).
4. This eliminates the need for our libp2j.so on the remote client, which is very good.
5. Since we are dealing with a byte[], it is completely heap based.

I've found the Memptr.java javadoc on windev01 devsrv01 and it states that "Represents the PROGRESS 4GL MEMPTR data type. Allows to convert the value to and from a Java byte array." So they use byte arrays internally. I'll make the changes to use the MemoryBuffer class.

1. implement context-local reset (for State-reset mode). This is a little tricky, as I think we need to take a snapshot of all ContextLocal data and restore it after the agent is done with the task.

Document this as a separate task. Make sure to put an estimate of the time needed. If I understand correctly, we don't need this for this customer project, right?

2. improve the appserver agent launching (i.e. automate appserver launching at server startup, use a separate driver or some other idea)

How do the appserver agents get launched today? What is the effort to improve this?

3. implement async requests

The plan on this was to do this work along with the sockets task, right? That is OK, if nothing has changed.

This one is dependent on Stanislav's work:
4. create and populate the temp-table on the remote side, based on the received metadata and result set. Note that when the caller and the appserver are in the same JVM, no parameter processing is done. The temp-table DMO is passed directly, and I couldn't find any issues related to this.

Is this just from 4GL (in JVM1) to 4GL (in JVM2)? Or is there something that Open Java Client caller can use that is like dynamically creating a temp-table?

And a last question related to table parameters: for OUTPUT and INPUT-OUTPUT mode, do we provide ResultSet access to the received data? From the docs, looks like the Open Java Client does provide a mechanism to access the new data via a ResultSet. If we implement this, it will be read-only

Please confirm that the Open Java Client's returned ResultSet is also read-only. I presume so, but we don't want to be surprised. I do think we need to provide support for this feature, but I hope it can be read-only.

Code Review 0625b

I am fine with the proposed code. There a a couple of minor coding standard issues (explicit imports versus using * in Persistence.java AND a throws clause on the wrong line of the TempTableResultSet.init() method definition). Neither of these things is that critical.

If Eric is OK with the code, then please get it regression tested.

0626a.zip has passed regression testing.

Great! Please distribute the changes and check it in.

see the javadoc for AppServerTest.java class for details about connecting to the appserver from Java code. Please let me know if I need to be more detailed than this.

This will do as a good start.

Greg Shah wrote:

Great! Please distribute the changes and check it in.

Committed to bzr revision 10361.

About State-reset: the good news is that 4GL doesn't save any state created by the connect or startup procedures, for the agent. After the agent disconnects, its entire context is reset (this includes shared vars, sessions attrs, need to double check transient DB connections). At a first glance, a call to SecurityContextStack.getContext().getEffectiveContext().cleanup(); would seem to be enough, but this is too aggressive, as we do have some context to be preserved, like the AppServerManager.local (related to the Agent's state), and maybe other state related to perm or temp DB connections. What I'm thinking of doing is to add a ContextLocal.isResetAllowed method which by defaults returns true, and let special ContextLocal instance override it to return false. Also, add a SecurityContext.reset method, which cleanup all ContextLocal instances with their isResetAllowed returning true.

I don't have a problem with this, but how does the app-created stuff (e.g. shared vars) get cleaned up if we completely bypass normal context cleanup? In particular, it seems that globally scoped resources might stay around indefinitely.

The SharedVariableManager's ContextContainer instances will get cleaned up (i.e. removed) too during reset. What I want to do is to call ContextLocal.cleanup first and after that remove the entry from the SecurityContext.tokenMap, as if this has never been initialized. On subsequent call of a ContextLocal.get(), it will notice that the token map has no entry and it will call initialValue(), which will create a new instance.

And a side note about how 4GL behaves: a new global shared var defined in the startup procedure will get removed after the Agent has performed the first disconnection finished its first request (i.e. the global shared var will be seen on the first request until the first disconnection, but after the first disconnection on subsequent requests will dissapear and a "Shared variable has not yet been created" error will be raised). Idea is, the context is reset after the agent finishes a request, and not before disconnects.

Code Review 0628a

Overall it looks very good. A couple of things:

1. SecurityManager.resetContext() should probably be only callable from selected portions of the P2J runtime (using RestrictedUseException).

2. The impact of the change on SessionManager.activeSession() appears to be larger than just the AppServer state-reset case. I am worried about the ramifications of that change.

1. SecurityManager.resetContext() should probably be only callable from selected portions of the P2J runtime (using RestrictedUseException).

OK

2. The impact of the change on SessionManager.activeSession() appears to be larger than just the AppServer state-reset case. I am worried about the ramifications of that change.

I don't think I understand this one. Context reset is done in an Agent thread, and in it the activeSession holds the Session for this Agent (i.e. the connection to a P2J Client). Are you thinking of something else?

I think I understand your point now. These changes don't affect the cleanup that happens during normal termination processing, but rather they just add a (partial) reset capability to clear all the context (that should be cleared) while the context is still active.

I am fine with the 0701a code. Please check it in and distribute it.

Greg Shah wrote:

I am fine with the 0701a code. Please check it in and distribute it.

Committed to revision 10362.

1. in automatic mode, at server startup, it will pick up all appservers configured with auto_start=true in the directory and start them
2. in manual mode, start an appserver (or additional agent if already started) by specifying an additional parameter for the server startup; ServerDriver.main will pickup this parameter and will establish a connection to the remote server, to send the command to start the appserver. This is needed because when starting the appserver, I need its live information from the running P2J server. I don't think it is OK to rely on reading the data from the directory.xml and just start a client, because that may be different from what is live.

My problem at this time is how do I build the (generic) command to start the P2J client associated with the appserver agent. Beside environment variables and startup dir, I need some other info like library path, classpath and other bootstrap config related to client startup. From the running P2J server, using the location of the jar file containing i.e. ServerDriver class, we can build the library path and classpath (making sure the proper separators are used). At this time, I'm a little reluctant to hardcode the JVM parameters in the Java code, plus I'm not sure where to keep the info we set in the client.xml file.

Code Review 0705b

Generally, I like the improvements and the approach. It seems a bit cleaner than the previous code.

1. SessionManager.config() needs javadoc.

2. SessionManager.config() should probably return a copy of the config so that caller can't modify the stored/in-use values.

3. AppServerLauncher.launch(account, appserver), line 226:

      synchronized (launchers)
      {
         launcher = launchers.get(appServer);
         if (launcher == null)
         {
            first = true;
            launcher = new Launcher(builder(account, appServer));
            launchers.put(appServer, launcher);
         }

         launcher.success = null;
      }

The direct access of launcher.success outside of a synchronized (launcher) block seems dangerous.

Generally, the use of this success "flag" is a bit confusing. Even after reading your comment, it is not clear why one instance of this flag can be safely shared for multiple agent launches, especially since it seems that it can be set from multiple threads/out of order.

Generally, the use of this success "flag" is a bit confusing. Even after reading your comment, it is not clear why one instance of this flag can be safely shared for multiple agent launches, especially since it seems that it can be set from multiple threads/out of order.

The idea is once the command to start Agent A1 is sent, currently there is no way of knowing the notification that an Agent has started comes from A1 and not from another Agent A2 started by someone else. This should be safe, as I envisioned this notification not like being used to notify that the started Agent originates from the last used command, but more of something like "another agent has been successfully added to the pool".

About the other points, you are correct, I'll fix them.

And something which was on my mind, but I forgot to mention: our GuestAccess plugin should pick up the sent access:subject:id=<account> and override the setting from the directory.xml. More, this should be picked up by the custom login plugin we will be adding for the server project.

I am fine with the 0708b. You can check it in and distribute it.

0708b is committed to bzr revision 10363.

Please summarize the remaining work that is to be done (and the plan for the work).

I think it is just these two items:

1. implement async requests (to be implemented as part of the sockets work)
2. create and populate the temp-table on the remote side, based on the received metadata and result set (to be done after Stas finishes the dynamic temp-table support)

If I understand correctly, neither of these items will cause a problem for the Java-access to the appserver. Is that right? In other words, the customer should already have everything needed to make their modifications to their Java code that called their appserver code. Or is there some remaining work on #1638?

Greg Shah wrote:

...
If I understand correctly, neither of these items will cause a problem for the Java-access to the appserver. Is that right? In other words, the customer should already have everything needed to make their modifications to their Java code that called their appserver code. Or is there some remaining work on #1638?

You are correct, only the 2 items you mentioned are left to be implemented for the appserver support. And yes, Java-access to the appserver is not affected by these two.

The changes look fine. Check them in when they pass testing.

0110e.zip has passed runtime testing.

Check it in and distribute.

0110e.zip was committed to bzr rev 10434.

• all the appserver entry-points ran persistent are saved in AgentPool.persistentProcedures; all these get deleted implicitly when the connection is terminated (with all the associated static resources).
• in case of persisent procedures, the Finalizable.deleted and the Scopeable.scopeDeleted will take care of cleaning up the data, when the proc gets deleted. For such implementations, their finalize and scopeFinished work is delayed and executed when the procedure gets deleted.

Code Review 0326c

The changes look good. They can be runtime regression tested.

0326c.zip was committed to bzr rev 10502.

Code Review ca_upd20150416b.zip

It is good. You can check it in.

Greg Shah wrote:

Code Review ca_upd20150416b.zip

It is good. You can check it in.

Committed revision 10838.