FWD » Core Development » Runtime Infrastructure

We need testcases that explore the following as comprehensively as possible. The intention is to provide complete coverage of the 4GL compatibility of this "runtime infrastructure" support.

This entry is still being edited.

See Runtime Infrastructure Testcases for status of the testcases.

• Appserver Tests
  • see #1608, #2124, #3854, #6424, #6488, #6417, #4398, #8661
  • accessed from 4GL code
  • accessed from "Open Java Client"
  • accessed from "Open .NET Client"
  • classic (including stateless, state-free, state-reset, state-aware modes)
  • PASOE (both session-managed and session-free modes)
  • synchronous and asynch calls
  • all invocation and parameter combinations
• LOG-MANAGER Tests
  • see #3853
  • LOG-MANAGER system handle
    • Attributes
      • LOGFILE-NAME
      • LOGGING-LEVEL
      • LOG-ENTRY-TYPES
      • LOG-THRESHOLD
      • ENTRY-TYPES-LIST
      • NUM-LOG-FILES
      • QUERY:BASIC-LOGGING
    • Methods
      • WRITE-MESSAGE()
      • CLEAR-LOG()
      • CLOSE-LOG()
  • LOGGING-LEVEL is set to 2, 3
    • 0 ("none"), means no logging
    • 1 ("errors"), this would log only 4GL error messages and ignores the LOG-ENTRY-TYPES setting
    • 2 ("basic"), includes "errors" plus categories defined in LOG-ENTRY-TYPES
  • LOG-ENTRY-TYPES is set to one of the following
    • "QryInfo:2,4GLMessages" (this is used in non-appserver mode)
    • "QryInfo:2,ASDefault" (used in appserver mode)
    • "4GLTrace"
    • "4GLMessages"
    • At least in 1 location it is set to an arbitrary value.
    • 4GLTrace logs on execution of internal procedures, functions, events etc...
    • 4GLTrans logs on transactions and sub-transactions
    • QryInfo logs queries (open query and for each)
    • ASPlumbing logs unspecified appserver events
    • DB.Connects logs database connects/disconnects
    • 4GLMessages logs MESSAGE statement output (the same way that appserver and batch does this without the LOG-MANAGER)
    • session:debug-alert IS explicitly set so 4GLMessages does also include stack traces (yikes!)
    • ASDefault is the same as ASPlumbing + DB.Connects
• PROFILER Tests
  • See #3819.
  • Cover all types of callable program, with all forms of parameter usage:
    • OO method
    • constructor
    • destructor
    • property getter
    • property setter
    • class events
    • external proc
    • internal proc
    • user defined function
    • UI trigger
    • database trigger
  • Check that the generated output matches the expected raw profiler format.
  • All methods and attributes of the PROFILER system handle.
• REST Tests
  • See #3855 and #4089, these tests also cover the WebHandler tests (see WebSpeed/WebHandler Tests).
  • Details about questions we need answered for the REST support. We need these answered for each mode (classic, PASOE, web handler):
    • parameters (input and output):
      • all type combinations - https://documentation.progress.com/output/ua/OpenEdge_latest/index.html#page/dvwsv%2Fparameter-mapping.html%23
      • mismatch between the value received by the REST call and the actual 4GL parameter definition (i.e. you send an character value and the 4GL parameter is an integer). Is it some automatic conversion happening - from i.e. "12" string to 12 integer? If yes, what happens if this automatic conversion is not possible? We need all data-type combinations here.
      • there is mention of TABLE and DATASET allowed in the body - we need examples for this; again, with all possible data-types at the TABLE/DATASET field level.
      • all ways of passing the parameters - via the URI path (/customer/{customerName}), via the URI query parameter, the request's body.
    • error handling
      • parameter validation (number, position, mode - input/output).
      • the REST is configured to call some 4GL program, but that program is not found on the PROPATH
      • authentication failure
      • other possible ways for a REST call to fail?
      • what if there is an ERROR, QUIT, STOP, ENDKEY condition generated during the call? How is the remote side informed of this?
    • can the RETURN-VALUE be exposed to the caller?
    • tests for all supported HTTP verbs
    • how is /customer/{custname} mapping the custname to a 4GL parameter?
    • examples with REST annotations - targeting both external and internal procedures
    • can internal procedures be targeted only by using REST annotations?
    • JSON message formats
  • Security - for each mode (classic, PASOE, web handler):
    • how is 4GL authenticating a REST call?
    • how is the Agent handling this REST call running - in Stateless mode, something else?
  • Configuration, deploy - for each mode (classic, PASOE, web handler):
    • how is the exposed REST service configured
    • what artifacts are needed to run the REST services (the .paar file? something else?)
    • what configuration files are generated and needed to run the REST services
    • what is the OpenEdge configuration available for REST (number of Agents, log files, etc). This should be a document describing how a REST gets configured and deployed, what available options are for the user to setup the Agents, etc.
    • tests to prove that all modes behave the same, at the 'executor side'. What I need to know is if the parameter transport, validation, error handling, etc is all the same when executing the REST call, regardless of the deployment approach (classic, PASOE, web handler).
  • We need tests for both the REST consumer (client) and the producer (server), all of this in 4GL.
  • Regarding PASOE: we need to determine in which mode the REST requests are ran (I assume session-free), when handled by the PASOE Agent. Also, are there SESSION attributes or methods (like REMOTE) which are dependant on executing a REST request?
  • Data Object Services examples and documentation for:
    • testing with both ABL classes and external programs implementing the operations
    • what artifacts are generated and needed to work with the Data Object Service
    • what is the structure of the Progress Data Service Catalog?
    • what are the annotations for each service
    • what at are parameters for each service (OO method or internal procedure)
    • how are RETURN ERROR and conditions handled (Both at runtime and as a response to the requestor).
    • what are the URIs for each service - what is their structure
    • how are parameters handled - format at the requestor side, how is processed at the service side. This is important as TABLE and DATASET are used as parameters, and we need to know the JSON (I assume) format for (de)serializing them, and how the field data-type maps from 4GL to JS.
    • differences between /rest and /web/pdo transports
    • what are the HTTP verbs for each operation and how do they map to the service - what happens if a service is invoked with a different verb?
    • the API specification for OpenEdge.BusinessLogic.BusinessEntity and any other 4GL classes/interfaces which are used to implement the DataObjectService in 4GL.
    • can a Data Object Service be executed without JSDO?
    • how is the response handled by the requestor side (via AJAX? synchronous? asynchronous? something else?)
    • security and authentication for a request - how is it implemented?
    • how are the Agents executing the request behaving? Are they State-free? Is SESSION:REMOTE attribute set? Other SESSION attributes being used when executing the request?
    • how is a Data Object Service configured - what options are available? Number of Agents? something else?
    • when using a class or external program for the service, are they singleton - if so, when are they initiated?
    • is an Agent bound to only one service? Because if singleton is used, then this state must be common for all Agents executing that request; can more than one Agent handle the same singleton class/persistent external program?
    • examples should use minimal UI (is better without UI). I'd like to have some pure Javascript code detailing how these calls are performed.
  • Regarding JSDO - the documentation is pretty complex (https://documentation.progress.com/output/pdo/index.html#page/pdo%2Foverview-of-progress-data-objects%2C-services%2C-and.html) and the JSDO APIs even more so (https://documentation.progress.com/output/pdo/index.html#page/pdo%2Fjsdo-properties%2C-methods%2C-and-events-reference.html%23).
  • We need to understand how the OpenEdge JSDO classes (in Javascript) communicate with the 4GL side (so a Kendo UI app can integrate easily with FWD), and this is only possible by testing (and documenting) how each and every JSDO class and API behaves in regard to CRUD operations, transaction support, error management, etc.
  • For the classic appsrv, the session-managed modes (State-reset, State-aware. Stateless) are already supported by FWD. But I need some confirmation that an Agent executing a SOAP/REST service will always run in State-free mode - is this configuration automatic in OpenEdge - i.e. it automatically sets the mode to State-free and it can't be overridden by another configuration? If it can be overridden, how will the Agent behave?
  • On the other hand, for #3854, we need to make sure how the PASOE Session-managed and Session-free work, so we will need some tests for session-managed, too.
  • We need an example of a openapi.openedge.export annotation for internal procedures, functions (if is possible in OpenEdge) and class methods (with void and non-void return type) - these will be REST services, with 2 or more PATH parameters.
• Security Tests
  • make sure to address all features from #3752, #3810, #4108, #4380, #6422, #6423, #6399, #6419, #6420, #6421, #6422
  • handles and system handles including all attributes and methods
    • AUDIT-CONTROL
    • AUDIT-POLICY
    • CLIENT-PRINCIPAL
    • SECURITY-POLICY
  • AUDIT-ENABLED()
  • CAN-DO()
  • CREATE CLIENT-PRINCIPAL
  • DECRYPT()
  • ENCODE()
  • ENCRYPT()
  • GENERATE-PBE-KEY()
  • GENERATE-PBE-SALT()
  • GENERATE-RANDOM-KEY
  • GENERATE-UUID
  • GET-DB-CLIENT()
  • GUID
  • HEX-DECODE()
  • HEX-ENCODE()
  • MD5-DIGEST()
  • MESSAGE-DIGEST()
  • RANDOM()
  • SET-DB-CLIENT()
  • SET-USERID()
  • SHA1-DIGEST()
  • SSL-SERVER-NAME()
  • USERID
• SOAP Tests
  • see #3310 (GES: I tried to make this list from the task, it needs review by CA)
  • what artifacts are required to run the web services in 4GL (WSDL?). What is the role of each artifact, so we can determine what we need at conversion and/or runtime from it.
  • how is the mapping from the web service to the actual 4GL external program being made? I assume the WSDL will have this knowledge? Can internal procedures/functions be mapped?
  • how are parameters passed and mapped to 4GL parameters?
  • what is the SOAP structure for this call? How can the header and body look?
  • examples for SOAP request and response XML messages
  • error handling - how is the SOAP FAULT error generated by the server and what does it contain?
  • items from #3855-2 (but for SOAP not REST)
  • Regarding PASOE: we need to determine in which mode the SOAP requests are ran (I assume session-free), when handled by the PASOE Agent. Also, are there SESSION attributes or methods (like REMOTE) which are dependant on executing a SOAP request?
  • There are the SESSION:CURRENT-REQUEST-INFO and SESSION:CURRENT-RESPONSE-INFO, which return a Progress.Lang.OERequestInfo instance. These are used only in PASOE I think, and we need some tests to know how each property is formatted, and possible values.
  • There are also the GetClientPrincipal() and SetClientPrincipal() methods, they look like simple getter/setter, and the only way to set a value is via the setter (default is unknown). These are assumptions from the docs, if they are correct, then nothing special is needed for testing.
  • the .xpxg can have a SubAppObject node (as a child of AppObject)
  • raised errors (QUIT, STOP, ERROR, etc)
  • some tests for the before-table - is it possible at all?
  • return values from procedure - do you have this? If not, please add.
  • extent (fields or vars), if is possible.
  • what happens if the WSDL is not found, when trying to access it via the URL?
  • what happens if the SOAP request is not a valid XML?
  • what happens if there is more than one operation in the SOAP body, like #3310-38
  • what is the CreateAO operation for a sub-service (look in the WSDL)? Can this ever be used in a request?
  • serialization options (serialize-name is the default for xml/json)
• WebSpeed/WebHandler Tests
  • For WebHandler support, see the REST Tests for the test descriptions. The original task #3855, #4089 was shared between REST and WebHandler.
  • For WebSpeed: TBD

I expect these tests to be split into smaller functional groupings that can be run on their own. I don't expect a single set of tests which includes all of these categories.

Where is the startup guide on how to write tests, add them to testcases and run them in Harness?

Galya B wrote:

Where is the startup guide on how to write tests, add them to testcases and run them in Harness?

As documented in Testing Process, different techniques are used for different purposes.

If your test code is written in the 4GL AND it can be executed using a single threaded test process, then we use ABLUnit. You will find details about this in 4GL Unit Testing and in Writing 4GL Testcases (especially in ABLUnit Test Cases).

The Harness is used for more specialized purposes. It has tools for automating interactive ChUI code and for executing REST/SOAP calls. It also is used for cases where we want to run tests on multiple threads and coordinate behavior between those tests. There is documentation in Documentation and the multi-threaded test approach is detailed in #9270.

Should I write 4GL tests for testcases that work with only one directory.xml that is derived from the template?

Galya B wrote:

Should I write 4GL tests for testcases that work with only one directory.xml that is derived from the template?

That is not required. If you need any kind of custom cfg, it is OK. We can even have more than one cfg, if needed.

Greg Shah wrote:

Galya B wrote:

Should I write 4GL tests for testcases that work with only one directory.xml that is derived from the template?

That is not required. If you need any kind of custom cfg, it is OK. We can even have more than one cfg, if needed.

How do I convey that to those who will run the tests in the future? I mean we need a standard way of dealing with multiple directory configs. Should I just add the directory.xml.template to each directory that needs different configs?

Greg Shah wrote:

Testcases Needed

• Appserver Tests
  • see #1608, #2124, #3854, #6424, #6488, #6417, #4398, #8661
  • accessed from 4GL code

We have tests for this in testcases, ABLUnit style.

• accessed from "Open Java Client"

This can be done as JUnit directly probably - just different jars used for Progress&FWD 'open client'.

• accessed from "Open .NET Client"

Well, this has to be a .NET client so another technology into the stack.

• classic (including stateless, state-free, state-reset, state-aware modes)
• PASOE (both session-managed and session-free modes)

As far as the client is concerned the only difference between 'classic' & PASOE is the connect URL.

• synchronous and asynch calls
• all invocation and parameter combinations

Not sure about all but we've tried to cover as many as possible :)

• LOG-MANAGER Tests
  • see #3853
  • LOG-MANAGER system handle

We have a number of tests for this one, ABLUnit style but this is only 'client side'... maybe we can craft something for the appsrv using 4GL direct call or soap/rest.

• PROFILER Tests

Same as before we have touched some of those in testcases, ABLUnit style.

• REST Tests

We have something for WebHandler and REST services, the REST services is more or less being replaced by the WebHandler approach although there are many still using them - a pain in the a** to maintain the service definition and deploy those :(

As far as I remember the tests for consuming REST services were done just by using SOAPUI, using the Http client we can write test ABLUnit style if those fits better in the CI pipelines.

• SOAP Tests

Same as for REST, we have tests as SOAPUI project and some ABLUnit style - consume web services from 4GL, if this is available in FWD those tests can be used already.

Galya B wrote:

Greg Shah wrote:

Galya B wrote:

Should I write 4GL tests for testcases that work with only one directory.xml that is derived from the template?

That is not required. If you need any kind of custom cfg, it is OK. We can even have more than one cfg, if needed.

How do I convey that to those who will run the tests in the future? I mean we need a standard way of dealing with multiple directory configs. Should I just add the directory.xml.template to each directory that needs different configs?

Add them into the project with explicit names and document the setup process and specific cfg files in the Appserver Tests page. We are working on ways to standardize the deployment, but until then this will do.

I reviewed this task and the Appserver_Tests wiki and noticed that we already have multiple tests covering different areas, scattered across several locations. This raises the question:
Should we add new tests to the existing suites, following the same patterns, or should we look for a way to centralize them under a single tool or framework?

In my opinion, the most effective way to properly test AppServer functionality across native and converted code, in all modes and with all possible parameters, would be to develop a Java Open Client that calls the 4GL/converted backend. As far as I know, both Progress and FWD support the Java Open Client, making it the best common entry point for both platforms.

The idea would be to expose endpoints in the Open Client and invoke the business logic through generated proxies. A client capable of receiving requests, executing 4GL/converted logic, and returning responses would allow us to:

• Easily implement regression testing via Harness.

• Perform load and performance testing using tools like JMeter.

With this approach, I don’t think we would need additional dependencies such as ABLUnit or Ant for testing.

A similar solution could also be implemented using .NET, which is compatible with both platforms as well (Linux with FWD and Windows with 4GL).

Please let me know what you think of this approach and if a new one is needed at all :)

I reviewed this task and the Appserver_Tests wiki and noticed that we already have multiple tests covering different areas, scattered across several locations. This raises the question:
Should we add new tests to the existing suites, following the same patterns,

Please list the related test areas you have found and we will discuss which get extended, what gets added or moved.

or should we look for a way to centralize them under a single tool or framework?

We cannot centralize them under a single tool/framework.

Appserver, by its very nature is about connectivity from multiple client types. We current support access from both the Java Open Client (our version of it) and directly from 4GL code. In the future we will support access from .NET. We can't just write all tests in the open client because then we would not test access from the 4GL.

One thing we can do is implement a common set of backend 4GL code which is used across all tests. For each test, I expect some minimal duplicated code in Java Open Client (JUnit 5) and in 4GL (ABLUnit) while most of the logic should be in the common back-end code.

The same tests written for 4GL "create server" will need to be executed via OpenClient, too (in both classic and pasoe modes).

So for 4GL I think is best to use unit tests, and for OpenClient I think also via JUnit - so they can be executed via the harness, too.

The point is: is not only about what happens on the appserver side when executing the call, but also what happens with i.e. exceptions/errors being thrown to the caller, timeouts, etc.

• that cover 4gl access in testcases/pasoe_test
• rest tests in testcases/appsrv/rest
• soap tests in testcases/appsrv/soap

• convert the existing tests in testcases/pasoe_test to abl unit, if possible.
• write 4gl procedures that use different parameter types, maybe use some of the existing ones in pasoe_test.
• write a java open client that uses the 4gl procedures.
• write a .net open client that uses the 4gl procedures.

The tests need to be executable both in windows/4GL and linux/FWD.
The purpose of the tests is to establish a baseline with the 4gl env regarding the appserver behavior in different scenarios, configurations and access technologies (direct 4gl, open client java/.net) and make sure that on our side the behavior is the same.

Is this accurate?

• the connection including the errors that might be raised at this step.
• running the procedures with various parameter types and modes.
• timeouts/errors raised from the procedure.
• synchronous and async calls.

The same project could be used to test the PASOE mode (minimal changes to the connection step), I might have some issues in creating an running a PASOE appserver on my env, but we'll se when we get there.

I expect that such a project could be easily converted and run with FWD with minimal configuration challenges.

Constantin, I am trying to write some tests that explore the different session management with classic appserver (state-free, state-aware etc). With all the operating modes, persistent procedures are persisted on the appserver until it is deleted and the non persistent procedures are not kept on the appserver, which is fine, but I need a way to check if a request is within the same session or on the same appserver, in other words, prove that the context is the same as the last call. There are no session ids for classic and shared variables are not shared no matter the operating mode.
Do you think there is a reliable way to check if the context is actually reset or if something gets preserved between sessions with classic and 4gl access?

Artur Școlnic wrote:

Do you think there is a reliable way to check if the context is actually reset or if something gets preserved between sessions with classic and 4gl access?

• use a logical parameter at the program invoked via this appserver
• depending on the parameter's value:
  • when true, run program A which has a def new global shared var v1 as char. This saves the shared var in the agent's context until it gets reset.
  • when false, run program B which has def shared var v1 as char. - this will work only if the request is executed on an agent which already executed program A, and didn't have its context reset.

I already tried with shared variables, I don't think they get saved on the appserver.
I have a single agent for a state-aware appserver, on it a program that defines a new shared variable is ran, then a program that increments it.
I get the Shared variable has not yet been created error, as if the first program did not persist it in the agesnt's context or it was erased.

Artur Școlnic wrote:

I already tried with shared variables, I don't think they get saved on the appserver.

Are you sure you are using def new global shared variables? Because def new shared var scope is 'on the stack' and not 'global'.

Otherwise, the appserver config needs to have min=max=initial=1 for the agents for this to be relevant.

p1.p

def new global shared var shared_var as char.

p2.p

def shared var shared_var as char.
define output parameter out as char.
shared_var = shared_var + "1".
out = shared_var.

 RUN procedures/p1.p ON SERVER h.

 RUN procedures/p2.p ON SERVER h (output calls). // throws the error

Artur Școlnic wrote:

And I configured only one agent in the pool range for all the tests, so it would be obvious when the context is preserved.

What's the operating mode for the broker?

State-aware, I think it is the only one that guarantees that the requests are handled by the same agent.

Ok, it was an issue in my project, I was running the wrong program, sorry for the confusion. The variable is persisted on the appserver as expected, thanks.

I want to mention that I am committing the tests to 8922a/AppserverTests as I write them, so they can be found and reviewed there.

Artur Școlnic wrote:

I want to mention that I am committing the tests to 8922a/AppserverTests as I write them, so they can be found and reviewed there.

Please don't commit tests to FWD branches - these are private and are not made public; do archive 8922a as 'dead'.

Use the new testcases project and commit them to a new appservers folder. Note the existing tests in appsrv/ (which also has REST/SOAP/web support, but we are not migrating these yet) and pasoe_test/ folder, which should be migrated to unit tests. Consider looking into these to see what was tested there.