Harness

The goal of the task is to expose API to allow external tools to access Harness Mutexes and Semaphores in order to synchronize and communicate in parallel tests scenarios.

The initial idea is to expose a TCP socket and allow external apps to lock/unlock resources with a simple protocol initially defined in #9270

Harness would expose a TCP socket on a predefined port after providing a dedicated command line option to allow clients to connect.
Each connected client would start a new Driver thread that would become a new actor in test scenario, thus interact/synchronize with a test plan.

• six characters constituting a header, in the first version of protocol that would six ascii digits providing a number of bytes of an actual message
• actual message as ascii string.

• Client would connect to the socket and would send a request, and getting response back with possible blocking waiting for resources.
• Every command from client is a request with two possible responses OK or ERROR from Harness server
• Commands may have parameters, parameters are separated from command name and each other by vertical bar ( | ) sign - ASCII code 124.
• Semaphores are identified by name as defined in Harness Testcase
• Executing command may block client test execution e.g. resource is allocated and needs to be released
• Disconnection from server should be treated as an unconditional error

Examples:

000002OK
000005ERROR
000021ERRORWith description
000011POSTEStest1

Note there is no newline between messages, so message stream would like following (upstream and downstream):

000018POSTES|test1|false000016WAITES|sem1|1800000016WAITES|sem2|1800
000002OK|000002OK|000025ERROR|Timeout waiting for semaphore

• POSTES|<name>|<reset> - Post event semaphore, <reset> can be true/false
• RESETES|<name> - Reset Event Semaphore
• WAITES|<name>|<timeout> - Wait Event Semaphore identified by name up to <timeout> seconds - this may block
• ACQUIREMTX|<name> - Acquire mutex identified by name - this may block indefinitely
• RELEASEMTX|<name> - Release mutex identified by name

• TESTSET|<name> - set Harness TestSet name
• TEST|<name> - set Harness Test name

In order to activate Remote Access a following target needs to be specified in Harness TestPlan or in TestSet:

   <target type="remote" port="8889" threads="4" />

Marian - what is a progress of ABL part ?

Tomasz Domin wrote:

Marian - what is a progress of ABL part ?

We're testing that locally with a 'echo' server implementation right now, I will push that in testcases tomorrow morning first thing.

Marian Edu wrote:

Tomasz Domin wrote:

Marian - what is a progress of ABL part ?

We're testing that locally with a 'echo' server implementation right now, I will push that in testcases tomorrow morning first thing.

I've pushed the Harness API 4GL code on testcases. Only thing that is probably going to be used in test cases is support.harness.HarnessApi, apart the Connect method (port, [host = localhost]) there are only SendMessage methods that are merely overloads for sending a 'harness message' with optional timeout - default to no time-out. The method return is void - either completes with no error (OK response) or it will throw - when harness response is ERROR or time-out specified and passed.

Otherwise we have a mockup server and client UI implemented in 4GL in support/harness/test that we've used to test the whole thing, hope I've got the specs correctly but please let me know if there is anything wrong with it.

Marian Edu wrote:

We're testing that locally with a 'echo' server implementation right now, I will push that in testcases tomorrow morning first thing.

Otherwise we have a mockup server and client UI implemented in 4GL in support/harness/test that we've used to test the whole thing, hope I've got the specs correctly but please let me know if there is anything wrong with it.

Marian - what is support.harness.protocol.* referred in client ?
I tried to build client with FWD to connect it with Harness and I failed Caused by: java.lang.ClassNotFoundException: Can not find class for accessing Connect [OO_METH_VOID] @243:24 (107374183429)

Tomasz Domin wrote:

Marian - what is support.harness.protocol.* referred in client ?

That seems to have been a left over from previous refactor, the APPBuilder kinda hidden it under analyse-suspend preprocessor directives :(

Removed those from test client/server and added the convert file list.

I tried to build client with FWD to connect it with Harness and I failed Caused by: java.lang.ClassNotFoundException: Can not find class for accessing Connect [OO_METH_VOID] @243:24 (107374183429)

That doesn't seem to be related to the bad 'using' statements mentioned before although it just might, don't know how conversion handle bad wildcard 'using' statements although those were not really used.

Marian Edu wrote:

Tomasz Domin wrote:

Removed those from test client/server and added the convert file list.

I tried to build client with FWD to connect it with Harness and I failed Caused by: java.lang.ClassNotFoundException: Can not find class for accessing Connect [OO_METH_VOID] @243:24 (107374183429)

That doesn't seem to be related to the bad 'using' statements mentioned before although it just might, don't know how conversion handle bad wildcard 'using' statements although those were not really used.

Marian, are you able to build harnessClient.w with FWD ? I am still getting the error as above.

Tomasz Domin wrote:

Marian, are you able to build harnessClient.w with FWD ? I am still getting the error as above.

I had to modify code in a following way (and I am still getting the third error):

=== modified file 'support/harness/HarnessApi.cls'
--- old/support/harness/HarnessApi.cls    2024-12-31 12:54:35 +0000
+++ new/support/harness/HarnessApi.cls    2025-01-08 20:38:06 +0000
@@ -23,10 +23,10 @@
     define public event OnDisconnect void ().

     method public void Connect(harnessPort as integer):
-        this-object:Connect(harnessPort, ?).
+        this-object:Connect2(harnessPort, ?).
     end method.

-    method public void Connect(harnessPort as integer, harnessHost as character):
+    method public void Connect2(harnessPort as integer, harnessHost as character):
         harnessConn = new HarnessConnection().   
         harnessConn:OnDisconnect:Subscribe(onConnectionDisconnect).
         harnessConn:Connect(harnessPort, harnessHost). 

=== modified file 'support/harness/test/harnessClient.w'
--- old/support/harness/test/harnessClient.w    2025-01-06 17:57:11 +0000
+++ new/support/harness/test/harnessClient.w    2025-01-08 20:40:34 +0000
@@ -239,6 +239,7 @@

 on choose of connectButton 
     do:
+        define variable harnessPort as integer no-undo.
         if valid-object(harnessApi) and harnessApi:Connected then 
         do:
             run disconnect.
@@ -250,8 +251,10 @@
                 harnessApi = new HarnessApi().
                 harnessApi:OnDisconnect:Subscribe('onDisconnect').
             end.
-                
-            harnessApi:Connect(portFillIn:input-value) no-error.
+            
+            harnessPort = portFillIn:input-value.    
+            
+            harnessApi:Connect(harnessPort) no-error.

             if harnessApi:Connected then 
             do:

1. There is a problem with overloaded class methods - when looking for exact match the first one is selected - in this case Connect(harnessPort as integer, harnessHost as character):, so I had to rename overloaded method.
2. when looking for class method for expression harnessApi:Connect(portFillIn:input-value) it looks for connect(Widget) instead of connect(integer)
3. when conversion succeeds it seems modes are missing in converted code for one of constructors: error: no suitable method found for newInstance(Class<HarnessMessage>,object<HarnessCommand>,character)
[javac] harnessMsg.assign(ObjectOps.newInstance(HarnessMessage.class, HarnessCommand.getEnum(defaultFrameFrame.widgetCommandComboBox().getScreenValue()), (character)defaultFrameFrame.widgetResourceNameFillIn().inputValue()));

There are definitively issues in conversion, nevertheless I've changed the 4GL code to help the converter pickup the right method (widget input-value data type doesn't seem to be inferred from the data type). Committed rev #1685 that seems to convert right and the converted code has no errors, can't tell if it's correct or not nor if it's actually working but the conversion errors are gone.

Marian Edu wrote:

There are definitively issues in conversion, nevertheless I've changed the 4GL code to help the converter pickup the right method (widget input-value data type doesn't seem to be inferred from the data type). Committed rev #1685 that seems to convert right and the converted code has no errors, can't tell if it's correct or not nor if it's actually working but the conversion errors are gone.

Marian, I've pushed updated sources to xfer repository: Committed revision 1689.
Note I've updated build_ablunit.xml to allow two sessions of ant pct running in parallel - for that to happen I had to allow redefinition of a temporary directory.

I've added Harness ABL remote API to harness project and refactored it in the way, that harness if needed should be checked out into testcases as testcases/support/harness.

Tomasz Domin wrote:

I've added Harness ABL remote API to harness project and refactored it in the way, that harness if needed should be checked out into testcases as testcases/support/harness.

Hi Tomasz, I'm not sure if testcases is the right place to have that 4GL code but alas is the only repository we have access to and beside we are going to need it in testcases :)

I did updated the project to pick-up your changes, I see there are two additional message types added and you like to have specific methods for each action - on top of the more generic SendMessage, no problem with that. In the message body I see you've added | as separator between command and body data, no problem with that either only I think the constructor that takes raw data as input should be updated to take that into account, update the substring to skip the | separator - use 2 instead of 1 on the substring call on line #48 of HarnessMessage.

Marian Edu wrote:

Tomasz Domin wrote:

I've added Harness ABL remote API to harness project and refactored it in the way, that harness if needed should be checked out into testcases as testcases/support/harness.

Hi Tomasz, I'm not sure if testcases is the right place to have that 4GL code but alas is the only repository we have access to and beside we are going to need it in testcases :)

Thank you Marian
My idea is to move 4GL Harness ABL API code to harness project and then include harness project optionally in testcases project (via link or checkout) with a directory structure like this:

-testcases
--support
---harness
----abl <- here to put 4GL Harness ABL API, classes would be in @support.harness.abl@ package
----example <- here to put 4GL Harness ABL based example tests

Note I didnt want to change PROPATH of testcases project.
What do you think ?

I have a question where to put tests that require harness inside testcases project, as I think there are too much folders in root of testcases project. There is already a folder called flushing_and_triggers.