FWD » Core Development » Base Language

This task is to implement the core classic WebSpeed implementation for "CGI Wrapper" usage where a .p or .w can be referenced via HTTP and it results in the invocation of the procedure via CGI. The procedure can include any non-interactive 4GL processing and it writes its results as output to the WEB stream (#6432). This output is returned back as the HTTP response.

I understand that the core logic for the dispatching is in web-disp.p which is part of Possenet (and newer versions in the ADE tools).

We will NOT implement any support (at this time/in this task) for the following:

• Static HTML compiled to a procedure
• Embedded SpeedScript
• HTML Mapping

Marian: Please correct me if I have a misunderstanding on any of this.

Greg Shah wrote:

This task is to implement the core classic WebSpeed implementation for "CGI Wrapper" usage where a .p or .w can be referenced via HTTP and it results in the invocation of the procedure via CGI. The procedure can include any non-interactive 4GL processing and it writes its results as output to the WEB stream (#6432). This output is returned back as the HTTP response.

You are correct, you basically get the output stream, a couple of preprocessor definitions that writes to the said stream and a bunch of API's that gives you access to the web request information - as query string, form data, CGI variables.

I understand that the core logic for the dispatching is in web-disp.p which is part of Possenet (and newer versions in the ADE tools).

This is what they use internally, but there is no need to do the exact same thing - years back like in another life I had this little webspeed replacement called ganimede (https://sourceforge.net/projects/ganimede/) and things haven't changed much in that area since then.

We will NOT implement any support (at this time/in this task) for the following:

• Static HTML compiled to a procedure
• Embedded SpeedScript
• HTML Mapping

Yes, this is covered by the compiler mainly, you could probably pull that out too in conversion phase but imho it's not worth it - that was not largely adopted so I wouldn't bother much about it, it's very old technology anyway so better wait for a client that needs that.

Marian: Please correct me if I have a misunderstanding on any of this.

None that I could think of, that seems about right but as said you probably shouldn't bother about Possenet code nor the web-disp.p.

#6432 must be implemented at the same time as this task. We probably don't need 2 tasks for this.

The ADE tools 12.2 (and later) are released under the Apache 2 license and are safe to use. Where we previously implemented the web-disp.p from Possenet, we now should implement the version from 12.8.4. We can even convert it and include that converted code in FWD itself if that makes sense.

With #9516, we will have converted ADE code for web-disp.p and other related programs.

We will need a handler (similar to legacy REST) which will be used to (call and?) dispatch these requests. But, there is a part missing in FWD: for legacy classes (skeleton implementations), these are automatically managed in FWD via annotations and loaded into SourceNameMapper.

For programs, we will need a way to automatically register them with SourceNameMapper (avoiding a name_map.xml file) and into the propath.

From what I see, the program name is set in a:

/* Name of the program we're about to run */
DEFINE NEW GLOBAL SHARED VARIABLE AppProgram        AS character FORMAT "x(40)":U 
    NO-UNDO.

which exists in web-disp.p. This in turn is used by the "web-notify" trigger.

web-disp.p seems to be a persistent program which lives? in the CGI app.

We need to understand how the dispatch is being executed in OE (what is the CGI program actually executing/running).

Don't we already have a manually written Java version of web-disp.p? If that code is cleaner/more efficient than the converted version, then we may just want to ensure it has all the same logic as the latest web-disp.p from 12.8.4. If it is not cleaner/more efficient, then we can consider using the converted version.

I agree with your other points.

Also, the URL to run something will look like http://webserver/cgi-bin/webapp.cgi/login.w .

At this time (without having done any digging into how the backend in OE would work), I would think that we will have a dedicated 'web' appserver, on which the requests will be posted. But https://docs.progress.com/bundle/openedge-webspeed-117/page/Web-object-states.html states that webspeed agents works in both stateless and state-aware mode (and this can 'bind' an agent to a request).

So they have a default of 512 agents https://docs.progress.com/bundle/openedge-application-and-integration-services-117/page/WebSpeed-agent.html running these. It is unclear yet how the WEB-NOTIFY is received by these - I assume web-disp.p is launched automatically by these agents and the program is not persistent (is kept in a REPEAT loop), and this waits for web-notify events to be posted.

There are a number of new global shared vars defined in web-disp.p which I assume would need to be pre-populated once an webspeed agent was chosen to execute the request, but before the web-notify would be posted to it.

Greg Shah wrote:

Don't we already have a manually written Java version of web-disp.p? If that code is cleaner/more efficient than the converted version, then we may just want to ensure it has all the same logic as the latest web-disp.p from 12.8.4. If it is not cleaner/more efficient, then we can consider using the converted version.

I think what you have in mind is the 'legacy web handler' OpenEdge.Web.WebHandler class. We need the web-disp.p, with all its 'global shared vars' which get assigned with info about the currently executed request, beside the runtime. My worry is that once you have a 'global shared var' in web-disp.p, this can be read from any application's program, there is no restriction (plus lots of super-procedure info which adds another layer of complexity). So we need these, as web-disp.p is the root executing program on the webspeed agent.

A comment from Constantin via email:

My main worry is about the 'WEB' stream support (I assume this would target the HTTP response stream for the current request, but we need some tests to find how an explicit 'close' will affect this stream, if multiple streams are opened for it, etc) and the behavior of the actual CGI executable OE is running on the request.

It isn't OE running the CGI executable. It would be whatever web server (e.g. Apache or IIS) is configured. That web server would have some CGI module to execute the identified .p, .w, .r. That CGI module probably runs some batch mode version of _progres OE command line launcher. Doing so would make it easy to redirect STDOUT/STDERR for the child process and read the result back as the response.

If I'm right, then the 4GL code just implements named WEB special stream to point to that redirected unnamed output stream (STDOUT).

Greg Shah wrote:

That web server would have some CGI module to execute the identified .p, .w, .r.

This is the part I meant, whatever gets the requests first, prepares some state, and after that runs web-disp.p.

That CGI module probably runs some batch mode version of _progres OE command line launcher.

The command I see configured is -p web\objects\web-disp.p -weblogerror - there is no -b to run in batch.

Doing so would make it easy to redirect STDOUT/STDERR for the child process and read the result back as the response.

If I'm right, then the 4GL code just implements named WEB special stream to point to that redirected unnamed output stream (STDOUT).

Even so, in FWD at this time I would rather send this to some internal FWD stream, which is then written to the HTTP response stream.

Greg: the agents are configured similar to classic, so this will be a 'kind of classic appserver' I think. The 512 I mention in #6506-11 is related to 'concurrent requests at the broker' (i.e. Jetty qtp threads in FWD), see https://community.progress.com/s/article/000047740 So this makes things easier, as we can re-use the classic appserver support.

I'm trying to setup webspeed to run some tests but I'm having trouble.

Marian: is this something which can be easily done, or point me to some documentation? I can start the webspeed agents, but I think I need the web server for running the CGI part.

the agents are configured similar to classic, so this will be a 'kind of classic appserver'

There is also the PASOE WEB transport which is supposed to be the PASOE version of WebSpeed (if I understand correctly).

The global shared vars are initialized via the WEB-CONTEXT system handle - this has methods to access misc CGI vars, via the ADE tools in programs related to webspeed. This resource is not implemented in FWD, but it has some conversion support. My worry was that the global shared vars were just automatically injected into the state somehow.

So, with this in mind, things look simpler. The other unknown part is that the webspeed agents are just running web-disp.p and are waiting for WEB-NOTIFY event notifications. This will need some changes in how these agents are started.

+Galya

Constantin: what do you think I should approach first? After I get a better understanding of what needs to be done, I'll probably have some more questions.

I'm trying to find some online docs with the configuration for webspeed, but I can't find anything to summarize all broker properties.

For now, I see this in ubroker.properties:

    workDir=@{WorkPath}
    srvrStartupParam=-p web/objects/web_disp.p -cpstream iso8859-1 -weblogerror
    operatingMode=Stateless
    wsRoot=/webspeed
    killAgentAfterTimeout=true
    connectingTimeout=60
    sessionTimeout=180
    requestTimeout=15
    autoTrimTimeout=1800
    maxClientInstance=512 // these are the qtp sessions in Jetty - not sure if we need to configure these
    initialSrvrInstance=5
    minSrvrInstance=5
    maxSrvrInstance=10

• configuration of webspeed and its agents in FWD. This would be in a single section, like we do for REST, and will include the details to start the webspeed agents (the FWD process account should be enough). Plus PROPATH. Plus if we run in PASOE mode or not?
• creating a handler (inherited from LegacyServiceHandler)
• the webspeed agents and the pool
  • PASOE looks like deviates from classic when dispatching webspeed requests - see https://community.progress.com/s/article/What-is-the-difference-between-Classic-WebSpeed-and-Webspeed-for-PASOE .
  • The modes for 'classic' according to https://docs.progress.com/bundle/openedge-webspeed-117/page/WebSpeed-application-states.html are Stateless and State-aware.
  • I'm not sure how the webspeed agent (classic or PASOE) would differ from a normal PASOE/classic appserver agent, as I can't yet get a working example running in OE. At the least, this agent looks like it starts a top-level procedure and waits for event notifications - so we will need to start this agent in this 'special mode'. Maybe we can set a flag at the classic agent to run in webspeed or normal mode?
• the handler posting tasks to the agent:
  • this will need to expose the HTTP request - see how WebServiceHandler does it. Sending the instances directly I think is OK. This will require WEB-NOTIFY implementation on the 'remote' side (i.e. Agent.java) to post an event to that agent and wait for the request to complete? I don't know how webspeed does this.
• the WEB-CONTEXT handle implementation - this will use the exposed HTTP request/respose.

Constantin Asofiei wrote:

That CGI module probably runs some batch mode version of _progres OE command line launcher.

The command I see configured is -p web\objects\web-disp.p -weblogerror - there is no -b to run in batch.

Extracted from one of the articles: "This is because Classic WebSpeed is not running in batch mode, even though it's running as a background process. The PASOE does run in batch mode.". Maybe it helps, maybe not.

A bit late to it but I'm with Constantin on this one, just forget about how Progress implemented the Webspeed runtime (classical or new web transport in PASOE). You already have support for the Web Handler, just use custom wrap-cgi include files that uses the WebRequest/WebResponse implementation and forget about converting the Webspeed 'runtime' :)

Marian, I would go to that approach, but there are lots of exposed global shared vars and maybe even session super-procedures. We will end up duplicating this code regardless (and the WEB-CONTEXT handler anyway is needed).

For projects that depend on web-disp.p, we don't know if any back-calls are done into this ADE runtime (plus shared var usage).

A bit late to it but I'm with Constantin on this one, just forget about how Progress implemented the Webspeed runtime (classical or new web transport in PASOE). You already have support for the Web Handler, just use custom wrap-cgi include files that uses the WebRequest/WebResponse implementation and forget about converting the Webspeed 'runtime' :)

I'm fine with that, so long as customer 4GL code does not have any direct dependencies upon those webspeed runtime files. In that case, we would need to have equivalents.

Constantin Asofiei wrote:

For programs, we will need a way to automatically register them with SourceNameMapper (avoiding a name_map.xml file) and into the propath.

• ensure the ADE programs for webspeed are converted and running properly
• add them to the conversion of the customer application (so they are not included in FWD)
• any customization of web-disp.p (or in general terms, the 'webspeed handler') can be managed easily by providing at the customer app conversion the webspeed handler files.

To me, this is more clean and future-proof than hard-coding the web-disp.p handler in FWD.

I don't want each customer to have to DIY their own implementation of WebSpeed. FWD needs to include WebSpeed, built in. We can discuss what parts are core vs optional, but at least the core should be there. My expectation is that WebSpeed applications will just work and the customer won't have to do anything other than convert their code and configure the runtime.

Most customers will not need anything other than the stock WebSpeed. In the less likely case that a customer currently customizes the ADE WebSpeed code, we will consider their needs on a case by case basis.

OK, so if we allow the webspeed to specify the program via startup procedures, then we can use the default web-disp.p from FWD and customize this as necessary.

But, IMO, it will take more time to hand-write the handler than just convert web-disp.p.

I'm fine with converting the web-disp.p.

So I have a setup running to check webspeed tests. I need to write a test to check all WEB-CONTEXT methods, how they get the state from the HTTP request (the CGI vars) - but this will be when we will implement the handle.

I've checked what happens if multiple streams are opened for 'web', and they all can write, and closing a 'web' stream does not close the actual HTTP response stream. So except some flushing issues which may happen when multiple streams are used, I don't see any major issues.

Greg, I'm not sure what you mean by the 'unnamed output'. This test shows that unless STDOUT is redirected, it will not reach the 'web stream':

output to "web".
message "this goes to log file".
put unformatted "this is web!".
output close.

put unformatted "this is not web.".
message "this goes to log file also".

Greg, I'm not sure what you mean by the 'unnamed output'. This test shows that unless STDOUT is redirected, it will not reach the 'web stream':

Interesting. That is different that I would have expected, since standard CGI reads the STDOUT of the spawned child process to get the generated output that should be returned to the web browser.

OE must be fiddling with that and explicitly routing the special web stream while leaving the STDOUT redirected to the log.

For the moment I'm working with a normal appserver agent to simulate a webspeed one, but I'm planning to add, if needed, a node in the appserver configuration that marks if the current agent is a webspeed one or not. However, I'm not sure right now where this would come into play (i.e., what part of the code will use this information). Should the webspeed stuff only be allowed to be executed by a webspeed agent? Assuming that, is the following table correct?

Another possibility is that I've misunderstood this part and I need to review it.

No, I want to move this configuration in the webspeed specific node.

I'm OK to use the 'normal' agent during the work until we implement the webspeed agent and pool, but the solution is to have a separate configuration.

Radu Apetrii wrote:

And for the app server creation, may I use AppServerManager.startAppServer() or would you recommend against it?

We need a new entry point for the webspeed agent launch, as this needs to start a program and not 'listen in a loop' (it will look more like launching a normal FWD client). Please check some tests what happens if there are QUIT, STOP or other kinds of ERROR conditions (is the webspeed agent terminated?) when something gets executed on the webspeed agent.

Do the agents live across multiple requests or are they launched for each new request?

Greg Shah wrote:

Do the agents live across multiple requests or are they launched for each new request?

Certainly live across multiple requests, it can even be 'bound' to one client to have that 'state-aware' that even supports transactions spawning multiple requests. As for QUIT/STOP/ERROR those need to be handled (if not done already in the 4GL code executed) and the agent process returns back into the wait-for loop instead of being ended.

Constantin Asofiei wrote:

Radu Apetrii wrote:

And for the app server creation, may I use AppServerManager.startAppServer() or would you recommend against it?

We need a new entry point for the webspeed agent launch, as this needs to start a program and not 'listen in a loop' (it will look more like launching a normal FWD client).

Ok, then, I will create some new classes: WebspeedAgent, WebspeedAgentPool, WebspeedAgentStopper (maybe?), WebspeedAppServerLauncher (again maybe?), and I'll see what else. I see that the normal agent class is in p2j/util package. Should I place these new ones somewhere else? For the handler that deals with HTTP requests I created a p2j/admin/server/webspeed package. I don't know if this helps or not with the webspeed agents.

Let's discuss this a little on a meet. Do you have time?

Constantin Asofiei wrote:

From what I see, the program name is set in a:
[...]

which exists in web-disp.p. This in turn is used by the "web-notify" trigger.

web-disp.p seems to be a persistent program which lives? in the CGI app.

We need to understand how the dispatch is being executed in OE (what is the CGI program actually executing/running).

Just a relevant quote from their forums (https://community.progress.com/s/question/0D54Q00008GMM0PSAX/shared-variables-in-pasoe):

In Classic WebSpeed, all ABL code is run the web-disp.p call stack

They've missed the "in".

You can check SESSION:FIRST-PROCEDURE and see if it's web-disp.p (make sure to disable any startup procs).

Constantin, I've got a quick question. Should an instance of WebspeedHandler be created for each app server defined with webspeed, or will it just be one webspeed handler overall? I vaguely remember something about the first option, but I thought it would be better to ask. If that's the answer, indeed, then I presume that the variables from the handler won't be static (as they are currently in WebServiceHandler).

Radu Apetrii wrote:

Constantin, I've got a quick question. Should an instance of WebspeedHandler be created for each app server defined with webspeed,

Yes, one instance for each deployed config, as each one has its own configuration (basepath, etc).

If that's the answer, indeed, then I presume that the variables from the handler won't be static (as they are currently in WebServiceHandler).

Yes, those can't be singletons, as there will be different handlers for each webspeed config.

One more thing. When I add the webspeed node-attribute on the app server configuration, I receive this error when starting the server. Do I need to add something somewhere? (I believe the last error is relevant)

java.lang.RuntimeException: class 'container' name ''
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:310)
    at com.goldencode.p2j.directory.XmlRemapperIO.load(XmlRemapperIO.java:169)
    at com.goldencode.p2j.directory.XmlRemapper.load(XmlRemapper.java:159)
    at com.goldencode.p2j.directory.RamRemapper.bind(RamRemapper.java:250)
    at com.goldencode.p2j.directory.DirectoryService.rollBack(DirectoryService.java:5182)
    at com.goldencode.p2j.directory.DirectoryService.<init>(DirectoryService.java:316)
    at com.goldencode.p2j.directory.DirectoryService.createInstance(DirectoryService.java:338)
    at com.goldencode.p2j.main.StandardServer.bootstrap(StandardServer.java:1129)
    at com.goldencode.p2j.main.ServerDriver.start(ServerDriver.java:526)
    at com.goldencode.p2j.main.CommonDriver.process(CommonDriver.java:522)
    at com.goldencode.p2j.main.ServerDriver.process(ServerDriver.java:228)
    at com.goldencode.p2j.main.ServerDriver.main(ServerDriver.java:955)
Caused by: java.lang.RuntimeException: class 'container' name 'server'
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:310)
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:306)
    ... 11 more
Caused by: java.lang.RuntimeException: class 'container' name 'standard'
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:310)
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:306)
    ... 12 more
Caused by: java.lang.RuntimeException: class 'container' name 'appservers'
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:310)
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:306)
    ... 13 more
Caused by: java.lang.RuntimeException: wrong attribute 'webspeed' in class 'appserver' node 'app_server'
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:288)
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:306)
    ... 14 more

Yes, change the dir_schema.xml

Constantin Asofiei wrote:

Yes, change the dir_schema.xml

It worked, thanks!

Now the next question: When calling the method LegacyServiceHandler.initializeHandler() (the same method as in WebServiceHandler.initialize() for example) from WebspeedHandler.initialize(), the program reaches pool.initialize(), and in there, it tries to get the name of the app server. This is where the problem comes from. To get the app server name, the following line is called:

return Utils.getDirectoryNodeString(null, type + "/appserver", null, false);

The webspeed configuration node will be a container which will have one or more configurations. So that pool.initialize() needs to receive an argument, under which you will look for appserver . Currently type meant that e.g. web/appserver exists, but now it needs to exist as webspeed/config1/appserver. So webspeed/config can be specified as an argument.

Constantin Asofiei wrote:

So webspeed/config can be specified as an argument.

And where will this come from? Should this also be specified in the app server definition? If so, will this be a different thing from the basepath node (a.k.a. should there be two nodes)?

This is an (incomplete) example of how an webHandler configures:

        <node class="container" name="webHandler">
          <node class="boolean" name="enabled">
            <node-attribute name="value" value="TRUE"/>
          </node>
          <node class="string" name="appserver">
            <node-attribute name="value" value="appservername"/>
          </node>
          <node class="string" name="alias">
            <node-attribute name="value" value="fwdprocesscertificatealais"/>
          </node>
        </node>

For webspeed, it will look like this:

        <node class="container" name="webspeedHandler">
          <node class="container" name="configBroker1">
             <node class="boolean" name="enabled">
               <node-attribute name="value" value="TRUE"/>
             </node>
             <node class="string" name="appserver">
               <node-attribute name="value" value="webspeedappserver1"/>
             </node>
             <node class="string" name="alias">
               <node-attribute name="value" value="fwdprocesscertificatealais"/>
             </node>

          </node>
          <node class="container" name="configBroker2">
             <node class="boolean" name="enabled">
               <node-attribute name="value" value="TRUE"/>
             </node>
             <node class="string" name="appserver">
               <node-attribute name="value" value="webspeedappsrv2"/>
             </node>
             <node class="string" name="alias">
               <node-attribute name="value" value="fwdprocesscertificatealais"/>
             </node>
          </node>

        </node>

The appserver configuration will be marked 'for webspeed'.

As expected, I have another question. In the following code from StandardServer.initialize():

               if (rest || legacyWebHandler || soap || multiTenancy)
               {
                  // force loading of the services...
                  SourceNameMapper.registerServices();

                  if (RestHandler.useWebServiceAuthentication()       || 
                      SoapHandler.useWebServiceAuthentication()       ||
                      WebServiceHandler.useWebServiceAuthentication() ||
                      TenantHandler.useWebServiceAuthentication())
                  {
                     sm.legacyWebSm.startWebServiceContextThreads();
                  }
               }

Yes, that's needed for webspeed, also.

But, I should have been made this more extensible, currently you need to add new classes there, even for SessionHandler for the REST admin APIs was added.

I'm not sure I understand how do you plan to implement this but this is the situation in Progress:
- there is no 'clasic' webspeed product available anymore, implementing the webspeed broker the old way makes very little sense imho.
- there is nothing specific in the PASOE for webspeed apart the '[[OpenEdge.Web.CompatibilityHandler:https://github.com/AcornIT/ADE-Sourcecode/blob/master/src/netlib/OpenEdge/Web/CompatibilityHandler.cls]]' class that needs to be set as default handler for 'old' webspeed style web apps.
- there is no webspeed specific configuration in PASOE, all customisations of web-util.p are to be kept and those made to web-disp.p are to be migrated to web-handler.p.

The way I see it there shouldn't be no need to implement anything new in FWD, apart from the output to 'web' option to support the {&out} statements used in webspeed and the WEB-CONTEXT system handle... this is if you do plan to use the ADE code as-is and just convert.

- there is no 'clasic' webspeed product available anymore, implementing the webspeed broker the old way makes very little sense imho.

Isn't this just because PSC is forcing customers to move to 12.8 and PASOE? We are not removing classic appserver support and similarly, we would not remove classic webspeed support.

- there is nothing specific in the PASOE for webspeed apart the '[[OpenEdge.Web.CompatibilityHandler:https://github.com/AcornIT/ADE-Sourcecode/blob/master/src/netlib/OpenEdge/Web/CompatibilityHandler.cls]]' class that needs to be set as default handler for 'old' webspeed style web apps.

Are you saying that customers can use the PASOE web handler as a replacment for classic webspeed without any functional issues?

all customisations of web-util.p are to be kept

Is this a common thing?

The way I see it there shouldn't be no need to implement anything new in FWD

This is what I was asking in #6506-10. I thought we had already implemented the web handler in Java code, but if it isn't a complete or seamless replacement for web-disp.p then I would expect us to convert web-disp.p and ensure it runs properly.

A secondary question: should we convert web-handler.p instead of using our hand coded Java?

Greg Shah wrote:

A secondary question: should we convert web-handler.p instead of using our hand coded Java?

Greg, I'm not sure what you mean here - for the /web transport, we specify handlers which are implementations of OpenEdge.Web.WebHandler.

I thought we had our own version of web-handler.p written in hand coded Java.

Greg Shah wrote:

- there is no 'clasic' webspeed product available anymore, implementing the webspeed broker the old way makes very little sense imho.

Isn't this just because PSC is forcing customers to move to 12.8 and PASOE? We are not removing classic appserver support and similarly, we would not remove classic webspeed support.

Well, it looks to me there isn't any classic webspeed support yet... looks to me this is something new that is implemented now unless I'm reading all those messages wrong (as in new webspeed agent, new configuration and stuff).

Are you saying that customers can use the PASOE web handler as a replacment for classic webspeed without any functional issues?

Yes, all that Webspeed was able to do was to support GET/POST requests through the CGI interface - all of that can be done using a WebHandler, their simple implementation of CompatibilityHandler can be used for that purpose.

all customisations of web-util.p are to be kept

Is this a common thing?

If you are asking if customising web-util.p or web-disp.p is a common thing I really don't know, since the code was there to use people might have been doing so. Because for webspeed there were no hook event procedures that could be used they just changed the 'standard' code to do some extra stuff for initialisation, extend the API in web-util.p or whatever, not sure how supported was that by PSC though.

The way I see it there shouldn't be no need to implement anything new in FWD

This is what I was asking in #6506-10. I thought we had already implemented the web handler in Java code, but if it isn't a complete or seamless replacement for web-disp.p then I would expect us to convert web-disp.p and ensure it runs properly.

The WebHandler is just an interface in OO package, there is really nothing implemented there... one needs to handle all HTTP methods in implementation, but that is 4GL code not Java.

A secondary question: should we convert web-handler.p instead of using our hand coded Java?

If you want to go the 'ADE conversion' path then that is the reasonable thing to do, as said if customers have customised versions of those programs it will just be converted like any 4GL code.

Greg, for 'classic web-speed' where web-disp.p is used, I still think we need this approach to implement the WEB-NOTIFY event and the special agent which starts the web-disp.p program. This is the main difference between the 'classic web-speed' and the 'pasoe web-speed', as in PASOE we can just convert web-handler.p, the CompatibilityHandler.cls (And any other dependencies) and configure a path to be served by CompatibilityHandler.cls (or whatever other is the 'pasoe web-speed handler') in the FWD's configuration for web transport.

• for classic, a single instance of web-disp.p exists, running as startup-procedure on the webspeed agent. The stacktrace for a call looks like this:
  

  [25/01/29@13:26:22.414+0200] P-003612 T-004624 1 WS -- (Procedure: 'test1.p' Line:714) stack 1 test1.p
  [25/01/29@13:26:22.414+0200] P-003612 T-004624 1 WS -- (Procedure: 'test1.p' Line:714) stack 2 run-web-object web/objects/stateaware.p
  [25/01/29@13:26:22.414+0200] P-003612 T-004624 1 WS -- (Procedure: 'test1.p' Line:714) stack 3 run-web-object web/objects/web-util.p
  [25/01/29@13:26:22.414+0200] P-003612 T-004624 1 WS -- (Procedure: 'test1.p' Line:714) stack 4 SYSTEM-TRIGGER web\objects\web-disp.p
  [25/01/29@13:26:22.414+0200] P-003612 T-004624 1 WS -- (Procedure: 'test1.p' Line:714) stack 5 web\objects\web-disp.p
  

  • the persistent programs ran by the agent are:
    

    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1000 webutil/webstart.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1001 web/objects/stateaware.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1002 web/objects/web-util.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1003 web/objects/session.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1004 web/support/webinput.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1005 web/support/webtog.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1006 web/support/webradio.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1007 web/support/weblist.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1008 web/support/webedit.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1009 web/support/tagrun.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1010 web/support/webmsg.p
    [25/01/29@13:30:29.342+0200] P-002856 T-000928 1 WS -- (Procedure: 'test1.p' Line:707) persistent 1011 test1.p
    

    
    So even the test1.p program (the target) is ran persistent.
• for PASOE, an instance of the CompatibilityHandler.cls is created for each request (at least this is how /web does it.

I agree we can't change the execution environment by using PASOE always. Go ahead with the plan to convert web-disp.p (and its dependencies) and web-handler.p/CompatibilityHandler.cls (and their dependencies).

• Should this loop be added in Agent.startWebspeed()? This is a method that I added and is executed right after the agent is created.
• If so, is Agent.listen() of interest? Do I directly call that method or should I derive it somehow?

Thank you in advance!

Radu, there is no 'loop waiting for commands' - startWebspeed needs to normally launch the target program.

Radu, please commit your changes in a branch, if you haven't already.

• This is a valid request URL: https://localhost:7443/path/to/webspeed/WService=wsbroker1/test1.p
• And this is not a valid request URL: https://localhost:7443/path/to/webspeed/test1.p?

Radu Apetrii wrote:

Just a quick question here: will the webspeed requests always have the name of the webspeed agent/broker in their URI? For example:

• This is a valid request URL: https://localhost:7443/path/to/webspeed/WService=wsbroker1/test1.p
• And this is not a valid request URL: https://localhost:7443/path/to/webspeed/test1.p?

Lets make the broker name part of the basepath. If you recall, we discussed that the webspeed node will contain multiple webspeed 'broker' definitions, each one with its own config, and in FWD there will be multiple Jetty handlers, one for each 'broker'.

So if the path is path/to/webspeed/wsbroker1/test1.p, then wsbroker1 will have as basepath path/to/webspeed/wsbroker1.

Constantin Asofiei wrote:

So if the path is path/to/webspeed/wsbroker1/test1.p, then wsbroker1 will have as basepath path/to/webspeed/wsbroker1.

Amazing! I was looking into how to differentiate the brokers when a request is handled and this is exactly what I needed.

Radu Apetrii wrote:

Constantin Asofiei wrote:

So if the path is path/to/webspeed/wsbroker1/test1.p, then wsbroker1 will have as basepath path/to/webspeed/wsbroker1.

Amazing! I was looking into how to differentiate the brokers when a request gets handled and this is exactly what I needed.

Radu Apetrii wrote:

Radu Apetrii wrote:

Constantin Asofiei wrote:

So if the path is path/to/webspeed/wsbroker1/test1.p, then wsbroker1 will have as basepath path/to/webspeed/wsbroker1.

Amazing! I was looking into how to differentiate the brokers when a request gets handled and this is exactly what I needed.

Hmm, what a weird thing. This double-message behavior happened to (at least) Florin as well today.

Does this cause a compatibility issue? In other words, the URLs from the existing 4GL system must continue to work.

Greg Shah wrote:

Does this cause a compatibility issue? In other words, the URLs from the existing 4GL system must continue to work.

For classic webspeed, the OpenEdge URL is this to run something on webspeed: http://localhost/scripts/cgiip.exe/WService=wsbroker1/test.p Note the CGI exe app which is in the URL. Most likely application will already have a reverse-proxy to rewrite URLs. So, do we want to have this exact URL in FWD?

Constantin Asofiei wrote:

Greg Shah wrote:

Does this cause a compatibility issue? In other words, the URLs from the existing 4GL system must continue to work.

For classic webspeed, the OpenEdge URL is this to run something on webspeed: http://localhost/scripts/cgiip.exe/WService=wsbroker1/test.p Note the CGI exe app which is in the URL. Most likely application will already have a reverse-proxy to rewrite URLs. So, do we want to have this exact URL in FWD?

I'm not saying that is not possible to have it, it will just require to have a single handler for all webspeed brokers, and get the broker and test from the path. OTOH, I don't know if ADE web-disp.p and related code has any dependencies on CGI variables returning or using exactly this path, so on a second thought, Radu, I think we need to change the Jetty handler approach. Basepath will always be /scripts/cgiip.exe/ and last path will give the broker and target program. We will have a single Jetty handler, even if there are multiple configurations.

Constantin Asofiei wrote:

Greg Shah wrote:

Does this cause a compatibility issue? In other words, the URLs from the existing 4GL system must continue to work.

For classic webspeed, the OpenEdge URL is this to run something on webspeed: http://localhost/scripts/cgiip.exe/WService=wsbroker1/test.p Note the CGI exe app which is in the URL. Most likely application will already have a reverse-proxy to rewrite URLs. So, do we want to have this exact URL in FWD?

Please ask Scott for his thoughts on this. I generally like the cleaner approach you are suggesting. I just want to make sure we don't break applications. If the applications are all configurable and don't have any hard coded dependencies on the /scripts/cgiip.exe/WService=wsbroker1/, then we would want your approach. But if there is the possibility that customers could have hard coded references to the legacy URL components, then we will need to retain that as an option.

Constantin Asofiei wrote:

Radu, I think we need to change the Jetty handler approach. Basepath will always be /scripts/cgiip.exe/ and last path will give the broker and target program. We will have a single Jetty handler, even if there are multiple configurations.

Constantin, if you don't mind, I'll make the change after we receive Scott's answer, just in case there's something else we're missing.

From the reply for the email in the previous note, there is this note:

The app uses the Broker’s URL Address to determine what to include. Ex: /oe.rds/WService=wsdevt or in most PAS instances we have been configuring it to /web

So this means that we need to have a single handler (with multiple webspeed configurations), and let the base path of i.e. oe.rds be configured. This will let the handler to get all requests on this basepath, look in the first path of the target i.e. WService=wsbroker1, resolve the webspeed config name i.e. wsdevt or @wsbroker1, and send the request to that.

Followup question from Constantin:

For this one on 3.c:

c. web-context:get-config-value(name) - I found these possible config names in ADE classic web-speed files:

We utilize some of these which are configured through ubroker.properties.

Please send a list of the ones you use from ubroker.properties, and what is the intent with them, in the application.

Constantin Asofiei wrote:

From the reply for the email in the previous note, there is this note:

The app uses the Broker’s URL Address to determine what to include. Ex: /oe.rds/WService=wsdevt or in most PAS instances we have been configuring it to /web

So this means that we need to have a single handler (with multiple webspeed configurations), and let the base path of i.e. oe.rds be configured. This will let the handler to get all requests on this basepath, look in the first path of the target i.e. WService=wsbroker1, resolve the webspeed config name i.e. wsdevt or @wsbroker1, and send the request to that.

This will not impact the current directory configuration (#6506-51), right? In other words, I don't need to change anything in the directory.

Radu Apetrii wrote:

Constantin Asofiei wrote:

From the reply for the email in the previous note, there is this note:

The app uses the Broker’s URL Address to determine what to include. Ex: /oe.rds/WService=wsdevt or in most PAS instances we have been configuring it to /web

So this means that we need to have a single handler (with multiple webspeed configurations), and let the base path of i.e. oe.rds be configured. This will let the handler to get all requests on this basepath, look in the first path of the target i.e. WService=wsbroker1, resolve the webspeed config name i.e. wsdevt or @wsbroker1, and send the request to that.

This will not impact the current directory configuration (#6506-51), right? In other words, I don't need to change anything in the directory.

Not really, but you will need to have 'global' enabled and basepath settings:

       <node class="container" name="webspeedHandler">
          <node class="boolean" name="enabled">
            <node-attribute name="value" value="TRUE"/>
          </node>
          <node class="string" name="basepath">
            <node-attribute name="value" value="" />
          </node>

The basepath will be like /oe.rds/ or scripts/cgiip.exe/. The other paths, WService=<name>/<program>, are 'hard-coded', and the main webspeed handler in FWD will need to resolve the target from /Service=<name>/ path.

enable will be for both the 'main' handler and the specific webspeed configs.

Constantin Asofiei wrote:

The basepath will be like /oe.rds/ or scripts/cgiip.exe/. The other paths, WService=<name>/<program>, are 'hard-coded', and the main webspeed handler in FWD will need to resolve the target from /Service=<name>/ path.

And I assume I won't need a basepath node in the appserver configuration anymore.

Radu Apetrii wrote:

Constantin Asofiei wrote:

The basepath will be like /oe.rds/ or scripts/cgiip.exe/. The other paths, WService=<name>/<program>, are 'hard-coded', and the main webspeed handler in FWD will need to resolve the target from /Service=<name>/ path.

And I assume I won't need a basepath node in the appserver configuration anymore.

Correct. But please post a directory.xml once you have the config.

• Refactored WebspeedHandler to use one single handler as noted in #6506-78.
• Added support for the WEB-NOTIFY event.
• Added countdown latches for the threads that are handling webspeed requests.
• Other minor bug fixes found while testing.

I rebased 6506a with the latest trunk, the branch reaching revision 15760. I hope there aren't any conflict leftovers, but I'll double check just in case.

Constantin Asofiei wrote:

So this means that we need to have a single handler (with multiple webspeed configurations), and let the base path of i.e. oe.rds be configured. This will let the handler to get all requests on this basepath, look in the first path of the target i.e. WService=wsbroker1, resolve the webspeed config name i.e. wsdevt or @wsbroker1, and send the request to that.

• Before, when we had multiple handles, each instance would retrieve the app server name assigned to it. In other words, instance.initializeHandler() was called multiple times, each time for a different handler configuration.
• Now, we have one handle, but each broker has its own configuration (i.e. one app server name per broker). Thus, when I call instance.initializeHandler(), what app server name is supposed to be used? Is it alright if the program gets the first one and that's it?

Radu, yes, the handler will need to have different pools, for each webspeed configuration (and its appserver).

So, you need to add an override initializeHandler, where you create different pools, and save them at the webspeed handler. And make the LegacyServerHandler.handle methods instance instead of static, and override as needed in the webspeed handler.

Conclusion: refactor as needed so webspeed handler can use multiple pools, and its 'handle' override can post the work to the appropriate pool.

For the WEB stream: for the /web handler, the request/response and other info gets saved via WebHandler.initialize. This part needs to remain common for webspeed and web handler. Move the WorkArea context in another class, if needed.

Constantin, I have another question. Do I need to create a WebspeedService, like the ones for REST, SOAP, and the others? Take a look at the following piece of code from SourceNameMapper.registerServices():

                  case LegacyService.SOAP:
                     SoapHandler.addService(pls, extProg.pname, extProg.main.getMethod());
                     break;

                  case LegacyService.REST:
                     RestHandler.addService(pls, extProg.pname, extProg.main.getMethod());
                     break;

                  case LegacyService.WEBHANDLER:
                     WebServiceHandler.addService(pls, (Class) cls);
                     break;

I see that RestHandler.invoke() uses a RestService, and I'm not exactly sure how to replicate this in the WebspeedHandler.

Radu Apetrii wrote:

Constantin, I have another question. Do I need to create a WebspeedService, like the ones for REST, SOAP, and the others? Take a look at the following piece of code from SourceNameMapper.registerServices():

No, you do not need this - as any 4GL program can be ran via webspeed, not just whatever is configured for REST/SOAP.

Please put the webspeed code (for now, the handler but in the future probably some other code like the web-context support) in a src/com/goldencode/p2j/webspeed/ directory instead of src/com/goldencode/p2j/admin/server/webspeed/. Webspeed has nothing to do with our admin support so I don't want it mixed together.

Greg Shah wrote:

Please put the webspeed code (for now, the handler but in the future probably some other code like the web-context support) in a src/com/goldencode/p2j/webspeed/ directory instead of src/com/goldencode/p2j/admin/server/webspeed/. Webspeed has nothing to do with our admin support so I don't want it mixed together.

Noted. The change of directories will be visible/available in the next commit.

From Scott:

I have attached the openedge.properties file for our PasOE “pasdevt” instance:

[pasdevt.oeabl.WEB]
    adapterEnabled=1
    defaultCookieDomain=
    defaultCookiePath=
    defaultHandler=OpenEdge.Web.DefaultWebHandler
    srvrAppMode=production
    srvrDebug=0
    wsRoot=/oeabl/static/webspeed

You will notice that only the WEB Handler is enabled for the pasdevt instance.

I have also attached the openedge.properties.README file that gives an explanation of the values that can be assigned to each entry.

The specific files referenced are posted in #9839.

Followup question from Constantin:

I more interested in how you use them in the application; you mentioned that 'some of them you use in the application'. So, if you use 'defaultCookiePath', what's the expected value? Or 'fileUploadDirectory' or 'srvrAppMode'. I'm interested more in what you do with these values - what value is expected, and how you decide some logic in the app beside on this value.

From Scott:

To the best of my knowledge, we actually don’t reference any of them internally in our code but are defined for the startup and control of the actual PasOE runtime, specifying where different directories are mapped to and the security level of the run-time environment.

The exception to the above would be if we allow files to be uploaded in an application, the following two parameters would need to be defined for the agent to allow the upload to be received (within the size limit specified in the MaxSize parameter):

binaryUploadMaxSize=0
fileUploadDirectory=

Will – do you know of any parameters that are directly referenced in the ABL code?

From Will:

As far as I know we only use the two parameters Scott mentioned.

Two things that are complicating the process of completing these changes:

• #8661 a long running task where the core PASOE support is finished, it affects the handlers and needs to merge to trunk before 6506a
• #9687 where we are dropping support for Java 8 and Java 11, this means we move to Jetty 12 and the handlers have a different API so there is some rework (however, we may get 6506a into trunk before 9687a merges)

• Make sure that the uses of contexts and threads are correct. This is the reason I was facing some bugs, and thanks to Constantin, it is much clearer now. I'm adjusting things right now, and I will provide updates soon.
  • This is the main block of what's left to do in this task. By my estimation, the percentage would bump up to 80 after I finish adjusting the contexts and threads.
• Another thing is Constantin's #6506-91 note. This involves a bit of refactoring such that webspeed and web handler have a common ground. Currently, there are things duplicated/really similar between them.
  • After this gets completed, I will probably up the percentage to 90.
• The last thing on my list is some code cleanup and javadoc + history entries writing. That's the last 10%.

If I have any more updates, I will make sure to include them here.

• in Agent.startWebspeed, add a finally block after the catch, to also terminate the webNotifyListener thread
• in p2j.oo.web.WebHandler - remove the ContextLocal<WebHandlerHelper.WorkArea> local field and access the WorkArea only from the WebHandlerHelper - define the ContextLocal var there. More, clear and initialize are the same for WebHandler and WebspeedRequestHandler - the point is, make this code common in WebHandlerHelper , and remove WebspeedRequestHandler - I don't see a reason for it.
• please post a sample of the directory.xml configuration for webspeed (with multiple brokers).
• WebspeedHandler - I don't see how result.getError() != null can be true; in Agent.handleWebspeed, invocationResult is just created and returned.

Constantin Asofiei wrote:

and remove WebspeedRequestHandler - I don't see a reason for it.

May I also remove the handler parameter from the handleWebspeed chain? It's not used anywhere. I'm talking about AppServerEntry.handleWebspeed, Agent.handleWebspeed, AppServerHelper.handleWebspeed and AppServerManager.handleWebspeed.

Radu Apetrii wrote:

Constantin Asofiei wrote:

and remove WebspeedRequestHandler - I don't see a reason for it.

May I also remove the handler parameter from the handleWebspeed chain? It's not used anywhere. I'm talking about AppServerEntry.handleWebspeed, Agent.handleWebspeed, AppServerHelper.handleWebspeed and AppServerManager.handleWebspeed.

Yes, that is not needed for webspeed. For web/ is the actual target class which needs to be instantiated and execute the request.

And should I also add in Agent.handleWebspeed a call to WebHandlerHelper.clear() right before the return statement? The initialize is there, but the clear call is missing.

Radu Apetrii wrote:

And should I also add in Agent.handleWebspeed a call to WebHandlerHelper.clear() right before the return statement? The initialize is there, but the clear call is missing.

Yes, in a finally block.

• WebNotifyListener.stopListening just does webNotifyThread.interrupt() - you need to also set a volatile ? variable running to false (and also not post WEB-NOTIFY event if this var is false, and also running = true; while (running).
• openWebStream - please mark UnimplementeadFeature.missing or otherwise as TODOs the fact that we (at this time) do not support paged/page-size and INPUT FROM 'web'.
• GenericOutputStream(HttpServletResponse response) needs to be GenericOutputStream(java.io.OutputStream os) - no need to hard-code it for ServletOutputStream.

Constantin Asofiei wrote:

• WebNotifyListener.stopListening just does webNotifyThread.interrupt() - you need to also set a volatile ? variable running to false (and also not post WEB-NOTIFY event if this var is false, and also running = true; while (running).

This means that in WebNotifyListener.startListening(), the while condition will change from while(true) to while(running), right? The variable will be set to true before the while begins, it will become false in the stopListening method, and there will also be a check before sending the web-notify event.

Radu Apetrii wrote:

Constantin Asofiei wrote:

• WebNotifyListener.stopListening just does webNotifyThread.interrupt() - you need to also set a volatile ? variable running to false (and also not post WEB-NOTIFY event if this var is false, and also running = true; while (running).

This means that in WebNotifyListener.startListening(), the while condition will change from while(true) to while(running), right? The variable will be set to true before the while begins, it will become false in the stopListening method, and there will also be a check before sending the web-notify event.

Yes.

• Added implementations for GET-CGI-LIST(), GET-CGI-VALUE(), GET-CGI-LONG-VALUE() methods.
  • For GET-CONFIG-VALUE() I'm still a bit unsure how to correctly retrieve the values. In the 4GL environment, these are stored in a ubroker.properties file, but in our Java environment I don't know exactly. I might be able to retrieve a few of them via the directory, but I need to investigate this further.
• Added a few CGI variables in WebHandlerHelper.
• Addressed the review points from #6506-109.

In parallel, I was also working on getting a customer application running. I'm currently facing some difficulties, but I will provide updates as soon as possible.

• WebContextOps - anything not supported, like FORM/QUERY env, other CGI vars, etc need to be logged via UnimplementedFeature.missing
• use character.of to get constants instead of new instances
• using response.setStatus(HttpServletResponse.SC_OK); in WebSpeedHandler I don't think is OK - this should be set by the web-disp.p program in the header?
• In Agent$WebNotifyListener.notifyWebEvent, this code needs to create first the latch and after that notify:
  

           synchronized (this)
           {
              this.notify();
              requestLatch = new CountDownLatch(1);
           }
  

• AppServerDefinition - the basepath is read this.basepath = ds.getNodeString(appSrvPath, "basepath"); but dir_schema.xml does not include this.

• make a copy of the branch
• rebase to latest trunk - be careful here, there was lots of refactoring with #8661. You may need to just create a separate branch if rebase is not working.
• we will do testing after that

• Added getHttpRequest() function in WebHandlerHelper.
• Improved the handling of webspeed requests in Agent.
• Added implementations for getExclusiveId(), getConfigValue(), incrementExclusiveId(), getConfigName(), getCurrentEnvironment(), getSessionEnd(), getUtcOffset(),
  and the missing code for FORM and QUERY environments of getCgiValue(), getCgiLongValue(), getCgiList().
• Addressed the review from the previous note.

At this moment in time, the test from webspeed sample works, but there is some weird behavior if requests are executed more than once without the server being closed in between. This is the issue from #9516-70 that I'm trying to address now.

I also need to rebase the branch. I will do this soon.

Radu - the name_map.xml for the FWD changes to include web-disp.p and associated programs has the programs registered like:

  <class-mapping jname="com.goldencode.p2j.webspeed.adecomm.Osfext" pname="adecomm/_osfext.p">

  <class-mapping jname="com.goldencode.p2j.webspeed.adecomm.Osfext" pname="ade/src/adecomm/_osfext.p">

Note the ade/src/ was removed. This works only if the ADE webspeed code is not using a RUN ade/src/path/to/program.p or RUN src/path/to/program.p. Do you know if all RUN statements are relative to ade/src?

The propath for the webspeed agent in directory.xml has ade:ade/src. This would mean that these can be removed for the runtime.

Constantin Asofiei wrote:

Note the ade/src/ was removed. This works only if the ADE webspeed code is not using a RUN ade/src/path/to/program.p or RUN src/path/to/program.p. Do you know if all RUN statements are relative to ade/src?

As far as I've checked, this is the case, yes. There are no RUN statements that have ade/src/path or src/path.

The propath for the webspeed agent in directory.xml has ade:ade/src. This would mean that these can be removed for the runtime.

Understood.

I got the sample working with the web-speed files in FWD.

Greg - https://localhost:7443/scripts/cgiip.exe/WService=WSBroker1/goldencodewebspeed_htm.htm will not work in FWD, but it works in OE. How do we handle this? Do we rewrite automatically the .htm extension to .w?

Do we rewrite automatically the .htm extension to .w?

Yes. OE will automatically convert all .htm and .html extensions to .w. We must do the same.

After ./abl/goldencodewebspeed_htm.w is generated and incremental conversion is started, it will also be picked up if the pattern specified includes *.w files. The same file is stored in the file_signature incremental conversion database. There is no java file generated for the htm file (turned into .w), so no changes will be made if the htm is changed again (just the process of changing the htm into w).

The first thing to fix would be to convert the resulted .w file?

If an .htm or .html file is changed, then we must run the embedded 4GL preprocessor on that file. Running the E4GL preproc will generate a matching .w which then must be included in our conversion list.

The issue with the .w file not being included seems to be related to convertSourceNameToAstNames():

      for (String sourceFile: source.listFilenames())
      {
         astFiles.add(Configuration.getPathToConversionFolder(sourceFile, AstGenerator.AST_POSTFIX));
      }

getPathToConversionFolder() should be able to transform .htm/.html files into .w when called.

I see that 6065a is based on trunk/15757, but I've been testing with trunk/16277. I'll switch the branch and check if the issue is the same.

Dănuț Filimon wrote:

I see that 6065a is based on trunk/15757, but I've been testing with trunk/16277. I'll switch the branch and check if the issue is the same.

trunk issue is fixed by the #10109-194 patch. Conversion and incremental are working properly with 6509a.

Are you holding off on rebase to wait for code review?

The Jetty 12 changes came in trunk 16293 and I'm especially worried that we need to time to stabilize those with webspeed support. I'd rather get the rebase done sooner than later.

Constantin: I know this week is bad for your schedule. Can you get time early next week to do the code review on 6506a? We need to get it into trunk sooner rather than later.

Greg, you should be looking at 6506b, not 'a'. Sorry it is not clear in this task. In #10109-314 the 'b' branch is mentioned. 6506b is rebased with a relatively new trunk, including the one that had Jetty changes.

Every time you rebase, please post in the task to let watchers know the status. The post should be something like "Rebased 6506b from trunk rev XYZ, the latest revision is now ABC.".

Greg Shah wrote:

Every time you rebase, please post in the task to let watchers know the status. The post should be something like "Rebased 6506b from trunk rev XYZ, the latest revision is now ABC.".

My mistake was that I wrote in #10109 about the rebase process, not here (notes #10109-313 and #10109-276). Sorry about that.

• name_map.xml and related code: I think we need to add a fwdwebspeed/ prefix for these sources in name_map.xml. This requires to change the basepath to contain fwdwebspeed/ entry when webspeed is needed, but this ensures that any collisions from i.e. Possenet can not happen.
• WebHandlerHelper.java - what was the need to make the CGI_VARS set case-insensitive?
• the Admin Console needs to be updated with the new appserver configs (webspeed and basepath
• StreamFactory - openWebStream(boolean write, int pageSz, boolean paged) has:
  

  UnimplementedFeature.missing("Open Web Stream: paged and page-size are not supported yet.");
  

  
  We should not log this each and every time. Maybe do it only if pageSz != -1 || paged
• WebContextOps.java - instead of ÿ represents ASCII 255 character just write (char) 255?
• we need some tests with standalone 4GL temp-tables which collide with one or more src/com/goldencode/p2j/webspeed/dmo temp-tables (only by name lets say). And run this test together with the temp-table from builtin FWD webspeed support code. I'm worried about runtime issues here.
• Ovidiu - please review the persistence changes
• Greg, I'm leaving the e4gl changes to you.

Otherwise, 6506b requires rebase (again) to get the Jetty 12 support and move to Java 17.

Code Review Task Branch 6506b Revisions 16293 through 16295

Radu: Although Florin wrote the changes in the E4GL preprocessor and 4GL preprocessor, you'll have to address the issue below.

1. In the E4GLPreprocessor.getParser(), the new approach is inefficient.

• I dislike that we read the entire thing into memory and then make a copy of the array. For very large inputs, this will not scale well. Please consider alternatives such as reading from the end of the file to find the last new line and then remembering that location and only reading to that point.
• If we do keep the current array concept, please rewrite the lastNL search to walk backwards from the end of the array. It makes no sense to me that we would walk from the front of the array to find the last newline.

2. The change in text.g is better addressed in the preproc changes from 6859b. I'm not suggesting a change is needed in 6506b but if 6859b is merged to trunk first, then the text.g should be dropped from 6506b. Otherwise we'll have to clean it up on rebase of 6859b.

Radu, did you get a chance to address the review?

Constantin Asofiei wrote:

Radu, did you get a chance to address the review?

Not yet. The plan is to finish addressing it either later on today, or tomorrow. I'll let you know when the changes are committed.

This is taking a bit longer because I rebased the branch and I have an issue.

I rebased branch 6506b with trunk rev. 16346. That means that 6506b is now at rev. 16349. Then, I committed some changes in order to address what happened during rebase, so the branch reached rev. 16350. However, when I'm starting the server right now, I'm getting:

26/01/15 13:23:01.179+0200 |  SEVERE | com.goldencode.p2j.main.StandardServer | ThreadName:main, ThreadId:00000003, User:standard | 
java.lang.StringIndexOutOfBoundsException: String index out of range: 0
    at java.base/java.lang.StringLatin1.charAt(StringLatin1.java:48)
    at java.base/java.lang.String.charAt(String.java:1517)
    at com.goldencode.p2j.util.JarUtil.searchResourceJars(JarUtil.java:130)
    at com.goldencode.p2j.main.WebServer$WebContext.createContextHandler(WebServer.java:410)
    at com.goldencode.p2j.main.WebServer.getWebContentHandlers(WebServer.java:288)
    at com.goldencode.p2j.main.StandardServer$16.initialize(StandardServer.java:1635)
    at com.goldencode.p2j.main.ServerHookManager.hookInitialize(ServerHookManager.java:147)
    at com.goldencode.p2j.main.StandardServer.bootstrap(StandardServer.java:1174)
    at com.goldencode.p2j.main.ServerDriver.start(ServerDriver.java:566)
    at com.goldencode.p2j.main.CommonDriver.process(CommonDriver.java:522)
    at com.goldencode.p2j.main.ServerDriver.process(ServerDriver.java:233)
    at com.goldencode.p2j.main.ServerDriver.main(ServerDriver.java:1062)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at com.goldencode.p2j.main.FwdLauncher.main(FwdLauncher.java:106)

So I'm looking at why this error appears. Then, I'll have another commit with this fix + the changes from the two reviews.

Radu Apetrii wrote:

So I'm looking at why this error appears. Then, I'll have another commit with this fix + the changes from the two reviews.

Never mind, I just didn't have the directory configured properly. Now I'm trying to understand why the appserver isn't starting.

• Addressed Greg's point in E4GLPreprocessor.getParser(). No more content storing ⛔ + the search is done from the end of the file.
• Added the suggested 'if' in StreamFactory.openWebStream().
• Replaced that ASCII 255 character with (char) 255.

There are some things left which I will mention below:

name_map.xml and related code: I think we need to add a fwdwebspeed/ prefix for these sources in name_map.xml. This requires to change the basepath to contain fwdwebspeed/ entry when webspeed is needed, but this ensures that any collisions from i.e. Possenet can not happen.

I'm not sure I fully understand this. The sources mentioned in name_map.xml are already stored in a webspeed package, but I don't think this is what you meant.

WebHandlerHelper.java - what was the need to make the CGI_VARS set case-insensitive?

I saw in one of the customer applications that a certain get-value() was not returning the right value because the parameter name given to that function was something like <Name> and not <name>. I then checked the behavior in 4GL and saw that the function works the same for both <name> and <Name> parameters, thus I concluded that things are case-insensitive. For more details, you can check notes 236-239 from #10109.

the Admin Console needs to be updated with the new appserver configs (webspeed and basepath)

This is going to sound dumb, but what class handles that? My biggest guess is AdminService.java + AdminServiceImpl.java, but I would really like a confirmation if that's alright.

Ovidiu - please review the persistence changes

I believe we're still waiting on this because Ovidiu was not a watcher on the task. I updated that.

Radu,

I looked at the changes related to persistence which were manually written. They generally seem OK from my PoV. I think the methods getDmoBasePackages(), getDmoBasePackage(), and isDmoBasePackage() form DmoMetadataManager would benefit if their javadoc would be more explicit because now it is a bit of a puzzle, of course, not a showstopper.

The classes in com.goldencode.p2j.webspeed.dmo seem converted, and the same can be said for other classes in com.goldencode.p2j.webspeed where the DMOs are used. I did not go too deep into these. I assume the responsibility belongs to conversion and if that changes, these files will require reconversion.

Radu Apetrii wrote:

name_map.xml and related code: I think we need to add a fwdwebspeed/ prefix for these sources in name_map.xml. This requires to change the basepath to contain fwdwebspeed/ entry when webspeed is needed, but this ensures that any collisions from i.e. Possenet can not happen.

I'm not sure I fully understand this. The sources mentioned in name_map.xml are already stored in a webspeed package, but I don't think this is what you meant.

What I mean is to change name_map.xml so that instead of i.e. pname="adecomm/_osfext.p" you have pname="fwdwebspeed/adecomm/_osfext.p"; this in turn will require for the application which relies on the built-in FWD webspeed support to have fwdwebspeed/ added to PROPATH.

WebHandlerHelper.java - what was the need to make the CGI_VARS set case-insensitive?

I saw in one of the customer applications that a certain get-value() was not returning the right value because the parameter name given to that function was something like <Name> and not <name>. I then checked the behavior in 4GL and saw that the function works the same for both <name> and <Name> parameters, thus I concluded that things are case-insensitive. For more details, you can check notes 236-239 from #10109.

Thanks.

the Admin Console needs to be updated with the new appserver configs (webspeed and basepath)

This is going to sound dumb, but what class handles that? My biggest guess is AdminService.java + AdminServiceImpl.java, but I would really like a confirmation if that's alright.

Look how p2j.admin.AppserverInfo class is referenced in FWD - start from there and adjust the screen for classic appserver, and so on.

name_map.xml and related code: I think we need to add a fwdwebspeed/ prefix for these sources in name_map.xml. This requires to change the basepath to contain fwdwebspeed/ entry when webspeed is needed, but this ensures that any collisions from i.e. Possenet can not happen.

I'm not sure I fully understand this. The sources mentioned in name_map.xml are already stored in a webspeed package, but I don't think this is what you meant.

What I mean is to change name_map.xml so that instead of i.e. pname="adecomm/_osfext.p" you have pname="fwdwebspeed/adecomm/_osfext.p"; this in turn will require for the application which relies on the built-in FWD webspeed support to have fwdwebspeed/ added to PROPATH.

This may be OK temporarily but I don't want this to be a permanent solution. It would be better for this:

• Applications that need NO customization of the ADE, will "just work out of the box" without having to do any conversion of the ADE code.
• Applications that need to customize the ADE, will convert the files needed and ensure they are present in the propath in front of the standard ADE code. In addition, I would expect the jar file to be in the front of the classpath (before FWD).

If we need changes to how we handle the name map and/or propath for this to work, then let's create a task for it.

• Added support for webspeed and basepath fields in Admin Console related code.

I believe at the moment, we are waiting for Constantin to commit the change related to #6506-135. Afterwards, if everyone agrees, we can move this into Internal Test.

Radu, we don't have support for PASOE to run webspeed - so there is no need to have the fields in MultiSessionAppserverView.java

Constantin Asofiei wrote:

Radu, we don't have support for PASOE to run webspeed - so there is no need to have the fields in MultiSessionAppserverView.java

Understood. Should I remove them, or is it alright to leave those fields in there for the future?

Radu Apetrii wrote:

Constantin Asofiei wrote:

Radu, we don't have support for PASOE to run webspeed - so there is no need to have the fields in MultiSessionAppserverView.java

Understood. Should I remove them, or is it alright to leave those fields in there for the future?

I don't think we'll add them soon. Please remove them. The admin appserver views are read-only, right?

6506b rev. 16353 - Constantin's commit for the change suggested in DmoMetadataManager.java.
6506b rev. 16354 - I removed the fields from MultiSessionAppserverView.java.

The admin appserver views are read-only, right?

Yes, there's no way to change the fields values after they are set in the constructor.

Radu Apetrii wrote:

The admin appserver views are read-only, right?

Yes, there's no way to change the fields values after they are set in the constructor.

Ignore this note, I was looking at the wrong class. Yes, they are read-only, indeed.

Radu, I'm OK with the changes. Do we have wiki for the webspeed broker config?

Yes. I've renamed it to REST, SOAP and WEB.

Just a heads up: the readme in the root project file needs to be edited in the branch to contain the ADE files which have been included in FWD.

Greg - in readme we have this:

Files relicensed in that manner can be found in the following directories:
TODO: edit here as converted ADE code is added

I assume we do not want to list each file, just a folder where converted .java ADE files can be found, even if that folder has other FWD specific files. So for 6506b we will add src/com/goldencode/p2j/webspeed/

I assume we do not want to list each file, just a folder where converted .java ADE files can be found''

Correct.

6506b/16356 contains the change to readme file.

I rebased 6506b with trunk rev. 16420. The 6506b branch is now at rev. 16430. I will do the conversion for #11198 next.

Unfortunately, I've seen errors in two project conversions. We're dealing (again) with some character skipping when going from the .p to the .p.cache version. I'm working on a testcase right now, but I'm afraid we might have to take this back to the drawing board.

As a side note, before the testing process started, I rebased 6506b with trunk rev. 16431. The 6506b branch is now at rev. 16441.

Alright, so, here's a testcase that highlights the issue:

define var a as char.
define var b as char.

a = string(9,">>>>9")+"|" + b + "~n".

And the error:

     [java] start.p:4:35: expecting '"', found 'n'
     [java]     at com.goldencode.p2j.preproc.TextLexer.nextToken(TextLexer.java:265)
     [java]     at antlr.TokenBuffer.fill(TokenBuffer.java:69)
     [java]     at antlr.TokenBuffer.LA(TokenBuffer.java:80)
     [java]     at antlr.LLkParser.LA(LLkParser.java:52)
     [java]     at com.goldencode.p2j.preproc.TextParser.textBlock(TextParser.java:211)
     [java]     at com.goldencode.p2j.preproc.TextParser.text(TextParser.java:165)
     [java]     at com.goldencode.p2j.preproc.Preprocessor.<init>(Preprocessor.java:1021)
     [java]     at com.goldencode.p2j.uast.AstGenerator.preprocess(AstGenerator.java:1385)
     [java]     at com.goldencode.p2j.uast.AstGenerator.prepareDataStream(AstGenerator.java:1165)
     [java]     at com.goldencode.p2j.uast.AstGenerator.prepareLexer(AstGenerator.java:1757)
     [java]     at com.goldencode.p2j.uast.AstGenerator.parse(AstGenerator.java:1659)
     [java]     at com.goldencode.p2j.uast.AstGenerator.processFile(AstGenerator.java:1078)
     [java]     at com.goldencode.p2j.uast.ScanDriver.lambda$scan$0(ScanDriver.java:470)
     [java]     at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:510)
     [java]     at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:319)
     [java]     at com.goldencode.p2j.convert.TransformDriver.runScanDriver(TransformDriver.java:445)
     [java]     at com.goldencode.p2j.convert.TransformDriver.front(TransformDriver.java:301)
     [java]     at com.goldencode.p2j.convert.TransformDriver.executeJob(TransformDriver.java:1234)
     [java]     at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDriver.java:1301)
     [java] 
     [java] ./abl/start.p
     [java] Exception: start.p:4:35: expecting '"', found 'n'
     [java]         ClearStream state:  clearCount = 0; inComment = no; inString = yes; keepTildes = no
     [java] Scanning: " + b + "~
     [java] Scanning: LA(1)='n' LA(2)='"'
     [java]  scope 1 --- #0:0./abl/start.p{}{}
     [java]     fwd-version=(global,"FWD v4.0.0_p2j_6506b_16441")
     [java]     window-system=(builtin,"null")
     [java]     file-name=(builtin,"null")
     [java]     opsys=(builtin,"null")
     [java]     batch-mode=(builtin,"null")
     [java]     line-number=(builtin,"null")
     [java]     process-architecture=(builtin,"null")
     [java]     sequence=(builtin,"null")
     [java]  scope 2 --- #0:0./abl/start.p{}{}
     [java] =================================

Soo, this works fine with trunk, and fails with 6506b. However, if I add the changes from #6859 to the 6506b branch, the test works fine. Greg: is it alright to push for the merge of 6859b into trunk, or should I just extract the piece of code that solves this issue and add it to 6506b?

In the meantime, I will look at the other failing case to see what's happening.

Please add the test code from #6506-156 to the Preprocessor Testcases.

I will discuss 6859b with Octavian.

I retested the conversion on one of the two failing customer applications, this time with 6506b + the changes from #6859. Both conversion and runtime work fine now.

There is one more application that showed errors. One of the two errors is presented in note #6506-156, which is fixed by the same combination of branches (6506b + 6859b). I'm working right now on the second case.

This is the other test that was failing:

def var a as logical.

display
   "A:" when a
   "B"~  at 36
   with frame f.

And the error:

     [java] start.p:5:8: expecting '"', found ' '
     [java]     at com.goldencode.p2j.preproc.TextLexer.nextToken(TextLexer.java:265)
     [java]     at antlr.TokenBuffer.fill(TokenBuffer.java:69)
     [java]     at antlr.TokenBuffer.LA(TokenBuffer.java:80)
     [java]     at antlr.LLkParser.LA(LLkParser.java:52)
     [java]     at com.goldencode.p2j.preproc.TextParser.textBlock(TextParser.java:211)
     [java]     at com.goldencode.p2j.preproc.TextParser.text(TextParser.java:165)
     [java]     at com.goldencode.p2j.preproc.Preprocessor.<init>(Preprocessor.java:1021)
     [java]     at com.goldencode.p2j.uast.AstGenerator.preprocess(AstGenerator.java:1385)
     [java]     at com.goldencode.p2j.uast.AstGenerator.prepareDataStream(AstGenerator.java:1165)
     [java]     at com.goldencode.p2j.uast.AstGenerator.prepareLexer(AstGenerator.java:1757)
     [java]     at com.goldencode.p2j.uast.AstGenerator.parse(AstGenerator.java:1659)
     [java]     at com.goldencode.p2j.uast.AstGenerator.processFile(AstGenerator.java:1078)
     [java]     at com.goldencode.p2j.uast.ScanDriver.lambda$scan$0(ScanDriver.java:470)
     [java]     at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:510)
     [java]     at com.goldencode.p2j.uast.ScanDriver.scan(ScanDriver.java:319)
     [java]     at com.goldencode.p2j.convert.TransformDriver.runScanDriver(TransformDriver.java:445)
     [java]     at com.goldencode.p2j.convert.TransformDriver.front(TransformDriver.java:301)
     [java]     at com.goldencode.p2j.convert.TransformDriver.executeJob(TransformDriver.java:1234)
     [java]     at com.goldencode.p2j.convert.ConversionDriver.main(ConversionDriver.java:1301)
     [java] 
     [java] ./abl/start.p
     [java] Exception: start.p:5:8: expecting '"', found ' '
     [java]         ClearStream state:  clearCount = 0; inComment = no; inString = yes; keepTildes = no
     [java] Scanning: "B"~
     [java] Scanning: LA(1)=' ' LA(2)=' '
     [java]  scope 1 --- #0:0./abl/start.p{}{}
     [java]     fwd-version=(global,"FWD v4.0.0_p2j_6506b_16441")
     [java]     window-system=(builtin,"null")
     [java]     file-name=(builtin,"null")
     [java]     opsys=(builtin,"null")
     [java]     batch-mode=(builtin,"null")
     [java]     line-number=(builtin,"null")
     [java]     process-architecture=(builtin,"null")
     [java]     sequence=(builtin,"null")
     [java]  scope 2 --- #0:0./abl/start.p{}{}
     [java] =================================

Thankfully, this is fixed as well with 6506b + 6859b. I think this means that 6506b is good to go once 6859b reaches trunk. Greg, if you think that checking these two tests is not enough and I should ask for the reconversion of this app to be 100% sure, I will. This is the last item on the testing list.

Now that 6859b got into trunk, I will rebase 6506b and re-test the conversion of a customer application. Assuming this goes well, I think we can merge 6506b afterwards.

I rebased 6506b with trunk rev. 16452. The 6506b branch is now at rev. 16462. I'm preparing the conversion test right now.

Update: sorry for this post, it was not meant to be for this task.

In 16465, did you intend to remove this change from text.g?

** 035 FER 20250806          Added an option in QSTRING rule to allow the ("~") sequence inside a string.
**                           This is a known quirk in 4GL.

Greg Shah wrote:

In 16465, did you intend to remove this change from text.g?

[...]

Yes, because in 6859b I added support for both "~" and '~':

**     AOG 20251203          Added support for "~" and '~'. Those are translated as escaped double/single
**                           quotes inside strings.

This is an example that fails with 6506b rev. 16466:

{&OUT} '    for (var i=1~; i < a~; i++) ~{~n'.

Let me know how I can help further.

Code Review Task Branch 6506b Revisions 16465 and 16466

The changes look OK, though it is hard to reason about the ClearStream reordering. We'll have to see the testing results.

Radu Apetrii wrote:

This is an example that fails with 6506b rev. 16466:
[...]

Let me know how I can help further.

It looks like { is being treated as an include brace even when it's inside a string. I noticed that 6859b accounts for this in comments, but not for strings.

Greg, shouldn't we apply the same logic to strings?

Please see ClearStream:

         // process braces
         if (nextChar == '{')
         {
            if (notInComments)
            {
               // instantiate the braces parser, call it and handle any
               // nesting and hints issues
               processBraces();

               // reread the modified stream
               continue;
            }
            else
            {
               return nextChar;
            }
         }

Octavian Adrian Gavril wrote:

Radu Apetrii wrote:

This is an example that fails with 6506b rev. 16466:
[...]

Let me know how I can help further.

It looks like { is being treated as an include brace even when it's inside a string. I noticed that 6859b accounts for this in comments, but not for strings.

Greg, shouldn't we apply the same logic to strings?

Please see ClearStream:
[...]

Actually this is incorrect. This is an example were brace is used inside strings:

&IF DEFINED({1}) = 1 &then
   "{1} defined globally".
&else
   "{1} not defined globally".
&endif

I've reverted the previous reordering and implemented a new fix: when tilde mode is 1 or 2, the semicolon is returned immediately to skip alternative processing. Other characters, including {, are no longer skipped and will be processed normally. The new revision is 16467. The preprocessor tests are fine and both #10109-402 and #6506-169 are fixed.

This is a testcase for the error in #10109-409:

{&OUT} 'function("../`~'{&b}~'`","","c" + replace(/d/,~'~') + "e","f")` + chr(010) + '}' + chr(010) + '.

Again, this can probably be trimmed further, but it would require a bit of jiggery-pokery.

Radu Apetrii wrote:

This is a testcase for the error in #10109-409:
[...]

Again, this can probably be trimmed further, but it would require a bit of jiggery-pokery.

This is not compiling in OE. Is this expected?

Octavian Adrian Gavril wrote:

Radu Apetrii wrote:

This is a testcase for the error in #10109-409:
[...]

Again, this can probably be trimmed further, but it would require a bit of jiggery-pokery.

This is not compiling in OE. Is this expected?

Hmmm, something seemed suspicious when I first looked at it. Let me analyze the scenario and I'll come back with a response.

Radu Apetrii wrote:

Hmmm, something seemed suspicious when I first looked at it. Let me analyze the scenario and I'll come back with a response.

Sure. This is the error in OE:

Unable to understand after -- "function("../`''`","","c" + replace(/d/,'') + "e","f")` + chr(010) + 
 ".

I think the error points to the same code area as the one mentioned in #10109-409.

I see what's happening now. We're dealing with another skipped semicolon. Take a look at the following:

start.p

{&OUT} 'else~{' + 'function1("../`~'~{&a~}~'`","","b" + function2(/Child/,~'~') + "c","d");' + '~}' + 'break;'.

start.p.cache

 'else~{' + 'function1("../`~'{&a}~'`","","b" + function2(/Child/,~'~') + "c","d")` + '}' + 'break`.

Look at the end of function1 how the semicolon is ignored. Let me know if you need more info on this.

Radu Apetrii wrote:

I see what's happening now. We're dealing with another skipped semicolon. Take a look at the following:

start.p
[...]

start.p.cache
[...]

Look at the end of function1 how the semicolon is ignored. Let me know if you need more info on this.

Thanks, Radu! I'm looking into it.

I've added new changes in 6506b/16468. Relocated the clearCount logic to ensure it takes precedence over escaped quotes checks. Preprocessor tests are fine and #6506-177, #6506-169 + #10109-401 are fixed.

Octavian Adrian Gavril wrote:

Preprocessor tests are fine and #6506-177, #6506-169 + #10109-401 are fixed.

+ #10569-1 ✅

Things are looking pretty good at the moment. I think we can get 6506b rev. 16468 reviewed.

• Fixed an issue with the servlet mapping in WebspeedHandler.
  • This was encountered when looking at the return value for getPathInfo() which was null instead of what it was supposed to be.
• Fixed an issue when retrieving the value from the directory for supportsDefaultService.

The next thing on my list is #10109-410.

Alex/Greg/Constantin: has trunk warmed up (i.e. not freezing anymore)? After I receive the confirmation, I will put 6506b in the merge queue, assuming there are no more objections.

It is still frozen.