FWD » Core Development » Runtime Infrastructure

Currently, enabling server-side streams does not allow work with FRAMEs (i.e. redirected terminal). To make it work, this requires for FWD client to 'call back' to the FWD server to access the stream. I have some changes which I will commit to 11271a once I have them stable; the downside for this is that the FWD client has granular stream API calls, which in some cases may increase the network trips.

This also affects process streams, which will be on the server.

Alexandru: do you have some standalone tests for the behavior in #11184?

For streams to work server-side, we need to also bring the file-system to the server-side. But OS-GETENV converts in FWD as FileSystemOps.getProperty() and executes via FileSystemDaemon.getProperty - and, if file-system is server-side, then the FWD client's env is not available.

So I need to move OS-GETENV to emit via EnvironmentOps and execute via EnvironmentDaemon.

Alex, I've ran your image load test and I don't see any difference with server-side file-system - it takes ~0.5s in both cases. The latest changes are in 11271b rev 16485 (had to fix some other stuff related to server-side file-system, for APIs which are still called from client).

Constantin Asofiei wrote:

Alex, I've ran your image load test and I don't see any difference with server-side file-system - it takes ~0.5s in both cases. The latest changes are in 11271b rev 16485 (had to fix some other stuff related to server-side file-system, for APIs which are still called from client).

I've done some debugging now and there is something lets say 'out of order': ButtonGuiImpl.loadImage creates a java FileInputStream - so this does not use 'server-side' file-system or streams, even if enabled. I've tested in Swing for now, I'll check also in Web GUI.

Tested also in Web GUI, times are similar to Swing, in both cases.

Code Review Task Branch 11271a Revision 16469

1. Let's find a different name than IStream for the interface. I like the changes to abstract that, but the name is too dotnet. Perhaps BaseStream or something like that could be better?

2. The StreamDaemon on the server side is a singleton, which means we must synchronize all of the state access and modification. Please make the class fully thread-safe.

Is the testcase from #11271-7 a common pattern? Should we just complete the implementation of a server-side headless/redirected UI?

Hynek: What is the work effort to accomplish that?

Greg Shah wrote:

Code Review Task Branch 11271a Revision 16469

Sorry, the branch to review is 11271b rev 16492 as of now.

1. Let's find a different name than IStream for the interface. I like the changes to abstract that, but the name is too dotnet. Perhaps BaseStream or something like that could be better?

In 11271a, yes, I tried to extract APIs which need to be executed by the client-side. But this is not used in 11271b. This needs to be further refactored at some point, to know exactly which APIs are executed by the client and server side.

2. The StreamDaemon on the server side is a singleton, which means we must synchronize all of the state access and modification. Please make the class fully thread-safe.

In 11271b there is no state in StreamDaemon when on server-side: resolveWorkArea is overridden in ServerStreamDaemon to be context-local. ServerSideTerminal is also context-local.

Code review 11271b.

FileStream.restoreStateAndOpen always reopens with append=true. What if the original stream is opened with append=false?

resolveId will NPE on unknown streams — StreamDaemon.java

The new resolveId method returns primitive int, but wa.ids.get(stream) returns Integer which can be null. The original code in reverseLookup safely handled null via Integer id = ids.get(s).

Swapped Javadoc on serialization methods — Stream.java and FileStream.java

In both classes, the Javadoc for readExternal (diff lines 1017, 617) says "Serialize this stream's state" when it actually deserializes/restores. Conversely, writeExternal (diff lines 1079, 655) says "Restore this stream's state" when it actually serializes. The comments are transposed.

Should access to RemoteStreamHandler.remoteStreams be synchronized?

Double close in RemoteStreamHandler.invoke for file resource streams

1. Calls local.checkOutStream(id) (closes local file handle)
2. Calls remote.checkInStream(id, state) (opens on remote)
3. Calls method.invoke(remote, args) (close on remote)
4. Then falls through to method.invoke(handler, args) where handler local — executing close logic again on the already-checked-out local stream

FileSystemDaemon.singleton used without null guard in static methods.

RuntimeException wrapping in restoreStateAndOpen/saveStateAndClose bypasses 4GL error condition handling. Both methods catch IOException/ClassNotFoundException and wrap them in raw RuntimeException (diff lines 377–385, 417–419). If a file cannot be re-opened during stream migration (e.g., file deleted between check-out and check-in), this RuntimeException will propagate through the block execution stack. BlockManager.exceptionToCondition() does not map plain RuntimeException to any 4GL condition — it will be treated as an unhandled exception, likely surfacing as a STOP condition rather than an ERROR condition that ON ERROR phrases could handle.

sourceCharset can become null after deserialization
Stream.readExternal sets sourceCharset = javaSourceCS null ? null : Charset.forName(javaSourceCS). But the field's Javadoc says "This is never null".

I think "store" in RemoteStreamHelper should advance next stream id to be in sync with StreamDaemon.store().

Hynek Cihlar wrote:

Code review 11271b.

FileStream.restoreStateAndOpen always reopens with append=true. What if the original stream is opened with append=false?

Once the stream is opened, we are always appending - we can't reset its content.

The new resolveId method returns primitive int, but wa.ids.get(stream) returns Integer which can be null. The original code in reverseLookup safely handled null via Integer id = ids.get(s).

Fixed

Swapped Javadoc on serialization methods — Stream.java and FileStream.java

Fixed

Should access to RemoteStreamHandler.remoteStreams be synchronized?

No, ServerResourceManager is context-local thus the proxy using RemoteStreamHandler is context-local

Double close in RemoteStreamHandler.invoke for file resource streams

When streamClose is true and the stream is a local file resource, the code:

1. Calls local.checkOutStream(id) (closes local file handle)
2. Calls remote.checkInStream(id, state) (opens on remote)
3. Calls method.invoke(remote, args) (close on remote)
4. Then falls through to method.invoke(handler, args) where handler local — executing close logic again on the already-checked-out local stream

Well, this close logic will also remove the stream from the various registries. That's why I use this order: checkout/checkin doesn't affect the registries, and calling close on both remote and local ensures both sides have cleared everything.

FileSystemDaemon.singleton used without null guard in static methods.

We have a case in deleteWorker where a JVM shutdown hook is executed - see #11325, but otherwise this must never be null.

RuntimeException wrapping in restoreStateAndOpen/saveStateAndClose bypasses 4GL error condition handling. Both methods catch IOException/ClassNotFoundException and wrap them in raw RuntimeException (diff lines 377–385, 417–419). If a file cannot be re-opened during stream migration (e.g., file deleted between check-out and check-in), this RuntimeException will propagate through the block execution stack. BlockManager.exceptionToCondition() does not map plain RuntimeException to any 4GL condition — it will be treated as an unhandled exception, likely surfacing as a STOP condition rather than an ERROR condition that ON ERROR phrases could handle.

This must never happen. I'd rather abend if 'the OS did something with the file during switch' than treat this as a 4GL condition.

sourceCharset can become null after deserialization
Stream.readExternal sets sourceCharset = javaSourceCS null ? null : Charset.forName(javaSourceCS). But the field's Javadoc says "This is never null".

This is just a protection, it should never happen.

I think "store" in RemoteStreamHelper should advance next stream id to be in sync with StreamDaemon.store().

I need to double-check this.

Constantin, do you need to do a change in 11271b? I am asking about this to know if I put this into ChUI regression testing.

Teodor Gorghe wrote:

Constantin, do you need to do a change in 11271b? I am asking about this to know if I put this into ChUI regression testing.

Please go ahead and run ChUI.

Hynek, you were correct to fix the 'nextId' on 'store'. Also I found a regression in ChUI - RemoteStream was caching state which was not set properly when opening a process stream. The fixes are in 11271b rev 16502 (after rebase).

I'll run ChUI, ETF and SXe again.

Greg: I think we need to plan for server-side redirected terminal (and thus frames) rather sooner than later; I think for batch processes, we can use this.

I think we need to plan for server-side redirected terminal (and thus frames) rather sooner than later; I think for batch processes, we can use this.

I agree. The question from #11271-14 remains: what is the expected effort for that?

Constantin Asofiei wrote:

Hynek, you were correct to fix the 'nextId' on 'store'.

Actually this one was spotted with AI ;-).

Greg Shah wrote:

I think we need to plan for server-side redirected terminal (and thus frames) rather sooner than later; I think for batch processes, we can use this.

I agree. The question from #11271-14 remains: what is the expected effort for that?

There are not many static fields in the p2j.ui package remaining. After we clean this up and make them context-local, what we need to do is run a FWD client only on server-side; I would like to do this still via ClientDriver, which after authentication, if it sees server-side-resources=ALL, it switches automatically to this 'headless' mode (i.e. closes the socket, etc) and initializes ThinClient and other needed UI resources on server-side.

As for the effort: I think 3-4 weeks of work.

Constantin Asofiei wrote:

Greg Shah wrote:

I think we need to plan for server-side redirected terminal (and thus frames) rather sooner than later; I think for batch processes, we can use this.

I agree. The question from #11271-14 remains: what is the expected effort for that?

There are not many static fields in the p2j.ui package remaining. After we clean this up and make them context-local, what we need to do is run a FWD client only on server-side; I would like to do this still via ClientDriver, which after authentication, if it sees server-side-resources=ALL, it switches automatically to this 'headless' mode (i.e. closes the socket, etc) and initializes ThinClient and other needed UI resources on server-side.

As for the effort: I think 3-4 weeks of work.

Please create a new task for this.

• If any context does not have ALL server-side resources, then register the network server in 'initializeServer', for filesystem and stream resources. Refs #11271 #11259

I'll merge this now. Affects multisession FWD mode.

Branch 11271c was merged to trunk rev 16511 and archived.

Greg Shah wrote:

Is the testcase from #11271-7 a common pattern? Should we just complete the implementation of a server-side headless/redirected UI?

Hynek: What is the work effort to accomplish that?

See #9055 and 9055a where the initial support for running client-side UI in server context was implemented. It included very limited use cases like MESSAGE statement.

For full UI I suppose the UI code will have to be cleaned from static Java fields and proper use of security context. A rough estimate 15 MDs.

Branch 11271d fixes a conversion problem: FileSystemDaemon is used in conversion by .xpxg analysis (for OpenClient). This was merged to trunk rev 16516

Constantin, is there any reason for:

      if (wa.serverSideFs && !Paths.get(filename).isAbsolute())
      {
         if (filename.startsWith("." + getFileSeparator()))
         {
            filename = filename.substring(2);
         }
         filename = getWorkingDirectory().value.concat(getFileSeparator()).concat(filename);
      }

inside FileSystemOps.searchPath? This class does:

• lookupLegacyName which should use relative names like my/project/proc.p. It won't work with absolute path and/or getWorkingDirectory prefixed.
• ignoreClientSearch honoring that, at least in another project, has my/project/proc.p relative paths.
• searchResourceJars that searches a file in a jar, but relative to the root of the jar. Also, appending getWorkingDirectory doesn't help at all.
• if (filename.equalsIgnoreCase(quoterExecName)) won't ever match.

So this code basically makes FileSystemOps.searchPath a no-op. Am I misunderstanding something?

Alexandru Lungu wrote:

So this code basically makes FileSystemOps.searchPath a no-op. Am I misunderstanding something?

Alex, you are correct; this needs to be moved lower in the method, after 4GL and jar lookup.

I have some changes; I will commit them by EOD in 11242a where I tried to make filesystem,stream as a server-side resource work for the customer there.