FWD » Documentation

1. I've committed to bzr the remote_launchers.odt chapter, with edits/comments.
2. I think the styles are broken - instead of using LiberationSans, they are using Arial/Times New Roman?
3. add two spaces after each phrase (i.e. if a new pharse begins after a dot, the phrase must start with two spaces). Couldn't find this in the standard/dev docs, I think this was stated in some email some time ago.
4. do not use BOLD to emphasize inlined code, use Teletype style.
5. I think you should add some examples of how a broker setup (i.e. folder structure, files, rights) should look like
6. the ACLs need to be checked when all resource types are in place:
   

             <node-attribute name="values" value="com.goldencode.p2j.security.SystemResource"/>
             <node-attribute name="values" value="com.goldencode.p2j.security.AdminResource"/>
             <node-attribute name="values" value="com.goldencode.p2j.net.NetResource"/>
             <node-attribute name="values" value="com.goldencode.p2j.directory.DirectoryResource"/>
   

   
   i.e. check if MAJIC can use the remote brokers safely. See the ACLs mentioned in the spawner setup chapter, and post here if you need other ACLs to be setup.

1.On my machine font is Arial for "Text Body" and "Heading n" sections. For "Preformatted Text" section the font is "DejaVu Sans Mono".
I checked also other chapters (files) inside the book and I found same fonts no LiberationSans font!

2. Could I use the converted Majic from my last regression test or I have to do a Majic conversion first in order to have the latest release?

I found LiberationSans font in spawner_setup.odt may I use this chapter as a reference for style?

Marius Gligor wrote:

I found LiberationSans font in spawner_setup.odt may I use this chapter as a reference for style?

Is the other way around, spawner_setup.odt is wrong - please fix the style for this file (I think copy-pasting the text into a file with correct style should work).

BTW, something else: not sure why, but if in your remote_launcher.odt file I use "Text body" as style, it gets me to use Times New Roman instead of Arial...

2. Could I use the converted Majic from my last regression test or I have to do a Majic conversion first in order to have the latest release?

I think you need a new conversion. Also, you will need to setup the directory (i.e. broker process account, brokers node), if you haven't already done so.

Please be more explicit. Which font is correct LiberationSans or Arial?

Marius Gligor wrote:

Please be more explicit. Which font is correct LiberationSans or Arial?

Arial.

I can't tell why that is happening... do you have the customer libs in place? do a diff with the standard majic install on devsrv01:~/testing/majic/ and see what you are missing.

No I have no customer libs. Also on the old environment I have no such libs. Maybe something has changed in the mean time.
I will try do download more files from devsrv01

I've fixed the Majic server start-up. Something was wrong on my environment.

Marius Gligor wrote:

So I have two options:
1. To fix this issue.
2. Try to configure a remote launcher and see what's happen.

You have to work on both, make sure that both built-in launcher and remote launcher work properly.

I've started to do broker configurations in gso server directory.
1. I added a broker container to /server/gso node
2. On node /security/account/processes I added a new process account with name p2j_proc which is the same like p2j_proc account from P2J server.

Now I want to configure the broker client.
Using the p2j_proc process account I think that we could use the same broker key store as in P2J.
As trust sore we could use server timco-trust.store file alias gso-p2jserver with password timco-trust

Please let me know if is correct or not and if not how to proceed.

Marius Gligor wrote:

I've started to do broker configurations in gso server directory.
1. I added a broker container to /server/gso node
2. On node /security/account/processes I added a new process account with name p2j_proc which is the same like p2j_proc account from P2J server.

Now I want to configure the broker client.
Using the p2j_proc process account I think that we could use the same broker key store as in P2J.

You can't use this, as the certificate in the directory won't match.

Please let me know if is correct or not and if not how to proceed.

The easy way is to run the ServerDriver -c with the GSO directory, to regenerate the certificates/private keys and save the into key stores.

I generated new certificates in gso_directory.xml as you recommended.

Broker client.xml file SSL configuration looks like. The password keyentry is for p2j_proc process account.

<security>
      <certificate validate="true" />
      <truststore filename="broker-trust.store" />
      <truststore alias="gso" />
      <keystore filename="broker-key.store" />
      <keystore processalias="p2j_proc_alias" />
      <authentication type="program"/>
   </security>

   <access>
      redacted
   </access>

I have a few questions:

1. "The servers' certificates [gso, gyr, lcq, mcn, osc, rfq] were saved in the [broker-trust.store] key store."
Is OK to use truststore alias "gso" ?

2. "Any configuration set at the client.xml or server.xml files or via bootstrap config arguments will have priority over the in-directory keys or certificates."
Should I change gso_server.xml to use in-directory keys or certificates or I have to change the configuration to use values from gso_server.xml?

LE: GES redacted keystore password entries.

Using the broker bootstrap configuration resulted after new certificates generation and staring the gso server
without SSL configuration in gso_directory.xml I've got the following exception when broker try to connect.

Jul 29, 2014 1:33:48 PM BrokerCore.start() 
INFO: {main} Connecting (9) ...
Jul 29, 2014 1:33:48 PM BrokerCore.connect() 
SEVERE: {main} Connection Exception.
java.security.UnrecoverableKeyException: Cannot recover key
    at sun.security.provider.KeyProtector.recover(KeyProtector.java:328)
    at sun.security.provider.JavaKeyStore.engineGetKey(JavaKeyStore.java:138)
    at sun.security.provider.JavaKeyStore$JKS.engineGetKey(JavaKeyStore.java:55)
    at java.security.KeyStore.getKey(KeyStore.java:792)
    at sun.security.ssl.SunX509KeyManagerImpl.<init>(SunX509KeyManagerImpl.java:131)
    at sun.security.ssl.KeyManagerFactoryImpl$SunX509.engineInit(KeyManagerFactoryImpl.java:68)
    at javax.net.ssl.KeyManagerFactory.init(KeyManagerFactory.java:259)
    at com.goldencode.p2j.security.TransportSecurity.<init>(TransportSecurity.java:221)
    at com.goldencode.p2j.security.SecurityManager.getTransportSecurity(SecurityManager.java:7302)
    at com.goldencode.p2j.security.SecurityManager.getSecureSocketContext(SecurityManager.java:1722)
    at com.goldencode.p2j.net.SessionManager.createQueue(SessionManager.java:1024)
    at com.goldencode.p2j.net.LeafSessionManager.connectDirect(LeafSessionManager.java:201)
    at com.goldencode.p2j.main.BrokerCore.connect(BrokerCore.java:389)
    at com.goldencode.p2j.main.BrokerCore.start(BrokerCore.java:143)
    at com.goldencode.p2j.main.ClientDriver.start(ClientDriver.java:223)
    at com.goldencode.p2j.main.CommonDriver.process(CommonDriver.java:423)
    at com.goldencode.p2j.main.ClientDriver.process(ClientDriver.java:112)
    at com.goldencode.p2j.main.ClientDriver.main(ClientDriver.java:290)

On the server side I've got the following error:

07/29/2014 13:33:20 EEST] (SessionManager.listen():INFO) {00000000:00000001:gso} Server ready
[07/29/2014 13:33:57 EEST] (Incoming.run():WARNING) {Incoming SSL Connector} initialization failure
javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
        at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1476)
        at sun.security.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1488)
        at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:70)
        at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1876)
        at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1785)
        at java.io.ObjectOutputStream.<init>(ObjectOutputStream.java:247)
        at com.goldencode.p2j.net.NetSocket.<init>(NetSocket.java:79)
        at com.goldencode.p2j.net.RouterSessionManager$Incoming.run(RouterSessionManager.java:1389)
        at java.lang.Thread.run(Thread.java:744)

It seems that the SSL handshake fails. Do you have any idea why?

Marius Gligor wrote:

It seems that the SSL handshake fails. Do you have any idea why?

Look into the gso-server.xml file and update the SSL config to match the new configuration (keys, stores, etc).

Finally I generated certificates once again and I did all configurations. I checked all aliases on key stores and trust store and are OK.
One step forward but unfortunately still not working. Now I have another exception:
Client:

Jul 29, 2014 4:14:53 PM BrokerCore.start() 
INFO: {main} Connecting (9) ...
Jul 29, 2014 4:14:53 PM BrokerCore.connect() 
SEVERE: {main} Connection Exception.
java.lang.Exception: Authentication failed
    at com.goldencode.p2j.net.SessionManager.createQueue(SessionManager.java:1042)
    at com.goldencode.p2j.net.LeafSessionManager.connectDirect(LeafSessionManager.java:201)
    at com.goldencode.p2j.main.BrokerCore.connect(BrokerCore.java:389)
    at com.goldencode.p2j.main.BrokerCore.start(BrokerCore.java:143)
    at com.goldencode.p2j.main.ClientDriver.start(ClientDriver.java:223)
    at com.goldencode.p2j.main.CommonDriver.process(CommonDriver.java:423)
    at com.goldencode.p2j.main.ClientDriver.process(ClientDriver.java:112)
    at com.goldencode.p2j.main.ClientDriver.main(ClientDriver.java:290)

[07/29/2014 16:08:21 EEST] (SessionManager.listen():INFO) {00000000:00000001:gso} Server ready
[07/29/2014 16:14:53 EEST] (SecurityManager:SEVERE) {00000000:0000001E:gso} Error sending AUTHRESULT java.net.SocketException: Broken pipe

Could be generated by some missing of rights in server directory?

Marius, debug into SecurityManager.authenticateLocal - this is the server-side auth code.

I did a remote debug in SecurityManager.authenticateLocal and on the server part seems to have no problems.
The process account is OK and has enough rights to authenticate on server. The user certificate is also read without any errors.
Finally the result is send back to client (line 2833) with the following values:

authResult = 0 (AUTH_ACTION_DONE)
authAction = 0 (AUTH_RESULT_SUCCESS)

 
     // send the AUTHRESULT packet to the client
      if (!sendAuthResult(out, authResult, authAction))
      {
         // error sending the result; do not keep the session
         decision = false;
      }

Debuging on client side I found the cause of the error. Server certificate validation fails and client close the connection.
When the server send back the authentication information the socket is closed and a SocketException (Broken pipe) is throw.
I have to check why this happen and fix the problem. Seems to be a configuration issue.

Do you have the root CA in the trust store? The certificate alias in the broker's xml needs to be for the broker's certificate, not server...

On the broker trust store I have the server certificates and on key store users private keys.
On the server side I have server certificates in trust store and CA root in key store

Fixed. I changed the alias in broker configuration to CA root (timco-root).
No I have another exception related to exported BrokerServerServices. I have to fix this in server directory.

Search for ServerPropertiesInspector in ACL chapter - I think the same ACL is needed for BrokerServerServices.

I added the following node to security/acl/net

<node class="container" name="002350">
  <node class="resource" name="resource-instance">
    <node-attribute name="reftype" value="TRUE"/>
    <node-attribute name="reference" value="com.goldencode.p2j.main.BrokerServerServices"/>
  </node>
  <node class="netRights" name="rights">
    <node-attribute name="permissions" value="'0101'B"/>
  </node>
  <node class="strings" name="subjects">
    <node-attribute name="values" value="all_others"/>
  </node>
</node>

Marius Gligor wrote:

I added the following node to security/acl/net

Make sure to update the ACL chapter with this config.

Try the ediavc process - don't expect it to do anything (or even complete successfully), just check if it can log in. check the directory for the entry point, where you can place a breakpoint to check if it is reached - if so, then login/startup was correct.

You can start it via the ServerDriver's -b argument.

Code Review 0730a

The change is fine. You don't need to regression test this one. If you have tested both local and remote cases, you can check this in and distribute it.

Committed revision 10587.

Notes 15 through 29 suggest that our existing documentation is insufficient for the proper setup of the remote launcher. What documentation changes are needed to ensure that the (multiple) problems encountered can be avoided by future readers?

LE: I understand that one of the ACLs needed was missing, that doc update was already listed in note 30. I am talking about the other changes.

Now I'm working to do batch client tests. Than I'll go to update and finalize documentation.
I think no other updates are needed to documentation.

I created a bootstrap configuration file for process user ediavc using the yesterday generated key stores and trust stores.
I added also a node inside server directory to access exported interface com.goldencode.p2j.util.BatchProcess.
Starting a batch process for edivac login succeeded and no errors I found inside batch log files.
I think that are enough tests for batch clients running Majic on my machine.
Now I'm working to update and finalize the documentation for remote launchers.

I committed the updated and revised documentation. Please let me known if I have to do other changes.
Committed revision 65.

Marius, I've looked at the remote_launcher.odt file, done some edits and I think it is OK now. Please post here all the ACLs and other configuration you had to do for MAJIC, to make the remote brokers work. Also, update the acl_cfg.odt file with the ACLs you added.

Greg: please take a final look at the remote_launcher.odt file and check if it looks OK to you.

Here are the configurations that I made in order to tests brokers with Majic.

1. On client side:
- a configuration file config.xml having the following content:

<node type="client">
  <client>
    <mode broker="true" />
  </client>

  <net> 
    <server host="192.168.1.10" />
    <server port="3333" /> 
    <server secure_port="3334" /> 
    <connection secure="true" /> 
  </net>

  <security>
      <certificate validate="true" />
      <truststore filename="broker-trust.store" />
      <truststore alias="timco-root" />
      <keystore filename="broker-key.store" />
      <keystore processalias="p2j_proc_alias" />
      <authentication type="program"/>
   </security>

   <access>
      redacted
   </access>

   <remote>
      <retry count="10" />
      <retry seconds="5" />
      <spawner file="/home/mag/p2j/spawner/spawn" />
   </remote>
</node>   

- broker-trust.store file containing server and root CA authority certificates . This file is the same as server timco-trust.store file.
- broker-key.store containing users private keys. The key entry for process alias is used by setting the appropriate password in access:password:keyentry.
- On the machine where broker is running p2j.jar and dependencies should be installed and shared by all OS users and broker. Also the spawner tool should be available for broker only.

2. Server side gso-directory.xml configurations:

- the list of brokers was added to /server/gso node:

<node class="container" name="brokers">
 <node class="broker" name="broker1">
   <node-attribute name="account" value="mag"/>
 </node>
 <node class="broker" name="broker2"/>
</node>

<node class="process" name="p2j_proc">
  ...
  <node-attribute name="broker" value="broker1"/>
</node>

- ACL settings:
On node /security/acl/net I added:

<node class="container" name="002350">
  <node class="resource" name="resource-instance">
    <node-attribute name="reftype" value="TRUE"/>
    <node-attribute name="reference" value="com.goldencode.p2j.main.BrokerServerServices"/>
  </node>
  <node class="netRights" name="rights">
    <node-attribute name="permissions" value="'0101'B"/>
  </node>
  <node class="strings" name="subjects">
    <node-attribute name="values" value="all_others"/>
  </node>
</node>

On /security/acl/system I added the node bellow only when I used P2J standard server directory.xml in P2J project
For Majic this node already exists inside gso-directory.xml having the name="000400"

<node class="container" name="000500">
   <node class="strings" name="subjects">
     <node-attribute name="values" value="all_others"/>
   </node>
   <node class="systemRights" name="rights">
     <node-attribute name="check" value="true"/>
   </node>
   <node class="resource" name="resource-instance">
     <node-attribute name="reference" value="accounts"/>
     <node-attribute name="reftype" value="TRUE"/>
   </node>
 </node>

LE: GES redacted keystore passwords.

Marius Gligor wrote:

2. Server side gso-directory.xml configurations:

- the list of brokers was added to /server/gso node:
- ACL settings:
On node /security/acl/net I added:

Please add this ACL to the acl_cfg.odt. The rest is covered.

acl_cfg.odt updated. Committed revision 67.

I have reviewed and edited the remote_launchers.odt. It is committed as bzr revision 69. I had to copy/paste the contents into a document that properly was extended from book_chapter instead of chapter.odt (which is not a proper template). I changed the heading levels and cleaned up a variety of items.

I have also added some specific questions/requests in the document. Please look for the GES: comments in red and make the changes needed in each case. Please leave my comments behind in your next checked in version. Have them there will make my next review easier.

I updated the documentation. Committed revision 70.
Please read my proposals regarding working directory which could eliminate this limitation.

I reviewed the latest chapter. The result is good. I cleaned it up and checked it in as bzr rev 71.

My only question is this:

A possible improvement here is to do some changes in the current implementation and provide a URL instead of address on redirection. Further the URL should be resolved by a DNS.

Please create a new task in the UI project and describe the changes that would be needed to support DNS names in our redirection URLs. Make Constantin and myself watchers. We won't work on this right now, but you make a good point that we probably need to address eventually.

Code Review 0802a

The code is fine. I don't see much downside to providing this feature.

Constantin: any objections?

Greg Shah wrote:

Constantin: any objections?

1. the javadoc for BrokerCore.BROKER_WD is incorrect.
2. setting a common working dir for all OS accounts handled by a certain remote spawner can pose security risks (in terms of a user should not have access to i.e. payroll reports generated by another user), and this should be stated explicitly in the documentation. More, there might be collisions if the same report path+name is used by two users at the same time.
3. I don't see how the per-user working dir configuration (from the directory.xml) is passed by the ClientBuilder to the remote broker. If a remote broker is used, it always passes the ${remote.working.directory} string, which gets replaced by the broker side, with the setting in the client.xml file (the remote:working:directory config). I think the userDir returned by the ClientBuilderParameters.getWorkingDirectory should be sent instead of the replacement string if this value is not null, i.e. in ClientBuilder.buildCommand we should have this:
   

         // Working directory
         command.add(userDir == null || userDir.trim().length() == 0 ? BrokerCore.PARAM_WD : userDir);
   

   
   instead of this:
   

         // Working directory
         command.add(remote ? BrokerCore.PARAM_WD : userDir);
   

   
   This will default to the broker's working dir only if there is not an explicit setting in the directory.

I would like to fix the workingDirectory issue and I coming with a lot of remarks:

1. We have the same problem on both local and remote launch: A global workingDirectory parameter for many OS accounts.
As a result we cannot set a specific working directory on each OS user account. However using a relative path basically we are able to define a per user account having the limitation that the relative path is the same and must exists on all OS accounts. (see. 3)

2. If workingDirectory is an absolute path means that working directory is the same for all OS users. Because the working directory is shared by all OS users each user should have enough rights to access it. As Constantin already mentioned on note 50.2 for security reasons and in order to avoid possible conflicts on file names is better to avoid using of absolute paths for this parameter.

3. If workingDirectory is a relative path means that working directory is user OS account specific because the working directory is relative to OS user home directory. The relative path must exists on all OS accounts used by local and remote launchers. A special case is using of "." as working directory which make the user home directory to become the working directory.

4. On initial implementation we have only local launchers. The launcher and spawned client are running on the same platform and working directory parameter is set accordingly. Having remote launchers (brokers) however it's possible to have brokers running on other platform and the workingDirectory parameter from server directory could have a wrong format for platform on which remote clients are launched. That's why I proposed to delegate the set of working directory to remote launchers (brokers).

5. Regarding Constantin note 50.3
- The userDir cannot be null because the default value when missing is "./" (see ClientBuilderOptions)

- The placeholder should be sent only for remote launchers and the statement should looks like:

// Working directory
if (remote)
{
   command.add(userDir == null || userDir.trim().length() == 0 ? BrokerCore.PARAM_WD : userDir);
}
else
{
   command.add(userDir);
}

- However In my opinion the workingDirectory parameter from server directory should be used for local spawn only. Using a workingDirectory from server directory for remote launchers could introduce restrictions regarding the platforms on which the brokers could runs.

6. As a conclusion I think that the best solution is to use relative paths only for working directory and for remote launchers to set the working directory always on the broker side.

Marius Gligor wrote:

I would like to fix the workingDirectory issue and I coming with a lot of remarks:

1. We have the same problem on both local and remote launch: A global workingDirectory parameter for many OS accounts.
As a result we cannot set a specific working directory on each OS user account. However using a relative path basically we are able to define a per user account having the limitation that the relative path is the same and must exists on all OS accounts. (see. 3)

There is a catch here: ProcessBuilderOptions overrides the getNode, to search for per-user then per-server settings. ClientBuilderOptions does not, searches only per-server. We need to standardize this, and let the getNode search per-user/per-server for both cases. This will allow setting the workingDirectory (and other config) for each P2J account. As I recall, the clientConfig node (at least for ProcessBuilder) is read in a "cascade" approach: each clientConfig attribute is checked individually, and is searched first in the per-user than the per-server config. So you can have common clientConfig options set at the server, and each user can have only the working dir explicitly set. Thus please move the getNode APIs from ProcessBuilderOptions to the parent class.

4. ... That's why I proposed to delegate the set of working directory to remote launchers (brokers).

As long as the working dir can't be set for each user, delegating the working dir to the remote launcher is not enough.

6. As a conclusion I think that the best solution is to use relative paths only for working directory and for remote launchers to set the working directory always on the broker side.

Again: setting it on the broker side is OK in terms of easing the installation for simple cases, but is not enough for more complex cases, when each user needs to have its own working dir. Using relative/full paths I don't think is a real problem, in P2J terms: this is just a configuration issue, and depends on how the admin will configure the brokers. P2J needs to allow both full and relative paths, and let the admin use both, without any additional setting. More, if a broker name has remote launchers running on different OS types (i.e. mixed linux and Windows OS) and a certain user can be handled by more than one OS type, then this is again a administration/configuration problem: the admin must be aware that mixing OS types will pose problems in terms of setting the working dir and needs to setup the working dir/user accounts accordingly.

Sorry but I'm totally confused. The working directory is for OS accounts not for P2J accounts.
When a web client is spawn no P2J accounts are used only the user OS account is used.

Marius Gligor wrote:

Sorry but I'm totally confused. The working directory is for OS accounts not for P2J accounts.
When a web client is spawn no P2J accounts are used only the user OS account is used.

You are correct, I forgot that the P2J account is not known when the web client logs in (that was the reason that ClientBuilderOptions is searching only for per-server configs). But my reasoning still applies: we should find a way to configure the working dir for each OS account, even for web clients. Maybe a special section in the directory, to setup a mapping of OS account/OS type to working dir?

OTOH, now your reasoning about the relative paths makes sense... if the spawner can use/determine the home dir for that user, then is just a matter of appending the relative path to the home dir. Else, if the path is fixed, then use that without any preprocessing. But I feel more comfortable giving more control to the admin, than having P2J work in a certain way, which may not satisfy all possible cases.