FWD » Core Development » Database

The current code provides two ways to make a database connection in FWD:

• Using the session based auto-connect (#3930) that is configured in the directory.
• Executing a CONNECT statement in 4GL code.

The auto-connect is a weird replacement in FWD for the -db command line parameter which is how it is done in OE. That might be specified in a .pf file or explicitly on the command line, but using -db (and the implicit "special" database name as first parameter of a command) is the way that OE allows a database to be connected for a session. This mechanism has the advantage over the CONNECT statement because it enables all 4GL programs (including the startup procedure) to access the database.

It is also nice because it allows customization of which database instance will be connected, where the FWD directory auto-connect is not as flexible. So the intention here is to add support in bootstrap configuration for connecting databases. When we add this support, it should support the same options that we support in the CONNECT statement. This might be things like hostname, port/servicename and so forth.

One thing that makes this less straightforward to implement is that in the 4GL, you can have many -db sections on the command line (each section can have it own related parameters like -db some-database.db -H host -S port -ssl -P password. In this example, the -H host is an option that relates to the -db some-database.db. In addition, there can be more than one -db and each of them can have their own database parameters. So there can be more than one -H too. The current bootstrap cfg approach doesn't provide for arrays at the key level, so we either need to add such a concept or do something dirty like adding a numeric suffix to these and honoring n number of these. Yuck, I don't that.

Are -H, -S and -P used only for the db? If so, it's best to have an array of custom objects with all the configured details for the db instead of having its properties spread out as separate params.

Also I've found an interesting behavior of -pf we don't seem to support (https://docs.progress.com/bundle/abl-reference/page/STARTUP-PARAMETERS-attribute.html): In the character response of the getter SESSION:STARTUP-PARAMETERS pf should be presented as -pf dbconnect.pf,-db sports2020,-H pclsmith,-S 5000,(end .pf), in other words its content should be read on startup and saved in a more complex structure instead of a string.

If all dbs should be auto-connected simultaneously, then I guess precedence is not that important. Nevertheless, it may need to be tested for conflicting settings.

Are -H, -S and -P used only for the db? If so, it's best to have an array of custom objects with all the configured details for the db instead of having its properties spread out as separate params.

At runtime, these can be used for multiple purposes. They are all used during a CONNECT statement which can be for a database, a TCP "client" socket, an appserver or a web service.

At startup (the command line), it is possible these are only available when specified with the -db, we will need to check. I don't think there is any natural connection to a socket, web service or appserver during startup so it may just be with database connections.

I have started working on this task.

Currently, after I have checked the FWD implementation of client-side bootstrap configuration.

I am thinking to create a new class type of StartupParameter, DBStartupParameter, which behaves like a single StringStartupParameter, but has a list of combination of (host-name, service-name, password, etc). The alternative for this is to add this into ConfigItem, but is counterintuitive since this is an actual startup parameter.
This will allow reading multiple -db arguments.
For the client configuration structure, I am thinking of having a different XML node name because it is easier and more natural, like in the following example:

   <client>
      <driver type="gui_swing"/>

      <db-connections>
         <connection
            physical-database-name="hotel.db" 
            host-name="localhost" 
            service-name="1234" 
            logical-database-name="hotel" 
         />
         <connection
            physical-database-name="hotel2.db" 
            host-name="localhost" 
            service-name="1235" 
            logical-database-name="hotel2" 
         />
      </db-connections>
   </client>

Committed revision 16347 on task branch 8834a:
- Added support for reading list items from Bootstrap client configuration.

What needs to be done next:
- on startup, the Connection.connect(...) should be called (I think)
- add some more parameters and experiment with validation.
- implement reading list items from DirectoryService.
- think about ClientCore.processTemporaryClient, it should also copy config list items?

Committed revision 16348 on task branch 8834a:
- Added support for database startup parameter.
- AUTOCONNECT is now working properly with -db parameters from client bootstrap.

Currently, I am starting to implement the -pf parameter, what Galya said on #8834-3.

For the client configuration structure, I am thinking of having a different XML node name because it is easier and more natural

Please consider that bootstrap cfg is sometimes but not always encoded in XML. A very common use case is to pass bootstrap cfg "overrides" via the command line for starting the client or the server. See Command Line Overrides.

With this in mind, how would the extra structure be handled?

I worry about adding something custom here. It has the potential to make the configuration code more fragile.

Is there a more generic alternative?

Also I think I need to find an equivalent structure for directory.xml??? What should be done about default existing connections?

Eric: Please answer these.

The client bootstrap should override default configuration?

Yes

Greg Shah wrote:

For the client configuration structure, I am thinking of having a different XML node name because it is easier and more natural

Please consider that bootstrap cfg is sometimes but not always encoded in XML. A very common use case is to pass bootstrap cfg "overrides" via the command line for starting the client or the server. See Command Line Overrides.

Yes, that is a problem when I have encountered when I have started implementing (the command line gets very complicated).
I think I should rethink the actual configuration to be just a string, like how it is in the directory.xml

• Current config example (which is being implemented in rev 16348: Show/HideShow/Hide

  <?xml version="1.0"?>
  
  <!-- Client side bootstrap configuration for the standard client application
       with the two way certificate validation
  -->
  
  <node type="client">
     <net>
        <connection secure="true" />
        <server host="localhost" />
        <server secure_port="3334" />
        <server insecure_port="3434" />
     </net>
     <client>
        <driver type="gui_swing"/>
  
        <db-connections>
           <connection
              physical-database-name="book" 
           />
        </db-connections>
     </client>
     <security>
        <truststore filename="../server/srv-certs.store" />
        <truststore alias="standard" />
     </security>
     <access>
        <password truststore="..." />
     </access>
  </node>
  

• New proposed version: Show/HideShow/Hide

  <?xml version="1.0"?>
  
  <!-- Client side bootstrap configuration for the standard client application
       with the two way certificate validation
  -->
  
  <node type="client">
     <net>
        <connection secure="true" />
        <server host="localhost" />
        <server secure_port="3334" />
        <server insecure_port="3434" />
     </net>
     <client>
        <driver type="gui_swing"/>
  
        <db-connections>
           <connection value="book -H localhost -S 1234 -U user -P pwd"/> <!-- -H -S -U -P args are optional -->
           <connection value="book2 -H localhost -S 1235 -U user -P pwd"/> <!-- -H -S -U -P args are optional -->
        </db-connections>
     </client>
     <security>
        <truststore filename="../server/srv-certs.store" />
        <truststore alias="standard" />
     </security>
     <access>
        <password truststore="..." />
     </access>
  </node>
  

Greg Shah wrote:

Also I think I need to find an equivalent structure for directory.xml??? What should be done about default existing connections?

Eric: Please answer these.

There are two configuration options which is in directory.xml (before 8834a):
- server.default.runtime.database-connections which contains a list of database connection strings, which are meant to be passed to CONNECT statement (similar with second client bootstrap cfg).
- load_at_startup parameter from each database p2j schema node.

Greg Shah wrote:

The client bootstrap should override default configuration?

Yes

This is not being done on latest revision but it makes sense. I will commit the change for this today.

Committed revision 16349 on task branch 8834a:
- Override directory.xml server auto-connect configuration when client database auto-connect options are being used.

Teodor, please test how cfgOverrides can handle the spaces, in directory.xml - is it enough to use " in the string?

Ok, I see. There are lots of places where the split(" ") is being used and I think I shall replace that with a more advanced parsing method (do not split the string if is is enclosed in quotes).
Also, during testing, we need to make sure that customer applications configuration is compatible with these changes (example, there are some parameter files which has OE connection strings).

Committed revision 16351 on task branch 8834a:
- Added support for quoted cfgOverrides.
- Additional fixes like making sure that parameter file gets parsed first before any other parameter.

In order to use a client:db-connections:connection which has spaces, you need to provide the option in escaped quotes, like this:

<node class="string" name="cfgOverrides">
   <node-attribute name="value" value="&quot;client:db-connections:connection:book=-H localhost -P 2294&quot; client:driver:background=true/>
</node>

• in ParameterFileStartupParameter.readParameterFile - the file needs to be read from the FWD client side, not server's (unless we have server-side resources). So please see how InputStreamWrapper is used by i.e. SourceLobFile.
• does existing directory configurations still work, or will they need to be adjusted?

Constantin Asofiei wrote:

• does existing directory configurations still work, or will they need to be adjusted?

I have tested the initial directory configuration according to https://proj.goldencode.com/projects/regression/wiki/Cross-Customer_Test_Plan_Tracking, but I have found only one application which needs directory.xml modified (because of .pf implementation). See #9438-7.

Constantin Asofiei wrote:

• in ParameterFileStartupParameter.readParameterFile - the file needs to be read from the FWD client side, not server's (unless we have server-side resources). So please see how InputStreamWrapper is used by i.e. SourceLobFile.

The code is not being executed at client side?

Teodor Gorghe wrote:

Constantin Asofiei wrote:

• in ParameterFileStartupParameter.readParameterFile - the file needs to be read from the FWD client side, not server's (unless we have server-side resources). So please see how InputStreamWrapper is used by i.e. SourceLobFile.

The code is not being executed at client side?

You are correct. If this is happening also when the cfgOverrides is being processed from directory.xml (it can have its own -pf file spec), then we are OK.

Constantin Asofiei wrote:

You are correct. If this is happening also when the cfgOverrides is being processed from directory.xml (it can have its own -pf file spec), then we are OK.

Yes, this is still happening since BC.processOverrides is being called before StartupParameters.initFromBootstrap.

Harness, reports and navigation passed!