FWD » Deployment

This is a place for discussion of the JSON used mostly by ./deploy/server/prepare_dir.sh which primarily creates the directory.xml and server.xml files (as is currently implemented in the Hotel samples). Recently prepare_dir.sh added the feature to be able to configure brokers, which included brokerx_client.xml file creation, and addition of process certificates and stores.

From #4369 the discussion came forth about setting up the cluster configuration via enhancements to prepare_dir.sh, as well as additions to the JSON. The Stateless Server Clustering wiki has some places where configuration of nginx was presented and I wanted to have a place to discuss overall JSON formatting, which can be done here. I'll bring the discussion points here.

For the configuration of clustering, the ./deploy/server/prepare_dir.sh can be updated to look for a new section in the JSON, which could look something like:

  "cluster": {
    "geode_locator_host": "locator",
    "geode_locator_port": "10334" 
  }

And a directory_cluster.xml.template:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<remapper-storage>
  <node class="container" name="">
    <node class="container" name="server">
      <node class="container" name="default">
        <node class="boolean" name="cluster-mode">
          <node-attribute name="value" value="TRUE"/>
        </node>
        <node class="container" name="geode">
          <node class="string" name="locatorHost">
            <node-attribute name="value" value="{geode_locator_host}"/>
          </node>
          <node class="integer" name="locatorPort">
            <node-attribute name="value" value="{geode_locator_port}"/>
          </node>
        </node>
      </node>
    </node>
  </node>
</remapper-storage>

The prepare_dir.sh would attempt to read, and process the values if found.

# Handle cluster portion as appropriate
if $(_getval "cluster" "$file"); then
   locator_host=$(sub_getval "cluster" "geode_locator_host" "locator" $infile)
   locator_port=$(sub_getval "cluster" "geode_locator_port" "10334" $infile)
   sed \
       -e "s/{geode_locator_host}/$locator_host/g" \
       -e "s/{geode_locator_port}/$locator_port/g" \
       directory_cluster.xml.template > directory_cluster.xml
   rc=$?
   if [ $rc -ne 0 ]; then
      echo "ERROR: Could not create directory_cluster.xml rc=$rc" 
      exit $rc
   fi
   directory_copy "directory_cluster.xml" "/server" "directory_tmp.xml" "/server" 
   rm directory_cluster.xml
fi

This is pretty easy to add to Hotels as a sample. Let me know if this is to be done now.

The other end is the configuration of other things like nginx and the compose file. These could be read from the same JSON as we use to setup directory and server.xml. But I am trying to design how to handle the JSON for this purpose for a project, but a discussion here for clustering would bear the same fruit for both projects.

If we start including compose and other configuration data in the same JSON as we are currently using for the directory (see the hotel_gui ./deploy/etc/prepare_dir_pg_broker_allinone.json for a sample) things are a little cluttered and could use some better design. For example, if we intermingle the compose information with the main JSON, we get fields like ip4_address, which is used to assign an IP address to a service when macvlan is used. Same thing with swarm_node if you are configuring the Swarm Stack.

Would it be better to break out the other config items to a compose or nginx section to keep that data separate, or perhaps put them under the cluster section to group them together?

  "cluster": {
     "geode_locator_host": "locator",
     "geode_locator_port": "10334" 
     "nginx": {
        "upstream": "backend_server",
        "upstream_server": "FWD_server",
        "upstream_port": 7443
     }
  }

We could also go the route of an entire JSON dedicated to the other configurations, like an nginx.json and a compose.json, which would make configuration more granular, but also create duplicated effort and possible misconfiguration, as you have to have each JSON contain all data required. The script could pull together what it needs by pulling some config values from the application JSON and some from another JSON as needed.

Any opinions would be appreciated.

Rules of thumb:

1. One JSON file.
2. Group related values together, using the structure of the file to make the result more meaningful.
3. Reuse values that already exist, where possible (minimize redundancy).

I think that should answer your questions.

Please note that at this time we aren't making changes related to #4369.

• The hotel.input.linux now has an equivalent hotel_input_linux.json
• The hotel.input.linux.pg now has an equivalent hotel_input_linux_pg.json

When the json_template.sh runs, you have the option to save a JSON that would then be used by prepare_dir.sh to configure the directory. This contains most all the values you would need to configure directory.xml and server.xml.

The prepare_dir_pg_broker_allinone.json from the Hotel GUI project is below. It contains most all the information required to create a Docker configuration, given the paths to the configuration as /opt/hotel/ which corresponds to the ./deploy/ directory.

{
  "server_xml_file": "/opt/hotel/etc/server.xml",
  "directory_xml_file": "/opt/hotel/etc/directory.xml",
  "spawner_path": "/opt/spawner/spawn",
  "client_start_dir": "/opt/hotel/client",
  "dateFormat": "mdy",
  "numberGroupSep": ",",
  "numberDecimalSep": ".",
  "p2j_entry": "start.p",
  "pkgroot": "com.goldencode.hotel",
  "propath": ".:common:",
  "search_path": ".:common",
  "path_separator": ":",
  "file_separator": "/",
  "case_sensitive": "TRUE",
  "os_user": "fwd",
  "server_log": "/opt/hotel/logs",
  "client_log": "/opt/hotel/logs",
  "embedded_host": "hotelsrv",
  "admin_port": 7443,
  "from_port" : 7449,
  "to_port"   : 7459,
  "secure_port": 3333,
  "insecure_port": 3433,
  "server_ipv4_address": "192.168.2.51",
  "server_swarm_node": "fwddev2",
  "dbnames": ["hotel"],
  "hotel": {
    "dbtype": "postgres",
    "collation": "en_US@iso88591_fwd_basic",
    "dbhost": "hoteldb",
    "dbport": 5432,
    "ipv4_address": "192.168.2.50",
    "swarm_node": "fwddev1" 
  },
  "brokers": ["broker1","broker2"],
  "broker1": {
    "client_xml_file": "/opt/hotel/etc/broker1_client.xml",
    "broker_process_name": "broker1_process",
    "broker_process_description": "broker1_process",
    "broker_process_name_alias": "broker1",
    "broker_name": "broker1",
    "broker_os_user": "fwd",
    "broker_process_acl": "000510",
    "spawner_path": "/opt/spawner/spawn",
    "broker_client": "hotelbrk1",
    "broker_server": "hotelsrv",
    "storepath": "/opt/hotel/security",
    "logpath": "/opt/hotel/logs",
    "fwd_lib": "/opt/fwd/build/lib",
    "ipv4_address": "192.168.2.52",
    "swarm_node": "fwddev3",
    "from_port" : 7449,
    "to_port"   : 7459
  },
  "broker2": {
    "client_xml_file": "/opt/hotel/etc/broker2_client.xml",
    "broker_process_name": "broker2_process",
    "broker_process_description": "broker2_process",
    "broker_process_name_alias": "broker2",
    "broker_name": "broker2",
    "broker_os_user": "fwd",
    "broker_process_acl": "000515",
    "spawner_path": "/opt/spawner/spawn",
    "broker_client": "hotelbrk2",
    "broker_server": "hotelsrv",
    "storepath": "/opt/hotel/security",
    "logpath": "/opt/hotel/logs",
    "fwd_lib": "/opt/fwd/build/lib",
    "ipv4_address": "192.168.2.53",
    "swarm_node": "fwddev4",
    "from_port" : 7449,
    "to_port"   : 7459
  }
}

Greg Shah wrote:

Rules of thumb:

1. One JSON file.
2. Group related values together, using the structure of the file to make the result more meaningful.
3. Reuse values that already exist, where possible (minimize redundancy).

I think that should answer your questions.

Those are the rules that I had mostly adhered to, but #2 is where there are some issues with the current format that prevent future expansion. For example, how best to represent 2 servers? My guess is to have similar to "dbnames", and have a "servers" section, and most all the data from the current main level would be moved into that section, and given the name "localhost" or similar (perhaps the embedded_host?), to signify the only server in a configuration. (If we use embedded host, I could jettison that field, and use the read-in server name for that value).

   "servers": ["localhost"],
   "localhost": {
      "server_xml_file": "/opt/hotel/etc/server.xml",
      "directory_xml_file": "/opt/hotel/etc/directory.xml",
      "spawner_path": "/opt/spawner/spawn",
      "client_start_dir": "/opt/hotel/client",
      "dateFormat": "mdy",
      "numberGroupSep": ",",
      "numberDecimalSep": ".",
      "p2j_entry": "start.p",
      "pkgroot": "com.goldencode.hotel",
      "propath": ".:common:",
      "search_path": ".:common",
      "path_separator": ":",
      "file_separator": "/",
      "case_sensitive": "TRUE",
      "os_user": "fwd",
      "server_log": "/opt/hotel/logs",
      "client_log": "/opt/hotel/logs",
      "embedded_host": "hotelsrv",
      "admin_port": 7443,
      "from_port" : 7449,
      "to_port"   : 7459,
      "secure_port": 3333,
      "insecure_port": 3433,
      "server_ipv4_address": "192.168.2.51",
      "server_swarm_node": "fwddev2" 
   }

and if you needed multiple servers, of course the "names" of the servers would need to change, as well as the paths being unique for each server (because the configuration is done on a single location, and shared across boundaries with volume usage). TODO: Add a new config value to differentiate between the "configured" filenames, and the physical filenames prepare_dir.sh uses to actually create the files. I added a potential "cfgdir" so that the tool knows where to place generated files.

So below might be an example of 2 servers, each with 1 DB to itself, and 1 DB shared across the servers, and each with 2 brokers. From the below, you could see the configurations, but how the containers are configured we need more information, because there is the opportunity to use the Docker image for the server that contains the database locally, or you might want to have the databases in separate containers. How best to represent that information? And I'm not sure if I created the best naming of multiple servers.

{
   "servers": ["hotel1srv","hotel2srv"],
   "hotel1srv": {
      "cfgdir": "./hotel1_cfg" 
      "server_xml_file": "/opt/hotel/etc/server.xml",
      "directory_xml_file": "/opt/hotel/etc/directory.xml",
      "spawner_path": "/opt/spawner/spawn",
      "client_start_dir": "/opt/hotel/client",
      ...
      "embedded_host": "hotel1srv",
      "admin_port": 7443,
      "from_port" : 7449,
      "to_port"   : 7459,
      "secure_port": 3333,
      "insecure_port": 3433,
      "server_ipv4_address": "192.168.2.50",
      "dbnames": ["hotel1","hotelprices"],
      "hotel1": {
        "dbtype": "postgres",
        "collation": "en_US@iso88591_fwd_basic",
        "dbhost": "hotel1db",
        "dbport": 5432,
        "ipv4_address": "192.168.2.51" 
      },
      "hotelprices": {
        "dbtype": "postgres",
        "collation": "en_US@iso88591_fwd_basic",
        "dbhost": "hotelpricesdb",
        "dbport": 5432,
        "ipv4_address": "192.168.2.52" 
      },
      "broker1": {
         "client_xml_file": "/opt/hotel/etc/broker1_client.xml",
         "broker_process_name": "broker1_process",
         "broker_process_description": "broker1_process",
         "broker_process_name_alias": "broker1",
         "broker_name": "broker1",
         "broker_os_user": "fwd",
         "broker_process_acl": "000510",
         "spawner_path": "/opt/spawner/spawn",
         "broker_client": "hotel1brk1",
         "broker_server": "hotel1srv",
         "storepath": "/opt/hotel/security",
         "logpath": "/opt/hotel/logs",
         "fwd_lib": "/opt/fwd/build/lib",
         "ipv4_address": "192.168.2.53",
         "from_port" : 7449,
         "to_port"   : 7459
      },
      "broker2": {
         "client_xml_file": "/opt/hotel/etc/broker2_client.xml",
         "broker_process_name": "broker2_process",
         "broker_process_description": "broker2_process",
         "broker_process_name_alias": "broker2",
         "broker_name": "broker2",
         "broker_os_user": "fwd",
         "broker_process_acl": "000515",
         "spawner_path": "/opt/spawner/spawn",
         "broker_client": "hotel2brk2",
         "broker_server": "hotel2srv",
         ...
         "ipv4_address": "192.168.2.54",
         "from_port" : 7449,
         "to_port"   : 7459
      }
   },
   "hotel2srv": {
      "cfgdir": "./hotel2_cfg" 
      "server_xml_file": "/opt/hotel/etc/server.xml",
      "directory_xml_file": "/opt/hotel/etc/directory.xml",
      "spawner_path": "/opt/spawner/spawn",
      "client_start_dir": "/opt/hotel/client",
      ...
      "embedded_host": "hotel2srv",
      "admin_port": 7443,
      "from_port" : 7449,
      "to_port"   : 7459,
      "secure_port": 3333,
      "insecure_port": 3433,
      "server_ipv4_address": "192.168.2.55",
      "dbnames": ["hotel","hotelprices"],
      "hotel2": {
        "dbtype": "postgres",
        "collation": "en_US@iso88591_fwd_basic",
        "dbhost": "hotel2db",
        "dbport": 5432,
        "ipv4_address": "192.168.2.56" 
      },
      "hotelprices": {
        "dbtype": "postgres",
        "collation": "en_US@iso88591_fwd_basic",
        "dbhost": "hotelpricesdb",
        "dbport": 5432,
        "ipv4_address": "192.168.2.52" 
      },
      "brokers": ["broker1","broker2"],
      "broker1": {
         "client_xml_file": "/opt/hotel/etc/broker1_client.xml",
         "broker_process_name": "broker1_process",
         "broker_process_description": "broker1_process",
         "broker_process_name_alias": "broker1",
         "broker_name": "broker1",
         "broker_os_user": "fwd",
         "broker_process_acl": "000510",
         "spawner_path": "/opt/spawner/spawn",
         "broker_client": "hotel2brk1",
         "broker_server": "hotel2srv",
         "storepath": "/opt/hotel/security",
         "logpath": "/opt/hotel/logs",
         "fwd_lib": "/opt/fwd/build/lib",
         "ipv4_address": "192.168.2.57",
         "from_port" : 7449,
         "to_port"   : 7459
      },
      "broker2": {
         "client_xml_file": "/opt/hotel/etc/broker2_client.xml",
         "broker_process_name": "broker2_process",
         "broker_process_description": "broker2_process",
         "broker_process_name_alias": "broker2",
         "broker_name": "broker1",
         "broker_os_user": "fwd",
         "broker_process_acl": "000515",
         "spawner_path": "/opt/spawner/spawn",
         "broker_client": "hotel2brk2",
         "broker_server": "hotel2srv",
         ...
         "ipv4_address": "192.168.2.58",
         "from_port" : 7449,
         "to_port"   : 7459
      }
   }
}

Please note that at this time we aren't making changes related to #4369.

Agreed. The notes are valid in that task, if it came time to implement.

I didn't make these rules up on a whim. They are based on fundamental best practices.

Those are the rules that I had mostly adhered to, but #2 is where there are some issues with the current format that prevent future expansion. For example, how best to represent 2 servers? My guess is to have similar to "dbnames", and have a "servers" section, and most all the data from the current main level would be moved into that section, and given the name "localhost" or similar (perhaps the embedded_host?), to signify the only server in a configuration.

The moment you have to say "we are breaking that rule", you'd better have a good reason to do it. In this case, there is no good reason for your proposed design. Thus, you should conclude that it needs rework to match the rules.

Having an array of names as one object and then having to use those to read the actual objects is no bueno. JSON provides the mechanism for defining objects within objects. That is one of the things meant by "use the structure of the file". It not only simplifies the usage of the file but it also reads better, making "the result more meaningful".

{
   "servers":
      [
         {
            "name": "hotel1srv",
            ...
         },
         {
            "name": "hotel2srv",
            ...
         },
      ],
   "databases":
   [
      {
         "name": "hotel1",
         ...
      },
      {
         "name", "hotel2",
         ...
      },
      {
         "name": "hotelprices",
         ...
      }
   ]
}

Note how I use this same design to solve the database problem AND this also eliminates redundancy (e.g. 2 copies of hotelprices). Please follow the rules. They will help you.

Notice how I eliminated the extra names and just have the objects in the arrays. The top level code just reads the arrays and has some objects to process. Easy-peasy.

I also see a LOT of duplication of text. How many times do we have to type "broker1"? I will point out that it is so many times that you have misconfigured broker2 by having it still have a broker_name of broker1. MINIMIZE the configuration.

Greg Shah wrote:

The moment you have to say "we are breaking that rule", you'd better have a good reason to do it. In this case, there is no good reason for your proposed design. Thus, you should conclude that it needs rework to match the rules.

It should go without saying that I opened this task for input into the design. I did use the past tense and indicated the current design was preventing future expansion. I will say having a pre-knowledge of the number of elements of an item (databases, brokers, servers) provided a level of coding convenience in knowing whether to skip a section in total, just stick with the first as the default, etc.

I also see a LOT of duplication of text. How many times do we have to type "broker1"? I will point out that it is so many times that you have misconfigured broker2 by having it still have a broker_name of broker1. MINIMIZE the configuration.

The duplication is really driven by the directory configuration, and whether or not we can just use the same value for everything in every field. Because this is for all customers, it's probably a good idea to give total control over all fields, and not force name = alias = process name, etc. There might be an appetite to name brokers by alphabetic names, Alpha, Bravo, Catfish, Doofis, etc. It's probably not even wise for me to give sequential numbers to them as an inference.

I also see a LOT of duplication of text. How many times do we have to type "broker1"? I will point out that it is so many times that you have misconfigured broker2 by having it still have a broker_name of broker1. MINIMIZE the configuration.

The duplication is really driven by the directory configuration, and whether or not we can just use the same value for everything in every field. Because this is for all customers, it's probably a good idea to give total control over all fields, and not force name = alias = process name, etc. There might be an appetite to name brokers by alphabetic names, Alpha, Bravo, Catfish, Doofis, etc. It's probably not even wise for me to give sequential numbers to them as an inference.

I don't see the value to this. Keep it simple for the users/admins and make it less fragile/error prone. If we need to enhance it later to break things out, we can but for now this duplication is a huge maintanence headache that has real costs. We aren't going to take thoses costs in exchange for "somebody later on may want more flexibility".

For example, all of these can be calculated from the "name" field that I added.

         "broker_process_name": "broker2_process",
         "broker_process_description": "broker2_process",
         "broker_process_name_alias": "broker2",
         "broker_name": "broker1",

Notice also that the structure itself of the brokers inside the server means that we can calculate the server for the broker instead of having this separate entry "broker_server": "hotel2srv".

I also don't understand why a server with brokers also has these values:

      "from_port" : 7449,
      "to_port"   : 7459,

Thin this all down and clean it up.

Greg Shah wrote:

I don't see the value to this. Keep it simple for the users/admins and make it less fragile/error prone. If we need to enhance it later to break things out, we can but for now this duplication is a huge maintanence headache that has real costs. We aren't going to take thoses costs in exchange for "somebody later on may want more flexibility".

For example, all of these can be calculated from the "name" field that I added.

Thanks fine. It was really coming being able to configure:

security/acl/system/{broker_process_acl}/subjects/{broker_process_name}
security/accounts/processes/{broker_process_name}/{broker_process_description}
security/accounts/processes/{broker_process_name}/{broker_process_name_alias}
security/accounts/processes/{broker_process_name}/{broker_name}
server/default/brokers/{broker_name}/{broker_os_user}

Notice also that the structure itself of the brokers inside the server means that we can calculate the server for the broker instead of having this separate entry "broker_server": "hotel2srv".

Good point. I haven't gotten too close to the multi-server configuration, but configuring the directory for the regression testing application is on the radar.

I also don't understand why a server with brokers also has these values:

[...]

Thin this all down and clean it up.

Hmm... I guess you are not familiar with the ability to specify port ranges? :-) If you are referring to the fact that it is defined in the server itself, like I said, I haven't really gotten to the multi-server configuration. The fact that the brokers would be "nested" in the servers would definitely provide for "normalization" of some values.

I also don't understand why a server with brokers also has these values:

If you have brokers, the server doesn't need port ranges.

Greg Shah wrote:

I also don't understand why a server with brokers also has these values:

If you have brokers, the server doesn't need port ranges.

Do you mean the server doesn't have to define port ranges if the broker does?

Roger Borrello wrote:

Greg Shah wrote:

I also don't understand why a server with brokers also has these values:

If you have brokers, the server doesn't need port ranges.

Do you mean the server doesn't have to define port ranges if the broker does?

That would be my expectation.

Greg Shah wrote:

Roger Borrello wrote:

Greg Shah wrote:

I also don't understand why a server with brokers also has these values:

If you have brokers, the server doesn't need port ranges.

Do you mean the server doesn't have to define port ranges if the broker does?

That would be my expectation.

Is that your expectation of how Sergey implemented? I will configure it properly, not matter what. Wouldn't the server need to know something about the valid ports that are going to be utilized?

I have another pass at an example JSON setup, but I'm still questioning how the databases are setup. In some respects, they should be independent of the servers, as in how the containers are setup. But in others, there is a relationship for which servers are connecting to which databases. It might be that the "databases" should be at the same level as "servers", and "servers" would link to databases by name, but would that name be the hostname? In containers the hostname is the container name.

{
  "servers": [
    {
      "name": "hotel1srv",
      "cfgdir": "./hotel1_cfg",
      "server_xml_file": "/opt/hotel/etc/server.xml",
      "directory_xml_file": "/opt/hotel/etc/directory.xml",
      "spawner_path": "/opt/spawner/spawn",
      "client_start_dir": "/opt/hotel/client",

      "embedded_host": "hotel1srv",
      "admin_port": 7443,
      "secure_port": 3333,
      "insecure_port": 3433,

      "server_ipv4_address": "192.168.2.50",

      "databases": ["hotel1", "hotelprices"],

      "brokers": [
        {
          "name": "hotel1broker1",
          "client_xml_file": "/opt/hotel/etc/broker1_client.xml",
          "broker_os_user": "fwd",
          "broker_process_acl": "000510",
          "spawner_path": "/opt/spawner/spawn",
          "broker_client": "hotel1brk1",
          "storepath": "/opt/hotel/security",
          "logpath": "/opt/hotel/logs",
          "fwd_lib": "/opt/fwd/build/lib",
          "ipv4_address": "192.168.2.53",
          "from_port": 7449,
          "to_port": 7459
        },
        {
          "name": "hotel1broker2",
          "client_xml_file": "/opt/hotel/etc/broker2_client.xml",
          "broker_os_user": "fwd",
          "broker_process_acl": "000515",
          "spawner_path": "/opt/spawner/spawn",
          "broker_client": "hotel1brk2",
          "storepath": "/opt/hotel/security",
          "logpath": "/opt/hotel/logs",
          "fwd_lib": "/opt/fwd/build/lib",
          "ipv4_address": "192.168.2.54",
          "from_port": 7449,
          "to_port": 7459
        }
      ]
    },

    {
      "name": "hotel2srv",
      "cfgdir": "./hotel2_cfg",
      "server_xml_file": "/opt/hotel/etc/server.xml",
      "directory_xml_file": "/opt/hotel/etc/directory.xml",
      "spawner_path": "/opt/spawner/spawn",
      "client_start_dir": "/opt/hotel/client",

      "embedded_host": "hotel2srv",
      "admin_port": 7443,
      "secure_port": 3333,
      "insecure_port": 3433,

      "server_ipv4_address": "192.168.2.55",

      "databases": ["hotel2", "hotelprices"],

      "brokers": [
        {
          "name": "hotel2broker1",
          "client_xml_file": "/opt/hotel/etc/broker1_client.xml",
          "broker_os_user": "fwd",
          "broker_process_acl": "000510",
          "spawner_path": "/opt/spawner/spawn",
          "broker_client": "hotel2brk1",
          "storepath": "/opt/hotel/security",
          "logpath": "/opt/hotel/logs",
          "fwd_lib": "/opt/fwd/build/lib",
          "ipv4_address": "192.168.2.57",
          "from_port": 7449,
          "to_port": 7459
        },
        {
          "name": "hotel2broker2",
          "client_xml_file": "/opt/hotel/etc/broker2_client.xml",
          "broker_os_user": "fwd",
          "broker_process_acl": "000515",
          "spawner_path": "/opt/spawner/spawn",
          "broker_client": "hotel2brk2",
          "storepath": "/opt/hotel/security",
          "logpath": "/opt/hotel/logs",
          "fwd_lib": "/opt/fwd/build/lib",
          "ipv4_address": "192.168.2.58",
          "from_port": 7449,
          "to_port": 7459
        }
      ]
    }
  ],

  "databases": [
    {
      "name": "hotel1",
      "dbtype": "postgres",
      "collation": "en_US@iso88591_fwd_basic",
      "dbhost": "hotel1db",
      "dbport": 5432,
      "ipv4_address": "192.168.2.51" 
    },
    {
      "name": "hotel2",
      "dbtype": "postgres",
      "collation": "en_US@iso88591_fwd_basic",
      "dbhost": "hotel2db",
      "dbport": 5432,
      "ipv4_address": "192.168.2.56" 
    },
    {
      "name": "hotelprices",
      "dbtype": "postgres",
      "collation": "en_US@iso88591_fwd_basic",
      "dbhost": "hotelpricesdb",
      "dbport": 5432,
      "ipv4_address": "192.168.2.52" 
    }
  ]
}

Wouldn't the server need to know something about the valid ports that are going to be utilized?

What makes you think that?

At this moment the server directory defines the port ranges for local clients and the remote clients simultaneously. The server has host to port map and uses host to get available ports. The logic is done via WebClientsManager and this should be known to all reviewers at least.

Greg Shah wrote:

Wouldn't the server need to know something about the valid ports that are going to be utilized?

What makes you think that?

I'm not sure how the server divvies up to the brokers sessions, but I would believe the server would supervise control and reject too many clients before even getting to the brokers

Why are you putting databases back inside of the server object? Keep the core database definitions separated (and not duplicated). Then just list the ones which need to be configured in the server.

Sergey Ivanovskiy wrote:

At this moment the server directory defines the port ranges for local clients and the remote clients simultaneously. The server has host to port map and uses host to get available ports. The logic is done via WebClientsManager and this should be known to all reviewers at least.

In broker mode we would not expect to have local clients.

Yes, if at least one broker is set, then no local clients are spawned but only remote clients in respect to the server.

Greg Shah wrote:

Why are you putting databases back inside of the server object? Keep the core database definitions separated (and not duplicated). Then just list the ones which need to be configured in the server.

Edited #11272-13 to reflect servers and databases at the same level. The databases need to have definitions back to which servers are connecting to them. How does that look?

Roger Borrello wrote:

Greg Shah wrote:

Why are you putting databases back inside of the server object? Keep the core database definitions separated (and not duplicated). Then just list the ones which need to be configured in the server.

Edited #11272-13 to reflect servers and databases at the same level. The databases need to have definitions back to which servers are connecting to them. How does that look?

Good.

Greg, I do not understand why do you set port ranges for each broker client? It is not given by our implementation. Port range is given by the server directory.

               "from_port" : 7449,
               "to_port"   : 7459

Those ranges are "logically" associated with each broker. So the person writing the JSON cfg file would expect to put them "in" the broker section. It doesn't matter where it ends up in the FWD cfg, the JSON should make sense to the people creating it.

Greg Shah wrote:

Those ranges are "logically" associated with each broker. So the person writing the JSON cfg file would expect to put them "in" the broker section. It doesn't matter where it ends up in the FWD cfg, the JSON should make sense to the people creating it.

Some customers have their directory.xml.template with the below so the range can be specified in the JSON:

    <node class="container" name="server">
      <node class="container" name="default">
        <node class="container" name="webClient">
          <node class="container" name="portsRange">
            <node class="integer" name="from">
              <node-attribute name="value" value="{from_port}"/>
            </node>
            <node class="integer" name="to">
              <node-attribute name="value" value="{to_port}"/>
            </node>
          </node>

I am not sure why this is necessary to change, even if they have brokers. Where would the port range be defined if they have brokers? Isn't it still in the same place? The specification I added to the JSON was specifically for the Swarm Stack, because the port definitions are to be declared in the YAML for each broker.

Greg Shah wrote:

Those ranges are "logically" associated with each broker. So the person writing the JSON cfg file would expect to put them "in" the broker section. It doesn't matter where it ends up in the FWD cfg, the JSON should make sense to the people creating it.

From this it follows that we should make changes in the code because now our implementation supposes to get them from the web client part of the directory. Roger shown the example in #11272-24 (the previous note above).

Greg, I can do the changes if your requirements are to set port ranges for each broker in 9709d. It is just enough to implement WebClientsManager.createResourceQueue(String host, portRanges) and to store ranges in this object.

Sergey Ivanovskiy wrote:

Greg, I can do the changes if your requirements are to set port ranges for each broker in 9709d. It is just enough to implement WebClientsManager.createResourceQueue(String host, portRanges) and to store ranges in this object.

But you need to specify requirements where to store these values in the server directory under the broker node or on the client side but in this case it needs to deliver them to the server side when the broker client is registered. Please specify requirements.

Sergey Ivanovskiy wrote:

Greg, I can do the changes if your requirements are to set port ranges for each broker in 9709d. It is just enough to implement WebClientsManager.createResourceQueue(String host, portRanges) and to store ranges in this object.

So would there be the below per broker?:

    <node class="container" name="server">
      <node class="container" name="default">
        <node class="container" name="brokers">
          <node class="broker" name="{broker_name}">
            <node class="container" name="portsRange">
              <node class="integer" name="from">
                <node-attribute name="value" value="{from_port}"/>
              </node>
              <node class="integer" name="to">
                <node-attribute name="value" value="{to_port}"/>
              </node>

With port ranges we also have agent ranges too, debuggers and jmx agents.

This task is meant to design an external JSON file that can be used to configure a FWD system (application server, clients/brokers, databases). There was no previous intention to change how we store things in the directory.

If you think that the directory design in this area is not a good/clean design, then document the current approach and propose changes.

Roger Borrello wrote:

Greg Shah wrote:

This task is meant to design an external JSON file that can be used to configure a FWD system (application server, clients/brokers, databases). There was no previous intention to change how we store things in the directory.

If you think that the directory design in this area is not a good/clean design, then document the current approach and propose changes.

I'm guessing that if the broker needed directory entries, we'd have known it by now. Of course that's an uneducated guess.

I did put together a migration tool, to convert JSON files from the "old" format. Would this need to be updated for each project?

There is no documentation for using the JSON and no project uses it other than the one customer you most closely work with. So there is no migration needed except for them.

Greg Shah wrote:

There is no documentation for using the JSON and no project uses it other than the one customer you most closely work with. So there is no migration needed except for them.

Really, every customer uses JSON for prepare_dir.sh, and also now there's a prepare_env.sh that Constantin created. I'm happy to allow each project take care of their own. The one I created was for Hotel.

Roger Borrello wrote:

Greg Shah wrote:

There is no documentation for using the JSON and no project uses it other than the one customer you most closely work with. So there is no migration needed except for them.

Really, every customer uses JSON for prepare_dir.sh, and also now there's a prepare_env.sh that Constantin created. I'm happy to allow each project take care of their own. The one I created was for Hotel.

No, they don't. No one uses it.

That is the problem. We don't have docs or a generic process implemented for all projects. This is just something you hacked into Hotel and that one customer's environment. SO: now we need to make this generic and applicable to all projects. And it will take effort to migrate it back into those projects because they aren't making all the assumptions you've made in the past.

Greg Shah wrote:

This task is meant to design an external JSON file that can be used to configure a FWD system (application server, clients/brokers, databases). There was no previous intention to change how we store things in the directory.

If you think that the directory design in this area is not a good/clean design, then document the current approach and propose changes.

Greg, it is not clear for me. Now the broker setup uses the web client settings given by

 <node class="container" name="server">
      <node class="container" name="default">
        <node class="container" name="webClient">
          <node class="container" name="portsRange">
            <node class="integer" name="from">
              <node-attribute name="value" value="{from_port}"/>
            </node>
            <node class="integer" name="to">
              <node-attribute name="value" value="{to_port}"/>
            </node>
          </node>

<node class="container" name="server">
      <node class="container" name="default">
        <node class="container" name="brokers">
          <node class="broker" name="{broker_name}">
            <node class="container" name="portsRange">
              <node class="integer" name="from">
                <node-attribute name="value" value="{from_port}"/>
              </node>
              <node class="integer" name="to">
                <node-attribute name="value" value="{to_port}"/>
              </node>

Sergey Ivanovskiy wrote:

Greg Shah wrote:

This task is meant to design an external JSON file that can be used to configure a FWD system (application server, clients/brokers, databases). There was no previous intention to change how we store things in the directory.

If you think that the directory design in this area is not a good/clean design, then document the current approach and propose changes.

Greg, it is not clear for me. Now the broker setup uses the web client settings given by
[...]
Thus each broker client has the same port range as defined in the directory. Should I make changes to support per broker setup of web client ports range? I can do these changes in 9709e branch task #9709 supposing these settings proposed by Roger
[...]

This indeed is a limitation that doesn't make much sense. It can work for the scenario that Roger has implemented with using a the same standard container for each broker. But other customer environments might not be able to operate if every broker has the same port range.

I prefer not to make more changes right now. How much work is needed? I'm worried that we have a lot of dependencies on this limitation and that the work is larger than I would like.

Although this limitation seems to be a problem, for now it

At first glance it takes no more than the work day or less.

Go ahead with the changes.

Greg Shah wrote:

No, they don't. No one uses it.

That is the problem. We don't have docs or a generic process implemented for all projects. This is just something you hacked into Hotel and that one customer's environment. SO: now we need to make this generic and applicable to all projects. And it will take effort to migrate it back into those projects because they aren't making all the assumptions you've made in the past.

There is prepare_dir.sh usage in other projects, and Constantin and Radu and others had been able to make their way through my hacks.

Greg Shah wrote:

Go ahead with the changes.

Created 9709e and committed these proposed changes as rev 16469. Please review. This is provided for the notification and link, the discussion will be continued in #9709.

Roger Borrello wrote:

Greg Shah wrote:

No, they don't. No one uses it.

That is the problem. We don't have docs or a generic process implemented for all projects. This is just something you hacked into Hotel and that one customer's environment. SO: now we need to make this generic and applicable to all projects. And it will take effort to migrate it back into those projects because they aren't making all the assumptions you've made in the past.

There is prepare_dir.sh usage in other projects, and Constantin and Radu and others had been able to make their way through my hacks.

Radu is trying (at my direction) to use it in that webspeed project. If it exists anywhere else, it is not being used as far as I know.

The bottom line: it needs work to make it generic and to get it used in a wider range of projects. There is no migration needed right now.