Directory¶
The directory is a special database optimized for read access, providing management and querying of configuration information on the application server level. The information contained in the directory is managed by the application server but it's available for reading from both the client and server applications. Client applications do not have direct access to the directory. Queries to the directory are made only by using the directory service that runs inside the application server.
Overview¶
The directory has a hierarchical structure and the default implementation used by FWD Directory Service uses an XML file as data storage back-end. In a conversion project which is organized in the standard manner, this file is located at {PROJECT_ROOT}/deploy/server/directory.xml. However, the location of this file can be overridden in the server's bootstrap configuration. It is possible to specify a relative filename, in which case the path is relative to the working directory of the server ({PROJECT_ROOT}/deploy/server/ in the standard setup. It is also possible to specify an absolute path to the file. Regardless of the location of the file, the server process must have read/write permissions to the file.
Much of the main information stored in the directory is dedicated to security purposes. For example the directory contains sections related to accounts, authentication, access control lists, but there are also configuration related to database and logging.
Definitions¶
A directory is a hierarchy of typed objects. The objects from the directory form a tree. There are no restrictions between the type of a node and the type of its parent node or between the types of the sibilings nodes.
Types of directory objects have names and are predefined. Types define what information can be stored in an instance of the type, what it is made of, what is mandatory or optional and whether nodes of a type may have children or should always remain leaves.
A directory object is a named set of attributes, each with associated value(s). All attributes have unique names which serve as the keys to values and are of predefined primitive data types. Attribute values may be single or multiple. There is also a type of object without any attributes. For attributes which support multiple values, values consist a set, so each value is unique. The order of the values in the set is undefined.
Directory Paths¶
Every directory object is a node in the directory tree and as such, has an ID. Every node can be identified by a string which describes how to find the object in the tree from its root. The path is made of links separated by '/' character like in file system directories. Every link names a node in the subtree of its parent. Sibling objects all have unique names, although the same names can be reused at different levels of the tree. The root object has the ID of "". Here are few examples.
/server /server/default/logging/level /server/default/adminPort /security/config /security/accounts/users/admin
Link names are case-insensitive. That means that object names must differ in more than just string case. Link names can be up to 256 characters in length. Link names may only consist of alpha-numeric characters, dashes, periods and underscores.
Types of Directory Objects¶
The following table shows the primitive types of objects from the directory:
| Primitive Data Type | String representation |
|---|---|
| INTEGER | Decimal number |
| BOOLEAN | “true” and “false” |
| STRING | Character string |
| DOUBLE | Decimal number with a “floating” decimal point |
| BYTEARRAY | Hexadecimal string |
| BITFIELD | String of 0s and 1s, enclosed in single quotes followed by 'B' suffix |
| BITSELECTOR | String of 0s and 1s, enclosed in single quotes followed by 'B' suffix |
| DATE | String formatted as yyyy-mm-dd |
| TIME | String formatted as hh:mm:ss |
Besides these primitive types, an object from the directory can have as type a predefined class from the directory object classes. A directory object class is a named collection of records, one record per attribute, plus some properties defined for the class.
| Attribute record details: | - primitive type of attribute - name of attribute - mandatory vs optional flag - single vs multiple values flag |
| Class properties: | - class name - leaf vs any node flag |
The following table shows the predefined directory object classes organized by categories:
| Category | Class name | Details |
|---|---|---|
| Internal classes | container | The simplest directory object class. The only function of containers is to be a parent to its children objects. Class defines no attributes. |
| terminal | The simplest directory object class. The only function of terminals is to represent a valid object ID. Class defines no attributes. |
|
| metaclass | This class describes directory object classes. |
|
| Mandatory single valued | integer | Named integer value class. Mandatory. |
| boolean | Named boolean value class. Mandatory. |
|
| string | Named string value class. Mandatory. |
|
| bytes | Named bytearray value class. Mandatory. |
|
| Optional single valued | integerOption | Named integer value class. Optional. |
| booleanOption | Named boolean value class. Optional. |
|
| stringOption | Named string value class. Optional. |
|
| bytesOption | Named bytearray value class. Optional. Multivalued. |
|
| Optional multiple valued | integers | Named integer value class. Optional. Multivalued. |
| booleans | Named boolean value class. Optional. Multivalued. |
|
| strings | Named string value class. Optional. Multivalued. |
|
| bytess | Named bytearray value class. Optional. Multivalued. |
|
| Security Accounts Classes | These classes will be detailed in the following chapters where the security related sections of the directory will be presented. |
|
| Security Audit Classes |
||
| ACL Classes |
||
| Miscellaneous Security Classes |
Working with Directory Objects¶
The information from a directory is stored in objects and their attributes. Directory objects are accessible for enumerations, creation, deletion and renaming. Object attributes are accessible for enumerations, reading, writing, creation, deletion and cloning.
The directory service provides a general purpose interface which supplies functions such as reading, writing, enumerating and searching for directory objects. For every call to the directory service, the access rights of the caller are checked to ensure that the caller is allowed to execute the requested operation. This access control function is implemented with a security manager resource plug-in.
Accessing Directory Objects¶
FWD directory is mostly used for information lookups. The following table lists the main operations related to reading configuration information from the directory.
| Operation | Details |
|---|---|
| Enumerating Objects | Any object in the directory can be a parent for zero or more other objects. Enumerating allows for getting an array of children for a given directory object. |
| Searching Directory | Arbitrary directory search is not provided, as it is of no value in FWD environment where the directory tree structure is well defined. |
| Enumerating Attributes | For any given directory object, it is possible to enumerate all attributes. Security policy may limit visibility of attributes. Enumeration always returns only those attributes that are visible to the caller according to the current security context. |
| Retrieving Attribute Values | One can query the attribute name and type, find out the number of values and query any particular value of the attribute. |
Editing Directory Objects¶
The information stored in the directory could be edited. The following table lists the main operations related to writing configuration information to the directory.
| Operation | Details |
|---|---|
| Modifying Attribute Values | This operation is used to set a value of an attribute, set values to all attributes of an object or delete a attribute value. |
| Modifying Objects | This operation is used to create an object providing all mandatory attributes with their values. Also an object could be deleted but only if all it's children were deleted first. |
| Batch Editing | FWD provides a Batch Editing feature that deals with concurrent updates made to the directory. This feature protects the users of the FWD directory from seeing partial updates being made to the directory. |
Note: How to implement/code directory operations in the application server is documented in the FWD Developer Guide.
FWD Directory Features¶
Apart from directory editing, FWD provides features related to live directory reloading and backup.
| Feature | Details |
|---|---|
| Safe Directory Reloading | This feature allows dynamic directory reloading from the backing XML file and, therefore, is available only for configurations with the XML back-end. This is useful for quick manual directory file edits without the need to recycle the running server. |
| Safe Directory Backup | This feature is designed to allow safe directory backups into another XML file and, therefore, is available only for configurations with the XML back-end. This is useful for backing up the directory without the need to recycle the running server. |
Directory Format
The format of the XML file of the directory is centered around nodes having a primitive or predefined data type nodes, nested inside nodes having the container type. All the directory nodes are included inside a <remapper-storage> tag the first one being the root node.
The tags used in the directory XML file are <remapper-storage>, <node> and <node-attribute>. Below is an example of the directory file including the root node.
<remapper-storage>
<node class="container" name="">
...
</node>
</remapper-storage>
Starting with the root node the directory hierarchy is represented by nested node tags. The configuration options are stored in leaf nodes having nested <note-attribute> child nodes holding the values of this options. The intermediate nodes from the root node to the leafs could be seen as categories of options. The main sections of the directory are the security, server and backup sections, this being the name of the nodes having the root node as parent. The following example shows the main sections of the directory.
<remapper-storage>
<node class="container" name="">
<node class="container" name="security">
...
</node>
<node class="container" name="server">
...
</node>
<node class="container" name="backup"/>
</node>
</remapper-storage>
The security section include all configurations related to security and the subsections could be identified in the following example.
<node class="container" name="security"> <node class="container" name="audit"> ... </node> <node class="container" name="acl"> ... </node> <node class="container" name="accounts"> ... </node> <node class="container" name="config"> ... </node> <node class="container" name="certificates"> ... </node> </node>
The server section include subsections with configurations for each server and also a default subsection with default values for all servers, values that can be overwritten in the subsection dedicated to each server. In the following example there are three subsections, one for the default values and other two sections for two defined servers.
<node class="container" name="server">
<node class="container" name="default">
<node class="boolean" name="adminEnabled"> ... </node>
<node class="container" name="chuiApplet"> ... </node>
<node class="integer" name="adminPort"> ... </node>
<node class="string" name="dmoindex"> ... </node>
<node class="container" name="runtime"> ... </node>
<node class="container" name="logging"> ... </node>
</node>
</node>
<node class="container" name="serverA">
...
</node>
<node class="container" name="serverB">
...
</node>
</node>
The properties are stored in leaf nodes along with their values, using the format shown below.
<node class="container" name="server">
<node class="container" name="default">
...
<node class="integer" name="adminPort">
<node-attribute name="value" value="7443"/>
</node>
...
</node>
This is another example also from the server section including options related to logging:
<node class="container" name="server">
<node class="container" name="default">
...
<node class="container" name="logging">
<node class="string" name="level">
<node-attribute name="value" value="WARNING"/>
</node>
<node class="container" name="handlers"> ... </node>
<node class="container" name="loggers"> ... </node>
</node>
</node>
...
</node>
Directory Search Algorithms¶
Many entries in the directory can be specified in different locations and the runtime has a search algorithm which obeys a well defined precedence order to determine which value is found first. Any value can be set for all users on all servers, for all users on a specific server, for specific groups or specific users or any combination of the above. The most specific element found will be the first honored for the given user/group and server.
Account Search Algorithm¶
This algorithm implements a search method that allows values to be found that are account (user or process) specific or group specific within the current server or a global default for all servers.
| Step | Description | Path |
|---|---|---|
| 1 | The algorithm iteratively looks up the directory node under: | /server/<server_id>/runtime/<user_or_process_account>/<option> |
| 2 | If there is no match on the previous step than, for each group the user or process account belongs to, the algorithm iteratively looks up the directory node under: | /server/<server_id>/runtime/<group>/<option> |
| 3 | If no user/process or group nodes are present, then this is checked: | /server/<server_id>/runtime/default/<option> |
| 4 | If no /server/<server_id>/runtime node exists, this is checked (it is the global default area for all servers): |
/server/default/runtime/<user_or_process_account>/<option> |
| 5 | If there is no match on the previous step than, for each group the user or process account belongs to, the algorithm iteratively looks up the directory node under: | /server/default/runtime/<group>/<option> |
| 6 | Finally, if no user/process or group nodes are present in the global default area, then this is checked: | /server/default/runtime/default/<option> |
Server Search Algorithm¶
This algorithm implements a search method that allows values to be found that are specific for the current server or a global default for all servers.
| Step | Description | Path |
|---|---|---|
| 1 | The algorithm looks up after the directory node: | /server/<server_id>/<option> |
| 2 | If there is no match on the previous step, this is checked: | /server/default/<option> |
Mixed Search Algorithm¶
In mixed mode, the search starts with the per-account lookup and, if not found, it continues with the per-server lookup. See the account and the server search algorithm sections for more details.
Command Line Tools¶
FWD offers a few command line utilities with the purpose of directory editing. These tools are useful for making specific changes on the directory using the command line but also for scripts that can automate directory changes, especially when the changes are voluminous. These utilities are designed to run independently of the P2j server application.
Besides these command line utilities, managing the directory could be done via the administration client. The specific features of the administration client related to directory edit are documented in part 4 of this book
WARNING: NEVER edit the file manually/directly while the server is running, make edits using the features of the administration client.
The following table lists the available directory utilities along with their purpose.
| Utility | Details |
|---|---|
| diff | Compare two directory files or branches of the same or different directories |
| copy | Copy the input directory or a branch to a copy point of the output directory |
| remove | Remove a directory branch recursively |
| clear | Remove, conditionally or unconditionally a value from a specific attribute or the whole attribute |
| set | Add or to set a value to the specified attribute |
WARNING: Except the diff, all other utilities change the specified directory file. Please make it a rule to create a backup copy of the file before applying any of the utilities.
Using diff¶
Use diff to compare the two directory files or branches of the directory files. The output can be redirected to a file for analysis. The diff utility compares the contents of the directories, which does not depend on the order of sections in the xml file.
Usage:
java com.goldencode.p2j.directory.DirectoryDiff <parameters>
where <parameters> is a list of parameters with the following syntax:
left-dir.xml right-dir.xml [ path | left-path [ right-path] ]
You have to specify two directory files. With no other parameters, the whole directories will be compared. You can also give a path. Then this directory path will specify a branch of the directories to compare. You can also specify different paths for the first and the second directory and only those branches will be compared.
The output will be self-explainable. The first directory is referred to as the Left directory and the second directory is referred to as the Right directory. This is a sample output:
L '/security/audit/logsize' (integer) attribute 'value' has different value(s): 4048 R '/security/audit/logsize' (integer) attribute 'value' has different value(s): 8096
Using copy¶
Use copy to copy a branch of the input directory to the specified copy point of the output directory. Copy utility does not synchronize directory nodes, however. If the output file contains something, it will be preserved, not removed.
Usage:
java com.goldencode.p2j.directory.DirectoryCopy copy <parameters>
where <parameters> is a list of parameters with the following syntax:
<source-xml> <source-path> <target-xml> <target-path>
All four parameters are required. The target-xml may specify the same file as the source-xml, but be careful to specify the non-overlappping branches in this case!
The target file will be created if it doesn't exist.
Using remove¶
Use remove to remove a branch from a directory.
Usage:
java com.goldencode.p2j.directory.DirectoryCopy remove <parameters>
where <parameters> is a list of parameters with the following syntax:
<target-xml> <target-path>
The meaning of the parameters is obvious.
Using clear¶
Use clear to scan the specified branch of a directory file for all nodes of a specified directory class, and then remove either the whole attribute specified by its name or a matching value of the attribute.
Usage:
java com.goldencode.p2j.directory.DirectoryCopy clear <parameters>
where <parameters> is a list of parameters with the following syntax:
<target-xml> <target-path> node-class attr-name [attr-value]>
The target-xml directory file gets scanned recursively starting from the target-path. All nodes of the node-class are inspected. If attr-value is not specified, the attribute attr-name is removed unconditionally regardless of its current value or values. If the attr-value is specified, then only that value will be removed. If the attribute is multivalued, the other values remain untouched; otherwise the attribute is removed.
Example 1. To clear the protected="false" attribute from all user accounts in test-directory.xml, type:
java com.goldencode.p2j.directory.DirectoryCopy clear test-directory.xml /security/accounts/users user protected false
Example 2. To clear account bogus from all ACLs of the same directory file, type:
java com.goldencode.p2j.directory.DirectoryCopy clear test-directory.xml /security/acl strings values bogus
Example 3. To clear all user accounts from the alias attribute, type:
java com.goldencode.p2j.directory.DirectoryCopy clear test-directory.xml /security/accounts/users user alias
Using set¶
Use set to add a value for an attribute or set the existing value to a given one for all nodes of the specified directory class in a branch of a directory file. When setting the existing attribute, you can specify the index at which the value will be set. This is a conditional operation and it will be a no-op if there are less values than the index implies (index n is valid if there are at least n values).
Usage:
java com.goldencode.p2j.directory.DirectoryCopy set <parameters>
where <parameters> is a list of parameters with the following syntax:
<target-xml> <target-path> node-class attr-name attr-value [value-index]
The target-xml directory file gets scanned recursively starting from the target-path. All nodes of the node-class are inspected. If value-index is not specified, the attribute attr-name gets the attr-value added. If the value-index is specified, then that value will be replaced if there are enough values. If the attribute is multivalued, the other values remain untouched.
Example. To set the enabled="true" attribute for all process accounts, type:
java com.goldencode.p2j.directory.DirectoryCopy clear test-directory.xml /security/accounts/processes process enabled true
Directory implementations¶
The logical view of the directory does not define how the objects are stored. In general, any storage can be used, but some storage types are more convenient then others, because they allow tree structures to be handled easier. Reasonable implementation choices are XML file and LDAP server.
| Implementation | Pros | Cons |
|---|---|---|
| XML file | - easy to implement - very good read performance |
- the directory cannot be easily distributed to another system - update of the large directory may be time consuming |
| LDAP server | - integration with existing directory - update performance depends only on amount of changes, not on size of the directory |
- due to a high variety of LDAP structures, requires a mapping layer - average read performance |
The current implementation is a dedicated XML file for each server. A description of the XML file implementation format could be found in the Directory Format section above.
LDAP Server Implementation¶
The LDAP server implementation is using OpenLDAP. For configuring OpenLDAP as a directory implementation for FWD follow the step by step OpenLDAP configuration guide below.
WARNING: Please note that the LDAP implementation IS NOT FULLLY WORKING at this time.
OpenLDAP Prerequisites¶
Explanation below makes following assumptions:
| No | Prerequisite |
|---|---|
| 1 | OpenLDAP software is installed using standard tools (e.g. SuSE YaST2). |
| 2 | OpenLDAP configuration files are located in the /etc/openldap. |
| 3 | Data files, certificates, sample configurations, etc. are located in the ~/p2j/testcases/test/ldap directory. |
| 4 | Certification Authority (CA) certificate, server certificate and server key file are prepared and named respectively: gcd-root-ca-cert.pem, p2jserver-cert.pem, p2jserver-key.pem. |
| 5 | Currently user is logged in as a regular user. |
| 6 | Samples below assume the current user is 'user' at host 'host'. |
Warning: OpenLDAP does not allow storing certificates and key files (necessary to establish TLS connections) inside the LDAP directory. They must be stored in files accessible to the OpenLDAP server at startup. In order to allow unattended startup/shutdown of the OpenLDAP server, the key file must NOT be password protected (encrypted). The protection of these sensitive files must be provided at the system level, by assigning appropriate permissions.
OpenLDAP Preliminary Steps¶
| No | Step |
|---|---|
| 1 | Sample slapd.conf must be updated to match desired configuration:Entries below should contain desired domain name. Sample uses goldencode.com: ...
suffix "dc=goldencode,dc=com"
rootdn "cn=Manager,dc=goldencode,dc=com"
...
Also, in some cases following entry must be uncommented and changed to match real certificate: ...
# Uncomment and change to match actual certificate CN:
#sasl-regexp
# cn=p2j\ server,ou=it\ operations,o=golden\ code\ development\ corporation,l=atlanta,st=georgia,c=us
# cn=Manager,dc=goldencode,dc=com
...
The purpose of this expression is to let OpenLDAP assign rights of one CN to the other CN. In particular this might be necessary to provide read/write access to the FWD server when certificate with CN which does not match DN in the directory is used. Without it connected client will have only guest level rights. Refer to OpenLDAP documentation for mode details about writing sasl-regexp expressions, rights assignment and other administrative tasks. Note 1: slapd converts all data from CN of the certificate to lower case before performing match.Note 2: In some cases it might be helpful to comment out the statement, restart slapd in debug mode (by specifying -d -1 in command line) and try to connect to LDAP server using FWD server (or other utility such as StartupDirectory or DirectoryCopy) and watch certificate CN in the debug output. Then found CN just copy as is into the slapd.conf and insert back slash before each space (see example above). |
| 2 | Initialization file init.ldif must be updated to match actual domain name, organization name, etc. |
| 3 | Script file initDir must be updated to use correct administrative DN (most likely - one listed in the slapd.conf as rootdn). |
| 4 | LdapMapGen configuration file mapping.xml, server configuration files (required for the use of StartupDirectory, DirectoryCopy, and so on) must be updated to include proper LDAP parameters: URL, credentials, location of the mapping data (if necessary), etc. |
OpenLDAP Main Steps¶
| No | Step |
|---|---|
| 1 | Login as rootuser@host:~>su - root Password:***** host:~ # |
| 2 | Change directory to /p2j/testcases/test/ldap:host:~ # cd /p2j/testcases/test/ldap host:/p2j/testcases/test/ldap # |
| 3 | Create subdirectory:host:/p2j/testcases/test/ldap # mkdir /etc/openldap/pem host:/p2j/testcases/test/ldap # |
| 4 | Copy files:host:/p2j/testcases/test/ldap # cp *.pem /etc/openldap/pem host:/p2j/testcases/test/ldap # cp gcd.schema /etc/openldap/schema host:/p2j/testcases/test/ldap # cp slapd.conf /etc/openldap |
| 5 | Start LDAP server:host:/p2j/testcases/test/ldap # slapd -h "ldap:/// ldaps:///" At this point OpenLDAP may request password for the PEM file if server key file is password protected: Enter PEM pass phrase: host:/p2j/testcases/test/ldap # |
| 6 | Load initial LDAP configuration:host:/p2j/testcases/test/ldap # ./initDir Enter LDAP Password: ... host:/p2j/testcases/test/ldap # exit user@host:~> At this point OpenLDAP is completely configured and ready to load mapping data. |
| 7 | Loading mapping data:user@host:~> cd /p2j/testcases/test/ldap user@host:/p2j/testcases/test/ldap> java com.goldencode.p2j.directory.LdapMapGen mapping.xml -u Writing schema mapping to cn=mapping Total 71 mappings written Done. user@host:/p2j/testcases/test/ldap> At this point directory is ready to load initial data with StartupDirectory or DirectoryCopy utilities. |
© 2004-2017 Golden Code Development Corporation. ALL RIGHTS RESERVED.