Logging¶

• Logging
  • Introduction
  • FWD Logging
    • Interference with External Java Application logging
    • Regular Process Logging
  • 4GL Logging
    • LOG-MANAGER (-clientlog)
    • Output Redirection

Introduction¶

Logging in FWD can come from sources that correspond with logging in the 4GL language or the logging can be something specific to the FWD implementation. The 4GL language features include LOG-MANAGER or output to a redirected file. These are things where the logging is intended to largely be compatible with the logging that would be seen in the original 4GL environment. The FWD-specific logging is usually for things that don't have a counterpart in the 4GL.

See also Logging Configuration and Logging.

FWD Logging¶

Interference with External Java Application logging¶

Slf4j is a library providing api for logging. It's often used by Java libraries to allow the freedom of choosing a concrete implementation with its own configurations and integrations. Some popular implementations include logback-classic, log4j-slf4j-impl, slf4j-simple, slf4j-jdk14. Often these libraries are included in projects by frameworks or other libraries. If multiple slf4j implementations are in the classpath, the JVM binds to only one of them.

FWD has its own implementation of slf4j api, that allows 3rd party libs used in the framework to output to FWD's logs. The binding can be found in fwd-slf4j.jar, while the concrete implementation classes are part of p2j.jar.

If a Java application deployed in the same JVM with the FWD framework already has logging solution in place, there is a possibility that it provides its own slf4j implementation and causes conflicts with FWD's implementation, since both are running in the same JVM with the same classpath. If this occurs, the most probable outcome is that slf4j successfully binds to one of the implementations and outputs all 3rd party lib logs to either the FWD log files, or the external application's logs.

It was observed though that it may cause more critical issues with dependency injection frameworks, including the Spring framework. When Spring picks up FWD's implementation it will try to instantiate new logging beans in its own context, but this will lead to an IllegalStateException.

Solutions:

• If no framework is used and the Java application is not mature or has no logging solution implemented, then preferably use the CentralLogger provided by FWD's framework and remove any other slf4j implementation libraries from the deployment (including any nested dependencies).
• If a dependency injection framework is already in place, then remove fwd-slf4j.jar from the classpath (often this is the deploy/lib/ directory) after a fresh deployment. This will likely cause FWD's 3rd party libs to output to the framework's log files.

Regular Process Logging¶

Each server and client process in FWD produces a corresponding log file. This feature uses a custom logger com.goldencode.p2j.util.logging.CentralLogger based on the Java class java.util.logging.FileHandler. The implementation relies on server-client communication and requires security/acl/net configuration in directory:

  <node class="container" name="">
    <node class="container" name="security">
      <node class="container" name="acl">
        <node class="container" name="net">
          <node class="container" name="002400">
            <node class="strings" name="subjects">
              <node-attribute name="values" value="all_others"/>
            </node>
            <node class="netRights" name="rights">
              <node-attribute name="permissions" value="'0101'B"/>
            </node>
            <node class="resource" name="resource-instance">
              <node-attribute name="reference" value="com.goldencode.p2j.util.logging.CentralLogService"/>
              <node-attribute name="reftype" value="TRUE"/>
            </node>
          </node>
        </node>
      </node>
    </node>
  </node>

Any other logging configs are optional and can be omitted, because there are defaults. Any single attribute of the following defaults can be added to directory by itself to customize the behavior:

  <node class="container" name="">
    <node class="container" name="server">
      <node class="container" name="default">
        <node class="container" name="logging">
          <node class="boolean" name="serverSide">
            <node-attribute name="value" value="FALSE"/>
          </node>
          <node class="container" name="file">
            <node class="string" name="path">
              <node-attribute name="value" value="fwd_%m_%tY%tm%td_%tH%tM%tS_%uos_%pid_%uf_%as_%g.log"/>
            </node>
            <node class="integer" name="rotationLimit">
              <node-attribute name="value" value="500000"/>
            </node>
            <node class="integer" name="rotationCount">
              <node-attribute name="value" value="120"/>
            </node>
          </node>
          <node class="container" name="loggers">
            <node class="string" name="root">
              <node-attribute name="value" value="INFO"/>
            </node>
            <node class="string" name="spawner">
              <node-attribute name="value" value="WARNING"/>
            </node>
          </node>
        </node>
      </node>
    </node>
  </node>

Option ID	Data Type	Default Value	Required	Details
file/path	string	fwd_%m_%tY%tm%td_%tH%tM%tS_%uos_%pid_%uf_%as_%g.log	No	The file path as pattern. By default it's used by both server and client processes, so unique placeholders/specifiers are required. Supports custom placeholders and Java formatting specifiers. See below. The value can be absolute or relative path. Absolute paths are resolved on the host so make sure they work for client hosts as well. Relative paths are relative to the launch process dir, where server or client driver is started. When logging/serverSide is TRUE relative paths are resolved only relative to the server launch dir. If the path doesn't exist, the process will try to create it, so dynamic folders can be used, e.g. based on date %tY/%tm/%td/fwd_%m_%pid_%g.log.
file/rotationLimit	integer	500000	No	Maximum file size for each log file in bytes for the file handler. By default on reaching ~500 kilobytes the log file gets rotated. The minimum is 500000 similar to OE.
file/rotationCount	integer	120	No	Number of log file generations for the file handler. Keeping the number high will make sure no logs are overwritten. Take disk space limitation and root logger level into consideration when configuring.
loggers/<logger_name>	string	loggers/root=INFO, loggers/spawner=WARNING	No	The logging level for any arbitrary logger name (in a hierarchical namespace). This sets the level for the source of a log message. That means that the class that is logging will only log messages at that level (and all less verbose levels). By convention, the namespace is based on the package/class name hierarchy. This is not required since any arbitrary text can be inserted. However, FWD does use the package and optionally the class name when creating loggers. If these loggers have a name that begins with at least one of the specified logging levels in this section, the most specific (longest) matching name will specify the logging level for that logger. There may be any number of these entries. As a result, one may specify the logging level for a specific package (and by default all contained classes and sub-packages and their classes). In addition, a more specific class or package name will override the "parent" logging level with the one specified in that more specific entry.
serverSide	boolean	FALSE	No	Determines if client processes create log files on the server or on the client side. By default client processes create log files themselves on the client side.

Custom placeholders

• %pid - process id (for server and clients);
• %uf - FWD user (for clients);
• %uos - OS user (for clients);
• %as - appserver name (for clients);
• %m - mode - server/client;

Supports complete java.util.logging.FileHandler specifiers:

A pattern consists of a string that includes the following special components that will be replaced at runtime:

    "/" the local pathname separator
    "%t" the system temporary directory
    "%h" the value of the "user.home" system property
    "%g" the generation number to distinguish rotated logs
    "%u" a unique number to resolve conflicts
    "%%" translates to a single percent sign "%" 

The sequence number for rotation is specified by %g. A limitation of java.util.logging.FileHandler is that it cannot be formatted.

Supports java.util.Formatter date/time specifiers (see the example in the default config):

 The following date and time conversion suffix characters are defined for the 't' conversion. The types are similar to but not completely identical to those defined by GNU date and POSIX strftime(3c).

The following conversion characters are used for formatting times:
'H'     Hour of the day for the 24-hour clock, formatted as two digits with a leading zero as necessary i.e. 00 - 23.
'I'     Hour for the 12-hour clock, formatted as two digits with a leading zero as necessary, i.e. 01 - 12.
'k'     Hour of the day for the 24-hour clock, i.e. 0 - 23.
'l'     Hour for the 12-hour clock, i.e. 1 - 12.
'M'     Minute within the hour formatted as two digits with a leading zero as necessary, i.e. 00 - 59.
'S'     Seconds within the minute, formatted as two digits with a leading zero as necessary, i.e. 00 - 60 ("60" is a special value required to support leap seconds).
'L'     Millisecond within the second formatted as three digits with leading zeros as necessary, i.e. 000 - 999.
'N'     Nanosecond within the second, formatted as nine digits with leading zeros as necessary, i.e. 000000000 - 999999999.
'p'     Locale-specific morning or afternoon marker in lower case, e.g."am" or "pm". Use of the conversion prefix 'T' forces this output to upper case.
'z'     RFC 822 style numeric time zone offset from GMT, e.g. -0800. This value will be adjusted as necessary for Daylight Saving Time. For long, Long, and Date the time zone used is the default time zone for this instance of the Java virtual machine.
'Z'     A string representing the abbreviation for the time zone. This value will be adjusted as necessary for Daylight Saving Time. For long, Long, and Date the time zone used is the default time zone for this instance of the Java virtual machine. The Formatter's locale will supersede the locale of the argument (if any).
's'     Seconds since the beginning of the epoch starting at 1 January 1970 00:00:00 UTC, i.e. Long.MIN_VALUE/1000 to Long.MAX_VALUE/1000.
'Q'     Milliseconds since the beginning of the epoch starting at 1 January 1970 00:00:00 UTC, i.e. Long.MIN_VALUE to Long.MAX_VALUE.

The following conversion characters are used for formatting dates:
'B'     Locale-specific full month name, e.g. "January", "February".
'b'     Locale-specific abbreviated month name, e.g. "Jan", "Feb".
'h'     Same as 'b'.
'A'     Locale-specific full name of the day of the week, e.g. "Sunday", "Monday" 
'a'     Locale-specific short name of the day of the week, e.g. "Sun", "Mon" 
'C'     Four-digit year divided by 100, formatted as two digits with leading zero as necessary, i.e. 00 - 99
'Y'     Year, formatted as at least four digits with leading zeros as necessary, e.g. 0092 equals 92 CE for the Gregorian calendar.
'y'     Last two digits of the year, formatted with leading zeros as necessary, i.e. 00 - 99.
'j'     Day of year, formatted as three digits with leading zeros as necessary, e.g. 001 - 366 for the Gregorian calendar.
'm'     Month, formatted as two digits with leading zeros as necessary, i.e. 01 - 13.
'd'     Day of month, formatted as two digits with leading zeros as necessary, i.e. 01 - 31
'e'     Day of month, formatted as two digits, i.e. 1 - 31.

Any characters not explicitly defined as date/time conversion suffixes are illegal and are reserved for future extensions. 

4GL Logging¶

LOG-MANAGER (-clientlog)¶

The base functionality of the 4GL handle LOG-MANAGER is supported in FWD. 4GL errors and MESSAGE VIEW AS ALERT-BOX statements are logged by default (with logging level 1), when LOG-MANAGER is enabled. Logging level 1 also supports custom writes with LOG-MANAGER:WRITE-MESSAGE(character).

LOG-MANAGER logging is disabled by default for all processes except appservers, replicating the original 4GL behavior. The default file path for appservers is defined by the format legacy_client_%tY%tm%td_%tH%tM%tS_%uos_%pid_%as.log that gets resolved to legacy_client_20230721_174120_osuser_142656_appservername.000001.log. The sequence number is automatically added to the file name before the extension.

Logging can be enabled by defining client:cmd-line-option:clientlog in client or directory configs or programmatically in 4GL code by using the LOG-MANAGER:LOGFILE-NAME handle. The specified directory should exist!

Here is the full list of configs:

Startup param	Programmatic getter	Programmatic setter	Description	Standalone clients - client bootstrap	Web and scheduled clients - directory.xml
-clientlog	LOG-MANAGER:LOGFILE-NAME	LOG-MANAGER:LOGFILE-NAME	Absolute or relative path to log file. Enables logging.	client:cmd-line-option:clientlog	clientConfig/cfgOverrides
-logginglevel	LOG-MANAGER:LOGGING-LEVEL	LOG-MANAGER:LOGGING-LEVEL	0 off; 1 errors; 2 (default sometimes) basic; 3 verbose; 4 extended	client:cmd-line-option:logginglevel	clientConfig/cfgOverrides
-logthreshold	LOG-MANAGER:LOG-THRESHOLD	none	Enables file rotation. Defines max file size: 0 (default) for no limit; 500,000 - 2,147,483,647 for bytes (488 KB - 2 GB)	client:cmd-line-option:logthreshold	clientConfig/cfgOverrides
-numlogfiles	LOG-MANAGER:NUM-LOG-FILES	none	Max number of log files between 000001 - 999999. Files get deleted, when exceeded.	client:cmd-line-option:numlogfiles	clientConfig/cfgOverrides
-logentrytypes	LOG-MANAGER:LOG-ENTRY-TYPES	LOG-MANAGER:LOG-ENTRY-TYPES	Comma separated list of entry types (optional :logginglevel), example: QryInfo:2,4GLMessages	client:cmd-line-option:logentrytypes	clientConfig/cfgOverrides
-debugalert	SESSION:DEBUG-ALERT	SESSION:DEBUG-ALERT	ABL stack trace for error messages (ABL errors and .NET Exceptions) and Alert-box messages. Affects LOG-MANAGER log level and entry types.	client:cmd-line-option:debugalert	clientConfig/cfgOverrides
-inp	none	none	Max number of input character for a single statement. Default 15000. Max 2147483647.	client:cmd-line-option:inp	clientConfig/cfgOverrides

FWD enhancement of OE functionality is the support of formatting placeholders in the file name specified with clientlog:

• %uos - OS user;
• %uf - FWD user;
• %pid - client process ID;
• %as - appserver name

Supports java.util.Formatter date/time specifiers (check FWD logging configs above):

client:cmd-line-option:debugalert=true adds the 4GL stacktrace and creates a lot more log lines. It can have performance implications.

Rolling log files is disabled by default and can be enabled with startup param logthreshold.

Supported entry types will be 4GLMessages, 4GLTrace, QryInfo, ASDefault, ASPlumbing, DB.Connects.

Example configurations:

• Standalone clients - in client bootstrap (client.xml) add:
  

  <node type="client">
     <client>
          <cmd-line-option clientlog="legacy_client_%uf_%pid_%uos.log" />
     </client>
  </node>
  

• Web processes - in directory config (directory.xml):
  

    <node class="container" name="">
      <node class="container" name="server">
        <node class="container" name="default">
          <node class="container" name="clientConfig">
            <node class="string" name="cfgOverrides">
              <node-attribute name="value" value="client:cmd-line-option:clientlog=legacy_client_web_%uf_%pid_%uos.log"/>
            </node>
          </node>
        </node>
      </node>
    </node>
  

• Appserver and scheduled batch processes - in directory config (directory.xml) in the corresponding container:
  

    <node class="container" name="">
      <node class="container" name="server">
        <node class="container" name="default">
            <node class="container" name="appserver_process">
              <node class="container" name="clientConfig">
                <node class="string" name="cfgOverrides">
                  <node-attribute name="value" value="client:driver:background=true client:cmd-line-option:clientlog=legacy_client_%uf_%pid_%uos_%as.log"/>
                </node>
              </node>
            </node>
        </node>
      </node>
    </node>
  

Server-side logging can be enabled with configurations in the directory. If the node server-side-resources has the value <node-attribute name="value" value="filesystem">, then server-side logging is enabled for that user/process.

As can be seen in the javadoc for Utils.getDirectoryNodeWorker(), there is a hierarchy of lookups that occurs:

1. If a user/process or group node is present:
   1. /server/<serverID>/runtime/<account_or_group>/server-side-resources.<project> (honors the project token, if configured)
   2. /server/<serverID>/runtime/<account_or_group>/server-side-resources
2. If no user/process or group nodes are present, then these server-specific nodes are checked:
   1. /server/<serverID>/runtime/default/server-side-resources.<project> (honors the project token, if configured)
   2. /server/<serverID>/runtime/default/server-side-resources
3. If no server node exists, these are checked (it is the global default area for all servers):
   1. /server/default/runtime/<account_or_group>/server-side-resources.<project> (honors the project token, if configured)
   2. /server/default/runtime/<account_or_group>/server-side-resources
4. Finally, if no user/process or group nodes are present in the global default area, then these are checked:
   1. /server/default/runtime/default/server-side-resources.<project> (honors the project token, if configured)
   2. /server/default/runtime/default/server-side-resources.

Output Redirection¶

Standard output redirection can be used in 4GL with batch processes only and allows clients running without a graphical environment to present the result of their work.

In FWD batch running in terminal / console or started from a bash command can have its output redirected the same way as in OE with > output.txt.

In FWD there is one unique batch type, that is the scheduled batch started by the server. Redirecting its output is required and has the default value of client_stdout_%timestamp%_%userid%_%pid%.log. It can be configured in directory with clientConfig/outputToFile. The value /dev/null is supported to allow explicitly disabling the redirect. The placeholders that can be used in the file name are %timestamp%, %userid%, %pid%.