Project

General

Profile

Bug #11796

Improve database export tool

Added by Ovidiu Maxiniuc 7 days ago. Updated 7 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
Due date:
% Done:

0%

billable:
No
vendor_id:
GCD
case_num:
version_reported:
version_resolved:
reviewer:
production:
No
env_name:
topics:

History

#1 Updated by Ovidiu Maxiniuc 7 days ago

  • reviewer deleted (Constantin Asofiei, Eric Faulhaber, Greg Shah)

The current implementation of ExportWorker is highly automated and tries to the directory to get the connection details for a configure-less execution. It also supports some command-parameters and reads from environment (System.getProperty()) additional optional parameters.

But there are a few areas in which it may be improved:
  • tenants: currently, dumping data for the tenants is supported more like a workaround. If the correct database connection is provided, the tool will dump the tenant-private database. A better way would be to use the landlord database and read from there the necessary connection data in a less configured solution;
  • ported meta tables. While the database is converted, some tables from _meta (standard.df) databases (whose content requires to be persisted) are moved to permanent database (in case of tenants, this is the shared database). Because the export tool uses DatabaseManager.getDmosMeta() for collecting the list of tables to be exported as .d files, these tables are skipped. The reason is that the method will use reflection to obtain the metadata for the DMO interfaces of tables from the base package. But, in case of these tables (currently they are _User, _Sec-Authentication-Domain and _Sec-Authentication-System), their DMO interfaces keep the path in _meta package, therefore will not be found while scanning the permanent package.

#2 Updated by Greg Shah 7 days ago

Do we have the option to configre everything on the command line so that this can be executed without a running FWD server or access to the directory?

#3 Updated by Ovidiu Maxiniuc 7 days ago

I am sorry, I must admit I was tricked by existing run configuration and assumed all the connection info was read from the only possible location: the directory.xml. In fact, that is the case only for a single process parameter.

The connection parameters are, as I actually reported during the stand-up, from a mix of command-line and environment. However, since the latte settings can be specified using -D syntax, ultimately, then can all be part of the command-line.

Here is a summary of all values that can be configured for this utility:
  • the following values are read from environment (using System.getProperty): connection.url, database_name, connection.password, connection.username, dmo_package, file.separator;
  • the following parameters are hardcoded: c3p0.maxStatementsPerConnection = 100, c3p0.minPoolSize = 4, c3p0.maxPoolSize = 20, c3p0.acquireIncrement = 2, CP_MAX_IDLE_TIME = 900, cache.use_second_level_cache = false and jdbc.batch_size = 1024;
  • from command line, as normal parameters the following are read: maxThreads (default: 4), exportPath (default: export), lobsPath (default: lobs), and encoding (no default, if present it overwrites cpstream from p2j.cfg.xml);
  • from cfg/p2j.cfg.xml, cpstream, numformat and dateformat for processed schema/namespace;
  • only the persistence/primary-key-name is read from directory.
Conclusions:
  • the application runtime is configured from (probably too) many places;
  • it can be more self-configuring, as described in my previous note. If no parameters specified, it could read all necessary data as the FWD server does, including spawning threads for dumping tenants using landlord database for connection details;
  • solution for overwriting in command line for all the above values should be implemented for maximum versatility. For example:
    • exporting only private data of a specific tenant;
    • executing the export of a database based on credentials/parameters without having a FWD server installation.

Also available in: Atom PDF