The following is an AI-assisted plan for rationalizing, updating, and filling out missing content in our persistence-related documentation, which has grown organically over time. There are some existing tasks included and referenced in this plan. I will be breaking out other, new tasks and making individual assignments from this starting point, with this as their parent issue.

If you review and find inaccuracies or missing ideas, or you wish to propose alternative approaches, please post here. Thanks.

Persistence / Database Documentation Overhaul¶

This issue coordinates a comprehensive overhaul of the database/persistence documentation reachable from the Documentation page. It is based on a systematic audit of every in-scope database/persistence wiki page, evaluated on three lenses: (1) explicit staleness/incompleteness markers (obsolescence banners, TODO/TBD, reviewer initials); (2) obsolete Hibernate/HQL content; and (3) version drift against the current supported baseline. It gathers the four "missing content" issues and the four "update existing docs" issues under a single plan, and adds the inline fixes that are not yet tracked by any issue.

Documentation versioning policy¶

This overhaul follows the project-wide documentation policy for handling multiple supported versions. It is stated here because it governs almost every disposition below.

• There is one canonical page per topic. Documentation is not split into per-version pages. Content is assumed common across versions by default.
• Only genuinely version-specific content (currently: content that applies to FWD v3.x but not v4) is called out. Such content stays inline in the canonical page, marked at its opening with the last FWD version it applies to and the last main-trunk revision to which it applies.
• Version-specific sections are collapsible and collapsed by default wherever current (v4) content would otherwise be cluttered by them. The presentation is deliberately biased for v4 readers, who are the large majority; v3 readers expand the relevant sections.
• Version-specific content is not deleted. Older-version material (Hibernate, PL/Java, PostgreSQL 9.x, Java 8, etc.) is retained as marked, collapsed sections so that customers still on v3 remain supported.
• The existing per-version page splits are a departure from this policy and are re-merged as part of this overhaul (see below).
• Sourcing the exact "last applicable main-trunk revision" for each marked section is deferred to the implementation of this overhaul (done by a human maintainer, or by an AI agent with access to the version-control system). This plan only identifies which sections need the marker.
• Internal-only capabilities that are not part of any supported customer release (for example, the internal PL/Java back door used for GCD testing) do not belong in the customer-facing books at all; they move to a private area of Redmine.

Marking and collapsing version-specific content¶

The pattern for a version-specific section, using Redmine's collapse macro (collapsed by default; the label lets a v4 reader skip it at a glance; the first line states precise applicability):

{{collapse(FWD v3.x only - click to expand)
*Applies through FWD v3.x. Superseded in FWD v4 (last applicable main-trunk revision: rNNNNN).*

...content that applies only to v3.x...
}}

rNNNNN is a placeholder filled in during implementation. For a page whose content is wholly v3-specific (no current counterpart to clutter), the same applicability line is placed at the top of the page and the body is left expanded; the collapse wrapper is used only where current content coexists on the page.

Current supported baseline (ground truth for the audit)¶

Two versions are supported today: v3 and v4. v3 is expected to be dropped at the next major release. The current baseline (the default, uncollapsed content) is:

• PostgreSQL 14 and higher; 17 is current/recommended. PostgreSQL 9.x-13 content becomes marked, collapsed v3 material.
• Minimum Java version 17. Java 5-11 content becomes marked, collapsed v3 material.
• Native UDFs. PL/Java is removed from the customer product; customer-facing PL/Java content becomes marked, collapsed v3 material (the internal-only PL/Java back door moves to a private area).
• The FWD ORM (com.goldencode.p2j.persist.orm) and FQL (FWD Query Language), the direct successor to HQL. Hibernate content becomes marked, collapsed v3 material.

Analysis summary¶

The audit covered roughly 60 in-scope pages across five books plus several un-parented (root-level) persistence pages. Problems and their remedies under the versioning policy:

Note: the FWD ORM deliberately reuses many Hibernate class and interface names (with new implementations under com.goldencode.p2j.persist.orm and related packages) precisely to ease the replacement of Hibernate. com.goldencode.p2j.persist.orm.SessionFactory and com.goldencode.p2j.persist.dialect.* are only two examples: these current FWD types are not to be confused with the obsolete Hibernate classes of the same or similar name.

Re-merging the existing version splits (in scope)¶

These pages were split by version in the past; consolidate each into a single canonical page, folding the older content in as a marked, collapsed v3 section:

• Database Configuration v3 into Database Configuration (the v3 hibernate config container becomes the collapsed v3 section).
• String Collation Service Provider Installation (deprecated; Java 8, p2jspi.jar) into H2 String Collation Service Provider Installation.
• Database Server Setup for SQLServer on Windows with CLR UDF (SQL Server 2012 + CLR/IKVM) into Database Server Setup for SQLServer on Windows (SQL Server 2022 current).

After each merge, the old page is retired (redirect/remove) so there is a single canonical page for the topic.

Target state by book¶

The persistence documentation stays within the existing books. Each book should present current (v4) content as its default bulk, with v3 specifics marked and collapsed.

Conversion Reference¶

Target: the schema chapters describe the FWD ORM (annotations, DmoMetadataManager/DmoMeta) as current, with the Hibernate mechanism retained as marked, collapsed v3 content; the runtime chapters describe FQL.

• Chapter 12 (Hibernate Mappings) is wholly v3 (the .hbm.xml mechanism, per #9665 note 2). Mark it at the top with the last applicable version/revision and retain it for v3 users; document the v4 annotation-based approach in the current chapters (Data Model Objects).
• Chapter 7 and Chapter 9: rewrite the common flow around the FWD ORM; fold the Hibernate architecture, UserType and Dialect passages into marked, collapsed v3 sections.
• Chapter 11: document the current DmoMetadataManager/DmoMeta mechanism as default; the old dmo_index.xml/.hbm mechanism becomes a marked, collapsed v3 section (per #9665 note 2).
• Chapter 2: update the grammar for FWD v4; collapse any v3-only grammar as a marked section. Chapter 10: fix the TODO, blank table caption and truncated sentence, and move the Hibernate-mapping / PL/Java-descriptor references into marked, collapsed v3 content.
• Rename HQL to FQL (canonical text) and verify examples in Chapter 13, Chapter 25, Chapter 26 and Chapter 27; complete Chapter 25's unwritten "TBD chapter/section" cross-references; resolve the TODO: ECF review flags (about 13) in Chapter 21; correct the stray Hibernate sentence in Chapter 28.
• Place the currently un-parented core reference pages (see "Orphaned pages" below): promote Data Model Objects and Datasets to numbered chapters here, and relocate QueryExecution to the Internals book (folding any user-facing content into Chapter 25).

Developer Guide¶

Target: post-conversion database development on the current stack; v3-only mechanisms retained as marked, collapsed content.

• Database Access: make the FWD ORM / FQL usage the default content; fold the Hibernate ORM machinery (session lifecycle, import org.hibernate.*, .hbm.xml upkeep, transparent write-behind) into marked, collapsed v3 sections; rename HQL to FQL in the current examples.
• Java Inside the Database (UDFs): the page title is itself a v3 anachronism - the only way FWD ever ran Java inside the database was via PL/Java - so rename the chapter to Built-in Functions Within Database Queries. Much of its content is generic and still applies to v4, but the page needs substantial refactoring: retain the generic material as default content, mark and collapse the PL/Java-specific content as v3, and document current native-UDF authoring as the default. Surface the Database User Defined Functions Overview content within this chapter via an !{{include}} macro rather than cross-referencing it; the page itself remains parented under Database Setup (see "Orphaned pages" below), so the content is authored once and shown in both places. The internal-only PL/Java back door moves to a private area.
• Non-Standard Save Cache Implementation: complete the TIMEZONE/EXTENT annotation TODO items and correct the MariaDB strict-dialect mappings flagged as "not currently correct".

Installation, Configuration and Administration Guide¶

Target: one canonical setup path per supported database, defaulting to PostgreSQL 14-17 / Java 17 / native UDFs, with the v3-era stack retained as marked, collapsed sections.

• Database Server Setup for PostgreSQL on Linux: make PostgreSQL 14-17 + native UDFs the default; convert the existing struck-through PL/Java sections and the PostgreSQL 9.5 / Java 1.8 instructions into properly marked, collapsed v3 sections (per #9659 note 1).
• Database Server Setup for PostgreSQL on Windows: add current PostgreSQL 14-17 / Java 17 content as default; move the PostgreSQL 9.5.15 / Java 1.7-1.8 / PL/Java v1.5.2 content into marked, collapsed v3 sections (currently unmarked).
• Database Server Setup for SQLServer on Windows: current SQL Server 2022 as default; fold the SQL Server 2012 + CLR/IKVM content (re-merged from the CLR UDF page) in as a marked, collapsed v3 section.
• Database Server Setup for SQLServer on Linux: finish the "work in progress" section.
• Database Server Setup for H2: fix the two TODO items (broken import link, obsolete referenced doc); make the current SPI approach default and collapse the Java Extension Mechanism (Java 8) instructions as v3; update the legacy hibernate/url config path to the orm container.
• Chapter 23 Database Checklists: current-baseline checklist as default; the PostgreSQL 9.1.x / PL/Java 1.4.3 / Java 6 / JDK 5 steps become a marked, collapsed v3 section.
• Native UDFs for PostgreSQL: surface the install/reinstall command (currently buried in an ant target) and extend to MariaDB and SQL Server, per #9715 note 2.
• Rewrite the unfinished draft Admin Console Record Locks into finished reference. Record Lock Monitoring is already embedded in the Monitoring chapter via an !{{include}} macro; keep that mechanism, parent it to Monitoring, and add a direct link from Admin Console Record Locks; defer any promotion until its features ship (#3222, #3307).
• Database User Defined Functions Overview is currently unreachable from any book; place it under Database Setup as the UDF section introduction, linking onward to Native UDFs for PostgreSQL and the collapsed v3 PL/Java content. Its content is also surfaced in the Developer Guide's Built-in Functions Within Database Queries chapter via an !{{include}} macro (see Developer Guide above), so the page stays canonical here while appearing in both books.
• Data Migration - Set Logging Properties: the guidance to enable the org.hibernate.SQL/org.hibernate.type loggers is v3; replace it with the current ORM loggers as default and collapse the v3 note.
• Multi-Tenancy: already linked as "Chapter 34" from this book's landing page - set its tree-parent to match; complete the TODO/TBD content ("runtime support not yet written", #6229); relates to #9658.

Directory Configuration Reference (within the Installation Guide)¶

• Database Configuration: complete the inline TODO items (document the new database-connections configuration, relocate the flagged global parameter, finish the "incomplete" SQLServer dialect note); re-merge Database Configuration v3 in as a marked, collapsed v3 section.
• Cache sizes: low priority; only a forward-looking note pending #8281 (CacheManager).

Conversion Handbook / Code Analytics¶

• Complete the blank "Unindexed Queries" section in Conversion Query Reports.
• Database Connections and Standard Code and Schema Reports are current; no action.

Orphaned pages: placement and reachability¶

Several in-scope persistence pages have no parent in the wiki tree. A full inbound-link scan of the entire wiki established how each is currently reachable from Documentation and what its home should be. "Reachable" means a chain of links (or an !{{include}}) that begins at Documentation.

The only genuinely unreachable pages - the must-fix items for navigation - are QueryExecution and Database User Defined Functions Overview. The rest are reachable today; their dispositions above are structural improvements, not repairs.

Missing content - new topics, chapters and pages¶

Driven by the four existing "missing content" issues. Each lands in the book noted, as current (v4) content within the single-canonical-page model.

1. Database import - #9658 (#9658 note 1). Fill the never-documented import topics: runtime-only import, importing _sec-auth* tables, multi-tenant import (with #9093), importing an empty database (#7894), interpreting import output, import performance/threading, encoding, common import errors and fixes, and the full syntax of the raw import command (not just import.sh). Installation Guide, near Data Migration.
2. Managing schema changes over time - #9660 (#9660 note 2). Document how to manually create and apply schema changes until the tooling in #8786 is ready. New chapter in the Installation Guide.
3. Word index support - #9661 (#9661 note 1). New page: support level, how it works, why there is no native full-text search, limitations, and how the extra indexes are re-indexed/managed over time (from #1587). Conversion Reference and/or Installation Guide.
4. Metadata support - #9662 (#9662 note 1, extended in note 2). Document the metadata (_meta) tables at conversion and runtime: p2j.cfg.xml configuration, runtime activation, what is persistent/editable/partial, supported tables and fields, and limitations. Conversion Reference and Developer Guide.

Encoding / collation (issue #9710)¶

Per #9710 note 1, promote Database Collation into its own chapter and complete it (its "UTF-8" section is an empty [[p2j:Database Collation#UTF-8|TODO]]). Cover supported encodings, how to choose one, limitations, OE-vs-modern-DBMS sorting implications, and how to add new encoding support. Consolidate the SPI pages as described under Re-merging above, refreshing the surviving H2 String Collation Service Provider Installation examples from Java 8 to Java 17 (default), with the Java 8 approach retained as a marked, collapsed v3 section.

Cross-cutting cleanups (apply throughout the persistence docs)¶

• Hibernate: make the FWD ORM the default content everywhere; retain Hibernate specifics only as marked, collapsed v3 sections. Do not delete.
• PL/Java: make native UDFs the default; retain customer-facing PL/Java as marked, collapsed v3 sections; move the internal-only back door to a private area.
• Versions: make PostgreSQL 14-17 and Java 17 the default; retain PostgreSQL 9.x / Java 8 details as marked, collapsed v3 sections.
• Terminology: rename HQL to FQL in current text and verify each example.
• Consistency: apply the marker + collapse convention uniformly, and re-merge the existing per-version splits.

Inline updates not yet tracked by any issue (create sub-issues)¶

The items below are needed but are not covered by #9658, #9659, #9660, #9661, #9662, #9665, #9710 or #9715. Recommend creating sub-issues (or folding them into this one as a checklist).

Relationship to existing issues¶

• Update existing docs - #9659 (database server install/config), #9665 (schema conversion chapter review), #9710 (encoding/collation), #9715 (native UDF reinstall).
• Add missing content - #9658 (import), #9660 (schema-change management), #9661 (word indexes), #9662 (metadata).
• This issue is the parent that sequences all of the above plus the untracked inline updates, enforces the versioning policy (single canonical page; marked, collapsed v3 sections; re-merge of the existing splits), and applies the cross-cutting cleanups.

I tried to make this the parent task of the existing, related issues, but the edit attempts indicate this is an invalid parent task for those issues, so we'll just leave them as related. I suppose a parent task needs to be created before its children tasks.