FWD » Build and Source Control

From Marian:

I know you've made a task for ABLUnit support, I did thought the issue is related to the custom JUnit engine somehow but it turns out we can not even get the `testcases` project to execute even simple ui screens - outside of the unit test framework.

What I've tried so far, with only limited success :(

• setup development environment for everyone on Windows
• we're using Eclipse as IDE - actually the Progress Developer Studio 12.2
• java 11
• normally I would like to use the 6410a branch since this is what I'm currently working on, I'm trying now with the 'trunk' to see if that works better
• building fwd (gradlew all) works, Gradle dependencies are not added to Eclipse classpath because of custom configurations but we add the jars from build/lib folder to the classpath afterwards manually
• install spawner inside the testcase project using 6410a as p2j works fine
• for java 11 I've updated the build.xml to add the libs from the `spi` folder to the app classpath
• had to update the scripts to remove the use of external libraries 'java.ext.dirs', added the spi lib in classpath

With that setup we can only start the server if we comment-out the `tstcases` database from the persistence section of `directory.xml`, it starts with the database entry but then on each client connection the client crash due to an sql exception saying the 'user_meta' table isn't found:

com.goldencode.p2j.persist.PersistenceException: Failed to execute [SELECT COUNT(*) FROM meta_user]
Caused by: java.sql.SQLException: A problem occurred while trying to acquire a cached PreparedStatement in a background thread.
at com.mchange.v2.c3p0.stmt.GooGooStatementCache.acquireStatement(GooGooStatementCache.java:571)
at com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkoutStatement(GooGooStatementCache.java:204)
at com.mchange.v2.c3p0.impl.NewPooledConnection.checkoutStatement(NewPooledConnection.java:321)
at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:553)
at com.goldencode.p2j.persist.Persistence.getSingleSQLResult(Persistence.java:2734)
at com.goldencode.p2j.persist.ConnectionManager.getAuthLevel(ConnectionManager.java:2111)
at com.goldencode.p2j.util.MetadataSecurityOps.getAuthLevel(MetadataSecurityOps.java:323)
at com.goldencode.p2j.util.SecurityOps.getDefaultUserid(SecurityOps.java:435)
at com.goldencode.p2j.persist.ConnectionManager.putConnected(ConnectionManager.java:4165)
at com.goldencode.p2j.persist.ConnectionManager.makeConnection(ConnectionManager.java:3630)
at com.goldencode.p2j.persist.ConnectionManager.connect(ConnectionManager.java:3422)

I did run `ant create.db` and `ant import.db` and it does say one record was created in that table so I'm kinda out of idea as of why is failing to find the table afterwards.

Anyway, if we comment out the database to run tests that do not need any database connection the server starts fine, we can even run a simple `hello world` procedure with both chui/gui clients - the only real issue there is the fact that the mouse seems not to be usable, we need to use the keyboard to close that alert-box dialog - mouse click doesn't do anything :(

For the ABLUnit engine, we've could convert some test cases and we can start the engine with a simple JUnit run configuration from within Eclipse (found that on the page edited by Vladimir) so this is great when it works - on Windows in the setup I've listed before running the tests with the 'chui' client kinda works - aka it runs all the tests and we can see the results, for 'gui' client on the other hand the JVM crash:

# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000065ac1bd8, pid=8816, tid=4144
#
# JRE version: Java(TM) SE Runtime Environment 18.9 (11.0.11+9) (build 11.0.11+9-LTS-194)
# Java VM: Java HotSpot(TM) 64-Bit Server VM 18.9 (11.0.11+9-LTS-194, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C  [p2j.dll+0x1bd8]

Current thread (0x000001a6635ae000):  JavaThread "main" [_thread_in_native, id=5260, stack(0x0000002473a00000,0x0000002473b00000)]

Stack: [0x0000002473a00000,0x0000002473b00000],  sp=0x0000002473afe4f0,  free space=1017k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [p2j.dll+0x1bd8]
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  com.goldencode.p2j.ui.client.chui.driver.console.ConsoleHelper.resume()V+0
j  com.goldencode.p2j.testengine.FWDEngineDescriptor.prepare(Lcom/goldencode/p2j/testengine/FWDEngineExecutionContext;)Lcom/goldencode/p2j/testengine/FWDEngineExecutionContext;+3
j  com.goldencode.p2j.testengine.AbstractFWDTestDescriptor.prepare(Lorg/junit/platform/engine/support/hierarchical/EngineExecutionContext;)Lorg/junit/platform/engine/support/hierarchical/EngineExecutionContext;+5
j  org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$prepare$2()V+9
j  org.junit.platform.engine.support.hierarchical.NodeTestTask$$Lambda$693.execute()V+4
j  com.goldencode.p2j.testengine.FWDTestEngine$FWDThrowableCollector.execute(Lorg/junit/platform/engine/support/hierarchical/ThrowableCollector$Executable;)V+1

I would like to get back the `testcases` project into a state where we can start the server/client and optionally be able to run unit tests on it since all the tests we have now are built as ABLUnit tests. It's possible the configuration files in the project are outdated and there are things there that are missing or are simply wrong - it will be great if someone can help us with this. If Java 11 support is not complete and we need to use Java 1.8 we can install that as an alternative, anything that can help us move forward with this and get into a stable environment would be great :(

Eric/Ovidiu can help with the persistence issue.

About the GUI failure in native code, I think that is the same GUI process launching issue as #7875. That still needs to be fixed.

In regard to Java 11, see Switching Between Java 8 and Java 11.

See revision 1480 which revamps the cfg/p2j.cfg.xml, build.xml, and build_db.xml to utilize up-to-date standards (similar to hotel_gui). Also, renamed the data/dump/tstcasedata directory to data/dump/tstcasesdb so that the sample configuration will import.

Greg Shah wrote:

Eric/Ovidiu can help with the persistence issue.

About the GUI failure in native code, I think that is the same GUI process launching issue as #7875. That still needs to be fixed.

When I logged that issue, I was only looking at GUI on Unix and ncurses was in the middle of it. We don't use ncurses on Windows, but perhaps there is still a character mode operation being performed in a Win environment. See around note #7676-39. I

Marian,

From the description of this thread I understand that you have configured PostgreSQL, initialized the/a cluster, created the database and installed the schema for tstcases database.

In case you need more details here, please find additional details in Database Server Setup for PostgreSQL on Windows and Native UDFs for PostgreSQL.

I cannot give you an exact answer regarding the exception above; this is my first encounter with it. It is generated in c3p0 which caches statements for a performance boost. I think the stack is incomplete. The root cause of the problem is probably listed next, like this:

Caused by: java.sql.SQLException: <root cause>
<continued stack>

Please confirm the existence of it and, if the case, publish it in this thread.

On the other topic: you cannot use mouse with the ChUI client. It does not receive mouse events. These are captured at a different level and the only operations possible are related to clipboard and design (accessible through right-click).

Greg Shah wrote:

Eric/Ovidiu can help with the persistence issue.

About the GUI failure in native code, I think that is the same GUI process launching issue as #7875. That still needs to be fixed.

We're not launching any process from those tests - no `os-command`, `unix`, `dos` or anything like that used.

Greg Shah wrote:

In regard to Java 11, see Switching Between Java 8 and Java 11.

Yes, more or less what we did - both the server/client processes are starting fine, just thought the VM core could be related to the fact that we run on Java 11, will try with 1.8 today to see if this is the case or is something else.

Marian Edu wrote:

Ovidiu Maxiniuc wrote:

From the description of this thread I understand that you have configured PostgreSQL, initialized the/a cluster, created the database and installed the schema for tstcases database.

Ovidiu, for our purposes the current H2 setup should be enough and we don't need to switch to PostgreSQL for testcases - at least not in our Windows based test environment.

I concur about H2 staying the default. For switching temporarily to PostgreSQL: it's scripted, see:
testcases_h2_to_pg.sh at the root of the testcases project.

Roger Borrello wrote:

See revision 1480 which revamps the cfg/p2j.cfg.xml, build.xml, and build_db.xml to utilize up-to-date standards (similar to hotel_gui). Also, renamed the data/dump/tstcasedata directory to data/dump/tstcasesdb so that the sample configuration will import.

Thanks Roger, did update to that revision and gave it a try:
- I presume the references to `tstcasedata` in `directory.xml.template` should also be updated to `tstcasesdb`
- had to take out the `${deploy.home}/lib` from app.classpath in `build.xml` as is probably the same as `${fwd.lib.home}`, otherwise creade.db/import.db task were failing because of very long command line (classpath)
- from `directory.xml` (template) had to take out the `osusers` container all together and the `ssoplugin` and `ssopluginoption` node attributes from the `auth-mode` in order to start the server, those might require a newer FWD revision than 6410a branch I'm using???

Now, when I try `ant create.db` it doesn't give any error but it also doesn't seem to do anything - `ant import.db` on the other hand it give this:

import.db.h2:
     [java] Using unnamed schema profile.
     [java] Failed to locate 'data/standard.dict' as a resource. Using developer mode: attempting to locate it as local file in ${cvtpath} location.Elapsed job time:  00:00:00.245
     [java]
     [java] ERROR:
     [java] com.goldencode.ast.AstException: Bogus registry resource .\cfg\registry.xml
     [java]     at com.goldencode.ast.XmlFilePlugin.<init>(XmlFilePlugin.java:232)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.lambda$static$0(PatternEngine.java:523)
     [java]     at com.goldencode.ast.AstManager.getPlugin(AstManager.java:646)
     [java]     at com.goldencode.ast.AstManager.save(AstManager.java:633)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.finish(PatternEngine.java:1250)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.java:1114)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.main(PatternEngine.java:2219)

I can see the .dict files in `cvt\data` folder, `standard.dict` is right there.

Running conversion it does look like the testcasesdb.df is being imported - there were some errors about `LOB-SIZE` in that definition file having an unknown (?) value while a number was expected, however removing the offending lines although it makes the errors on conversion go away it doesn't solve the issue we have with the database connection.

It's basically the same error like before when an select is ran against `meta_user` table, this is what I can see in the server log:

23/10/25 11:42:44.661+0300 |    INFO | com.goldencode.p2j.persist.DatabaseManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Connecting to [tstcasesdb]
23/10/25 11:42:44.706+0300 | WARNING | com.goldencode.p2j.persist.orm.PooledDataSourceProvider | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Can't find a setter for the specified c3p0 parameter c3p0.validate
23/10/25 11:42:44.736+0300 |    INFO | com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 2, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, dataSourceName -> 1hge31gazsatuxn1x5m0id|74e79bc0, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> org.h2.Driver, extensions -> {}, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, identityToken -> 1hge31gazsatuxn1x5m0id|74e79bc0, idleConnectionTestPeriod -> 0, initialPoolSize -> 3, jdbcUrl -> jdbc:h2:../db/tstcasesdb;DB_CLOSE_DELAY=-1;MV_STORE=FALSE, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 0, maxIdleTime -> 900, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 20, maxStatements -> 0, maxStatementsPerConnection -> 100, minPoolSize -> 4, numHelperThreads -> 3, preferredTestQuery -> null, privilegeSpawnedThreads -> false, properties -> {password=******, user=******}, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ]
23/10/25 11:42:44.741+0300 | WARNING | com.mchange.v2.resourcepool.BasicResourcePool | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Bad pool size config, start 3 < min 4. Using 4 as start.
23/10/25 11:42:47.001+0300 |    INFO | com.goldencode.p2j.persist.Persistence | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | UDF version for db local/tstcasesdb/primary is FWD version undefined; UDFs style: Java; search_path: [unknown]
23/10/25 11:42:47.043+0300 | WARNING | com.goldencode.p2j.persist.orm.Settings | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Duplicate database Settings for local/tstcasesdb/dirty
23/10/25 11:42:47.062+0300 |    INFO | com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Initializing c3p0 pool... com.mchange.v2.c3p0.ComboPooledDataSource [ acquireIncrement -> 3, acquireRetryAttempts -> 30, acquireRetryDelay -> 1000, autoCommitOnClose -> false, automaticTestTable -> null, breakAfterAcquireFailure -> false, checkoutTimeout -> 0, connectionCustomizerClassName -> null, connectionTesterClassName -> com.mchange.v2.c3p0.impl.DefaultConnectionTester, contextClassLoaderSource -> caller, dataSourceName -> 1hge31gazsatuxn1x5m0id|37bfd906, debugUnreturnedConnectionStackTraces -> false, description -> null, driverClass -> org.h2.Driver, extensions -> {}, factoryClassLocation -> null, forceIgnoreUnresolvedTransactions -> false, forceSynchronousCheckins -> false, forceUseNamedDriverClass -> false, identityToken -> 1hge31gazsatuxn1x5m0id|37bfd906, idleConnectionTestPeriod -> 0, initialPoolSize -> 3, jdbcUrl -> jdbc:h2:mem:tstcasesdb;db_close_delay=-1;mv_store=false;rtrim=true;query_cache_size=1024;db_close_on_exit=false;, maxAdministrativeTaskTime -> 0, maxConnectionAge -> 0, maxIdleTime -> 0, maxIdleTimeExcessConnections -> 0, maxPoolSize -> 15, maxStatements -> 0, maxStatementsPerConnection -> 0, minPoolSize -> 3, numHelperThreads -> 3, preferredTestQuery -> null, privilegeSpawnedThreads -> false, properties -> {password=******, user=******}, propertyCycle -> 0, statementCacheNumDeferredCloseThreads -> 0, testConnectionOnCheckin -> false, testConnectionOnCheckout -> false, unreturnedConnectionTimeout -> 0, userOverrides -> {}, usesTraditionalReflectiveProxies -> false ]
23/10/25 11:42:47.368+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Persisting record for the metatable MetaDb__Impl__
23/10/25 11:42:47.428+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Persisting record for the metatable MetaTrans__Impl__
23/10/25 11:42:47.430+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Persisting record for the metatable MetaStartup__Impl__
23/10/25 11:42:47.433+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populating [_Database-Feature]
23/10/25 11:42:47.457+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populated [_Database-Feature] in 24 ms
23/10/25 11:42:47.458+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populating [_Filelist]
23/10/25 11:42:47.460+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populated [_Filelist] in 1 ms
23/10/25 11:42:47.460+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populating [_Tenant]
23/10/25 11:42:47.460+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populated [_Tenant] in 0 ms
23/10/25 11:42:47.460+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populating [_Area]
23/10/25 11:42:47.465+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populated [_Area] in 5 ms
23/10/25 11:42:47.466+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populating [_Sequence]
23/10/25 11:42:47.466+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | No sequences were detected for tstcasesdb database.
23/10/25 11:42:47.467+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populated [_Sequence] in 1 ms
23/10/25 11:42:47.468+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populating [_File]
23/10/25 11:42:48.000+0300 |    INFO | org.reflections.Reflections | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Reflections took 32 ms to scan 1 urls, producing 11 keys and 27 values 
23/10/25 11:42:48.193+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Populated [_File] in 725 ms
23/10/25 11:42:48.290+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Metaschema database local/tstcasesdb/meta ready
23/10/25 11:42:48.290+0300 |    INFO | com.goldencode.p2j.persist.meta.MetadataManager | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Caching 9 template record ids
23/10/25 11:42:48.303+0300 |  SEVERE | com.goldencode.p2j.persist.Persistence | ThreadName:Conversation [00000001:bogus], Session:00000001, Thread:00000004, User:bogus | Error executing SQL statement:  SELECT COUNT(*) FROM meta_user
com.goldencode.p2j.persist.PersistenceException: Failed to execute [SELECT COUNT(*) FROM meta_user]
Caused by: java.sql.SQLException: A problem occurred while trying to acquire a cached PreparedStatement in a background thread.
    at com.mchange.v2.c3p0.stmt.GooGooStatementCache.acquireStatement(GooGooStatementCache.java:571)
    at com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkoutStatement(GooGooStatementCache.java:204)
    at com.mchange.v2.c3p0.impl.NewPooledConnection.checkoutStatement(NewPooledConnection.java:321)
    at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:553)
    at com.goldencode.p2j.persist.Persistence.getSingleSQLResult(Persistence.java:2734)
    at com.goldencode.p2j.persist.ConnectionManager.getAuthLevel(ConnectionManager.java:2111)
    at com.goldencode.p2j.util.MetadataSecurityOps.getAuthLevel(MetadataSecurityOps.java:323)
    at com.goldencode.p2j.util.SecurityOps.getDefaultUserid(SecurityOps.java:435)
    at com.goldencode.p2j.persist.ConnectionManager.putConnected(ConnectionManager.java:4165)
    at com.goldencode.p2j.persist.ConnectionManager.makeConnection(ConnectionManager.java:3630)
    at com.goldencode.p2j.persist.ConnectionManager.connect(ConnectionManager.java:3422)
    at com.goldencode.p2j.persist.ConnectionManager.connectDbImpl(ConnectionManager.java:3103)
    at com.goldencode.p2j.persist.ConnectionManager.connect_(ConnectionManager.java:680)
    at com.goldencode.p2j.persist.ConnectionManager.connect(ConnectionManager.java:581)
    at com.goldencode.p2j.persist.DatabaseManager.autoConnect(DatabaseManager.java:956)
    at com.goldencode.p2j.main.StandardServer.standardEntry(StandardServer.java:547)
    at com.goldencode.p2j.main.StandardServerMethodAccess.invoke(Unknown Source)
    at com.goldencode.p2j.util.MethodInvoker.invoke(MethodInvoker.java:156)
    at com.goldencode.p2j.net.Dispatcher.processInbound(Dispatcher.java:788)
    at com.goldencode.p2j.net.Conversation.block(Conversation.java:422)
    at com.goldencode.p2j.net.Conversation.run(Conversation.java:235)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Table "META_USER" not found; SQL statement:
SELECT COUNT(*) FROM meta_user [42102-200]

Marian Edu wrote:

Thanks Roger, did update to that revision and gave it a try:
- I presume the references to `tstcasedata` in `directory.xml.template` should also be updated to `tstcasesdb`

I must admit that I have never used this version of the testcases project, except for maybe some years ago. It looked like the configuration was incorrect when I made some changes. It looks like I should have made the DB name 'tstcasedata' instead of 'tstcasesdb' in the default configuration.

- had to take out the `${deploy.home}/lib` from app.classpath in `build.xml` as is probably the same as `${fwd.lib.home}`, otherwise creade.db/import.db task were failing because of very long command line (classpath)

Sorry... I'll take a closer look.

- from `directory.xml` (template) had to take out the `osusers` container all together and the `ssoplugin` and `ssopluginoption` node attributes from the `auth-mode` in order to start the server, those might require a newer FWD revision than 6410a branch I'm using???

I did not take the testcase to the runtime. I can when I try some that

Roger Borrello wrote:

Marian Edu wrote:

Thanks Roger, did update to that revision and gave it a try:
- I presume the references to `tstcasedata` in `directory.xml.template` should also be updated to `tstcasesdb`

I must admit that I have never used this version of the testcases project, except for maybe some years ago. It looked like the configuration was incorrect when I made some changes. It looks like I should have made the DB name 'tstcasedata' instead of 'tstcasesdb' in the default configuration.

I revised my update to take into account the DB name is different from the .df file. Checked into project as 1481.

- had to take out the `${deploy.home}/lib` from app.classpath in `build.xml` as is probably the same as `${fwd.lib.home}`, otherwise creade.db/import.db task were failing because of very long command line (classpath)

Sorry... I'll take a closer look.

I see why this is taking place. Typically I use "deployapp.prepare", which does not copy the FWD jars into the deploy/lib directory, only the application jar (and fwdspi.jar). So to have a complete classpath, I have to include the FWD jars in the classpath so the subsequent usage finds all jars. But the original way of copying all jars into the deploy/lib (FWD and application) leads to a very long classpath when you include both the app.classpath. I don't have a great solution for handling both because the original way is much more typical than the new way. For now, we'll stick with the original way, and remove the explicit inclusion of the FWD jars, and let the deploy.prepare position all the jars.

As an aside, the new way is so much easier to move from version-to-version of FWD. The scripts used to start the application just need to be aware of the configuration, and not just expect all the jars to be in ../lib. The hotel_gui app has been updated, but I'm not going to take on the testcases_v2 at this time, so as to get you going faster.

- from `directory.xml` (template) had to take out the `osusers` container all together and the `ssoplugin` and `ssopluginoption` node attributes from the `auth-mode` in order to start the server, those might require a newer FWD revision than 6410a branch I'm using???

I did not take the testcase to the runtime. I can when I try some that

I cannot find 6410a, just 6410b. I want to make sure I get things working for the environment you require.

Marian Edu wrote:

Now, when I try `ant create.db` it doesn't give any error but it also doesn't seem to do anything - `ant import.db` on the other hand it give this:

import.db.h2:
     [java] Using unnamed schema profile.
     [java] Failed to locate 'data/standard.dict' as a resource. Using developer mode: attempting to locate it as local file in ${cvtpath} location.Elapsed job time:  00:00:00.245
     [java]
     [java] ERROR:
     [java] com.goldencode.ast.AstException: Bogus registry resource .\cfg\registry.xml
     [java]     at com.goldencode.ast.XmlFilePlugin.<init>(XmlFilePlugin.java:232)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.lambda$static$0(PatternEngine.java:523)
     [java]     at com.goldencode.ast.AstManager.getPlugin(AstManager.java:646)
     [java]     at com.goldencode.ast.AstManager.save(AstManager.java:633)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.finish(PatternEngine.java:1250)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.run(PatternEngine.java:1114)
     [java]     at com.goldencode.p2j.pattern.PatternEngine.main(PatternEngine.java:2219)

I can see the .dict files in `cvt\data` folder, `standard.dict` is right there.

The FWD import tool expects the resources to be packed in your deployment jars, so that the deployment/import can be performed without the full conversion artefacts. In the case of standard.dict, after a failed attempt to locate it there as a stream, it will try to load it as a file from the standard conversion location. This is 'developer mode'. It seems that it was successful, otherwise the process would have stopped there with another error message. You confirmed the existence of 'external' resources in the above line.

In the case of registry.xml, the lookup does not include the developer mode, so it is a quick fail. The process stops here, no more file level lookup performed.

However, IF you have built your jar to include these resources (cfg and data) folder, please check whether the correct name of the jar(s) is/are provided to import tool. (The artefacts converted and eventually compiled may be packaged in multiple jars if the application is modular. We also have customers which pack the DMOs in a separate jar. This is convenient for import since only the necessary DMOs classes are packaged along with cfg and data). Note that Java will not complain if an invalid jar name occurs on classpath, but the application will stop is the resource is not found.

The latest revision I checked in (1482) should work out of the box for the post-14783 revision of trunk, so this branch will need a rebase. It will make things much easier.

Out of the box, you should be able to build, import, and run (with the appropriate changes to the directory.xml). My file-cvt-list.txt was:

ask.p
field_io/tests/field_import_utf8.p
field_io/tests/field_import_simple.p
field_io/tests/field_import_mixed_eol.p
field_io/test_fieldio.p
common/put_version_info.p
common/function_procedure_recursive.p
common/abl_procedure_for_tests.p
common/LogIt.cls
common/run_testcase.p
common/AblUnitClass.cls

I added ./ask.p to the project just to have a sample you can run and prompt for other programs. I added client scripts so that you could run as a Swing client from the ./deploy/client directory.

OK, the database connection issue is solved now - I've updated the configuration to use tstcasesdb instead of mixing it with different definition file/data dump so it much clear what is what. The errors on `ant import.db` were indeed coming from the fact that application jar was not build/deployed at the time - `ant convert.list` was creating the data files in `cvt` folder but as Ovidiu pointed out this has to be in the classpath when running `db.import`.

I can now start the server and have some success in running some sample code through the client, however there is definitively something wrong with the code we have (6410b rev #14790) or some configuration again. When I ran a simple 'hello world' procedure in GUI client the output looks a bit odd and the mouse does not work as reported previously, when trying a simple 'window' procedure the client crash and no user interface is shown:
1. It starts with a NPE on `Frame.doLayoutWorker`, the layout manager is null on `Frame:1930`.

java.lang.NullPointerException
    at com.goldencode.p2j.ui.client.Frame.doLayoutWorker(Frame.java:1930)
    at com.goldencode.p2j.ui.client.gui.FrameGuiImpl.doLayoutWorker(FrameGuiImpl.java:707)
    at com.goldencode.p2j.ui.client.Frame.doLayout(Frame.java:1761)
    at com.goldencode.p2j.ui.client.Frame.postprocess(Frame.java:4421)
    at com.goldencode.p2j.ui.client.WidgetRegistry.pushDefinition(WidgetRegistry.java:766)
    at com.goldencode.p2j.ui.chui.ThinClient.lambda$52(ThinClient.java:10982)
    at com.goldencode.p2j.ui.chui.ThinClient.eventBracket(ThinClient.java:18606)
    at com.goldencode.p2j.ui.chui.ThinClient.eventDrawingBracket(ThinClient.java:18548)
    at com.goldencode.p2j.ui.chui.ThinClient.pushOneDef(ThinClient.java:10973)
    at com.goldencode.p2j.ui.chui.ThinClient.pushScreenDefinition(ThinClient.java:10916)

2. Leaving from `doLayoutWorker` if layout manager is null moves the error on `reset` method of a `FillIn` widget (FillIn:2423) where `config.appFormat` is null so calling methods on it leads to NPE:

java.lang.NullPointerException
    at com.goldencode.p2j.ui.client.FillIn.reset(FillIn.java:2423)
    at com.goldencode.p2j.ui.client.Frame.resetAll(Frame.java:5733)
    at com.goldencode.p2j.ui.chui.ThinClient.markFrames(ThinClient.java:14084)
    at com.goldencode.p2j.ui.chui.ThinClient.applyChanges(ThinClient.java:16547)
    at com.goldencode.p2j.net.Protocol.applyChanges(Protocol.java:403)
    at com.goldencode.p2j.net.Dispatcher.processInbound(Dispatcher.java:700)
    at com.goldencode.p2j.net.Conversation.block(Conversation.java:422)
    at com.goldencode.p2j.net.Conversation.waitMessage(Conversation.java:348)
    at com.goldencode.p2j.net.Queue.transactImpl(Queue.java:1221)
    at com.goldencode.p2j.net.Queue.transact(Queue.java:682)
    at com.goldencode.p2j.net.BaseSession.transact(BaseSession.java:273)
    at com.goldencode.p2j.net.HighLevelObject.transact(HighLevelObject.java:221)
    at com.goldencode.p2j.net.RemoteObject$RemoteAccess.invokeCore(RemoteObject.java:1466)
    at com.goldencode.p2j.net.InvocationStub.invoke(InvocationStub.java:144)
    at com.sun.proxy.$Proxy1.standardEntry(Unknown Source)
    at com.goldencode.p2j.main.ClientCore.start(ClientCore.java:493)

3. Checking for a valid `appFormat` leads to another NPE on `blankScreenValue` n `FillInGuiImpl:1799`) so I guess this would take a while and doesn't look like the path to follow :(

java.lang.NullPointerException
    at com.goldencode.p2j.ui.client.gui.FillInGuiImpl.blankScreenValue(FillInGuiImpl.java:1799)
    at com.goldencode.p2j.ui.client.FillIn.getScreenValue(FillIn.java:2500)
    at com.goldencode.p2j.ui.client.FillIn.getText(FillIn.java:2446)
    at com.goldencode.p2j.ui.client.gui.FillInGuiImpl.getInitialCursorPos(FillInGuiImpl.java:1345)
    at com.goldencode.p2j.ui.client.FillIn.activate(FillIn.java:997)
    at com.goldencode.p2j.ui.client.FillIn$LocalFocusListener.onFocusGained(FillIn.java:4335)
    at com.goldencode.p2j.ui.client.event.FocusEvent.dispatch(FocusEvent.java:145)
    at com.goldencode.p2j.ui.client.widget.AbstractWidget.processFocusEvent(AbstractWidget.java:1086)
    at com.goldencode.p2j.ui.client.widget.AbstractWidget.processEvent(AbstractWidget.java:2281)
    at com.goldencode.p2j.ui.client.widget.TitledWindow.processEvent(TitledWindow.java:279)
    at com.goldencode.p2j.ui.client.gui.WindowGuiImpl.processEvent(WindowGuiImpl.java:1637)
    at com.goldencode.p2j.ui.chui.ThinClient.processEventsWorker(ThinClient.java:19977)
    at com.goldencode.p2j.ui.chui.ThinClient.pop(ThinClient.java:18643)
    at com.goldencode.p2j.ui.chui.ThinClient.processProgressEvent(ThinClient.java:21517)
    at com.goldencode.p2j.ui.chui.ThinClient.processEventsWorker(ThinClient.java:19781)
    at com.goldencode.p2j.ui.chui.ThinClient.pop(ThinClient.java:18643)
    at com.goldencode.p2j.ui.chui.ThinClient.eventBracket(ThinClient.java:18626)
    at com.goldencode.p2j.ui.chui.ThinClient.eventDrawingBracket(ThinClient.java:18548)
    at com.goldencode.p2j.ui.chui.ThinClient.applyWorker(ThinClient.java:18292)
    at com.goldencode.p2j.ui.chui.ThinClient.sendEntry(ThinClient.java:15299)
    at com.goldencode.p2j.ui.chui.ThinClient.sendEntry(ThinClient.java:15261)
    at com.goldencode.p2j.ui.chui.ThinClient.sendEntry(ThinClient.java:15240)
    at com.goldencode.p2j.ui.chui.ThinClient.sendEntry(ThinClient.java:15221)
    at com.goldencode.p2j.ui.client.FocusManager.focusUndefImplicit(FocusManager.java:620)
    at com.goldencode.p2j.ui.chui.ThinClient.waitForWorker(ThinClient.java:14487)
    at com.goldencode.p2j.ui.chui.ThinClient.lambda$84(ThinClient.java:14263)
    at com.goldencode.p2j.ui.chui.ThinClient.lambda$104(ThinClient.java:19666)
    at com.goldencode.p2j.ui.chui.ThinClient.doInteractive(ThinClient.java:19638)
    at com.goldencode.p2j.ui.chui.ThinClient.doInteractive(ThinClient.java:19666)
    at com.goldencode.p2j.ui.chui.ThinClient.waitForWorker(ThinClient.java:14262)
    at com.goldencode.p2j.ui.chui.ThinClient.waitFor(ThinClient.java:14205)
    at com.goldencode.p2j.ui.chui.ThinClient.waitFor(ThinClient.java:14154)

Please post the testcase. The results seem quite wrong (and unexpected for anything simple).

• not post-processed by AspectJ
• the wrong .class files were packed in jar (the correct files are found in build/classes.aop folder).

Ovidiu Maxiniuc wrote:

Form the posted stacks, I guess the executed classes were either:

• not post-processed by AspectJ

Bingo, forgot to add the AspectJ nature to the new 'trunk' project in Eclipse :(

Thanks a bunch Ovidiu.

Roger/Marian, I committed the following:

=== modified file 'deploy/server/directory.xml.template'
--- deploy/server/directory.xml.template        2023-10-30 11:33:26 +0000
+++ deploy/server/directory.xml.template        2023-10-31 09:27:39 +0000
@@ -1143,7 +1143,7 @@
           <node class="boolean" name="override">
             <node-attribute name="value" value="TRUE"/>
           </node>
-          <node class="string" name="cfgOverrides">
+          <node class="string" name="cfgOverrides.example">
             <node-attribute name="value" value="client:cmd-line-option:startup-procedure=ask.p"/>
           </node>
           <node class="string" name="defaultOsUser">
@@ -1228,7 +1228,7 @@
               <node-attribute name="value" value="."/>
             </node>
             <node class="string" name="p2j-entry">
-              <node-attribute name="value" value="ask.p"/>
+              <node-attribute name="value" value="com.goldencode.testcases.NoProgramSpecified.execute"/>
             </node>
           </node>
         </node>

------------------------------------------------------------
revno: 1500
committer: tw
branch nick: testcases
message:
  Reverted template p2j-entry, changed startup-procedure=ask.p into example (refs #7957)

Reason: developers at our company (and the rest of the world: it's public facing) had a hampered experience following wiki: Testcases .
To my knowledge, p2j-entry should contain a FQN (not a legacy code reference), or are both supported?

Also this fragment:

            <node class="integer" name="p2j_port">
              <node-attribute name="value" value="[p2j-port]"/>
            </node>

I also committed an enabled=FALSE for all auxilary services.
Reason:
23/10/31 11:32:03.485+0100 | SEVERE | com.goldencode.p2j.util.AppServerLauncher [AppServerLauncher.launchWorker()] | ThreadName:Launcher for process 'appserver_process'. [00000000:standard], Session:00000000, Thread:00000003, User:standard | Appserver 'app_server' could not be started (121607ms)!
Which was harmless (process was disabled, cron entry wasn't) but noise should be prevented.

It might be that I disabled too much, it's debatable.
I prefer a 1st experience of a customer with FWD to have the lightest startup possible at default (least amount of threads and such).
If there are testcases and unit tests dependent on SOAP or REST, we'll have to reconfigure that use case (for example by a small xmlstarlet script).
Or partially revert my change.

=== modified file 'deploy/server/directory.xml.template'
--- deploy/server/directory.xml.template        2023-10-31 09:47:15 +0000
+++ deploy/server/directory.xml.template        2023-10-31 10:46:27 +0000
@@ -1056,7 +1056,7 @@
           <node class="job" name="start_appserver">
             <node-attribute name="type" value="process"/>
             <node-attribute name="target" value="appserver_process"/>
-            <node-attribute name="enabled" value="TRUE"/>
+            <node-attribute name="enabled" value="FALSE"/>
             <node-attribute name="mode" value="now"/>
           </node>
         </node>
@@ -1093,7 +1093,7 @@
         </node>
         <node class="container" name="rest">
           <node class="boolean" name="enabled">
-            <node-attribute name="value" value="TRUE"/>
+            <node-attribute name="value" value="FALSE"/>
           </node>
           <node class="string" name="appserver">
             <node-attribute name="value" value="app_server"/>
@@ -1104,7 +1104,7 @@
         </node>
         <node class="container" name="soap">
           <node class="boolean" name="enabled">
-            <node-attribute name="value" value="TRUE"/>
+            <node-attribute name="value" value="FALSE"/>
           </node>
           <node class="string" name="wsdl">
             <node-attribute name="value" value="fwd.wsdl"/>
@@ -1118,7 +1118,7 @@
         </node>
         <node class="container" name="webHandler">
           <node class="boolean" name="enabled">
-            <node-attribute name="value" value="TRUE"/>
+            <node-attribute name="value" value="FALSE"/>
           </node>
           <node class="string" name="appserver">
             <node-attribute name="value" value="app_server"/>

revno: 1501
committer: tw
branch nick: testcases
timestamp: Tue 2023-10-31 12:26:13 +0100
message:
  Disabled all template p2j-entry auxilary processes because of SEVERE in server log and not needed for hello world example in wiki (refs #7957)

FQN (not a legacy code reference), or are both supported?

Both are supported.

Greg Shah wrote:

(p2j-entry) FQN (not a legacy code reference), or are both supported?

Both are supported.

Nice!

Roger/Galya (because of logging):
OK, I've got testcases working again for the customer.
There is one issue left, which might be caused by an older p2j version on xfer.
Roger, this line:

 bzr praise directory.xml.template | grep '"logging"'
1482 rfb@gol |         <node class="container" name="logging">

    <node class="container" name="server">
      <node class="container" name="default">
        <node class="container" name="logging">
          <node class="boolean" name="console">
            <node-attribute name="value" value="FALSE"/>
          </node>
          <node class="container" name="file">
            <node class="string" name="path">
              <node-attribute name="value" value="logs/%m%g.log"/>
            </node>
            <node class="integer" name="rotationCount">
              <node-attribute name="value" value="4"/>
            </node>
            <node class="integer" name="rotationLimit">
              <node-attribute name="value" value="500000"/>
            </node>
          </node>

23/10/31 13:15:06.464Z |  SEVERE | com.goldencode.p2j.directory.DirectoryService | ThreadName:main | class 'container' name ''
java.lang.RuntimeException: class 'container' name ''
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:310)
    at com.goldencode.p2j.directory.XmlRemapperIO.load(XmlRemapperIO.java:169)
    at com.goldencode.p2j.directory.XmlRemapper.load(XmlRemapper.java:159)
    at com.goldencode.p2j.directory.RamRemapper.bind(RamRemapper.java:250)
    at com.goldencode.p2j.directory.DirectoryService.rollBack(DirectoryService.java:5182)
    at com.goldencode.p2j.directory.DirectoryService.<init>(DirectoryService.java:316)
    at com.goldencode.p2j.directory.DirectoryService.createInstance(DirectoryService.java:338)
    at com.goldencode.p2j.main.ServerDriver.configureLogging(ServerDriver.java:594)
    at com.goldencode.p2j.main.CommonDriver.process(CommonDriver.java:574)
    at com.goldencode.p2j.main.ServerDriver.process(ServerDriver.java:1)
    at com.goldencode.p2j.main.ServerDriver.main(ServerDriver.java:1016)
Caused by: java.lang.RuntimeException: class 'container' name 'server'
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:310)
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:306)
    ... 10 more
Caused by: java.lang.RuntimeException: class 'container' name 'default'
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:310)
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:306)
    ... 11 more
Caused by: java.lang.RuntimeException: class 'container' name 'logging' failed to add child
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:314)
    at com.goldencode.p2j.directory.XmlRemapperIO.readNode(XmlRemapperIO.java:306)
    ... 12 more
23/10/31 13:15:06.466Z |  SEVERE | com.goldencode.p2j.directory.DirectoryService | ThreadName:main | Unable to bind to underlying back-end

For now I instruct the customer to delete this block "logging" completely, if they encounter this error.
Agreed?

cd xfer_testcases
bzr log -Sl1
 1501 Tijs Wickardt     2023-10-31

bzr log -Sl1 p2j
14790 Hynek Cihlar      2023-10-25 [merge]

Tijs Wickardt wrote:

For now I instruct the customer to delete this block "logging" completely, if they encounter this error.
Agreed?

It's ok if the logging container is missing, because there are defaults.
As for the version it should be >= r14534.

Also double check if there are no two "logging" containers or missing closing tags.

Thank you Galya.
But you can see rev 14790 >= r14534 , and grep '"logging"' gives only one hit.
Do you see anything wrong with the directory fragment?

Tijs Wickardt wrote:

Do you see anything wrong with the directory fragment?

Is it that you have the "console" in the container twice?

    <node class="container" name="server">
      <node class="container" name="default">
        <node class="container" name="logging">
          <node class="boolean" name="console">               <--- Once
            <node-attribute name="value" value="[log-console]"/>
          </node>
          <node class="container" name="file">
            <node class="string" name="path">
              <node-attribute name="value" value="logs/%m%g.log"/>
            </node>
            <node class="integer" name="rotationCount">
              <node-attribute name="value" value="4"/>
            </node>
            <node class="integer" name="rotationLimit">
              <node-attribute name="value" value="500000"/>
            </node>
          </node>
          <node class="boolean" name="console">               <--- Twice
            <node-attribute name="value" value="TRUE"/>
          </node>
          <node class="container" name="loggers">
            <node class="string" name="root">
              <node-attribute name="value" value="WARNING"/>
            </node>
            <node class="string" name="com.goldencode.p2j.util">
              <node-attribute name="value" value="INFO"/>
            </node>
            <node class="string" name="com.goldencode.p2j.net">
              <node-attribute name="value" value="INFO"/>
            </node>
            <node class="string" name="com.goldencode.p2j.persist">
              <node-attribute name="value" value="INFO"/>
            </node>
          </node>
        </node>

Roger Borrello wrote:

Tijs Wickardt wrote:

Do you see anything wrong with the directory fragment?

Is it that you have the "console" in the container twice?
[...]

Looks indeed suspicious.
Marian, can you fix?

root@ubuntu-desktop:/v1/XXXXX-dev.local/workspace/eclipse/xfer_testcases# bzr praise deploy/server/directory.xml.template | grep -i console
             |               <node-attribute name="reference" value="/console"/>
1498 marian. |           <node class="boolean" name="console">
             |             <node-attribute name="value" value="[log-console]"/>
1490 marian. |           <node class="boolean" name="console">

Tijs Wickardt wrote:

Looks indeed suspicious.
Marian, can you fix?

Yes, I saw that as well when trying to replicate the install on another workstation... was thinking to make the console log configurable (TRUE/FASE) but then forgot to remove the node already added previously :(

Will push an update in a second.

revno: 1482
committer: Roger F Borrello <rfb@goldencode.com>
branch nick: testcases
timestamp: Wed 2023-10-25 16:48:00 -0400
message:
  Added some helpful client-side procedures, and the ask.p program set as a default, just to see something work. Updated the directory template to be closer to the hotel_gui version, so as to have some consistency.

The directory template in testcases was exemplary.

Tijs, yes, this is obviously an issue.

Tijs Wickardt wrote:

Also this fragment:
[...]
I left that as is, but Marian should update wiki: Testcases . TIA.

Tell the truth don't know if that is the port number on which the 'app_server' will listen or is the actual port number of the server (secure or insecure not clear either), given the original value (3333) I would say this should be the 'secure port number of the server'... does anyone has a clue as of what this value represent? :)

The problem in PSDOE environment is that that port number is almost always used internally by the Eclipse/PSDOE so the server fails to start using the default ports.

Galya B wrote:

Tijs, yes, this is obviously an issue.

Yes, thanks.
We should be more careful with the template.

Galya B wrote:

The directory template in testcases was exemplary.

More than that, it's public facing, and used in wiki: Testcases , which is also public facing.

1. Be compatibale with p2j trunk on xfer
2. Have correct documentation in the wiki

Tijs Wickardt wrote (see previous comment):

I left that as is, but Marian should update wiki: Testcases .

So all in all it's good that testcases gets more TLC (like similarity to hotel_gui , and more flexibility in the setup), but the customer (developers at customer sites) should not be hampered in their work.
For my work (a.o. ramping up FWD developers for a specific company) a lot is scripted and dockerized.
If parameters are added to the template, I need to add them to the docker scripts.
So a heads up is greatly appreciated :)

Marian Edu wrote:

Tell the truth don't know if that is the port number on which the 'app_server' will listen or is the actual port number of the server (secure or insecure not clear either), given the original value (3333) I would say this should be the 'secure port number of the server'... does anyone has a clue as of what this value represent? :)

The problem in PSDOE environment is that that port number is almost always used internally by the Eclipse/PSDOE so the server fails to start using the default ports.

The original value was 3333 , and inside the customer docker container that was fine.
To avoid complexity, you can simply instruct at the wiki to substitute it for 3333.
I've already done that at the customer docker scripts.

Marian Edu wrote:

Tell the truth don't know if that is the port number on which the 'app_server' will listen or is the actual port number of the server (secure or insecure not clear either), given the original value (3333) I would say this should be the 'secure port number of the server'... does anyone has a clue as of what this value represent? :)

The problem in PSDOE environment is that that port number is almost always used internally by the Eclipse/PSDOE so the server fails to start using the default ports.

A great mystery, indeed! I have found the code in the server.sh most confusing, so have generally updated it in hotel and other apps to be more like:

portbase=3333
portstep=100
...
sport=$(($portbase + $instance))
iport=$(($portbase + $instance + $portstep))
port="net:connection:secure=true net:server:secure_port=$sport net:server:insecure_port=$iport" 

Does that help?

Marian Edu wrote:

Tell the truth don't know if that is the port number on which the 'app_server' will listen or is the actual port number of the server (secure or insecure not clear either), given the original value (3333) I would say this should be the 'secure port number of the server'... does anyone has a clue as of what this value represent? :)

Yes, see portbase="${portbase:=3333}" in server.sh in testcases.
The default is 3333.
They should match. I recently made portbase overridable at the command line (having a port conflict at an AWS environment).

Roger Borrello wrote:

Does that help?

Yes, this is what I've thought as well... I'll update the templates to use the same [sport], [iport] placeholders for secure/insecure ports, how does that sound?

Marian Edu wrote:

Roger Borrello wrote:

Does that help?

Yes, this is what I've thought as well... I'll update the templates to use the same [sport], [iport] placeholders for secure/insecure ports, how does that sound?

Me personally, I don't like the placeholders in general.
I'd rather have xpath expressions, which makes it totally unambiguous on where the nodes reside.
I script it using xmlstarlet.

Tijs Wickardt wrote:

Me personally, I don't like the placeholders in general.
I'd rather have xpath expressions, which makes it totally unambiguous on where the nodes reside.
I script it using xmlstarlet.

I can go either way, have no personal preference here, just followed what seemed like the current approach :)

Related to updating the Wiki, that one has no reference to template files but adding/editing sections directly in directory.xml which I think is not present anymore in bazaar (only the template it). So wether or not we use templates/placeholders or something else we should be consistent about it :)

Marian, you've committed template placeholders again.
So the customer has issues again.
I'll have to redo part of my work today.
Luckily it's still in my fingers, so it will only take 30 minutes.
I'll inform the customer.

Marian Edu wrote:

Related to updating the Wiki, that one has no reference to template files ...

It instructs:

copy deploy/server/directory.xml.template as deploy/server/directory.xml

Marian, do you think this is the proper way forward?
I see you also added placeholders to server.xml.template, unit_test.xml.template, and various client.xml.template files.
This all increases the amount of documentation a potential customer has to read when first doing something with FWD.
And, it creates a lot of work for me, because all the scripting inside dockerhub images need to be updated.
That wasn't needed a few weeks back.

I think the previous approach was better.
Make it as easy as possible for new customers.

Can we revert this?

Generally we are moving toward a more standardized project setup approach that will indeed have placeholders in the directory and built-in rocess to set these values from a since JSON configuration file.

Over time, I expect that some things you are scripting will not be scripted by you. In the short term we need to keep moving ahead and if that means a bit more scripting, then it may be needed.

Please note that the testcases project is not meant for customers. It is meant for internal use at GCD. Don't be fooled by the fact that it is on xfer. Perhaps I need to lock down access more to make this clear. It is going to be very complex because it will have to be able to run the full range of 4GL testing.

For a simple customer-facing project, we have Hotel GUI or Hotel ChUI.

It's a bit less serious than I thought.
At the customer docker scripts, build.properties.template and directory.xml.template are used.
For the clients and unit tests, the templates are not used, but the non-template counterparts are used.
So no need to revert everything.
I'll go fix the customer scripts again.

Please see my request at #7957-34 . Your commit, change the name of the placeholder, immediately broke things. If possible, notify me before you add or change placeholders.

Greg Shah wrote:

Generally we are moving toward a more standardized project setup approach that will indeed have placeholders in the directory and built-in rocess to set these values from a since JSON configuration file.

Sounds good, maybe. It also sounds like layer upon layer. Directory was introduced for server configuration. Placeholders were introduced because configuration was too hard. When there are many placeholders, we might have to automate that again.

Over time, I expect that some things you are scripting will not be scripted by you. In the short term we need to keep moving ahead and if that means a bit more scripting, then it may be needed.

True, but someone has to. Will do (in the meantime).

Please note that the testcases project is not meant for customers. It is meant for internal use at GCD. Don't be fooled by the fact that it is on xfer. Perhaps I need to lock down access more to make this clear. It is going to be very complex because it will have to be able to run the full range of 4GL testing.

Understood, but this customer needs the testcases and unit tests in that repository. It's part of their daily work, to guard against regressions.
I'm open to other options of course.

For a simple customer-facing project, we have Hotel GUI or Hotel ChUI.

That's indeed good for new customers.

Tijs Wickardt wrote:

Marian, you've committed template placeholders again.
So the customer has issues again.
I'll have to redo part of my work today.
Luckily it's still in my fingers, so it will only take 30 minutes.
I'll inform the customer.

Well, you did said you don't like them anyway :)
I did it to align the names with what Roger was using in shell scripts.

The other client/unit_test templates might or might not be used by others, internally we have to use those placeholders so we can update the port numbers for both server/clients in one go. We have the original configurations out of our git repository so each time we start from templates but if you think this is not needed, or too much trouble to do it, we can continue to keep the client/unit_test configurations as-is in bazaar, I did not removed those.

Marian Edu wrote:

Well, you did said you don't like them anyway :)

That's true :) But that's just my opinion. Greg says "keep 'm". It doesn't hurt me much, since Xpath ignores the contents, be it a real value or a placeholder.
In the future it will become JSON, which will be much better than XML (also because of the tooling, it's more human friendly).

I did it to align the names with what Roger was using in shell scripts.

Understood. Alignment is good.

The other client/unit_test templates might or might not be used by others, internally we have to use those placeholders so we can update the port numbers for both server/clients in one go. We have the original configurations out of our git repository so each time we start from templates but if you think this is not needed, or too much trouble to do it, we can continue to keep the client/unit_test configurations as-is in bazaar, I did not removed those.

No, it's fine. You need them, it's clear.
Just try to inform me before you add placeholders to directory.xml.template, and I (or someone at the customer FWD development team) will pick it up before you commit it.

Marian Edu wrote:

we can continue to keep the client/unit_test configurations as-is in bazaar, I did not remove those.

Yes, please keep them.

Tijs Wickardt wrote:

Just try to inform me before you add placeholders to directory.xml.template, and I (or someone at the customer FWD development team) will pick it up before you commit it.

Tijs, looking at the Wiki page and the directory.xml.template I would like to change the [client-start-dir] placeholder with a [project-dir] or something that will be the full path to the testcases project location. Then I can use that for spawner working folder ([project-dir]/deploy/client) and for the database URL ([project-dir]/deploy/db/testcasesdb...), what do you think?

And maybe we can add a placeholder for the database name just in case someone feels the urge to change that later on ;)

Marian: I'm fine by all changes. Better to tackle them all in one go.
For the database full path (H2), perhaps change the current placeholder to a dedicated placeholder (or even better: two: "[db-dir][db-name]") .
Els e.a.: revert to the testcases revision I mailed you. I will fix it tomorrow.

Tijs, looking at the Wiki page and the directory.xml.template I would like to change the [client-start-dir] placeholder with a [project-dir] or something that will be the full path to the testcases project location. Then I can use that for spawner working folder ([project-dir]/deploy/client) and for the database URL ([project-dir]/deploy/db/testcasesdb...), what do you think?

This needs to be discussed with Roger first. I would like this project to be in sync with our standard project templates, if possible.

Greg Shah wrote:

Tijs, looking at the Wiki page and the directory.xml.template I would like to change the [client-start-dir] placeholder with a [project-dir] or something that will be the full path to the testcases project location. Then I can use that for spawner working folder ([project-dir]/deploy/client) and for the database URL ([project-dir]/deploy/db/testcasesdb...), what do you think?

This needs to be discussed with Roger first. I would like this project to be in sync with our standard project templates, if possible.

It's restricting to use the same variable in multiple places, even if it is generic, like [project-dir]. Something that can be leveraged highly for the [client-start-dir] would be something like ~/.project-files where the value would be specific to each user signing on, since the ~ is resolved at the client. That would be completely different for the spawner, in which you would want to reference the same directory, no matter which client.

That is just one edge example, but to illustrate why it helps to have separate substitutions. Forgive me if I am not seeing the whole picture, as I jumped into this a little late.

Roger Borrello wrote:

It's restricting to use the same variable in multiple places, even if it is generic, like [project-dir]. Something that can be leveraged highly for the [client-start-dir] would be something like ~/.project-files where the value would be specific to each user signing on, since the ~ is resolved at the client. That would be completely different for the spawner, in which you would want to reference the same directory, no matter which client.

That is just one edge example, but to illustrate why it helps to have separate substitutions. Forgive me if I am not seeing the whole picture, as I jumped into this a little late.

I really have no strong feelings about this, just looking at the template and the Wiki page it looks to me that [client-start-dir] is [project-dir]/deploy/client and the database is in [project-dir]/deploy/db - this is set in build.xml as ${deploy.home}/db. As it is now in the directory.xml.template, using a relative path, it depends from where the client script was called. Unless it's a classpath search I don't really like relative paths but it might be just me :(