FWD » Build and Source Control

We are going to implement comprehensive 4GL compatibility testing which will be automated and executed both nightly using CI/CD and on-demand as part of our development process.

All tests will be implemented in the New Hotness (Testcases) project. To the degree that useful tests exist in the Old and Busted (Old UAST Testcases) project, they can be cleaned up and reworked into automated tests and then checked in to the new testcases project. Tests should be written based on Writing 4GL Testcases and any additional details decided in #6183.

Unless explicit approval has been provided in advance, it is expected that a single/unified database schemata and set of test data will be used across all test sets. A single reset of the database should be sufficient to run all test sets, in any order, without requiring any further database reset. That means each test set should only edit or depend on data that is unique and/or which it can maintain itself.

This will be a "master" task to organize and track multiple tasks that are needed.

#6854 base language tests
#6855 database/persistence tests
#6856 user interface tests
#6857 rework existing testcases to easily convert and execute specific project subsets
#6858 rework existing testcases to use ABLUnit
#6859 preprocessor tests
#6860 lexer tests
#6861 valid and invalid 4GL syntax (parsing)
#6862 build conversion regression testing tooling
#6863 implement CI/CD servers and the rules for each set of tests

Greg, I'm not sure if this has been discussed or not, but we need a way to standardize how to write tests for integrating in CI/CD. This includes:

• how to included them (automatically?) in CI/CD
• what features need to be used to validate the test (do we use OE features for unit testing?)
• any special setup (like database to use, or appservers connections)
• where to write output files or read input files

I ask because I was looking #6444-198 (the tests in testcases/6453-6444) and that... doesn't look right.

how to included them (automatically?) in CI/CD

If one adds tests to an existing test set, then I expect it would run automatically.

If one adds a new test set, then we need to setup a pipeline for it and allocate resources somewhere. There would be Jenkins scripting changes and probably also changes in the Testcases configuration.

what features need to be used to validate the test (do we use OE features for unit testing?)

Yes, it should all be valid 4GL code. Esoteric and/or complex features should be avoided. We should follow the rules in Writing 4GL Testcases and if there is any ambiguity or open question, we can come up with a solution in #6183.

any special setup (like database to use, or appservers connections)

This would need to be in the project configuration and possibly in the pipeline script.

where to write output files or read input files

We should discuss this in #6183.

I ask because I was looking #6444-198 (the tests in testcases/6453-6444) and that... doesn't look right.

Yes, it is not right. I've previously asked Igor to change that.