FWD » Core Development » Base Language

01/10/2023

02:15 PM Feature #6827: reduce the memory footprint of WrappedResource implementations: replace BDTs with ...

I'm still on #6821, so feel free to do anything you want here. Stanislav Lomany

02:13 PM Feature #6827: reduce the memory footprint of WrappedResource implementations: replace BDTs with ...

Stanislav, do you need help here? Constantin Asofiei

12:14 PM Feature #6821: java collection performance

Committed 6129b rev 14351: @P2JIndex.components()@ provides direct access to the components array in order to improve... Stanislav Lomany

11:13 AM Feature #6821: java collection performance

Greg Shah wrote:
> Don't we already know the exact list of rowids used for all @UPDATE@ statements? We also know th... Ovidiu Maxiniuc

10:27 AM Feature #6821: java collection performance

> Ultimately, we could iterate the tables individually and process (delete/update then invalidate) the records indivi... Greg Shah

08:40 PM Feature #6821: java collection performance

The reason @pruneSessionCache()@ is called on @afterBuf@ and @beforeBuf@ after a batch operation executed using SQL d... Ovidiu Maxiniuc

10:51 AM Feature #6826: reduce the BDT usage in FWD runtime

Stanislav Lomany wrote:
> No.
OK, I'm taking it, as is easier to look in the profiler and see what can be reduced.
Constantin Asofiei

10:50 AM Feature #6826: reduce the BDT usage in FWD runtime

No. Stanislav Lomany

10:50 AM Feature #6826: reduce the BDT usage in FWD runtime

Stanislav, did you start work on this? Constantin Asofiei

09:20 AM Feature #3853: implement LOG-MANAGER runtime

*Logging level rules:*
|Startup params/4GL setters|Auto set logging level|
|none|2|
|-logentrytypes (invalid)|0|... Galya B

07:38 AM Feature #6650: improve performance of Scopeable notification in TM.processScopeNotifications

6129b/14350 improves:
* OUTPUT parameter processing: track if an output parameter is associated with a table field, ... Constantin Asofiei

07:34 AM Support #4032: block/transaction processing context-local and other performance improvements

A batch of changes (including #4032-5) are in 6129b/14349 Constantin Asofiei

07:12 AM Feature #6824: lazy initialization of collection class members

A batch of changes (including #6824-4) are in 6129b/14348
Please review. Constantin Asofiei

01/09/2023

12:58 PM Feature #6821: java collection performance

Greg Shah wrote:
> I was thinking of the DMO in its buffer form, not the concept of the specific record that is co... Eric Faulhaber

07:33 AM Feature #6821: java collection performance

> PreselectQuery.components() - 430k
Committed 6129b rev 14345: replaced components iterator with @ArrayList.get()... Stanislav Lomany

06:58 AM Feature #6821: java collection performance

Eric Faulhaber wrote:
> Greg Shah wrote:
> > Eric Faulhaber wrote:
> > > Greg Shah wrote:
> > > > > Greg, are you... Greg Shah

06:33 AM Feature #6821: java collection performance

Stanislav Lomany wrote:
> > I would start with Java arrays first (look for @[]@ in the text).
>
> Constantin, are... Constantin Asofiei

08:57 AM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

Greg Shah wrote:
> I understand there is no performance difference in lambdas themselves. I'm wondering about the c... Constantin Asofiei

01/08/2023

01:36 PM Feature #6821: java collection performance

> I would start with Java arrays first (look for @[]@ in the text).
Constantin, are you suggesting to replace enha... Stanislav Lomany

01/07/2023

05:51 PM Feature #6821: java collection performance

The "for" cycle changes reviewed by Constantin were merged and committed as 6129b rev 14344. Stanislav Lomany

01/06/2023

05:53 PM Feature #6821: java collection performance

Greg Shah wrote:
> Eric Faulhaber wrote:
> > Greg Shah wrote:
> > > > Greg, are you suggesting to maintain for tem... Eric Faulhaber

03:41 PM Feature #6821: java collection performance

> Please backport this 3821c/14472 to 6129b and commit it there.
Committed as 6129b rev 14343. Stanislav Lomany

02:19 PM Feature #6821: java collection performance

Eric Faulhaber wrote:
> Greg Shah wrote:
> > > Greg, are you suggesting to maintain for temp-tables a map of Record... Greg Shah

02:18 PM Feature #6821: java collection performance

Do we assign a new @rowid@ when we copy temp-table records from one temp-table to another, like in table parameter pr... Greg Shah

02:07 PM Feature #6821: java collection performance

I made an assumption that temp-table record IDs are not unique across multiplex IDs, which is wrong, as you're saying... Stanislav Lomany

02:05 PM Feature #6821: java collection performance

Stanislav Lomany wrote:
> > Do I understand correctly that only the most recently added entry will be kept and any o... Eric Faulhaber

02:03 PM Feature #6821: java collection performance

Greg Shah wrote:
> One thing that is not obvious to me is how the cache is used. Since we are using temp-tables wit... Eric Faulhaber

02:01 PM Feature #6821: java collection performance

> Do I understand correctly that only the most recently added entry will be kept and any older entry is overwritten? ... Stanislav Lomany

01:56 PM Feature #6821: java collection performance

Greg Shah wrote:
> > Greg, are you suggesting to maintain for temp-tables a map of RecordBuffer.dmoClass -> Set of r... Eric Faulhaber

01:29 PM Feature #6821: java collection performance

One thing that is not obvious to me is how the cache is used. Since we are using temp-tables with this mechanism, it... Greg Shah

01:25 PM Feature #6821: java collection performance

> Greg, are you suggesting to maintain for temp-tables a map of RecordBuffer.dmoClass -> Set of records?
Not exac... Greg Shah

12:52 PM Feature #6821: java collection performance

@invalidateRecords@ method is called only for temp-tables. And called only by @ACCEPT-CHANGES@ method (I'm interested... Stanislav Lomany

12:25 PM Feature #6821: java collection performance

All usage of @Session.invalidateRecords()@ comes from @RecordBuffer.pruneSessionCache()@ which is only called in 2 pl... Greg Shah

12:05 PM Feature #6821: java collection performance

Stanislav Lomany wrote:
> > TableMapper$LegacyFieldInfo.getMutableInfo 1.8 million
>
> This one looks like cachin... Constantin Asofiei

11:39 AM Feature #6821: java collection performance

> TableMapper$LegacyFieldInfo.getMutableInfo 1.8 million
This one looks like caching may be slightly improved, but... Stanislav Lomany

02:14 AM Feature #6821: java collection performance

Stanislav Lomany wrote:
> > You also need checks if this is the single resource in the chain being deleted, in this ... Constantin Asofiei

03:05 PM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

I understand there is no performance difference in lambdas themselves. I'm wondering about the call to @LambdaFactor... Greg Shah

02:51 PM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

Greg Shah wrote:
> The dynamic generation of the lambdas for an external procedure is only done the first time the e... Constantin Asofiei

10:44 AM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

The dynamic generation of the lambdas for an external procedure is only done the first time the external proc is load... Greg Shah

10:22 AM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

Eric: Are you OK with the proxy and persistence changes? Greg Shah

10:22 AM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

Code Review 6819_20221223_6129b_14341.patch
I have no objections. Greg Shah

02:44 PM Support #4032: block/transaction processing context-local and other performance improvements

Constantin Asofiei wrote:
> * I think I'll do some experimenting if keeping the reference of @ProcedureManager$Proce... Constantin Asofiei

01/05/2023

02:37 PM Feature #6821: java collection performance

> You also need checks if this is the single resource in the chain being deleted, in this firstResource/lastResource ... Stanislav Lomany

06:20 AM Feature #6821: java collection performance

Constantin Asofiei wrote:
> Attached is a space-delimited file with all FOR EACH loops which have a List, ArrayList ... Constantin Asofiei

04:26 AM Feature #6821: java collection performance

Forgot to add the csv file. Constantin Asofiei

04:25 AM Feature #6821: java collection performance

Keep in mind that this:... Constantin Asofiei

04:25 AM Feature #6821: java collection performance

Stanislav Lomany wrote:
> > what branch and revision did you use for this patch?
>
> 3821c rev 14467.
The patch ... Constantin Asofiei

01:54 PM Support #4032: block/transaction processing context-local and other performance improvements

Attached is a trace file with @ContextLocal.get@ usage in a customer application. This is expanded on 3 levels, leve... Constantin Asofiei

08:10 AM Feature #3853: implement LOG-MANAGER runtime

> > Is this different than @MESSAGE VIEW-AS ALERT-BOX@? Or is this just the case where the 4GL handles the really lo... Greg Shah

08:10 AM Feature #3853: implement LOG-MANAGER runtime

> It is @ContextLocal@, which creates an instance of the log manager wrapper through @class Fallback extends ThreadLo... Greg Shah

07:57 AM Feature #3853: implement LOG-MANAGER runtime

Greg Shah wrote:
> We must solve this properly. The behavior needs to match the 4GL implementation while implementi... Galya B

07:51 AM Feature #3853: implement LOG-MANAGER runtime

Greg Shah wrote:
> Is this different than @MESSAGE VIEW-AS ALERT-BOX@? Or is this just the case where the 4GL handl... Galya B

07:49 AM Feature #3853: implement LOG-MANAGER runtime

Greg Shah wrote:
> > Our implementation of @LOG-MANAGER@ runs purely on the server.
>
> It is certainly preferab... Galya B

07:48 AM Feature #3853: implement LOG-MANAGER runtime

> Do we want also the 32767 chars limit for @LOG-MANAGER:WRITE-MESSAGE@?
Yes. Also, please implement the limit of... Greg Shah

06:47 AM Feature #3853: implement LOG-MANAGER runtime

A few observations in relationship to max log message length (tested):
* By default statement length is limited to... Galya B

04:47 AM Feature #3853: implement LOG-MANAGER runtime

Greg Shah wrote:
> Does the 4GL have an implicit set of settings for @LOG-MANAGER@ in an appserver process? If so, ... Galya B

03:50 AM Feature #3853: implement LOG-MANAGER runtime

Constantin Asofiei wrote:
> Galya: please post what logmanager related config needs to added.
4GL LOG-MANAGER con... Galya B

03:06 AM Feature #3853: implement LOG-MANAGER runtime

Greg Shah wrote:
> For @logthreshold@ and @numlogfiles@ and the other items from #3853-19, we should be able to conf... Constantin Asofiei

01/04/2023

02:44 PM Feature #6817 (WIP): replace INSTANTIATING-PROCEDURE and other handle<ExternalProgram> with the a...

Hynek Cihlar

11:23 AM Feature #6817: replace INSTANTIATING-PROCEDURE and other handle<ExternalProgram> with the actual ...

In Eclipse, there is a way to look for a field or local variable of a certain type. Other IDEs should have something... Constantin Asofiei

12:46 PM Feature #3853: implement LOG-MANAGER runtime

Greg Shah wrote:
> From Galya via email:
>
> > I've tested gui, batch, web gui clients and they are working well ... Constantin Asofiei

12:33 PM Feature #3853: implement LOG-MANAGER runtime

From Galya via email:
> I've tested gui, batch, web gui clients and they are working well with @LOG-MANAGER@. But ... Greg Shah

12:29 PM Feature #3853: implement LOG-MANAGER runtime

For @logthreshold@ and @numlogfiles@ and the other items from #3853-19, we should be able to configure these via 2 me... Greg Shah

08:04 AM Feature #3853: implement LOG-MANAGER runtime

Galya Bogdanova wrote:
> open a @-debugalert@ separate task (maybe #4065 or this one #3853) to migrate the logic to ... Galya B

07:55 AM Feature #3853: implement LOG-MANAGER runtime

Galya Bogdanova wrote:
> P.S.S. The easiest quickest patch to complete #5753 would be to just switch the dependency ... Galya B

04:58 AM Feature #3853: implement LOG-MANAGER runtime

*Current clientlog implementation and -debugalert*
Currently the code uses @-clientlog@ startup param to enable lo... Galya B

03:10 AM Feature #3853: implement LOG-MANAGER runtime

*Rotation and testing*
A substantial part of LOG-MANAGER's logic is related to handling rotation of log files and is... Galya B

08:54 AM Feature #6821: java collection performance

> what branch and revision did you use for this patch?
3821c rev 14467. Stanislav Lomany

05:17 AM Feature #6821: java collection performance

Stanislav Lomany wrote:
> > Related to map access (hashCode calls):
> > ** @HandleChain.delete@ 880k
>
> Maybe w... Constantin Asofiei

05:13 AM Feature #6821: java collection performance

Stanislav Lomany wrote:
> Please review the diff which replaces "for-each" loops with indexed "for" loops (I'll add ... Constantin Asofiei

08:32 AM Feature #6824 (WIP): lazy initialization of collection class members

Constantin Asofiei

01/03/2023

02:54 PM Feature #6821: java collection performance

> Related to map access (hashCode calls):
> ** @HandleChain.delete@ 880k
Maybe we don't need to perform @first/la... Stanislav Lomany

08:33 AM Feature #3853: implement LOG-MANAGER runtime

> Do we show the same error when LOG-MANAGER:WRITE-MESSAGE tries to write more than 15000 chars? If yes, then we'll h... Greg Shah

01/02/2023

05:21 PM Feature #6821: java collection performance

Please review the diff which replaces "for-each" loops with indexed "for" loops (I'll add headers later), specificall... Stanislav Lomany

12/28/2022

07:27 AM Feature #6410: implement additional built-in OO classes/interfaces

Greg Shah wrote:
> Yes, please use @PROVERSION@ and preprocessing to conditionally implement logic by OE version.
... Marian Edu

12/23/2022

11:55 AM Feature #6824: lazy initialization of collection class members

Some notes :
* @BufferImpl.callbacks@ can be lazily created, not all buffers are used in FILL operations.
* @Record... Constantin Asofiei

08:30 AM Feature #3853: implement LOG-MANAGER runtime

@LOG-MANAGER:WRITE-MESSAGE@ supports by default text up to 15000 characters (actually it should be less, since that i... Galya B

05:45 AM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

Constantin Asofiei wrote:
> Constantin Asofiei wrote:
> > Attached patch is built on top of #6939 patch. Eric, ple... Constantin Asofiei

12/21/2022

02:29 PM Feature #6827: reduce the memory footprint of WrappedResource implementations: replace BDTs with ...

I think the @object@ fields need to remain as-is, for the reference counting to work (which affects when these get de... Constantin Asofiei

02:28 PM Feature #6827: reduce the memory footprint of WrappedResource implementations: replace BDTs with ...

This code:... Constantin Asofiei

09:38 AM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

Constantin Asofiei wrote:
> Attached patch is built on top of #6939 patch. Eric, please review and use this one for... Constantin Asofiei

12/20/2022

02:19 PM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

Attached patch is built on top of #6939 patch. Eric, please review and use this one for your tests. Constantin Asofiei

04:40 AM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

I've tested with a 'static bytecode' (i.e. 10k .java files each with 300 lambdas, checked the disassembled .class and... Constantin Asofiei

07:47 AM Feature #7000: Replace old-style synchronization primitives with java.util.concurrent classes

Greg Shah wrote:
> Do you have specific areas seen in profiling where these old-style primitives are heavily involve... Igor Skornyakov

06:24 AM Feature #7000: Replace old-style synchronization primitives with java.util.concurrent classes

Do you have specific areas seen in profiling where these old-style primitives are heavily involved? Greg Shah

03:22 AM Feature #7000: Replace old-style synchronization primitives with java.util.concurrent classes

FWD code contains many places where old-style synchronization primitives (@synchronized@ blocks, @wait@/@notify@) are... Igor Skornyakov

02:58 AM Feature #7000 (New): Replace old-style synchronization primitives with java.util.concurrent classes

Igor Skornyakov

12/19/2022

06:08 PM Feature #3853: implement LOG-MANAGER runtime

> > The base implementation is already handled at conversion and it is stubbed out in the runtime.
>
> Does it mea... Greg Shah

06:35 AM Feature #3853: implement LOG-MANAGER runtime

Greg Shah wrote:
> The base implementation is already handled at conversion and it is stubbed out in the runtime.
... Galya B

11:28 AM Feature #6821: java collection performance

For @ArrayList.iterator()@, the top callers are:
* @PreselectQuery.components()@ - 430k
* @AnnotatedAst.notifyListe... Constantin Asofiei

10:26 AM Feature #6821: java collection performance

The ones that are most important are listed in #6821-4. We aren't suggesting changing things everywhere. Greg Shah

10:22 AM Feature #6821: java collection performance

That may be true. There're about 2k foreach loops, and about 10% of them can be updated. If we want, we can can limit... Stanislav Lomany

09:57 AM Feature #6821: java collection performance

Stanislav Lomany wrote:
> > Actually, I doubt that such optimization will result in any significant performance in a... Igor Skornyakov

09:56 AM Feature #6821: java collection performance

Small improvements over millions of calls add up. The fact that the profiling tools don't show obvious hotspots mean... Greg Shah

09:45 AM Feature #6821: java collection performance

Greg Shah wrote:
> > I would perform benchmarking/profiling first.
>
> This is what was done. See #6821-4.
Th... Igor Skornyakov

09:31 AM Feature #6821: java collection performance

> The point with all the Map and Set iterators is that they allocate lots of short-lived objects.
That's true. I m... Stanislav Lomany

09:25 AM Feature #6821: java collection performance

> Actually, I doubt that such optimization will result in any significant performance in all situations.
I did som... Stanislav Lomany

09:18 AM Feature #6821: java collection performance

> I would perform benchmarking/profiling first.
This is what was done. See #6821-4. Greg Shah

05:12 AM Feature #6821: java collection performance

Actually, I doubt that such optimization will result in any significant performance in all situations. See e.g. a dis... Igor Skornyakov

04:54 AM Feature #6821: java collection performance

Stanislav Lomany wrote:
> I assume we're taking about replacing
Correct.
> I found that there're not so many plac... Constantin Asofiei

04:45 AM Feature #6821 (WIP): java collection performance

Constantin,
> For example, using Java for-each loop for an an ArrayList instead of a @for@ and getting the element... Stanislav Lomany

09:08 AM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

> the performance decreased a lot (there may be some internal storage which affects the INVOKEDYNAMIC instruction at ... Greg Shah

09:08 AM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

From Constantin:
> More with the the lambda for ControlFlowOps, I think the lambda performance is directly proport... Greg Shah

12/16/2022

03:31 AM Bug #6999: OO methods/properties used in direct Java calls

Consider this type:... Constantin Asofiei

03:27 AM Bug #6999 (New): OO methods/properties used in direct Java calls

Constantin Asofiei

12/15/2022

09:28 AM Feature #6827 (WIP): reduce the memory footprint of WrappedResource implementations: replace BDTs...

Sorry, guys, I think I've reviewed all FWD classes implemening @WrappedResource@ but there're no usages of BDT. I've ... Stanislav Lomany

12/14/2022

01:26 PM Feature #6819: refactor FWD proxy implementation to use ReflectASM instead of Java Method reflection

The lambdas will created like this:... Constantin Asofiei

09:15 AM Feature #6410: implement additional built-in OO classes/interfaces

Yes, please use @PROVERSION@ and preprocessing to conditionally implement logic by OE version. Greg Shah

09:09 AM Feature #6410: implement additional built-in OO classes/interfaces

Greg Shah wrote:
> OE 11.7.x is in use right now. They can't move to OE 12 without facing serious issues which have... Marian Edu

07:42 AM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Greg Shah wrote:
> Can you do something low effort to work around this issue until we have 6129b changes ready?
I... Vladimir Tsichevski

07:32 AM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Can you do something low effort to work around this issue until we have 6129b changes ready? Greg Shah

12/13/2022

04:54 PM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Greg Shah wrote:
> Vladimir: Please check your failing scenario in 6129b and let us know what you find.
Yes, in... Vladimir Tsichevski

04:53 PM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Constantin Asofiei wrote:
> Check your classpath. ConvertedClassName no longer exists in 6129b.
Yes, I forgot to ... Vladimir Tsichevski

04:29 PM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Check your classpath. ConvertedClassName no longer exists in 6129b. Constantin Asofiei

03:43 PM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Greg Shah wrote:
> Vladimir: Please check your failing scenario in 6129b and let us know what you find.
I canno... Vladimir Tsichevski

02:04 PM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Our OO implementation is much more advanced in vranch 6129b than in 3821c. In 6129b, at this time we only use @Legac... Greg Shah

07:30 AM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Vladimir Tsichevski wrote:
> No, it is the Java sources which will not compile.
> Note: the name if the field ref... Marian Edu

07:24 AM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Marian Edu wrote:
> All classes in @OpenEdge@ and @Progress@ packages were created *manually*, this is the expecte... Vladimir Tsichevski

07:13 AM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Vladimir Tsichevski wrote:
> The fields in @OpenEdge.Core.DataTypeEnum@ are named after the corresponding Progress b... Marian Edu

06:59 AM Bug #6993 (Review): OO: incorrect field names in DataTypeEnum.java

Fixed in 3821c rev. 14438. Vladimir Tsichevski

06:45 AM Bug #6993 (WIP): OO: incorrect field names in DataTypeEnum.java

Vladimir Tsichevski

06:42 AM Bug #6993: OO: incorrect field names in DataTypeEnum.java

Please do fix this. Greg Shah

06:31 AM Bug #6993: OO: incorrect field names in DataTypeEnum.java

The fields in @OpenEdge.Core.DataTypeEnum@ are named after the corresponding Progress base data types, like @characte... Vladimir Tsichevski

06:24 AM Bug #6993 (Closed): OO: incorrect field names in DataTypeEnum.java

Vladimir Tsichevski