FWD » Core Development » Runtime Infrastructure

07/14/2023

04:43 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> Greg: I think we can merge this to trunk. Alexandru reported a 3.5% improvement for 744... Constantin Asofiei

07/11/2023

05:07 AM Bug #7498: allow a single Web client launch if the client presses ENTER key multiple times

In the FWD Web client login page, if the FWD client presses the ENTER key multiple times, each keystroke will be post... Constantin Asofiei

05:04 AM Bug #7498 (Rejected): allow a single Web client launch if the client presses ENTER key multiple t...

Constantin Asofiei

07/10/2023

05:34 AM Bug #7425: Avoid directory usage in static blocks

Alexandru Lungu wrote:
> I am OK with the documentation. Looks clear to me. However, I feel you can do just a bit mo... Dănuț Filimon

05:23 AM Bug #7425: Avoid directory usage in static blocks

I am OK with the documentation. Looks clear to me. However, I feel you can do just a bit more feedback on the "list o... Alexandru Lungu

03:18 AM Bug #7425: Avoid directory usage in static blocks

Alexandru Lungu wrote:
> Danut, please make sure you get this done correctly asap. I intend to experiment with some ... Dănuț Filimon

07/07/2023

09:04 AM Bug #7425: Avoid directory usage in static blocks

Danut, please make sure you get this done correctly asap. I intend to experiment with some fine-tuning using this con... Alexandru Lungu

07/06/2023

10:10 AM Bug #7425: Avoid directory usage in static blocks

I updated [[p2j:Database_Configuration#Persistence|Database Configuration]], this is subjected to changes since I am ... Dănuț Filimon

09:27 AM Bug #7425: Avoid directory usage in static blocks

I will update the [[p2j:Database_Configuration#Persistence|Database Configuration]] wiki with the proper configuratio... Dănuț Filimon

09:22 AM Bug #7425 (Test): Avoid directory usage in static blocks

Merged 7388a in trunk as rev. 14641 and archived. Alexandru Lungu

08:03 AM Bug #7425: Avoid directory usage in static blocks

Well, @BufferManager@ is per-session so it *may* be a problem. We can go the other way around sticking to @8192@ whic... Alexandru Lungu

07:55 AM Bug #7425: Avoid directory usage in static blocks

*Committed 7388a/rev.14642*. Increased cache sizes:
* @BufferManager@: from 10000 to 16384.
* @FastFindCache@ L2: f... Dănuț Filimon

07:42 AM Bug #7425: Avoid directory usage in static blocks

Danut, please do the same changes to @BufferManager@ and @FastFindCache@. Find the smallest power of 2 bigger than th... Alexandru Lungu

07:22 AM Bug #7425: Avoid directory usage in static blocks

Alexandru, @TemporaryAccountPool@ is OK, it gets fully initialized at server startup. I don't think even @bootstrap@... Constantin Asofiei

06:52 AM Bug #7425: Avoid directory usage in static blocks

Alexandru Lungu wrote:
> Danut, please create a second method only with class and size, which calls this 3 parameter... Dănuț Filimon

06:12 AM Bug #7425: Avoid directory usage in static blocks

*Review:*
* I am still a bit concerned of @TemporaryAccountPool.getInstance(); TemporaryAccountPool.bootstrap();@.... Alexandru Lungu

07/05/2023

02:45 PM Bug #7425: Avoid directory usage in static blocks

> If everything is ok, can we merge this by tomorrow EOD?
Yes Greg Shah

10:55 AM Bug #7425: Avoid directory usage in static blocks

I will do a second review asap. I am intending to do a second rebase + test + profile just to check that the performa... Alexandru Lungu

09:04 AM Bug #7425: Avoid directory usage in static blocks

*Committed 7388a/rev.14640*. Made changes based on the review in #7425-25, please note that:
* Fields were made non-... Dănuț Filimon

04:44 AM Bug #7425: Avoid directory usage in static blocks

Review for 7388a rev 14639 compared with 14633 (base trunk rev for the branch):
* @TemporaryDatabaseManager.initiali... Constantin Asofiei

07/04/2023

10:50 AM Bug #7425: Avoid directory usage in static blocks

Right, 7388a was the one I meant to write. Edited now. Alexandru Lungu

10:13 AM Bug #7425: Avoid directory usage in static blocks

Alexandru, I guess the branch is 7388a, with latest rev 14639. I'm reviewing this now. Constantin Asofiei

08:46 AM Bug #7425: Avoid directory usage in static blocks

You are right Danut, my bad on this one. It is fine to keep @null@ for @ds@ and let the directory service do the work... Alexandru Lungu

08:32 AM Bug #7425: Avoid directory usage in static blocks

Alexandru, you previously mentioned that "there are places where @Utils.getDirectoryNodeInt(null, ...);@ is used stra... Dănuț Filimon

07:46 AM Bug #7425: Avoid directory usage in static blocks

*Review of 7388a/rev. 14639*
* @TemporaryAccountPool@ still has a static block that does all kind of initializatio... Alexandru Lungu

03:55 AM Bug #7425: Avoid directory usage in static blocks

Please proceed with the reviewing/testing since I've yet to receive a review for #7425-11. The last change I've done ... Dănuț Filimon

03:47 AM Bug #7425: Avoid directory usage in static blocks

Danut, please let me know if there is something else to be addressed here. I will like to do a review and testing rou... Alexandru Lungu

06/30/2023

05:34 AM Bug #7425 (Review): Avoid directory usage in static blocks

Alexandru Lungu

06/28/2023

09:40 AM Bug #7425: Avoid directory usage in static blocks

Alexandru Lungu wrote:
> > Although there is a field called loggingEnabled in P2JQueryLogger, it is not actually use... Radu Apetrii

09:24 AM Bug #7425: Avoid directory usage in static blocks

> Although there is a field called loggingEnabled in P2JQueryLogger, it is not actually used, because P2JQueryExecuto... Alexandru Lungu

08:58 AM Bug #7425: Avoid directory usage in static blocks

Alexandru Lungu wrote:
> Radu, please check @P2JQueryLogger@. I feel like I've mismatched something in it while reba... Radu Apetrii

04:31 AM Bug #7425: Avoid directory usage in static blocks

Radu, please check @P2JQueryLogger@. I feel like I've mismatched something in it while rebasing. More specific, I am ... Alexandru Lungu

03:44 AM Bug #7425: Avoid directory usage in static blocks

I retested the customer application and found out that the @psCache@ is initialized before the @CacheManager@, which ... Dănuț Filimon

06/27/2023

10:07 AM Bug #7425: Avoid directory usage in static blocks

*Committed 7388a/rev14637.*
* Removed the @DatabaseStatistics.get@ call in @StandardServer@
* This will be the stan... Dănuț Filimon

04:31 AM Bug #7425: Avoid directory usage in static blocks

*Review of 7388a*
This also includes the review for #7388.
* In @StandardServer@, you no longer need @DatabaseSta... Alexandru Lungu

03:53 AM Bug #7425: Avoid directory usage in static blocks

Rebased 7388a from trunk rev. 14633. 7388a is now at rev. 14636. Alexandru Lungu

09:03 AM Bug #7291 (Closed): LOG-MANAGER to log on client-side

The last issue was fixed in task branch 7415c which was merged to trunk as rev 14631. Greg Shah

06/26/2023

01:22 PM Bug #7443 (Closed): performance of CentralLogger

Greg Shah

12:52 PM Bug #7443 (Test): performance of CentralLogger

Galya B

12:29 PM Bug #7443: performance of CentralLogger

Task branch 7443a was merged to trunk as rev 14633 and archived. Galya B

10:56 AM Bug #7443: performance of CentralLogger

Please merge 7443a to trunk. Good work! Greg Shah

10:11 AM Bug #7443: performance of CentralLogger

Greg: I think we can merge this to trunk. Alexandru reported a 3.5% improvement for 7443a (from a 3.5% loss without ... Constantin Asofiei

06:12 AM Bug #7443: performance of CentralLogger

Compilation error fixed with r14637. Galya B

04:47 AM Bug #7443: performance of CentralLogger

Attempting to profile with rev. 14636. I've got:... Alexandru Lungu

03:35 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> Galya, one more small optimization, please: this test in @LoggingUtil.addLeadingSymbols@... Galya B

03:32 AM Bug #7443: performance of CentralLogger

Galya, one more small optimization, please: this test in @LoggingUtil.addLeadingSymbols@ can be @leadingSymbolsCount ... Constantin Asofiei

03:13 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> we need to have only one history number per branch
You say that often, but when I add h... Galya B

02:44 AM Bug #7443: performance of CentralLogger

Eugenie Lyzenko wrote:
> I see the log call used when process *started*, not stopped. So it is not clear for me why ... Galya B

02:29 AM Bug #7443: performance of CentralLogger

Eugenie Lyzenko wrote:
> I see the log call used when process *started*, not stopped. So it is not clear for me why ... Galya B

06/23/2023

12:33 PM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> Eugenie: please review the native changes.
>
I have no objection for code itself. H... Eugenie Lyzenko

11:52 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> Alexandru: please do another round of performance testing with 7156b + 7443a. Please no... Constantin Asofiei

11:42 AM Bug #7443: performance of CentralLogger

Galya, this is a review for 7443a rev 14634:
* @CentraLogger@ - please leave only 006 number and remove 007 and 008 ... Constantin Asofiei

10:26 AM Bug #7443 (Review): performance of CentralLogger

r14634: Fixed Java (@java.util.logging.LogRecord#needToInferCaller@ set with reflection to stop heavy checks unused b... Galya B

08:52 AM Bug #7443: performance of CentralLogger

If it's not a trouble, can we have a perf test running now, while I wonder what else to change.
P.S. with updated ... Galya B

08:51 AM Bug #7443: performance of CentralLogger

r14633:
The appserver kept the spawner live, which kept the stderr parsing / logging loop on the server live. It p... Galya B

05:14 AM Bug #7443: performance of CentralLogger

Please do a profiling and see what else stands out which can be improved. Constantin Asofiei

05:11 AM Bug #7443: performance of CentralLogger

r14632 16% improvement of the @+0.45%@ degradation by replacing @String.format@ in formatting logging context.
I a... Galya B

02:56 AM Bug #7443: performance of CentralLogger

@CleanFormatter@ and the java logger init code is taken from an old trunk version.... Galya B

02:07 AM Bug #7443: performance of CentralLogger

Galya B wrote:
> If we blindly introduce "improvements" in iterations without any clearly expressed expectations, th... Constantin Asofiei

02:04 AM Bug #7443: performance of CentralLogger

I'll test it in a unbiased way now, in FWD. Wait for results. Galya B

02:02 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> Galya, I think your tests are biased because they do not use the FWD runtime. My point ... Galya B

01:53 AM Bug #7443: performance of CentralLogger

If we blindly introduce "improvements" in iterations without any clearly expressed expectations, this may not work as... Galya B

01:23 AM Bug #7462: fix 'AS' instead of '4GL' in LOG-MANAGER log lines and add an incompatible FWD and FWD...

OK, sorry about that. I keep forgetting that LOG-MANAGER needs more work. Constantin Asofiei

01:01 AM Bug #7462: fix 'AS' instead of '4GL' in LOG-MANAGER log lines and add an incompatible FWD and FWD...

This is not a bug, this is a unimplemented feature. AS proper support was supposed to be worked under #3853. Galya B

06/22/2023

10:41 AM Bug #7462: fix 'AS' instead of '4GL' in LOG-MANAGER log lines and add an incompatible FWD and FWD...

When used on 4GL client, LOG-MANAGER outputs this:... Constantin Asofiei

10:35 AM Bug #7462 (New): fix 'AS' instead of '4GL' in LOG-MANAGER log lines and add an incompatible FWD a...

Constantin Asofiei

08:56 AM Support #5568: implement application-specific TERMINFO database overrides

With the completion of #5167, we now have the option of using a statically linked ncurses installation rather than th... Greg Shah

06/21/2023

11:00 AM Bug #7443 (WIP): performance of CentralLogger

We know we need proper logging because there are 4GL language features that depend upon it. In addition, we must pro... Greg Shah

09:15 AM Bug #7443: performance of CentralLogger

Another summary:
* @CentralLogger@ (used in new trunk) is 300% slower than @System.err@ redirected to file (used in ... Galya B

09:02 AM Bug #7443: performance of CentralLogger

Here is the java logger to file with the new formatter. There is slight degradation of performance (+6.3%), because t... Galya B

08:48 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> For example, can @String.format@ be replaced with something else, which parses the forma... Galya B

08:41 AM Bug #7443: performance of CentralLogger

Biased is a synonym for isolated? I have no idea what settings you use in your tests, while I shared my samples. Aren... Galya B

08:34 AM Bug #7443: performance of CentralLogger

Galya, I think your tests are biased because they do not use the FWD runtime. My point was to test from within the F... Constantin Asofiei

08:13 AM Bug #7443: performance of CentralLogger

What I'm implying is: Console writes seem quite expensive, so the java logger performed poorly before, but the day wa... Galya B

08:10 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> From your results, are you saying that Java logger is 33seconds and CentraLogger is 14se... Galya B

08:09 AM Bug #7443: performance of CentralLogger

From your results, are you saying that Java logger is 33seconds and CentraLogger is 14seconds? Constantin Asofiei

08:07 AM Bug #7443: performance of CentralLogger

A little bit of history:
Once upon a time FWD used direct @System.err.print@ and @throwable.printStackTrace()@ sta... Galya B

08:02 AM Bug #7443: performance of CentralLogger

Here it is, the groundbreaking reality (in src.com.galya.LoggingTest):... Galya B

07:58 AM Bug #7443: performance of CentralLogger

I don't understand what you mean by System.err. Previously, FWD was using @java.util.logging@ for logging support on... Constantin Asofiei

04:39 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> using what was before CentraLogger
You mean @System.err@? server.log was "generated" by... Galya B

04:01 AM Bug #7443: performance of CentralLogger

Galya, please measure the following, for 1mm log lines:
* the LogTest modified program with CentralLogger and 7443a
... Constantin Asofiei

03:14 AM Bug #7443: performance of CentralLogger

Let's set the requirements first before jumping into fixing. If I make @CentralLogger#describeContext@ (24,755%) and ... Galya B

02:50 AM Bug #7443: performance of CentralLogger

Alexandru Lungu wrote:
> I got a result that is a bit slower than my trunk base-line before @CentralLogger@ (only +0... Galya B

02:42 AM Bug #7443: performance of CentralLogger

Alexandru, can you please try to run with the following configs:... Galya B

02:42 AM Bug #7443: performance of CentralLogger

> Alexandru, please commit this change to 7443a. I expect 7443a to be merged to trunk before 7026e.
Done! Committe... Alexandru Lungu

01:41 AM Bug #7443: performance of CentralLogger

Alexandru Lungu wrote:
> Constantin Asofiei wrote:
> > Alexandru Lungu wrote:
> > > I commented an WARNING (that ... Constantin Asofiei

06/20/2023

04:41 PM Bug #7443: performance of CentralLogger

Galya, Alexandru I think is right, formatting the message is expensive. Please see how this can be improved. I've a... Constantin Asofiei

10:00 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> Alexandru Lungu wrote:
> > I commented an WARNING (that is already fixed on another br... Alexandru Lungu

09:54 AM Bug #7443: performance of CentralLogger

Alexandru Lungu wrote:
> I commented an WARNING (that is already fixed on another branch)
Is this already in trunk... Constantin Asofiei

09:42 AM Bug #7443: performance of CentralLogger

I've done one set of profiling tests and got a slight improvement comparing to the state before #7443 (which was +3.1... Alexandru Lungu

09:14 AM Bug #7443: performance of CentralLogger

Code Review Task Branch 7443a Revision 14630
The change is good. I like that the code is in @Utils@ instead of @C... Greg Shah

08:48 AM Bug #7443: performance of CentralLogger

r14630 reverts @LogHelper#createThreadName@, requested in #7143-234. Galya B

04:23 AM Bug #7443: performance of CentralLogger

I pushed r14629 to fix the compilation error. I haven't rebuilt with @all@. Galya B

03:40 AM Bug #7443: performance of CentralLogger

7443a/rev. 14628 doesn't compile: ... Alexandru Lungu

01:47 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> Alexandru: please apply (locally) the 7443a changes to 7156b and run a new round of test... Alexandru Lungu

05:33 AM Bug #7425: Avoid directory usage in static blocks

This bootstrap is triggered only on server start-up from @StandardServer@ hooks. Should we consider other FWD usages ... Alexandru Lungu

04:55 AM Bug #7425: Avoid directory usage in static blocks

There are a few classes that already have an @initialize@ method, so to avoid confusion I @bootstrap@ for the method ... Dănuț Filimon

06/19/2023

11:50 AM Bug #7443: performance of CentralLogger

Galya B wrote:
> 7443a r14628 based on trunk r14627 tested and ready for review.
The changes look reasonable, but t... Constantin Asofiei

10:40 AM Bug #7443 (Review): performance of CentralLogger

7443a r14628 based on trunk r14627 tested and ready for review. Galya B

04:19 AM Bug #7443 (WIP): performance of CentralLogger

7443a created. Galya B

07:03 AM Bug #7425: Avoid directory usage in static blocks

Danut, try to make the initialization calls hierarchical. That is, do not list all initialization in @StandardServer@... Alexandru Lungu

07:01 AM Bug #7425: Avoid directory usage in static blocks

I found the following static initializers that use @directory.xml@:
* @TemporaryAccountPool@
* @WebHandler@: from t... Dănuț Filimon

06/16/2023

09:02 AM Bug #7443: performance of CentralLogger

Greg Shah wrote:
> What can't be slow or even as slow as a single map lookup is the @isLoggable()@.
Roger that. I... Galya B

09:01 AM Bug #7443: performance of CentralLogger

Galya B wrote:
> When invalidating the levels in all instances that will be a performance bottleneck, because all lo... Greg Shah

02:44 AM Bug #7443: performance of CentralLogger

When invalidating the levels in all instances that will be a performance bottleneck, because all loggers will need to... Galya B

06/15/2023

12:39 PM Bug #7443: performance of CentralLogger

> Galya, beside the FWD server startup, what are the condition under which a logger's level can be changed? Here I m... Greg Shah

12:38 PM Bug #7443: performance of CentralLogger

> 1. Have you done performance testing on Java @isLoggable@ implementation? Think about it carefully.
All of our p... Greg Shah

12:37 PM Bug #7443: performance of CentralLogger

Galya, beside the FWD server startup, what are the condition under which a logger's level can be changed? Here I mea... Constantin Asofiei

11:47 AM Bug #7443: performance of CentralLogger

I'm not sure if that got understood, but all loggers should listen to the change, this means a few hundred objects wr... Galya B

11:41 AM Bug #7443: performance of CentralLogger

What I'm trying to say is that listeners and more state are a bad idea. It should be implemented with @WeakReference@... Galya B

11:33 AM Bug #7443: performance of CentralLogger

Also Java logger has a tree :( I wanted to implement a tree as well, but didn't have the capacity of a Java team. Galya B

11:24 AM Bug #7443: performance of CentralLogger

I made an expensive mistake, that removed will improve things 3 times... Now @getLevel@ checks in 3 concurrent maps, ... Galya B

11:13 AM Bug #7443: performance of CentralLogger

Galya B wrote:
> Galya B wrote:
> > To me removing 50% of the calls sounds quite right. Or am I misreading your scr... Constantin Asofiei

10:59 AM Bug #7443: performance of CentralLogger

Galya B wrote:
> To me removing 50% of the calls sounds quite right. Or am I misreading your screenshot?
I didn't s... Galya B

10:58 AM Bug #7443: performance of CentralLogger

Also as I've said, I have a mistake to fix.
Galya B

10:57 AM Bug #7443: performance of CentralLogger

To me removing 50% of the calls sounds quite right. Or am I misreading your screenshot? Galya B

10:54 AM Bug #7443: performance of CentralLogger

Galya B wrote:
> Give me 1h to rework @TransactionManager@ to my liking and see if it fixes something.
Reworking co... Constantin Asofiei

10:52 AM Bug #7443: performance of CentralLogger

Do we have a branch here? Galya B

10:51 AM Bug #7443: performance of CentralLogger

Give me 1h to rework @TransactionManager@ to my liking and see if it fixes something. Galya B

10:49 AM Bug #7443: performance of CentralLogger

Constantin, I started with a similar implementation, where the state also included the level, but there are other cav... Galya B

10:46 AM Bug #7443: performance of CentralLogger

@isLoggable@ is meant to be used when there is a cost with resolving the arguments passed to the @log@ method. That'... Constantin Asofiei

10:44 AM Bug #7443: performance of CentralLogger

Even if I bend due to pure pressure and implement it, I don't recommend using @isLoggable@ just for the sake of looki... Galya B

10:29 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> It doesn't matter if the same logger name is being used by multiple logger instances, th... Galya B

10:20 AM Bug #7443: performance of CentralLogger

Greg Shah wrote:
> Bypassing everything but the logging level check has served us well. There is a performance reas... Galya B

10:20 AM Bug #7443: performance of CentralLogger

Galya, the problem is not @isLoggable@ being called excessively. @canLog@ relies too on @isLoggable@, so getting rid ... Constantin Asofiei

10:14 AM Bug #7443: performance of CentralLogger

Bypassing everything but the logging level check has served us well. There is a performance reason those checks were... Greg Shah

10:13 AM Bug #7443: performance of CentralLogger

This is the entire @logging@ node. The @loggers@ levels is the same from before testing with @CentralLogger@.... Constantin Asofiei

10:08 AM Bug #7443: performance of CentralLogger

Constantin, what is the logging level you tested with on this screenshot? Galya B

09:50 AM Bug #7443: performance of CentralLogger

Also let's not forget that resource consumption is part of execution of applications and this feature was not existin... Galya B

09:44 AM Bug #7443: performance of CentralLogger

There is no free lunch, it's either the CPU or the RAM. If I need to work on this task, I'll remove @isLoggable@ from... Galya B

09:40 AM Bug #7443: performance of CentralLogger

And we know the complexity of accessing string keyed map. Do you think the concurrency of the map adds to the CPU usage? Galya B

09:39 AM Bug #7443: performance of CentralLogger

Actually the cache map should have been the first lookup in the method. This is my bad. Galya B

09:33 AM Bug #7443: performance of CentralLogger

Constantin Asofiei wrote:
> We need to remove the map lookup. The log level needs to be cached at the logger instan... Galya B

09:32 AM Bug #7443: performance of CentralLogger

Galya B wrote:
> We can lower it to 1.8 million executions removing the unnecessary @if (LOG.isLoggable(Level.FINE))... Constantin Asofiei

09:31 AM Bug #7443: performance of CentralLogger

~1% of resources for 3.6 million executions sounds great to me!
Jokes aside...
We can lower it to 1.8 million e... Galya B

08:13 AM Bug #7443: performance of CentralLogger

@CentralLogger.isLoggable@ is called 3.6 million times when testing an application. In the profiler, this highlights... Constantin Asofiei

08:08 AM Bug #7443 (Closed): performance of CentralLogger

Constantin Asofiei

10:52 AM Bug #7425 (WIP): Avoid directory usage in static blocks

Danut, this is close to your work on #7388. I will do the review on caches there asap.
Please extend your research... Alexandru Lungu