FWD » Core Development » Runtime Infrastructure

04/21/2016

07:53 AM Feature #1842: implement the multi-source directory approach

It should really minimize per-site/per-system configuration and allow most things to be configured at the most approp... Greg Shah

07:42 AM Feature #1842: implement the multi-source directory approach

I've not come across this issue before - I like it and have been having similar (less well-structured) thoughts. Paul E

04/19/2016

05:42 AM Feature #1916: improve the runtime statistics gathered by P2J

Paul, I think the stats are enabled by setting a @block-execution-statistics@ attribute in the directory (which repre... Constantin Asofiei

04/18/2016

08:26 AM Feature #1916: improve the runtime statistics gathered by P2J

This sounds interesting. How do I enable gathering of these statistics? Paul E

04/12/2016

12:55 PM Feature #2973 (Closed): improve ContextLocal performance

Code Review Task Branch 3035a Revision 11006
The changes are fine.
I'm closing this task. If there are problem... Greg Shah

11:06 AM Feature #2973: improve ContextLocal performance

Greg Shah wrote:
> Code Review Task Branch 3035a Revision 11005
>
> 1. @keysInUse@ is missing javadoc.
>
> 2. ... Igor Skornyakov

10:44 AM Feature #2973: improve ContextLocal performance

Code Review Task Branch 3035a Revision 11005
1. @keysInUse@ is missing javadoc.
2. @ContextKey.toString()@ open... Greg Shah

09:39 AM Feature #2973: improve ContextLocal performance

Added @ContextKey@ re-use protection.
Committed to the task branch *3035a* revision 11005. Igor Skornyakov

07:03 AM Feature #2973: improve ContextLocal performance

Greg Shah wrote:
> > how many instances of the ContextKey are expected to be in use at the same time? Just up to an ... Igor Skornyakov

07:02 AM Feature #2973: improve ContextLocal performance

> how many instances of the ContextKey are expected to be in use at the same time? Just up to an order of magnitude.
... Greg Shah

07:00 AM Feature #2973: improve ContextLocal performance

Greg Shah wrote:
> We aren't checking that code is not in use. At construction of the @ContextKey@ we can simply as... Igor Skornyakov

06:56 AM Feature #2973: improve ContextLocal performance

We aren't checking that code is not in use. At construction of the @ContextKey@ we can simply ask the @SecurityConte... Greg Shah

05:47 AM Feature #2973: improve ContextLocal performance

Greg Shah wrote:
> The idea is simply to never set a @ContextKey.id@ that is already in use. No one wants the id to... Igor Skornyakov

05:33 AM Feature #2973: improve ContextLocal performance

The idea is simply to never set a @ContextKey.id@ that is already in use. No one wants the id to be mutable. Greg Shah

03:53 AM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> Of course this can be done very easily. However I do not think that it is a good idea. Firs... Igor Skornyakov

03:41 AM Feature #2973: improve ContextLocal performance

Greg Shah wrote:
> The problem is that one can create a @ContextKey@ instance which fails the @addToken()@ and then ... Igor Skornyakov

04/11/2016

07:15 PM Feature #2973: improve ContextLocal performance

The problem is that one can create a @ContextKey@ instance which fails the @addToken()@ and then later can call @getT... Greg Shah

06:32 PM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> Sorry Greg. I do not understand what you and Constantin mean exactly. The @SecurityContext... Igor Skornyakov

06:27 PM Feature #2973: improve ContextLocal performance

Greg Shah wrote:
> I'm referring to the wrapping case where an id gets reused because the long overflows and some lo... Igor Skornyakov

06:20 PM Feature #2973: improve ContextLocal performance

> May be it makes sense to add it to the 3035a branch as the change is very small (one line) and non critical?
Yes... Greg Shah

06:20 PM Feature #2973: improve ContextLocal performance

I'm referring to the wrapping case where an id gets reused because the long overflows and some long-running existing ... Greg Shah

06:16 PM Feature #2973: improve ContextLocal performance

Greg Shah wrote:
> Considering that the check for this is low cost and should not be in the most common use case (th... Igor Skornyakov

05:59 PM Feature #2973: improve ContextLocal performance

> but I think that 138 years is still a long time which as far beyond any reasonable estimation of a non-stop run dur... Greg Shah

04:07 PM Feature #2973: improve ContextLocal performance

Task branch 2973a was merged to the trunk revision 11001 and archived. Igor Skornyakov

04:02 PM Feature #2973: improve ContextLocal performance

Please merge to trunk. Thank you. Eric Faulhaber

03:40 PM Feature #2973: improve ContextLocal performance

Eric Faulhaber wrote:
> Please address the remaining code review issues.
All issues are fixed now.
> Constantin,... Constantin Asofiei

03:07 PM Feature #2973: improve ContextLocal performance

Please address the remaining code review issues.
Constantin, I don't think the changes made from this review warra... Eric Faulhaber

01:38 PM Feature #2973: improve ContextLocal performance

The @ContextKey.id@ is @long@ in revision 11012. Igor Skornyakov

01:29 PM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> I do not understand the argument regarding the @hashCode@ value - the additional (CPU) over... Constantin Asofiei

11:38 AM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> Regardless if we switch to long, the hash key will still be an integer value (this is wh... Igor Skornyakov

11:24 AM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> I understand the concern. However it is easy to calculate that if a new instance of the @Co... Constantin Asofiei

10:54 AM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> Review for 2973a rev 11008:
> # @ContextKey@
> - we are using @AtomicInteger@ as a gen... Igor Skornyakov

09:54 AM Feature #2973: improve ContextLocal performance

Another issue in @SecurityContext.transferAllTokens@ - removing an element from the map while the map is processed (v... Constantin Asofiei

09:39 AM Feature #2973: improve ContextLocal performance

Review for 2973a rev 11008:
# @ContextKey@
- we are using @AtomicInteger@ as a generator. My worries are about ver... Constantin Asofiei

08:29 AM Feature #2973: improve ContextLocal performance

Constantin, would you please review the latest version? Eric Faulhaber

04:08 AM Feature #2973: improve ContextLocal performance

Task branch 2973a was rebased from the trunk revision 11000. Pushed up to revision 11008. Igor Skornyakov

04/10/2016

06:22 PM Feature #2973: improve ContextLocal performance

I've restores the post-cleanup check.
Committed to the task branch 2973a revision 11006.
Automated regression t... Igor Skornyakov

04/09/2016

06:15 AM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> Constantin Asofiei wrote:
> > @tokenMap@ is guaranteed to not be changed by other threads.... Constantin Asofiei

03:28 AM Feature #2973: improve ContextLocal performance

I've performed majic regression testing of 2973a revision 11004.
# all CTRL-C tests passed
# from the main part the... Igor Skornyakov

04/08/2016

06:11 PM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> @tokenMap@ is guaranteed to not be changed by other threads. But is not guaranteed that... Igor Skornyakov

04:11 PM Feature #2973: improve ContextLocal performance

PS: if you take a look at @ContextLocal$Wrapper.cleanup@, you will see that after the var is cleaned, it is removed f... Constantin Asofiei

04:08 PM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> Sorry Constantin, I do not understand. If we are sure that @tokenMap@ is not changed during... Constantin Asofiei

03:51 PM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> > How this approach works with the situation when something was added to the @tokenMap@ ... Igor Skornyakov

03:35 PM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> How this approach works with the situation when something was added to the @tokenMap@ after... Constantin Asofiei

03:17 PM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> So, I think this should work:
> # this code is needed, as it is critical for @tokenMa... Igor Skornyakov

03:06 PM Feature #2973: improve ContextLocal performance

Igor, the code you commented from @cleanupWorker@ was responsible for two things:
# ensure that, after cleanup, @tok... Constantin Asofiei

12:52 PM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> After a close look at the @SecurityContext@ class I got an impression that if the @ tokenMa... Eric Faulhaber

12:07 PM Feature #2973: improve ContextLocal performance

After a close look at the @SecurityContext@ class I got an impression that if the @ tokenMap@ is a @ConcurrentMap@ we... Igor Skornyakov

11:57 AM Feature #2973: improve ContextLocal performance

Hynek Cihlar wrote:
> Igor Skornyakov wrote:
> > Hynek Cihlar wrote:
> > > Btw., have you considered making the lo... Igor Skornyakov

10:49 AM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> Hynek Cihlar wrote:
> > Btw., have you considered making the locking of reading operations... Hynek Cihlar

10:31 AM Feature #2973: improve ContextLocal performance

Hynek Cihlar wrote:
> Btw., have you considered making the locking of reading operations more optimistic? For exampl... Igor Skornyakov

10:22 AM Feature #2973: improve ContextLocal performance

Btw., have you considered making the locking of reading operations more optimistic? For example @StampLock@ allows th... Hynek Cihlar

09:08 AM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> The test run time is now 37 minutes 54 seconds.
> However the time spent in the @Security... Eric Faulhaber

08:54 AM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> However the time spent in the @SecurityContext.getToken()@ method decreased just to 55 seco... Constantin Asofiei

08:52 AM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
I've made the following changes:
# Replaced @ConcurrentHashMap@ with @HashMap@ as @tokenMa... Igor Skornyakov

07:13 AM Feature #2973: improve ContextLocal performance

I've used YourKit (sampling mode) to analyse server behaviour while running the customer test suite (**/*Test.java) (... Igor Skornyakov

06:09 AM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> This is a good idea. I will implement such test today. To make it more realistic - how may ... Constantin Asofiei

04:17 AM Feature #2973: improve ContextLocal performance

BTW if we are talking about a small map with is very infrequently updated but frequently read may be it makes sense t... Igor Skornyakov

04:02 AM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> Igor, following Eric's response, we can't go with the approach in note 37/38, as the num... Igor Skornyakov

03:45 AM Feature #2973: improve ContextLocal performance

Igor, following Eric's response, we can't go with the approach in note 37/38, as the number of @ContextLocal@ instanc... Constantin Asofiei

02:10 AM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> However if your tests show that @Map.get()@ is a bottleneck the reason can be that this map ... Eric Faulhaber

04/07/2016

06:02 PM Feature #2973: improve ContextLocal performance

Task branch 2973a was rebased from the trunk revision 10998. Pushed up to revision 11002. Igor Skornyakov

03:43 PM Feature #2973: improve ContextLocal performance

Greg Shah wrote:
>
> Hynek: can you report on @SyncCoordinatesAspect.contextData@ (see above discussion)?
I bel... Hynek Cihlar

03:23 PM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> Another headache is how to hide/ensure no one else can instantiate a @ContextLocal@ with... Igor Skornyakov

03:16 PM Feature #2973: improve ContextLocal performance

Greg Shah wrote:
> > This way, we can remove the map overhead completely. If the performance is still not improved, ... Constantin Asofiei

03:06 PM Feature #2973: improve ContextLocal performance

> This way, we can remove the map overhead completely. If the performance is still not improved, then the bottleneck ... Greg Shah

02:54 PM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> ... @tokenMap@ keys are always @ContextLocal@ instances, which are defined as static va... Constantin Asofiei

02:48 PM Feature #2973: improve ContextLocal performance

Greg, I think we should consider how many elements will @tokenMap@ hold, on maximum. @ContextLocal@ is instantiated ... Constantin Asofiei

02:40 PM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> Constantin Asofiei wrote:
> > The logic in @cleanupWorker@ does this:
> > * step 1: it so... Constantin Asofiei

02:31 PM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> The logic in @cleanupWorker@ does this:
> * step 1: it sorts or keys by their weight, w... Igor Skornyakov

02:26 PM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> OK, I will double check. Please not that it doesn't make sense to check the predicate twice... Constantin Asofiei

02:23 PM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> Followup for note 29: OTOH, this can be a fallback for the incorrect @cleanupWorker(bool... Igor Skornyakov

02:18 PM Feature #2973: improve ContextLocal performance

Eric Faulhaber wrote:
> In terms of performance, it was on the fastest end of the range of my recent timings for thi... Igor Skornyakov

02:07 PM Feature #2973: improve ContextLocal performance

Followup for note 29: OTOH, this can be a fallback for the incorrect @cleanupWorker(boolean reset, Map tm)@ filter co... Constantin Asofiei

02:02 PM Feature #2973: improve ContextLocal performance

Constantin Asofiei wrote:
> Review for 2973a rev 10999:
> # the @tokenMapLock.readLock()@ and @tokenMapLock.writeLo... Igor Skornyakov

02:02 PM Feature #2973: improve ContextLocal performance

Eric Faulhaber wrote:
> I ran the full search test group. When I disconnected BPM at the end, while the P2J server w... Constantin Asofiei

01:54 PM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> I understand that NPE happens because FileSystemOps.WorkArea.fs is @null@. This can be of c... Eric Faulhaber

01:52 PM Feature #2973: improve ContextLocal performance

In terms of performance, it was on the fastest end of the range of my recent timings for this test group, but not dra... Eric Faulhaber

01:42 PM Feature #2973: improve ContextLocal performance

Eric,
I understand that NPE happens because FileSystemOps.WorkArea.fs is @null@. This can be of course a result of m... Igor Skornyakov

01:37 PM Feature #2973: improve ContextLocal performance

Review for 2973a rev 10999:
# the @tokenMapLock.readLock()@ and @tokenMapLock.writeLock()@ can be saved in @Security... Constantin Asofiei

01:31 PM Feature #2973: improve ContextLocal performance

Eric Faulhaber wrote:
> I ran the full search test group. When I disconnected BPM at the end, while the P2J server w... Igor Skornyakov

01:29 PM Feature #2973: improve ContextLocal performance

I ran the full search test group. When I disconnected BPM at the end, while the P2J server was still running, I got t... Eric Faulhaber

12:00 PM Feature #2973: improve ContextLocal performance

Greg Shah wrote:
> > However it seems that null is used in a single place: at the SecurityContextStack.setEditing me... Igor Skornyakov

11:52 AM Feature #2973: improve ContextLocal performance

> However it seems that null is used in a single place: at the SecurityContextStack.setEditing method. May be it is b... Greg Shah

11:32 AM Feature #2973: improve ContextLocal performance

Igor Skornyakov wrote:
> If we really need @null@ values we can wrap it into @Optional@. However it seems that @null... Eric Faulhaber

11:20 AM Feature #2973: improve ContextLocal performance

If we really need @null@ values we can wrap it into @Optional@. However it seems that @null@ is used in a single plac... Igor Skornyakov

10:48 AM Feature #2973: improve ContextLocal performance

Eric Faulhaber wrote:
> There is a regression when launching the P2J server:
> [...]
The ConcurrentHashMap doesn... Igor Skornyakov

10:46 AM Feature #2973: improve ContextLocal performance

There is a regression when launching the P2J server:... Eric Faulhaber

09:51 AM Feature #2973: improve ContextLocal performance

Eric Faulhaber wrote:
> Code review 2973a/10997:
>
> Yes, this is what I was looking for in the scope of note 5, ... Igor Skornyakov

09:49 AM Feature #2973: improve ContextLocal performance

I would like to see some testing that detects if this has a measurable performance impact (and what that might be).
... Greg Shah

09:19 AM Feature #2973: improve ContextLocal performance

Code review 2973a/10997:
Yes, this is what I was looking for in the scope of note 5, thanks.
One question: why ... Eric Faulhaber

08:52 AM Feature #2973: improve ContextLocal performance

Task branch 2973a was rebased from the trunk revision 10996. Committed as revision 10997. Igor Skornyakov

07:55 AM Feature #2973: improve ContextLocal performance

I've implemented optimization mentioned in the node 5 along with some refactoring.
Committed to the task branch 29... Igor Skornyakov

05:40 AM Feature #2973: improve ContextLocal performance

Created task branch 2973a from the trunk revision 10995. Igor Skornyakov