Project

General

Profile

Bug #9667

Bottleneck in BaseRecord.getActiveUpdateDiffs

Added by Alexandru Lungu over 1 year ago. Updated over 1 year ago.

Status:
Test
Priority:
High
Target version:
-
Start date:
Due date:
% Done:

100%

billable:
No
vendor_id:
GCD
case_num:
version_reported:
version_resolved:
production:
No
env_name:
topics:

History

#1 Updated by Alexandru Lungu over 1 year ago

  • Assignee set to Artur Școlnic
  • Priority changed from Normal to High
In a profiled scenario (#9652), there are 46k calls to BaseRecord.getActiveUpdateDiffs:
  • 26.8k caused by DMO setters (Handler.invoke)
  • 19.6k cause by Validation.validateMaybeFlush
    • 5.5k from RB.endBatch
    • 14k from Handler.invoke
    • 2 from RB.flush
The total time is ~1s and 302ms is own time. There are:
  • 207k calls to BitSet.get (336ms)
  • 254k calls to BitSet.getNextBit (413ms)
  • 9 calls to BigDecimal.equals (<0.1ms)
This also implies:
  • 207k HashMap instances being created
  • ~254k Object[] instance being created and put in the maps.

All times are affected by YK overhead.

Possible improvements:
  • In most cases of Validation.validateMaybeFlush, the output of getActiveUpdateDiffs is not used!
  • When it is required, it is used to report change. However, that cases only report the change that took place on one single index affected by the setter. Thus, there is no need to actually store all changes in a map.
Lets refactor getActiveUpdateDiffs to extract only the diffs we are interested of in advance.
  • I suppose we can pass down the ormIndex and extentIndex we are interested of to getActiveUpdateDiffs.
  • Is there any point in using getActiveUpdateDiffs inside validateMaybeFlush and return it instead of computing them up-front, before the call to validateMaybeFlush? I suppose not, so:
    • compute getActiveUpdateDiffs before
    • refactor validateMaybeFlush to return true/false (to replace whether it was supposed to return active diffs or not).
    • if it returned false, then ignore the output of getActiveUpdateDiffs

I think we can rename getActiveUpdateDiffs to getActiveUpdateDiff as it will gain one diff based on propIndex and extent.

#3 Updated by Artur Școlnic over 1 year ago

  • Status changed from New to WIP

#4 Updated by Artur Școlnic over 1 year ago

  • Status changed from WIP to Review
  • reviewer Alexandru Lungu added

I committed the code to 9667a/15716, it is ready for review.

#5 Updated by Alexandru Lungu over 1 year ago

  • % Done changed from 0 to 90
  • Status changed from Review to WIP
Review of 9667a:
  • From BaseRecord.getActiveUpdateDiffs
    if(dirtyProps.get(actualIndex) || !Objects.equals(activePropPreviousValues[actualIndex], 
    +                                                        data[actualIndex]))
    
  • it should be && instead of ||
  • also mind the needed space after if keyword

#6 Updated by Artur Școlnic over 1 year ago

I made the changes, considering that they are trivial, I will move on with testing.

#7 Updated by Artur Școlnic over 1 year ago

  • Status changed from WIP to Review

I committed the changes to rev 15717.

#8 Updated by Alexandru Lungu over 1 year ago

  • % Done changed from 90 to 100
  • Status changed from Review to Internal Test
The changes look functionally right. There are still some formatting issues:
  • validation.validateMaybeFlush(); is indented 4 spaces instead of 3. Same applies to diff = null; following if (!validateMaybeFlush(currentRecord, false, false))
  • The offsets of the properties are the keys. comment is redundant for addUnreportedChanges
  • please also move the
    // NOTE: if the property was denormalized, the property index [ormIndex] already contains
                       //       the extent index
    • comment with extentIndex.
These are minor, so please proceed with testing after fixing the formatting issues. Please focus on debugging some examples with:
  • changing extent fields of temporary tables (using normalized extent tables): double-check the diff is generated right, for the correct index.
  • changing extent fields of persistent tables (using denormalized extent fields): double-check the diff is generated right, for the correct index.
  • batch assign, buffer-copy, buffer-copy with assign (with temp and persistent)
Afterwards:
  • 2 large customer application unit tests and ChUI regression tests.

#9 Updated by Artur Școlnic over 1 year ago

Alexandru Lungu wrote:

Afterwards:
  • 2 large customer application unit tests and ChUI regression tests.

All the testing passed.

#10 Updated by Alexandru Lungu over 1 year ago

  • Status changed from Internal Test to Merge Pending

Please merge 9667a to trunk after 9615a.

#11 Updated by Artur Școlnic over 1 year ago

Branch 9667a was merged into trunk as rev. 15737 and archived.

#12 Updated by Alexandru Lungu over 1 year ago

  • Status changed from Merge Pending to Test

Also available in: Atom PDF