FWD » Core Development » Database

From Constantin:

• AbstractTempTable.getMutableInfo - a plain array can be kept for cases when the LegacyFieldInfo instance is already resolved (thus each field has an unique index in this array).
• TableMapper.locatePerm - each schema can have an unique index in an ArrayList. This can get managed via removePermanentSchema and mapPermanentDMO, and the only issue is to compress the permanentDBs (which will be an ArrayList) if its size gets over a certain threshold
• something similar as above can be done for TempTableMapper.TT_CACHE and TempTableMapper.p2j, and also PermanentTableMapper.p2j and j2p maps (where possible).
• the point of the above is: if the API's caller has access to the DmoMeta, we can have an unique index which will be used instead of its name.

In TempTableMapper there is a private final Map<TempTable, LegacyTableInfo> p2j = new HashMap<>(); which should be replaced by an ArrayList.

I tried to index LegacyTableInfo elements in the follwing way, get the DmoMeta data member of TempTable using getDMOInfo method, after that get the DmoMeta id, and use that id as an index in the ArrayList I want to replace p2j.

The problem in current implementation is that TempTable does not have an override for equals and hashCode methods, this means that elements in p2j are mapped base on references of TempTable objects. And multiple elements with the same DmoMeta id , are introduced as keys in p2j, which obviously could not happen with ArrayList.

Also when running a converted application, in on case getAllLegacyFieldInfo method from TableMapper is called after removeClass of TempTableMapper with TempTable parameter. Please follow the table:

So removeClass deletes mapping from index 23 from ArrayList, remove from p2j mapping with reference 14234.
And getAllLegacyFieldInfo method called after removeClass, find a mapping with reference 1321 in p2j, and find a null at index 23 in ArrayList.

Created 7641a.

Committed on 7641a revision 14683.

fieldInfo in AbstractTempTable became an ArrayList, instead of HashMap.
permanetBDs in TableMapper became an ArrayList, instead of HashMap.
j2p in PermantTableMapper became an ArrayList, instead of Hashmap.

p2j from PermanentTempTable probably can not be switch to ArrayList because methods where p2j is used, returns the DmoMeta which we use to get an id or an index.
Switching p2j from TempTableMapper to ArrayList cause the problem from #7641-3

Eduard, fieldInfo (and other structures related to fields) can be 'frozen' as a Java array once this table info was built. This will remove the ArrayList overhead.

Made fieldInfo a simple Java array, also added to getMutableInfo a new parameter fieldCount, those meaning is number of tables' columns.

TableMapper.MutableFieldInfo getMutableInfo(Integer                    fieldIndex,
                                               int                        fieldCount,     
                                               boolean                    setter, 
                                               Supplier<TableMapper.MutableFieldInfo> field)
{
    if (!setter)
    {

       if (fieldInfo == null || fieldInfo.length <= fieldIndex)
       {
          return null;
       }

       return fieldInfo[fieldIndex];
    }

    if (fieldInfo == null)
    {
       fieldInfo = new TableMapper.MutableFieldInfo[fieldCount + 1];
    }

    TableMapper.MutableFieldInfo info = fieldInfo[fieldIndex];

   if (info == null)
   {
     fieldInfo[fieldIndex] = field.get();
   }

   return fieldInfo[fieldIndex];
}

I am getting fieldCount parameter from DmoMeta, getFieldCount method.

Rebased 7641a to trunk. 14711.

Committed refactoring the code on 7641a revision 14715.

I continued improving 7641a, which is now at rev. 14720. Basically, I moved SchemaHelper to persist, instead of orm package. I also cached the schema id at persistence level, instead of database level. Finally, I ensured that there is no map.get() call through AbstractTempTable, TableMapper and such.

Profiling is not quite good, out of an unknown reason. With rev. 14717 I've got 8.210s of execution time and with rev. 14720 I've for 8.230s of execution time. The base-line is 8.193s. Even so, I've got quite mixed results with rev. 14820 (first run in 8.149, second in 8.259s, one run in 8.329s and another in 8.067s).

I will do a final review tomorrow morning.

Alexandru Lungu wrote:

As this doesn't provide consistent improvements in this state, I am pending this for further review / advice.

I'll do some profiling when I get a chance.

Constantin - I wanted to remember you of this task, considering you have worked recently on performance matters. 7641a is (very) old and needs to be rebased.
My point is that we shall include this on the performance plan eventually and "revive" the effort. Also, avoid from extensive changes on Map.get() calls - lots are already done in 7641a.

I am adding this to the performance plan.

So, I tried to rebase, but I get that error with key already present in map. devsrv01 is too slow to do the rebase there. I think if it's worth creating a new 7641b branch and cherry pick the commits from 7641a.

Alexandru Lungu wrote:

I think we already invested to much effort on too little results for #7641. I will reject this task. Stefanel, please archive the associated task branches.

Done, I archived 7641a and 7641b as dead.