FWD » Core Development » Database

Currently, BaseRecords are cached in the session cache, which has a few drawbacks:
- fairly small capacity;
- one instance per each session (short lived);
- multiple cache instances can hold the same record?..
- session reclaiming moves cached records from the old cache to the new one (additional overhead);

I think the session cache should be smaller and used only for dirty records and records loaded in (recently) active buffers. The main caching of the latest images of the DMOs (non dirty) should be deferred to a database specific, shared (by all sessions) cache.
FastFindCache is shared and has solid invalidation logic, I think it should be backed by an equally potent records cache. A session should be able to cache a record and, if it is not changed, another one should be able to retrieve it from the same cache with minimal overhead.

A database specific records cache could be 'simulated', for testing purposes, by increasing the existing session case capacity and enabling infinite session reclaiming, this means that the cache will persist for a long duration and will not be reinitialized. To have an understanding of the potential performance increase, I performed the following experiment using a customer project:

1. Execute one 1500 records lookup API
2. Execute a series of unrelated APIs
3. Execute the first API again

Time for the 3rd step.
Infinite session-lifespan + large session cache: 1.76s
Default settings: 2.56s

In this scenario I get a consistent 30% increase, I expect a shared cache to yield similar results.

- multiple cache instances can hold the same record?..

Yes, different sessions can cache different versions of the same DMO. Sometimes, they cache the exact same version for tables that are were just read (not updated)

- session reclaiming moves cached records from the old cache to the new one (additional overhead);

Not cached records, but records that are loaded in active buffers. The overhead is negligible IMHO. The heavy part was searching inside all buffers.

I think the session cache should be smaller and used only for dirty records and records loaded in (recently) active buffers.

This is what is happening now. 1024 may be big if you look at it that way, but it provided a good memory/performance ratio by now.

The main caching of the latest images of the DMOs (non dirty) should be deferred to a database specific, shared (by all sessions) cache.

• by nature DMOs are per-session. Each time you look-up in this cache you suggest, you will require a cloning of that data. It may be negligible.
• you need synchronization on access. If all sessions query/update this collection, then there may be a lot of time wasted on consulting this shared cache. FFC is a bit more relaxed IMHO, comparing to session cache.
• What if a session is opening a transaction, fetches a record, changes it, releases (so it determines a flush), somehow the records gets evicted and then that record is READ again. There is no mechanism AFAIK in FWD to understand if the READ DMO was changes in the current transaction or not (aka safe to be shared).

PS: please also check #9053.

Alexandru Lungu wrote:

• by nature DMOs are per-session.

Can't this be changed? I don't see why a dmo should belong to a specific session, it should be the representation of a record in the DB, which can be accessed by multiple sessions.

• you need synchronization on access. If all sessions query/update this collection, then there may be a lot of time wasted on consulting this shared cache. FFC is a bit more relaxed IMHO, comparing to session cache.

I am proposing a shared cache backed up by the FFC, so only for RAQ results.

• What if a session is opening a transaction, fetches a record, changes it, releases (so it determines a flush), somehow the records gets evicted and then that record is READ again. There is no mechanism AFAIK in FWD to understand if the READ DMO was changes in the current transaction or not (aka safe to be shared).

Isn't this what the dirty state means? A record that is not the latest representation of the DB data?

Can't this be changed? I don't see why a dmo should belong to a specific session, it should be the representation of a record in the DB, which can be accessed by multiple sessions.

A transactional database can have different representation of the same record at the same time.

Isn't this what the dirty state means? A record that is not the latest representation of the DB data?

Well, it is the latest representation of the DB data from the POV of that transaction. This "latest representation" shouldn't be leaked. I presume the cache should contain only COMMITTED DB data; this is what I want to point out. When a FIND is executed, data is either committed or not ... can't tell.

Alexandru Lungu wrote:

A transactional database can have different representation of the same record at the same time.

Yes, but a single 'true' or committed state.

I presume the cache should contain only COMMITTED DB data;

yes

this is what I want to point out. When a FIND is executed, data is either committed or not ... can't tell.

We keep the current session cache implementation for uncommitted data and a shared cache for committed data.
I think we would first consult the session cache in case we are in a transaction that recently changed the record, but did not commit the changes yet. If not found there, then we consult the shared cache for the 'true' record. If not found in the shared cache, then it must be the first find or the record was invalidated due to a data change (so no time based invalidation of any kind)

I managed to write an initial implementation, which is far from perfect, but stable enough to recreate the test from #10583-3.

Trunk reclaiming enabled: 1.83s
Trunk reclaiming disabled: 3.32s
10583 (reclaiming disabled): 1.45s

I registered a 20% improvement over trunk with session reclaiming and 56% (2x) over trunk with no reclaiming.
AFAIK we have customers that have the reclaiming disabled, so for them a shared cache would make a huge difference.
Also any request that exceeds the current (1024 records) buffer will be faster due to the larger cache.

I will request a review once the branch is fully stable.

• Each FFCache instance gets a new cache for storing records.
• Any valid RecordIdentifier in FFC also has the associated record in the records cache
• The new records cache stores only non dirty records (at the moment of insertion)
• The shared cache does not store a copy of the DMO, the overhead for this approach outweighs the benefits
• The shared cache will serve only non-dirty DMOs, since one can become dirty after inserting
• All access to the shared cache is synchronized
• All the dirty DMOs and the ones retrieved from P2JQueries other than RAQ, will still be cached in the session cache as before.
• Session reclaiming is disabled, there is no longer need for copying the 'active' records from an old session to a new one.

For measuring the impact of cached synchronization, I executed 20 concurrent (same) API calls that request 2k records
10583a:
time: 0m20.320s
time: 0m6.023s
time: 0m5.754s
time: 0m6.232s
time: 0m5.931s

trunk:
time: 0m19.398s
time: 0m5.982s
time: 0m5.561s
time: 0m11.928s
time: 0m5.201s

There maybe is a hit of under 10%, concurrency testing is tricky, but I wonder how often will 20 sessions try to retrieve from the cache the same dmo.

When it comes to the general performance improvement, it is about 20-25% across all lookup type APIs, when chain executing, so one after the other, simulating a real life scenario.
For a single API call that exceeds the current cache size by 500 records (1500 in total, which is reasonable), the improvement is ~30%.

I commented out some code regarding session reclaiming, but we will need to decide how to handle it. As a side note, 10583a is not compatible with reclaiming.

The results are promising, but we need to employ cautious optimism until the review and exhaustive testing results :)

The code is in 10583a/16170.
Alexandru, please review.

I analyzed the impact on memory consumption. There is an increase in the retained memory, each cached record will retain an additional ~800 bytes of heap, for the current limit of 100k cached record this amounts to ~80Mb of total additional consumed memory. I think is is an insignificant trade-off for the improvement we are seeing. I will continue to monitor the memory allocation for the duration of this task, but it seems reasonable so far.

Alexandru Lungu wrote:

• I don't think that //initialize.accept(session); is right in SessionFactory. This was responsible of reconnecting the DMOs from active buffers to the new session. Without these, buffers will continue to use untracked records that may be STALE.

I will find another way to achieve this. My goal there was to eliminate the need for copying the records from one cache to another. Additionally, the initialize method was breaking the current implementation of the shared cache.

• Having isFromRAQ and setFromRAQ makes the session implementation quite fragile. Session in FWD was based on the Hibernate Session in the old days and it continues to provide the same principles. Hibernate (or maybe other ORMs) do not have a dependency on the query type when consulting the cache. That is why I don't think it is a good approach to do so in FWD ORM.

I need to know whether a query is RAQ at the Session cache level for cache management purposes.

• DMOs being per session is a hard-constraint. This shared cache should only hold snapshots. Otherwise, caching a record and then changing it will automatically change the cached DMO (because the cached object is mutable).

Caching a copy of the record is out of the question, the overhead is way too taxing.
I understand that the shared cache can have dirty records in it, but I do not serve them. Additionally, I think we can be pretty sure that a dirty record state is a fleeting moment, usually the changes are quickly committed.
I don't think that this separation of concerns is such a bad idea. We keep the dirty records in the session cache, seeing how only the session instance should have access to it's dirty records, and keep the non-dirty records in the shared cache. If a cached record is requested, but it has uncommitted data in it, we simply do not serve it, it will be retrieved from the session cache if the changes are from this session or from the DB if it is the first find.

• Checking out a DMO should also do a clone to avoid doing changes on the cached instance.

Can't do that either due to the same copying overhead (plus memory consumption).

The only problem I see with the current implementation is that two (ore more) concurrent sessions can retrieve the same dmo from the shared cache and concurrently modify it, which is a big problem, but it is easily solvable by removing the cached record at the retrieve step and inserting it back when the session commits. I think this way there won't be any dirty records in the cache at all. I will try this approach and report back with the results.

Artur Școlnic wrote:

it is easily solvable by removing the cached record at the retrieve step and inserting it back when the session commits.

This indeed solved the shared record problem, but it also negated any performance improvement. I will think of something else. We need to store just the references to the records, but also support a safe mutability across multiple sessions.

I changed the approach a bit. Now dmo references are being cached in the shared cache and in the session cache, on retrieval, if the record is not found in the session cache, it is retrieved from shared cache as a copy of the dmo.
If the dmo is dirty or stale, it is not retrieved from shared cache.
The branch is stable and the testing passes, but there is a fundamental potential issue. A record that is non dirty and non stale is not necessarily committed, and we can't know in the FWD layer if a record is committed or not. We will have to come up with a clever mechanism of ensuring that only 'true' committed records are served by the shared cache.

Artur Școlnic wrote:

We will have to come up with a clever mechanism of ensuring that only 'true' committed records are served by the shared cache.

I suppose a session hook could work, it would store Runnables that insert in the cache and will execute only on commit. I will try it and report back.

I am documenting here a live discussion with Artur. The solution with exposing record to the shared cache on commit or session close time makes sense. There is still a doubt on whether the actual DMO instances should be added to the share cache of clones. Ideally we would simply put the DMOs, if the closing session would not need that DMOs anyway. There is still uncertainty whether exposing at commit time is right, because the DMOs can still be in-use by the current session.

Anyway, we concluded that we should take a step back and design a more complex multi-session suite to validate the approach using harness. Tomasz can help on feedback on how to syncrhonize FWD clients for these tests. Artur, to be noted, we know we lack a proper multi-session test-suite, so this is the time to put some incipient effort into it. Ideally, the test suite will pass on trunk and guarantee that the current FWD is sound. With that in mind, we can also test 10538 approached and incrementally extend the suite with flaws we spot at dev time (e.g. #10583-17, #10583-15, etc.).

Overall: I feel like back and forth implementation-review process should materialize in some long running tests. Any question/implementation detail/uncertainty should be rather proved with a harness multi-user test. My dream is to have such suite integrated in the CI/CD and have a better coverage on FWD for multi-user tests. (#9270) Greg, we had some discussions in the testing meetings previously about this and so #9270 was finished. Can Artur take the lead into this area of multi-session functional and performance testing (at least for the persistence layer)?

My dream is to have such suite integrated in the CI/CD and have a better coverage on FWD for multi-user tests. (#9270) Greg, we had

some discussions in the testing meetings previously about this and so #9270 was finished.

Yes, the approach is done and I think all the code is in Testcases. But we still need to finish some documentation in Writing Multi-User Testcases. Tomasz will make that a priority.

Can Artur take the lead into this area of multi-session functional and performance testing (at least for the persistence layer)?

Yes, please!

Greg Shah wrote:

Yes, please!

Ok, I will get on it then.
As for 10583a, the implementation is completed. At this point it just should be thoroughly tested.

Will these tests also handle cross-session dirty share (i.e. testing for #8388)?

Greg Shah wrote:

Will these tests also handle cross-session dirty share (i.e. testing for #8388)?

AFAIK, cross-session dirty-share is not going to be supported anymore, so there is no point in testing that. We have Majic tests for cross-session and I think that suffices.
#8388 is about intra-session dirty-share and I have unit tests for it. Multi-user tests are only to prove that #8388 doesn't leak records by mistake to other sessions.
With this being said, the baseline for the harness will not be from 4GL as we need to check results when cross-session is disabled (to match production environments)

The harness suggested now is about:

• locking mechanisms
• transaction isolation
  • FWD still has the FFC that leaks record in between sessions incorrectly
  • DMOs should not leak in between sessions
  • invalidation shouldn't occur if two sessions work on the same table concurrently
• etc.

I posted the current test suite in #10664, 10583a passes it.