FWD » Core Development » Database

In 6129a, the prepared statement cache in TempTableDataSourceProvider is still configured to 100 - this was not enough, I will increase it in 6129a to 65536. In 3821c, is configured to 1000.

We need to allow configuration of all caches in the directory, plus document them.

This includes the query_cache_size=1024 for temp-tables in H2Helper.setCommonInMemoryProperties.

Alexandru Lungu wrote:

This is already implemented in 7388a.

We need also the documentation.

Alexandru, in #7669, I start seeing DynamicQueryHelper cache level #1 is full. Dumped 1 old entries - (some 750k message logs, this uses the default 65536, there is no config in directory.xml). Is there a way to get info when a cache size is small enough that it starts evicting entries aggresively? Maybe some JMX instrumentation? This may help to find some good values to tune the cache size.

OTOH, for this specific case, it may just be that the parsed predicate is 'one of a kind', in that it uses some unique value which is never reused, so caching it is moot anyway.

Alexandru Lungu wrote:

Danut, please point out to the documentation in this task and mark it as Review when finished.

The documentation for the cache sizes is available in Database Configuration. Currently the CacheManager stores and uses the cache sizes defined in the directory configuration, each cache size is retrieved internally when creating caches (LRUCache or Map). If we want to use the CacheManager to set the size for the property mentioned in #6815-2, we can simply make CacheManager.getCacheSize method public. Should I go ahead and do that?

OTOH, for this specific case, it may just be that the parsed predicate is 'one of a kind', in that it uses some unique value which is never reused, so caching it is moot anyway.

Maybe we can make them "more of a kind" :) I don't know if this is possible without context.

Is there a way to get info when a cache size is small enough that it starts evicting entries aggresively? Maybe some JMX instrumentation? This may help to find some good values to tune the cache size.

I think we have some specific JMX on some caches ... or maybe not. Now that we use the CacheManager, I see a really neat implementation: add dynamic JMX information for each cache generated by the CacheManager. This way we can instrument all caches for fine tuning. This is not implemented, but can be done. I don't quite have the inspiration now to design and implement this, so I will ask Danut to tackle this.

In my head, having a fat JMX storing a map Cache Name -> Cache Statistics is the way to go. Take for example QueryProfiler that for each query is computing statistics like cache hit / miss / row count etc. I see cache statistics as storing the number of elements / evicts cache hit / etc. To avoid any potential overhead, maybe we can deliver InstrumentedLRUCache from cache manager if we want to do the instrumentation of simply LRUCache if in production. Constantin, any feedback on this?

Committed 6815a/rev.14917. Removed unnecessary methods and changes the container accessed for the instrumentation and cache-sizes. The container accessed is at the same level as the persistence container and is called cache-config.

There were more methods instrumented, but I just noticed an issue that invalidates some of the results so I solved it in 6815a/rev.14918.

What other statistics can be achieved? I suspect the ratio is the most important one, so lets get it done. Also, we need a size, so we can understand in what circumstances were that statistics achieved.
Secondly, please address the export function. We want to have an easy to read format (table-like). Maybe use the Textile format we also use for Redmine (for easy copy-paste).
Mind that I didn't review the changes yet; please address the features above first.

Finally, we need to segregate the GET into GET hit and GET miss. We also need the timings separated. Also, we need different options for "instrument" and "profile". We can allow instrumentation without considerable overhead, but profiling may introduce too much of an overhead. Lets have them enabled separately.

I added monitoring for the configured and the in use cache size (latest size of the cache), GET hit/miss and modified the export method to print the results in the Textile format but the resulted table is too big to be posted on Redmine due to the number of instrumented methods. Can we consider redirecting the output to a csv file by adding an additional parameter to toggle between the formats or disregard the Textile format altogether?

Alexandru Lungu:

Also, we need different options for "instrument" and "profile".

I think instrumentation should always be enabled so that method calls can be counted, profiling should only monitor the execution time of the method and can be an additional parameter and when it is used, to calculate the average method call time based on the counted calls.

Dănuț Filimon wrote:

I added monitoring for the configured and the in use cache size (latest size of the cache), GET hit/miss and modified the export method to print the results in the Textile format but the resulted table is too big to be posted on Redmine due to the number of instrumented methods. Can we consider redirecting the output to a csv file by adding an additional parameter to toggle between the formats or disregard the Textile format altogether?

Why is it too big? Shouldn't it be similar to #6815-11, but with some extra columns?
Anyway, you have only the possibility to use print with our FWD JMX. CSV can do to be easily imported in Spreadheets.

I think instrumentation should always be enabled so that method calls can be counted, profiling should only monitor the execution time of the method and can be an additional parameter and when it is used, to calculate the average method call time based on the counted calls.

• the CacheManager should easily return bare caches when instrumentation is disabled
• the CacheManager should deliver instrumented caches when instrumentation is enabled,
• the cacheManager should deliver instrumented caches with profiling enabled only when both instrumentation and profiling are enabled.

I think these settings can go in the directory.xml together with the sizes. After the size definition, we can enable instrumentation and profiling eventually (but they are disabled by default). Also, if you have a cache configuration in directory.xml, but no size, let the default size kick in.

Alexandru Lungu wrote:

Why is it too big? Shouldn't it be similar to #6815-11, but with some extra columns?
Anyway, you have only the possibility to use print with our FWD JMX. CSV can do to be easily imported in Spreadheets.

Not only get and put methods are instrumented, methods like containsKey, containsValue, getOrDefault, putAll, putIfAbsent, remove, replace are methods that are also instrumented. If you also want to create an "Average" between call count and execution time we will have additional columns. I am currently looking into removing some of the methods that are not called based on previous instrumentation results.

The instrumentation implies overhead which is not desired in production systems. Instrumentation (and profiling) is meant to be used by us / system administrators that want to tune the caches in testing environments. Thus, please disable the instrumentation by default.

• the CacheManager should easily return bare caches when instrumentation is disabled
• the CacheManager should deliver instrumented caches when instrumentation is enabled,
• the cacheManager should deliver instrumented caches with profiling enabled only when both instrumentation and profiling are enabled.

I think these settings can go in the directory.xml together with the sizes. After the size definition, we can enable instrumentation and profiling eventually (but they are disabled by default). Also, if you have a cache configuration in directory.xml, but no size, let the default size kick in.

I took it in account and this is already handled.

Danut, please merge some of the statistics in one: get, getOrDefault can be merged. The same as put, putAll and putIfAbsent. The others can be standalone: containsKey, containsValue, remove and replace.