FWD » Bugs

I've noticed as I've been debugging a TRPL error during runtime conversion, that the class names for TRPL expressions keep advancing. The convention for compiled TRPL expressions is CE (for "Compiled Expression") followed by a number suffix, which advances sequentially. In trying to track down an NPE in a specific init-rule in brew.xml, I noticed the class name for the same compiled expression kept changing, each time with a higher numeric suffix.

Currently, we cache each compiled expression instance, such that if we encounter the same expression multiple times, we fetch the cached instance of the compiled version and use that instead of taking the performance and memory hit of compiling and instantiating it again. Each instance is cached at a certain scope, which can be a rule set, a named function, or the pattern engine itself. The cache is a two-level cache. The outer level is a weak hash map whose keys are the scope objects (rule sets, named functions, pattern engine). For each scope, there is a map at the inner level of the cache, which has infix expression strings as its keys and the compiled expression instances as its values.

We cache the instances of the compiled expressions rather than the classes, because the initialization of these objects uses information (such as the libraries they reference and the variables that are in scope at the time), which is only available and known at the time we are parsing/compiling the expression.

This approach works OK for the most part (though it must also leak PermGen memory, since we never unload the classes themselves as the scope keys are garbage collected), as long as you only have one instance of the pattern engine and of each unique rule and named function. This is how it works during static conversion (for which it was designed).

Problem is, now we instantiate PatternEngine for every dynamic conversion operation and the rule sets and named functions are loaded as new instances constantly. So, now the cache is not useful across dynamic conversion runs (different instances of the pattern engine and all other scope key objects). Not only do we have the original class leak, but we are compiling the same expressions into new classes and instantiating them all the time. While the instances will be garbage collected when the scope objects are freed, the classes will add up in PermGen heap.

I don't want to unload the classes, since there are a finite number of them if we fix the re-compilation problem. Thus, PermGen should reach a steady state eventually. I think the fix lies in correcting the cache behavior, but I have to think more on how best to do that.