public final class PatternEngine extends RuleContainer
ConfigLoader
object is used to load a profile into the engine,
or a profile may be configured programatically using the public methods
of this class.
A profile can be defined to perform a variety of analysis or transformation
applications. This is done via the run()
method. The logical program
flow (assuming a pre-defined profile is being used) is as follows:
In cases where multiple file lists must be processed using a single profile, the caller may choose to use the following idiom instead:
initialize(java.lang.String)
once only.
processFiles(java.util.Iterator<java.io.File>)
one or more times.
finish()
once only.
The pattern engine can walk either an arbitrary list of ASTs (default mode) or can traverse a call graph (a tree, really). In call graph mode, the engine uses one or more file specifications to load persisted call graph profiles and then walks the call trees defined by these profiles. If no call graph profiles are specified, it will try to load the project- specific root node list, which is a list of source file paths which represent all known program entry points for the target application.
Note: at this time, the pattern engine is not
designed to process multiple profiles in parallel. However, it IS possible
to run multiple pattern engines (with different profiles if desired)
SEQUENTIALLY as long as the initialize
method is called at
the beginning and the finish
method is called at the
end of each run AND in each case a different instance of the pattern
engine is used. Note that usage of the run()
method naturally
handles the proper bracketing of initialize
and
finish
. Multiple instances of this class should NOT be
created at the same time. The singleton pattern is not explicitly
enforced at this time, as this limitation may be lifted with further
development.
RootNodeList
Modifier and Type | Class and Description |
---|---|
static class |
PatternEngine.StorageKey
Container for an arbitrary category and id that is supposed to be used as a key for the
storedObjects map. |
Modifier and Type | Field and Description |
---|---|
private java.util.Set<java.lang.String> |
activatedFlags
The flags which must be set at the rule-set for the rule-set to be processed.
|
private Rule |
activeRule
Rule currently being (or most recently) processed
|
private java.util.List<FileList> |
astSpecs
List of AST filespecs used to locate persisted AST files.
|
private boolean |
conditionalRuleSets
Flag indicating that conditional rule-set processing is activated or not.
|
private java.lang.String |
directory
Default root directory used to locate persisted AST files.
|
private java.lang.String |
fileSpec
Filespec used to locate persisted AST files relative to a directory.
|
private org.apache.tinkerpop.gremlin.structure.Graph |
graph
Graph DB instance.
|
private boolean |
honorHidden
Honor hidden processing mode flag.
|
private long |
jobStartTime
System time at beginning of run.
|
private java.util.List<java.lang.String> |
newProgramFiles
List of newly created AST files by the rules processed by this engine.
|
private GraphNodeFilter |
nodeFilter
In callgraph walking mode, this will be used to filter the nodes which will be processed.
|
private java.util.Set<java.lang.Long> |
processedAsts
Set of visited AST IDs for use with call graph mode to prevent dups
|
private boolean |
readOnly
Operate in read-only mode; do not copy source AST for each ruleset
|
private AstSymbolResolver |
resolver
Object used by rules to resolve expression variables and functions.
|
private java.util.Map<PatternEngine.StorageKey,java.lang.Object> |
storedObjects
Map to store resulting objects like ASTs or streams during rules pipeline processing.
|
private java.util.List<Aast> |
targetAsts
List of ASTs to be processed.
|
private java.util.List<java.lang.String> |
targetNames
List of explicit target AST names.
|
private java.util.Set<java.lang.String> |
targetPaths
Ordered set of target paths of files to be processed.
|
private java.util.Map<java.lang.String,java.lang.Object> |
varInitializers
Mapping of user variable names to initializer expression strings or values.
|
private java.util.Iterator<Aast> |
view
Current AST results view from last ruleset processing in pipeline.
|
RULE_ASCENT, RULE_DESCENT, RULE_INIT, RULE_NEXT_CHILD, RULE_NONE, RULE_POST, RULE_WALK
MSG_DEBUG, MSG_NONE, MSG_STATUS, MSG_TRACE
Constructor and Description |
---|
PatternEngine()
Default constructor.
|
PatternEngine(java.lang.String profile,
boolean conditional)
A constructor for runtime conversion use to support a limited use mode, which does not
support some options, such as callgraph mode.
|
PatternEngine(java.lang.String directory,
java.lang.String fileSpec)
Constructor which accepts a directory and filespec to use instead of
those (if any) contained in a pattern profile.
|
Modifier and Type | Method and Description |
---|---|
boolean |
activeFlags(java.util.Set<java.lang.String> flags)
Check if the given flags are part of the flags registered in the
activatedFlags set. |
void |
addAstSpec(FileList list)
Add a list of persisted AST files to be processed into ASTs, which
will in turn be applied against a pipeline of rulesets.
|
void |
addAstSpec(java.lang.String directory,
java.lang.String fileSpec,
boolean force)
Add a directory and file specification pair which will allow the engine
to locate a set of persisted AST files to be processed into ASTs, which
will in turn be applied against a pipeline of rulesets.
|
void |
addConditionalFlag(java.lang.String flag)
Adds a flag to the
activatedFlags set, which will be checked only when
conditionalRuleSets is activated. |
void |
addNewProgramFile(java.lang.String file)
Add the given filename as being newly generated, so it will be registered and processed
by other rule-sets, downstream.
|
void |
addTarget(java.lang.String name)
Add the specified AST name as a target.
|
void |
addTargetAst(Aast ast)
Add AST to be processed against a pipeline of rule sets.
|
private Aast |
apply(RuleSet ruleSet,
Aast source)
Apply the given ruleset against the specified source AST, using the
rules and configuration information stored within the ruleset.
|
private void |
applyGlobal(int type)
Runs the global rules defined for a pipeline and DOES NOT recursively
call any rules in any contained ruleset.
|
java.util.Iterator<FileList> |
astSpecs()
Get an iterator over the ast specifications (
FileList objects). |
void |
clearNewProgramFiles()
Clear the list of newly, generated,
program file names . |
void |
clearStoredObjects()
Clear the map of objects currently stored with this pattern engine.
|
void |
finish()
Perform cleanup/termination post-processing, including applying all
global, user-defined post-rules, calling each pattern worker's
PatternWorker.finish() method, and nulling out the symbol resolver. |
private void |
gatherTargetPaths()
Assemble the master set of filenames to be processed.
|
java.util.Map<PatternEngine.StorageKey,java.lang.Object> |
getAllStoredObjects()
Get an unmodifiable map of all objects currently stored with this pattern engine.
|
java.util.List<java.lang.String> |
getNewProgramFiles()
Get the list of newly added program file names.
|
java.lang.Object |
getStoredObject(java.lang.String id)
Get the object stored under the given id.
|
java.lang.Object |
getStoredObject(java.lang.String category,
java.lang.String id)
Get the object stored under the given category and id.
|
void |
initialize(java.lang.String profile)
Initialize the pattern engine by creating a new symbol resolver
instance, registering default pattern workers, loading the specified
profile, ensuring an AST spec is available, and applying all global,
user-defined initialization rules.
|
boolean |
isConditionalRuleSets()
Check if conditional rule-sets is activated.
|
boolean |
isFlagActivated(java.lang.String flag)
Check if the given flag is part of the
activatedFlags set. |
boolean |
isHonorHidden()
Gets the flag that determines if the hidden status of an AST node
should be honored.
|
static void |
main(java.lang.String[] args)
Command line driver to launch the pattern engine, using a pre-defined
pattern profile.
|
private java.lang.String |
prepareFlag(java.lang.String flag)
Prepare the given flag by trimming any whitespace and converting it to uppercase.
|
private Aast |
processAst(Aast ast)
Apply the pipeline of rulesets currently loaded against a single AST.
|
private Aast |
processAst(java.lang.String astName)
Apply the pipeline of rulesets currently loaded against a single AST.
|
void |
processFiles(java.util.Iterator<java.io.File> iter)
This will apply the pipeline of rulesets defined in the currently
loaded profile to every AST it finds based upon the given iterator
that lists the
File objects that must be processed (and
the inherent order in which these files must be processed). |
private void |
registerDefaultWorkers()
Registers all default pattern workers.
|
void |
registerVariable(java.lang.String name,
java.lang.Class<?> type,
java.lang.String expression)
Override the superclass' implementation of this method to allow for
replacement of variables' initializer expressions with values provided
by the user.
|
void |
removeConditionalFlag(java.lang.String flag)
Removes a flag from the
activatedFlags set. |
void |
run()
Perform a discrete pipeline pattern run against a set of target ASTs.
|
void |
run(java.lang.String profile)
Run the pattern engine once using the specified configuration profile.
|
(package private) void |
setActiveRule(Rule activeRule)
Set the rule which is currently active.
|
void |
setConditionalRuleSets(boolean state)
Enable or disable conditional rule-sets mode.
|
void |
setGraph(org.apache.tinkerpop.gremlin.structure.Graph graph)
Set the Graph DB instance used by this pattern engine.
|
void |
setHonorHidden(boolean hide)
Sets the flag that determines if the hidden status of an AST node
should be honored.
|
void |
setReadOnly(boolean readOnly)
Set state of read-only mode.
|
void |
setVariableInitializers(java.util.Map<java.lang.String,java.lang.Object> initializers)
Store a mapping of user variable names to initializer expression
overrides (in infix notation), as extracted from the command line
arguments passed to this process.
|
void |
storeObject(PatternEngine.StorageKey key,
java.lang.Object object)
Store the given object under the given key.
|
void |
storeObject(java.lang.String id,
java.lang.Object object)
Store the given object under the given id.
|
void |
storeObject(java.lang.String category,
java.lang.String id,
java.lang.Object object)
Store the given object under the given category and id.
|
private static void |
syntax(java.lang.String msg,
int rc)
Emit a syntax statement and optional error message to
stderr , then exit the process with the specified return
code. |
private java.lang.String |
toNormalizedPath(java.lang.String path)
Normalize the specified path if possible, to eliminate unnecessary,
embedded, relative references and to prepend the project's home
directory, if
path is not absolute. |
private void |
walkGraph(org.apache.tinkerpop.gremlin.structure.Vertex node)
Walk the call tree recursively, depth-first, from the specified node.
|
addRule, addRuleSet, apply, cleanup, getDebugLevel, getEnclosingScope, getFunction, getFunctionContainer, getWorker, hasType, include, isContainer, putFunction, registerVariableValue, registerWorker, registerWorker, ruleList, setDebugLevel
private final java.util.List<FileList> astSpecs
private java.util.List<java.lang.String> newProgramFiles
private java.util.List<Aast> targetAsts
private java.util.List<java.lang.String> targetNames
private java.lang.String directory
private java.lang.String fileSpec
private AstSymbolResolver resolver
private java.util.Iterator<Aast> view
private java.util.Set<java.lang.Long> processedAsts
private org.apache.tinkerpop.gremlin.structure.Graph graph
private boolean honorHidden
private boolean readOnly
private java.util.Map<java.lang.String,java.lang.Object> varInitializers
private java.util.Set<java.lang.String> targetPaths
private Rule activeRule
private long jobStartTime
private final java.util.Map<PatternEngine.StorageKey,java.lang.Object> storedObjects
private boolean conditionalRuleSets
private final java.util.Set<java.lang.String> activatedFlags
private GraphNodeFilter nodeFilter
public PatternEngine()
public PatternEngine(java.lang.String directory, java.lang.String fileSpec)
directory
- Directory used to find ASTs to load; will override any
provided in the pattern profile if not null
.fileSpec
- File specification used to find ASTs to load; will override
any provided in the pattern profile if not null
.public PatternEngine(java.lang.String profile, boolean conditional) throws ConfigurationException
profile
- Name of the profile to be loaded. It is up to the ConfigLoader
implementation to understand what to do with this name.conditional
- If true
, enable conditional rule-set loading for runtime query mode.java.lang.IllegalStateException
- if invoked when not in runtime use mode.ConfigurationException
- if there is an error loading the profile.public void addConditionalFlag(java.lang.String flag)
activatedFlags
set, which will be checked only when
conditionalRuleSets
is activated.
Use setConditionalRuleSets(boolean)
to activate these flags before starting
conversion.
flag
- the conditional flag to add.public void removeConditionalFlag(java.lang.String flag)
activatedFlags
set.
Use setConditionalRuleSets(boolean)
to deactivate these flags after conversion is
finished.
flag
- the conditional flag to add.public void setConditionalRuleSets(boolean state)
When activated, only rule-sets marked with flags part of the activatedFlags
set will be included.
state
- The new state of the conditional rule-sets mode.public boolean isConditionalRuleSets()
conditionalRuleSets
flag.public boolean isFlagActivated(java.lang.String flag)
activatedFlags
set.flag
- The flag to check.true
if conditional rule-set
processing
is not activated or the flag is part of the activatedFlags
set.
false
if the flag is not part of the activatedFlags
set.public boolean activeFlags(java.util.Set<java.lang.String> flags)
activatedFlags
set.
The flags will be checked only when conditionalRuleSets
is activated.
flags
- The flags to check.true
if at least one flag is part of the
activatedFlags
set or if conditionalRuleSets
is not
activated.
false
if no flag from the given set is part of the
activatedFlags
set.public void setGraph(org.apache.tinkerpop.gremlin.structure.Graph graph)
graph
instance
will enable callgraph processing mode.graph
- The graph DB instance.public boolean isHonorHidden()
true
if a node's hidden flag should be honored.public void setHonorHidden(boolean hide)
hide
- true
if a node's hidden flag should be honored.public void setReadOnly(boolean readOnly)
In read-only mode, the source AST is used as both the source and copy AST for each rule-set processed. Rule-sets should never modify the AST being walked when read-only mode is enabled. Note that currently there is no enforcement of this contract. It is on the "honor system" only.
readOnly
- true
to enable read-only mode; false
to disable it.public void setVariableInitializers(java.util.Map<java.lang.String,java.lang.Object> initializers)
initializers
- Key/value pairs of variable names to initializer expression
strings. The strings must represent valid expressions in infix
notation.public void addAstSpec(java.lang.String directory, java.lang.String fileSpec, boolean force) throws ConfigurationException
directory
- Relative or absolute directory in which to begin a recursive
search for AST files matching the file specification defined
by fileSpec
. If relative, the path will be
prepended with the P2J_HOME
path.fileSpec
- A regular expression indicating a filename mask to match when
searching for persisted AST files to process for the current
pattern matching session.force
- true
to force the spec to be added.ConfigurationException
- if P2J_HOME
has not been defined for the current
process.public void addAstSpec(FileList list)
list
- An existing, non-null list of files.public void addTarget(java.lang.String name)
name
- An AST name.public void addTargetAst(Aast ast)
ast
- AST to be processed.public java.util.Iterator<FileList> astSpecs()
FileList
objects). Items are iterated in the
order in which they were added.FileList
objects used to
find ASTs to process.public void registerVariable(java.lang.String name, java.lang.Class<?> type, java.lang.String expression)
Register a new user variable with this container. If the variable's
name is already in use within this rule container's scope, it cannot
be re-used and this method will fail. If both type
and
expression
are provided, type
must match the
return type of expression
.
registerVariable
in class RuleContainer
name
- Name by which this variable will be referenced in expressions.type
- Data type of the variable. May be null
if and
only if expression
is not null
and expression
's return type is not
null
.expression
- Expression which will be used to initialize the variable each
time it is reset
by the pattern engine. May be null
if and only
if type
is not null
.SymbolException
- if the variable's name is already in use in this container's
scope.ExpressionException
- if type
is a class which is not assignable from
expression
's return type.java.lang.IllegalArgumentException
- if type
is not provided and it cannot be
determined from expression
.SymbolResolver.resetVariables(com.goldencode.expr.Scope, boolean)
public void run()
public void run(java.lang.String profile) throws ConfigurationException, AstException
profile
- Name of the profile to be loaded. It is up to the ConfigLoader
implementation to understand what to do with
this name.ConfigurationException
- if any error occurs loading the specified configuration
profile.AstException
- if any error occurs loading a persisted AST.public void processFiles(java.util.Iterator<java.io.File> iter) throws AstException
File
objects that must be processed (and
the inherent order in which these files must be processed).
It is assumed that the caller is responsible for loading a profile before this method is invoked and for conducting termination processing afterwards.
iter
- An iterator of File
objects to process.AstException
- if any error occurs loading a persisted AST.public void initialize(java.lang.String profile) throws ConfigurationException
profile
- Name of the profile to be loaded. It is up to the ConfigLoader
implementation to understand what to do with
this name.ConfigurationException
- if any error occurs loading the specified configuration
profile.public void finish() throws AstException
PatternWorker.finish()
method, and nulling out the symbol resolver.AstException
public void storeObject(java.lang.String id, java.lang.Object object)
id
- Arbitrary object id.object
- Object to store.public void storeObject(java.lang.String category, java.lang.String id, java.lang.Object object)
category
- Arbitrary object category.id
- Arbitrary object id.object
- Object to store.public java.lang.Object getStoredObject(java.lang.String id)
id
- Object id.public void storeObject(PatternEngine.StorageKey key, java.lang.Object object)
key
- Storage key under which to store the object.object
- Object to store.public java.lang.Object getStoredObject(java.lang.String category, java.lang.String id)
category
- Object category.id
- Object id.public java.util.Map<PatternEngine.StorageKey,java.lang.Object> getAllStoredObjects()
public void clearStoredObjects()
void setActiveRule(Rule activeRule)
activeRule
- Currently active rule.public java.util.List<java.lang.String> getNewProgramFiles()
newProgramFiles
.public void clearNewProgramFiles()
program file names
.public void addNewProgramFile(java.lang.String file)
file
- The new program file name.private java.lang.String prepareFlag(java.lang.String flag)
flag
- The flag to be prepared.private void walkGraph(org.apache.tinkerpop.gremlin.structure.Vertex node) throws AstException
node
, then iterate through node
's children,
invoking this method recursively for each child found. Apply the
global next-child rule (with the parent node in scope) when moving
between any child (but NOT when moving down to the first child which is
handled by the descend rule OR when moving up from the last child which
is handled by the ascend rule). In other words, the next-child rule
will be triggered between the first and second child, the second and
third child... and when moving between the second-to-last and the last
child.node
- Call graph node at which to begin the walk. Contains the
project-relative path of the AST it represents.AstException
- if any error occurs generating the copy AST.private Aast processAst(java.lang.String astName) throws AstException
AstManager.loadTree(java.lang.String)
method. It
is copied for the first ruleset applied, and any edits are made to the
copy rather than to the original. Once that ruleset has been fully
processed, the copy becomes the source AST to the next ruleset in the
pipeline. In such a way, an AST may be incrementally refined by each
ruleset and passed on to the next in the pipeline for further
processing.astName
- Name of the AST persisted file to be processed.AstException
- if any error occurs generating the copy AST.apply(com.goldencode.p2j.pattern.RuleSet, com.goldencode.ast.Aast)
private Aast processAst(Aast ast) throws AstException
AstManager.loadTree(java.lang.String)
method. It
is copied for the first ruleset applied, and any edits are made to the
copy rather than to the original. Once that ruleset has been fully
processed, the copy becomes the source AST to the next ruleset in the
pipeline. In such a way, an AST may be incrementally refined by each
ruleset and passed on to the next in the pipeline for further
processing.ast
- AST to be processed.AstException
- if any error occurs generating the copy AST.apply(com.goldencode.p2j.pattern.RuleSet, com.goldencode.ast.Aast)
private Aast apply(RuleSet ruleSet, Aast source)
source
is the root node. Each AST node in the copy tree
is linked by ID to its original counterpart in the source tree, but
there are no shared object references between the trees, so the copy
nodes may be edited, moved, or otherwise modified by rules within the
ruleset without impacting the original, source AST.
Next, the engine's symbol resolver is configured to use the correct expression library (that of the current ruleset merged with the global expression library). The root source and copy ASTs are set into the resolver as the current source and copy AST nodes, respectively. The ruleset's user-defined init-rules, if any, are then applied.
Next, the ruleset's walk-rules are applied. Depending upon the input mode
of the ruleset, this means either the
entire source tree is walked (RuleSet.INPUT_TREE
), or only
those AST nodes added to the filtered set of results defined by the
last ruleset processed in this pipeline (if any) are visited, regardless
of the tree structure (RuleSet.INPUT_VIEW
). In either case,
at each node visited in the source tree, the symbol resolver is updated
with the current source node and the corresponding copy node. By
convention, the source node is considered "read-only"; rules must not
modify it. Rules may make edits to the copy node, including annotating
it, moving it, or removing it from the copy tree entirely. A reference
to the copy node may also be added to a collection of results
maintained by the symbol resolver. An iterator on this collection, is
stored as an optional input to the next ruleset in the pipeline.
Next, the root source and copy ASTs are again set into the resolver as the current source and copy AST nodes, respectively, and the ruleset's user-defined post-rules, if any, are then applied.
Finally, the (possibly edited) copy tree is returned, so that it can become an input to the next ruleset in the pipeline.
ruleSet
- Ruleset to be applied against an AST.source
- Source AST to be processed by the ruleset.private void applyGlobal(int type)
type
- Either RULE_INIT
or RULE_POST
.private void gatherTargetPaths() throws ConfigurationException
ConfigurationException
- if any error with the configuration is encountered.private java.lang.String toNormalizedPath(java.lang.String path) throws ConfigurationException
path
is not absolute. If path
is already absolute, it is not converted.path
- An absolute path in the project's home directory, or
path
itself if it is already an absolute path.ConfigurationException
- if the project home directory is not configured.private void registerDefaultWorkers() throws ConfigurationException
Currently, the default workers are instances of:
ConfigurationException
- if any exception is thrown by the default constructor of a
default worker. The exception is caught and chained as the
root cause of the ConfigurationException
.private static void syntax(java.lang.String msg, int rc)
stderr
, then exit the process with the specified return
code. This method is called from main(java.lang.String[])
when the user passes
command line parameters which are not viable.msg
- Optional error message; may be null
.rc
- Process return code used with System.exit()
.public static void main(java.lang.String[] args)
args
- Expected arguments (in order) are as follows:
directory
and filespec
parameters (see below) to specify persisted ASTs
which should be processed. If those parameters are not
provided, process the ASTs referenced in the
project's root node list.
filelist
parameters to specify persisted AST files which should
be processed.
true
(this honor hidden mode is off by
default)
variable=expression
(optional): a key/value
pair which overrides the initializer expression (if any)
of a user variable defined within the configuration
profile. variable
is the variable name and
expression
is the initializer expression
override (which must resolve to a value of the variable's
data type).
profile
(required): name of the pattern
configuration profile to be loaded for this run.
directory
(optional): starting directory used
for a recursive search for persisted AST files to process.
Overrides any such data loaded from the profile. If in
call graph mode, represents the directory
filespec
(required if and only if
directory
was provided): regular expression
(in double quotes) used as a filename mask in combination
with directory
to find persisted AST files to
process. Overrides any such data loaded from the profile.
filelist
(required if and only if
explicit file list mode is enabled): the remaining
arguments specify an arbitrary list of persisted AST files
to process. Overrides any such data loaded from the
profile.