CODEWITHSUNDEEP
clipsexpert-system
Salahuddin
Salahuddin

Reputation: 1719

Tools to check completeness and consistency of rule-base - CLIPS

I've developed a set of rules that should control the execution of a specific flight mission scenario. I've tested this rule set in simulation to check if the expected scenario will be executed, and everything was working as expected. Therefore, I'm sure that the rules are working successfully to do the "expected" scenario. What I need to do is to check if the rule set handles all the possible situations that may occur including those "unexpected" or "unseen" situations. I.e., the situations that should not happen from the first place, however they may happen because of some error or outside force. For example, a drone should not climb over a certain threshold, however, it may climb over this threshold because of a strong thrust of air or a faulty pressure sensor. My rule-base has round 33 rules and 6 templates that all have around 25 attributes. Trying considering all the combinations of these 25 attributes (which vary between integers and symbols with different numbers and values of allowed-symbols) is very complex and is very difficult to be done manually. Is there a tool that can automatically checks if all the possible combinations of the templates' attributes (i.e., all the possible situations) are covered by the rule set? Briefly, the tool should answer the question: Are there any missing roles that handle possible situations (or combinations) that I forgot to consider or didn't think of?

Thanks

Upvotes: 0

Views: 263

Answers (1)

Gary Riley
Gary Riley

Reputation: 10757

I'm not aware of any ready-to-use tools for CLIPS. If I recall correctly, when I developed an application for JRules, the table editor for rules supported completeness checking since it knew the rows of the table represented a grouped collection of rules and it could make some inferences by comparing the rows, but it didn't support completeness checking for individual rules written using the business or technical rule syntax. Since my application had a significant number of complex rules that couldn't be expressed using tables, I had to use the unit testing functionality and manually generate a representative set of test cases since there was no way to test every scenario.

With CLIPS, there aren't any high level table editors, so all your testing is limited to unit tests. There's a set of test cases (https://sourceforge.net/projects/clipsrules/files/CLIPS/6.30/feature_tests_630.zip) used for unit testing CLIPS functionality that you can use as a framework for unit testing other applications. To run the test cases, launch CLIPS and execute a (batch "testall.tst") command from the top level directory of the test cases. When the test cases complete, you can use a diff program to compare the contents of the Expected output directory to the Actual output directory. Individual test cases consist of batch files that when executed dump their output to a text file. That output is then compared to a text file containing the expected output. There is no guide on how to create test cases, but there are over 100 test cases provided so if you just use those as a template it's not that difficult to figure out how to write one.

Generally speaking, it's not possible to prove that any given program works correctly or even terminates, so given the limited amount of information you've given about your rules it's not possible to say whether their correctness can be proven. However, if your rules are relatively simple and can be represented as facts, you can use CLIPS itself to validate some aspects of your program. For example, the CLIPS animal program (https://sourceforge.net/p/clipsrules/code/HEAD/tree/branches/64x/examples/animal.clp) represents its rules as facts:

   (rule (if order is scales and
          rounded.shell is yes) 
         (then type.animal is turtle))

You can then write rules such as this one which checks to see if a rule condition can be satisfied:

(defrule VALIDATE::reachable
   (rule (name ?name) (validate yes)
         (if ?a ?c ?v $?))
   (not (question (variable ?a)))
   (not (rule (then ?a $?)))
   =>
   (printout t "In rule " ?name " no question or rule could be found "
               "that can supply a value for the variable " ?a ":" crlf
               "   " ?a " " ?c " " ?v crlf))

This is similar to the approach used by Drools Verifier (https://developer.jboss.org/wiki/DroolsVerifier) which converts rules to facts in order to do an analysis. Your actual program doesn't need to represent rules as facts, but if you do this for analysis there's lots of things you can check by using rules to reason about your rules. I was able to find the Drools Verifier in a couple of minutes using a search engine, so you can probably find other examples of this technique.

Upvotes: 3

Related Questions