orngCN2: Orange CN2 Rule Learning Module

Module orngCN2 implements several variations of well-known CN2 rule learning algorithm and some additional functions to ease the work with rules described in the last section.

All variations of CN2 are implemented by by wrapping orange.RuleLearner class. Note that orange.RuleLearner class contains many replacable components. Each CN2 learner class in this module changes some of these components to reflect the required behaviour. Thus, in the description of each class, we mention only components that differ from default values (see Rule Learning).

Original CN2 Learner

CN2Learner is a class that implements classical CN2 (see Clark and Niblett; 1988). It learns a set of ordered rules, which means that classificator must try these rules in the same order as they were learned.

Specific components

All components are the same as default values of class orange.RuleLearner.

Unordered CN2

CN2UnorderedLearner class implements CN2 unordered (see Clark and Boswell; 1991). It learns a set of unordered rules - classification from rules does not assume ordering of rules - and returns a CN2UnorderedClassifier. In fact, learning rules is quite similar to learning in classical CN2, where the process of learning of rules is separated to learning rules for each class, which is implemented in class' __call__ function. Learning of rules for each class uses a slightly changed version of classical CN2 algorithm.

Specific components

evaluator
Rules are evaluated with Laplace's rule of successin implemented in orange.RuleEvaluator_Laplace

CN2-SD (Subgroup discovery)

CN2SDUnorderedLearner class implements CN2-SD (see Lavrac et al.; 2004). It learns a set of unordered rules, which is the same as CN2UnorderedLearner. The difference between classical CN2 unordered and CN2-SD is selection of specific evaluation function and covering function, as it is mentioned in Specific components section.

Specific components

evaluator
Rules are evaluated with WRACCEvaluator class that implements weighted relative accuracy.
coverAndRemove
Removing of examples in CN2-SD is not exclusive but it lowers the weight of covered examples. By default, this class uses CovererAndRemover_multWeights with multiplication factor set to 0.7. Alternative class for "weighted" covering is CovererAndRemover_addWeights that follows the principle of additive correction of weights.

Miscellaneous functions

ruleToString

This function writes a string presentation of rule in human readable format. It has two parameters; the first is rule which is obvious and the second is showDistribution for selecting whether presentation should also contain the distribution of covered examples.

WRACCEvaluator

This class implements WRACC (weighted relative accuracy) evaluation function.

mEstimate

This class implements m-estimate of probability rule evaluation function. It has one parameter m.

CovererAndRemover_multWeights

This class implements "weighted" covering and removing of examples. The weights of examples are multiplied by parameter mult.

CovererAndRemover_addWeights

This class also implements "weighted" covering and removing of examples following additive weights principle.

References

Clark, Niblett. The CN2 Induction Algorithm. Machine Learning 3(4):261--284, 1989.

Clark, Boswell. Rule Induction with CN2: Some Recent Improvements. In Machine Learning - EWSL-91. Proceedings of the European Working Session on Learning., pages 151--163, Porto, Portugal, March 1991.

Lavrac, Kavsek, Flach, Todorovski: Subgroup Discovery with CN2-SD. Journal of Machine Learning Research 5: 153-188, 2004.