Meta Decision Trees














Introduction


Meta Decision Trees are a novel method for combining multiple classifiers. The difference between meta and ordinary decision trees (ODTs) is that MDT leaves specify which base-level classifier should be used, instead of predicting the class value directly (as ODTs). The attributes used by MDTs are derived from the class probability distributions predicted by the base-level classifiers for a given example. An example MDT, induced in the image domain from the UCI Repository, is given below. The leaf denoted by an asterisk (*) specifies that the IBk classifier is to be used to classify an example, if the entropy of the class probability distribution predicted by IBk is smaller than or equal to 0.002369.






IBk:Entropy <= 0.002369: IBk (*)
IBk:Entropy > 0.002369:
|   J48:maxProbability <= 0.909091: IBk
|   J48:maxProbability > 0.909091: J48






The original algorithm MLC4.5 for inducing MDTs is an extension of the C4.5 algorithm for induction of ODTs. We have also integrated the algorithm for inducing MDTs in the Weka data mining suite: an extension of J4.8 (the Java reimplementation of C4.5 integrated in Weka), named MLJ4.8, has been developed for this purpose.






Publications


  • Todorovski, L., Dzeroski, S. Combining Multiple Models with Meta Decision Trees. Proceedings of the Fourth European Conference on Principles of Data Mining and Knowledge Discovery, pages 54-64. Springer, Berlin, Germany, 2000.
abstract | full paper [ps] [pdf]


  • Todorovski, L., Dzeroski, S. Combining Classifiers with Meta Decision Trees. Machine Learning, volume 50, issue 3, pages 223-249, 2003.
abstract | full paper [ps] [pdf] © 2003 Kluwer Academic Publishers.


  • Zenko, B., Todorovski, L., and Dzeroski, S. A comparison of stacking with MDTs to bagging, boosting, and other stacking methods. Proceedings of ECML/PKDD01 Workshop: Integrating Aspects of Data Mining, Decision Support and Meta-Learning, pages 163-175. Albert-Ludwigs-Universität Freiburg, Germany, 2001.
abstract | full paper [ps] [pdf]


  • Zenko, B., Todorovski, L., and Dzeroski, S. A comparison of stacking with meta decision trees to other combining methods. Proceedings A of the Fourth International Multi-Conference Information Society IS'2001, pages 144-147. Jozef Stefan Institute, Ljubljana, Slovenia, 2001.
abstract | full paper [ps] [pdf]


  • Zenko, B., Todorovski, L., and Dzeroski, S. A comparison of stacking with meta decision trees to bagging, boosting, and stacking with other methods. Proceedings of the 2001 IEEE International Conference on Data Mining, pages 669-670. IEEE Computer Society, Los Alamitos, CA, USA, 2001.
abstract | poster paper [ps] [pdf]






Download


  • C implementation of MLC4.5 (Based on the C4.5 release 8 source code).
source
  • Java implementation of MLJ4.8. (To be incorporated into Weka data mining suite)
source | bytecode





Bernard Zenko
Created: November 2001, Updated: February 2003