Authors: Dragi Kocev, Sašo Džeroski

Contents

Abstract

Abbreviations

1. Introduction

   1. General perspective

   2. Motivation

   3. Contributions

   4. Organization

2. Background

   1. Machine learning tasks considered

      1. The task of predicting multiple targets

      2. The task of hierarchical multi-label classification

   2. Related work

      1. Ensemble learning

      2. Predictive clustering

      3. Predicting structured outputs

3. Predictive clustering trees for structured outputs

   1. Basic algorithm for learning PCTs

   2. Global prediction of structured outputs with PCTs

      1. PCTs for multiple target variables

      2. PCTs for hierarchical multi-label classification

   3. Local prediction of structured outputs with PCTs

4. Ensembles for predicting structured outputs

   1. Global prediction of structured outputs with ensembles of PCTs

      1. Bagging

      2. Random forests

      3. Random subspaces

      4. Bagging of subspaces

      5. Combining the predictions of individual PCTs

      6. Implementation issues

   2. Local prediction of structured outputs with ensembles of PCTs

   3. Computational complexity

      1. PCTs for local prediction

      2. Ensembles for local prediction

      3. PCTs for global prediction

      4. Ensembles for global prediction

      5. Summary and discussion

5. Experimental evaluation

   1. Experimental design

      1. Experimental questions

      2. Descriptions of the datasets

      3. Evaluation measures

      4. Experimental setup

   2. Results and discussion

      1. Multiple continuous targets

      2. Multiple discrete targets

      3. Hierarchical multi-label classification

   3. Summary of the results

6. Case studies

   1. Predicting vegetation condition

   2. Hierarchical annotation of medical images

   3. Predicting gene function

   4. Summary of the case studies

7. Further developments

   1. Predicting other types of structured outputs

      1. Distances for hierarchical classification

      2. Time series

      3. Prototypes, voting and variance

   2. Feature ranking for structured outputs

      1. Feature ranking using random forests

      2. Biomarker discovery using multi-target ranking

   3. Construction of ensembles of PCTs using beam search

      1. Beam search induction of PCTs

      2. Diversity in the beam

      3. Computational complexity

      4. Empirical evaluation

8. Conclusions and further work

   1. Conclusions

   2. Further work

9. Acknowledgements

10. References

11. Appendix: CLUS user manual