To fuse or not to fuse: Fuser versus best classifier

A sample from a class defined on a finite-dimensional Euclidean space and distributed according to an unknown distribution is given. We are given a set of classifiers each of which chooses a hypothesis with least misclassification error from a family of hypotheses. We address the question of choosing the classifier with the best performance guarantee versus combining the classifiers using a fuser. We first describe a fusion method based on isolation property such that the performance guarantee of the fused system is at least as good as the best of the classifiers. For a more restricted case of deterministic classes, we present a method based on error set estimation such that the performance guarantee of fusing all classifiers is at least as good as that of fusing any subset of classifiers.