Assignment 5: Reductions

Assignment description

In this assignment, you will implement three algorithms for reduction from multiclass classification to binary classification: OVA, AVA, and single-elimination binary tournament.

You will use the perceptron code you’ve developed for Assignment 4 as the baseline binary classifier, and will evaluate the quality of the reductions on the primary-tumor dataset.

Helper code

You will write AVA code in ava.py, and respectively for ova.py and binary_tournament.py.

Questions

Answer the questions below in a “answers.txt” plain file, “answers.md” Markdown, or “answers.pdf” PDF. I will not accept Microsoft Word, OS X Pages, or OpenOffice documents. (I prefer Markdown, so I can see it from your repository on Github directly)

In addition, submit whatever code you use to answer the questions below.

  1. What are the accuracies you obtain for the primary-tumor dataset for AVA, OVA, and binary tree tournament?

  2. Report the confusion matrix of these methods. Are they comparable?

  3. Given the semantics of the labels for the primary-tumor dataset, is “accuracy” a good measure of model quality? If not, what are the problems and possible alternatives?

Hints