CPSC 633 - Spring 2008
Project 2
due: Monday, Mar 18, 2008

Implement a neural-network program (the BackProp algorithm) and test
it on some representative databases from the UC Irvine Database
(http://archive.ics.uci.edu/ml/datasets.html).  You may use any
programming language you like.  

You will have to write a simple parser to read in various datafiles
and extract attributes.  Your program should be able to handle
continuous features as well as discrete.  For discrete inputs, there
are several different approaches to encoding them: 1-of-n, binary,
mapping into a single linear range...  For multi-class problems, you
can use a 1-of-n output encoding or multiple output nodes.  (Remember
that it might help to map class values to 0.1/0.9, rather than 0/1;
for perceptrons, you should use -1/+1.)  You will have to experiment
with these design choices and pick what works best for you.  Be sure
to explain your choices, and show some supporting experimental results
that demonstrate your choices work well (i.e. that give good
accuracy).

You will need to implement a stopping criterion based on tracking the
mean-squared error on a validation set.  Also, when reporting
accuracies, give confidence intervals based on cross-validation.

From past experience, you will probably have to adjust the learning
rate to get convergence for each dataset.  You might also consider
implementing (and testing) momentum, random re-start or other tricks
to improve convergence.

The program should take the number of hidden layers and size (number
of nodes) input parameters (assume full connectivity between layers).
You should evaluate the effect of different network sizes on accuracy.
You might also want to try different threshold functions.  Don't forget
to randomize the order of examples in the database.

You might also want to implement a perceptron (easier) for comparison.
You should also compare the performance of your neural network with
your decision tree, and interpret which is better on which databases
and why.

What to turn in: a written report that describes your implementation,
the algorithmic variations you tested, the testing methodology (e.g.
cross-validation), and then the results on several (at least 3) different
databases.  Be sure to interpret your results.  Are the accuracies good?
Why or why not?