CPSC 633: Machine Learning (Spring 2016)

Professor: Dr. Thomas R. Ioerger
Office: 322C HRBB
Email: ioerger@cs.tamu.edu
Office hours: Wed, 3:15-4:15, or by appointment

Class Time: Tues/Thurs, 3:55-5:10
Room: 108 CHEN
Course WWW page: http://www.cs.tamu.edu/faculty/ioerger/cs633-spr16/index.html
Textbook: Machine Learning. Tom Mitchell (1997). McGraw-Hill. pdf

Teaching Assistant: TBD office hours: TBD

Goals of the Course:

Topics to be Covered:

• decision trees, rule learning
• empirical methods - algorithm evaluation and comparison
• Bayesian classifiers (Naive Bayes, Gaussian models)
• nearest-neighbor classifiers
• feature selection/weighting
• perceptrons/linear discriminants
• neural networks
• support vector machines
• Bayesian inference
• ensemble methods

Prerequisites

We will be relying on core concepts in AI, especially heuristic search algorithms, optimization, and propositional logic Either the graduate or undergraduate AI class (or a similar course at another university) will count as satisfying this prerequisite.

In addition, the course will require some background in analysis of algorithms (big-O notation), and some familiarity with probability and statistics (e.g. standard deviation, confidence intervals, linear regression, Binomial distribution).

Projects and Exams

Your grade at the end of the course will be based on a weighted average of points accumulated during the semester, 50% for projects and 50% for the final exam. The maximum cutoff for an A will be 90%, 80% for B, and 70% for C.

The late-assignment policy for homeworks and projects will be incremental: -5%/per day, down to a maximum of -50%. If the project is turned in anytime by the end of the semester, you can still get up to 50% (minus points marked off).

Schedule

Academic Integrity Statement and Policy

Americans with Disabilities Act (ADA) Policy Statement

Links

• other good books on ML:
  • Duda, Hart, and Stork
  • Bishop
• links related to Instance-Based Learning and Feature Selection
  • Cover and Hart (1967) - proof that 1-NN error is bounded by twice the error of the Bayes Optimal Classifier
  • KD-trees (Andrew Moore)
  • A tutorial on Principal Components Analysis (Smith, 2002)
  • Relief (Kira and Rendell, 1992)
  • Wrappers for Feature Subset Selection (Kohavi and John, 1997)
  • An introduction to variable and feature selection (Guyon, 2003, JMLR)
  • NTGrowth (Aha and Kibler, 1989)
  • A Review and Empirical Evaluation of Feature Weighting Methods for a Class of Lazy Learning Algorithms (Wettsshereck, Aha, and Mohri, 1997)
• decision tree papers
  • (Quinlan, 1986) - Induction of Decision Trees
  • (Mingers, 1989) - An Empirical Comparison of Selection Measures for Decision-Tree Induction
  • (Mingers, 1989) - An Empirical Comparison of Pruning Methods for Decision Tree Induction
  • (Buntine and Niblett, 1992) - A Further Comparison of Splitting Rules for Decision-Tree Induction
  • (Murthy, Kasif, and Salzberg, 1994) - A System for Induction of Oblique Decision Trees
  • (Quinlan, 1996) - Improved Use of Continuous Attributes in C4.5
  • (Cohen, 1995) Fast Effective Rule Induction (RIPPER)
  • (Brunk and Pazanni) - the algorithm for Rule-Post Pruning is discussed section 5; see also section 4.2 in (Furnkranz, 1997). A Comparison of Pruning Methods for Relational Concept Learning.
• papers on class imbalance
  • (Japkowicz, AAAI-2000) - nice overview of techniques
  • (Drummond and Holte, ECML-2005) - futility of efforts
  • (Akbani et al., 2004) - application to SVMs
• Overview of Bayesian Inference - by Jun Liu and Charles Lawrence
• An Introduction to MCMC Methods for Machine Learning (Andrieu...Jordan, 2003) - overview of Bayesian sampling methods
• Chapter on Decision Trees from Tom Michell's Machine Learning book
• UCL Machine Learning Repository
• chi-square test
• KD-trees (Andrew Moore)
• wrappers (Kohavi and Sommerfield, 1995)
• Relief (Kira and Rendell, 1992)