Summary

The goal of this lecture is to familiarize the students with the overall pipeline of a Pattern Recognition System. The various steps involved from data capture to pattern classification are presented. The lectures start with a short introduction, where the nomenclature is defined. Analog to digital conversion is briefly discussed with a focus on how it impacts further signal analysis. Commonly used preprocessing methods are then described. A key component of Pattern Recognition is feature extraction. Thus, several techniques for feature computation will be presented including Walsh Transform, Haar Transform, Linear Predictive Coding, Wavelets, Moments, Principal Component Analysis and Linear Discriminant Analysis. The lectures conclude with a basic introduction to classification. The principles of statistical, distribution-free and nonparametric classification approaches will be presented. Within this context we will cover Bayesian and Gaussian classifiers, as well as artificial neural networks. The accompanying exercises will provide further details on the methods and procedures presented in this lecture with particular emphasis on their application.

Dates & Rooms:
Tuesday, 10:15 - 11:45; Room: A 2.16
Wednesday, 12:15 - 13:45; Room: 0.68

Lecturer

General Information

Tentative course syllabus: It provides a roadmap to the lectures throughout the semester.
The lecture video is available here (only accessible if you are inside the university network; if you want to watch the videos from home, consider to tunnel the connection).
A. Exam Dates
Monday 18.02.2013 (only in the morning)
Tuesday 19.02.2013
Wednesday 20.02.2013 (only in the morning)
Wednesday 20.03.2013
Thursday 21.03.2013
Tuesday 09.04.2013
Wednesday 10.04.2013
B. Signing up for the Exam
Reserving a slot for the exam is only possible after January 6th, 2013.
You must reserve a time-slot for the exam, independent of whether you have signed up at meinCampus. You can do so by:
either by personally visiting the secretaries at the Pattern Recognition Lab, at the 09.138 at Martenstr. 3, 91058 Erlangen,
or by sending them an email at Kristina Müller at mueller(at)cs.fau.de or at Iris Koppe at koppe(at)cs.fau.de . Make sure in your email to write your full name, student ID, program of Studies, birthdate, number of credits and type of exam (e.g. benoteter Schein, unbenoteter Schein, Prüfung durch meinCampus, etc.).

Slides

The updated slides will be posted on the web soon after the corresponding lecture is completed.

In order to prepare yourself for an upcoming lecture, look at the Opens internal link in current window slides of the previous Winter semester.

Introduction:	course outline
Key PR Concepts:	the pipeline of a PR system, terminology, postulates of PR
Sampling:	review of Fourier analysis, Nyquist sampling theorem
Quantization:	signal-to-noise ratio, pulse code modulation, vector quantization, k-means algorithm
Equalization and Thresholding:	histogram equalization, thresholding, binarization, maximum likelihood estimation, various thresholding algorithms (intersection of Gaussians, Otsu's algorithm, unimodal algorithm, entropy-based)
Noise Suppression:	Linear Shift Invariant transformations, convolution, mean filter, Gaussian filter, median filter
Edge Detection:	gradient-based edge detector, Laplacian of Gaussian, sharpening
Non-linear Filtering:	recursive filters, homomorphic filters, cepstrum, morphological operators, rank operators
Pattern Normalization:	size normalization, location normalization, pose normalization, geometric moments, central moments
Introduction to Feature Extraction:	curse of dimensionality, heuristic versus analytic feature extraction methods, projection on orthogonal bases, Fourier transform as a feature
Orthonormal Bases for Feature Extraction:	spectrogram, Walsh-Hadamard transform, Haar transform
LPC and Moments:	linear predictive coding, moments as features, Hu moments
Multiresolution Analysis:	short-time fourier transform, continuous wavelet transform, discrete wavelt transform, wavelet series
PCA, LDA:	introduction to analytic feature extraction, principal component analysis, eigenfaces, linear discriminant analysis, fisherfaces
OFT:	optimal feature transform, Mahalanobis distance, feature transform
Optimization Methods:	gradient descent, coordinate descent
Feature Selection:	objective functions for feature selection including entropy and KL-divergence, strategies for exploring the space of feature subsets including branch-and-bound
Bayesian Classifier:	introduction to classification, decision function, misclassification cost, misclassification risk, Bayesian classifier
Gaussian Classifier:	Gaussian classifier, linear vs. quadratic decision boundaries
Polynomial Classifiers:	polynomial classifier, discriminant functions
Non-parametric Classifiers:	K-nearest neighbor density estimation, Parzen windows
Artificial Neural Networks:	introduction to ANNs, ANN and classification, Radial Basis Function ANNs
Multilayer Perceptrons:	ANN layouts, feed-forward networks, perceptron, MLPs, back-propagation
Review:	end of lecture review, brief recap of what was covered in class

Supplemental Material

A nice review/tutorial on Fourier Analysis can be found at http://www.sunlightd.com/fourier/
Follow this link for additional information on Covariance Matrices.
A more detailed tutorial on Principal Component Analysis can be found here.
Here are some notes on Singular Value Decomposition.
Here is the pseudocode for feature selection using Branch and Bound.
You may want to read the following paper which describes how Branch and Bound can be used in Feature Selection. Due to copyright issues, a copy can not be placed on this web-site. P.M. Narendra and K. Fukunaga, "A Branch and Bound Algorithm for Feature Subset Selection, " IEEE Transactions on Computers, Vol. C-26, No. 9, 1977, pp. 917-922.
A general overview on the Branch and Bound methodology is presented here.
A detailed paper on Branch and Bound can be found in this web-page.

Contact

Secretary

Address

Introduction to Pattern Recognition [IntroPR]

General Information

Slides

Supplemental Material