Project: The project, to be done individually, or by groups of two, or exceptionally three students, will consist of a short, professional, without plagiarism, well- written report (8-10 pages). The goal will be to examine some of the numerous topics in the area of information theory that are not covered in class. Pick a topic below, read the written material, work the theoretical derivations, check and may be implement the algorithms.

The material will be presented in class: 30 minutes for a group of two students. The presentation should be at a level that your classmates will understand, and should briefly cover the main points of your paper.

Some of the papers require more effort/background than others, however, the amount of effort that is required for a satisfactory grade is uniform across the projects. As a result you do not need to worry about picking a hard/easy paper. Choose a subject below. You are required to send me your choices by february, 13.

1. Estimation of Rényi divergence. The material is available below. Concentrate on the paper “Nonparametric Divergence Estimation with Applications to Machine Learning on Distributions” by Poczos et al. (in Uncertainty in Artificial Intelligence, 2011). The other papers are given for reference and further details.
2. Blind Source Separation. Focus on the paper by Pham, “Mutual information approach to blind separation of stationary sources”. The paper by Cardoso gives an information theoretic) geometric view of the procedure.
3. Estimating Mutual Information. Study the paper by Kraskov et al., and look at the paper by Garbellay. A matlab code is available at http://bsp.teithe.gr/members/downloads/Mutin.html. Check and experiment with this code.
4. Information theory and statistics. Study the chapter 11 in the book by Cover and Thomas. Focus on the method of types and on the results on hypothesis testing.

The presentation will take place in class on march, 07.

	Parent directory
	RenyiNeighbourEstimation.zip
	BlindSourceSeparation.zip
	EstimMutualInformation.zip
	Chap11.pdf