Mathematical Methods in

Lecture Notes

Lecture 1: Definition of entropy and mutual information[pdf] [Latex sources]
Lecture 2: Convexity and properties of divergence [pdf] [Latex sources]
Lecture 3: Prefix/Instantaneous codes [pdf] [Latex sources]
Proof of Kraft inequality for non finite alphabet [pdf] [Latex sources]
Lecture 4: Optimality of Huffman Code, Uniqely decodeable code, AEP [pdf] [Latex sources]
Lecture 5: AEP and fixed-length lossless source coding [pdf] [Latex sources]
  Channel coding definitions [pdf] [Latex sources]
Lecture 6: Channel coding [pdf] [Latex sources]
Example: Binarry Erasure Channel [pdf]
   Optimality of Source Channel coding seperation [pdf] [Latex sources]
Lecture 7: Differential entropy and Gaussian Channel [pdf] [Latex sources]
Lecture 8: Gaussian Channel [pdf] [Latex sources]
Lecture 9:  Geometric intuition for the Gaussian channel and channel state information  [pdf]
Maximum Entropy princeple, channel information [pdf] [Latex sources]
   Rate distorstion [pdf] [Latex sources]
Lecture 10: MIMO [pdf] [Latex sources]

Cheating sheet- A summary of the equations [pdf] [Latex sources]
Appendix 1- Introduction to Probability and stochastic processes [pdf] [Latex sources]
Appendix 2- Introduction to convex analisys [pdf] [Latex sources]
Appendix 3- Introduction to analisys [pdf] [Latex sources]
Appendix 4- Noatation [pdf] [Latex sources]
Appendix 5- Law of large numbers [pdf] [Latex sources]

Here are some tips on how to write a good paper/lecture.

Instruction on how to scribe the lectures:

The lectures should be scribed in English using Latex.
The format should be as in Lecture 1. (The best way is to download the sources of Lecture 1 and to modify lec1.tex file)
Each lecture should include all the results (and their proofs) presented in the class. Additional meterial added by the students is welcome too.
The scribing may be done in pairs or by single students.
Scribing a lecture is not mandatory; however a student or a pair of student who will scribe a lecture will get up to 7 point bonus to the final grade.
Students are encouraged to add figures to the lecture notes. Figures need to be in an EPS format (figures may be generated by Inkscape Visio xfig or any other drawing program). Please include the source file of the figure.
The notation should be as in class. We use the same notation as in Cover's textbooks, with only one difference: we denote pmf as P(x) or P_X(x) rather than p(x).

In order to write in latex one needs three components

Miktex which compiles the .tex file on windows
A latex editor, such as Winedt or Latex Editor (LEd) or TeXnicCenter
Ghostcript which can be found at http://pages.cs.wisc.edu/~ghost/ or at http://pages.cs.wisc.edu/~ghost/doc/GPL/gpl854.htm.

Those who do not want to install latex can connect to the server Faraday (supported by our department) using a remote desktop.
It already has Winedt, Miktex and Ghostscript.

Useful links:

Basic instructions [pdf] [Latex source] written by Morag Agmon on how to start writing in latex using TeXnicCenter.
Basic instruction [html] written by Yuval Carmel on how to install Latex Editor (LEd)
Wiki-latex
Introduction to Latex [pdf] written by Yuval Carmon
Advanced topic on Latex [pdf] written by Morag Agmon