第一週

• A coherent introduction of the primary principles of single-user information theory

• Holiday

• Overview in the subjects of suprema, limits, probability and random processes (e.g., random variables, statistical properties of random processes, Markov chains, convergence of sequences of random variables, ergodicity and laws of large numbers, central limit theorem, concavity and convexity, Jensen’s inequality, Lagrange multipliers, and the Karush–Kuhn–Tucker (KKT) conditions for constrained optimization problems)　

• Information measures for discrete systems and their properties (self-information, entropy, mutual information and divergence, data processing theorem, Fano’s inequality, Pinsker’s inequality, simple hypothesis testing, Neyman–Pearson lemma, Chernoff–Stein lemma, and Rényi’s information measures)　

• Fundamentals of lossless source coding (i.e., data compression): discrete memoryless sources, fixed-length (block) codes for asymptotically lossless compression, AEP, fixed-length source coding theorems for memoryless and stationary ergodic sources, entropy rate and redundancy, variable-length codes for lossless compression, variable-length source coding theorems for memoryless and stationary sources, prefix codes, Kraft inequality, Huffman codes, Shannon–Fano–Elias codes, and Lempel–Ziv codes　

• Fundamentals of lossless source coding (i.e., data compression): discrete memoryless sources, fixed-length (block) codes for asymptotically lossless compression, AEP, fixed-length source coding theorems for memoryless and stationary ergodic sources, entropy rate and redundancy, variable-length codes for lossless compression, variable-length source coding theorems for memoryless and stationary sources, prefix codes, Kraft inequality, Huffman codes, Shannon–Fano–Elias codes, and Lempel–Ziv codes

• Holiday

• Fundamentals of channel coding: discrete memoryless channels, block codes for data transmission, channel capacity, coding theorem for discrete memoryless channels, calculation of channel capacity, channels with symmetric structures, lossless joint source–channel coding, and Shannon’s separation principle

• Fundamentals of channel coding: discrete memoryless channels, block codes for data transmission, channel capacity, coding theorem for discrete memoryless channels, calculation of channel capacity, channels with symmetric structures, lossless joint source–channel coding, and Shannon’s separation principle

• Midterm Exam

• Information measures for continuous alphabet systems and Gaussian channels: differential entropy, mutual information and divergence, AEP for continuous memoryless sources, capacity and channel coding theorem of discrete-time memoryless Gaussian channels, capacity of uncorrelated parallel Gaussian channels and the water-filling principle, capacity of correlated Gaussian channels, non-Gaussian discrete-time memoryless channels, and capacity of band-limited (continuous-time) white Gaussian channels

• Fundamentals of lossy source coding and joint source–channel coding: distortion measures, rate–distortion theorem for memoryless sources, rate–distortion theorem for stationary ergodic sources, rate–distortion function and its properties, rate–distortion function for memoryless Gaussian sources, lossy joint source–channel coding theorem, and Shannon limit of communication systems

• Fundamentals of lossy source coding and joint source–channel coding: distortion measures, rate–distortion theorem for memoryless sources, rate–distortion theorem for stationary ergodic sources, rate–distortion function and its properties, rate–distortion function for memoryless Gaussian sources, lossy joint source–channel coding theorem, and Shannon limit of communication systems

• General information measure: Information spectrum and Quantile and their properties

• Advanced topics of losslesss data compression: Fixed-length lossless data compression theorem for arbitrary channels, variable-length lossless data compression theorem for arbitrary channels

• Holiday