Appendix B:Overview in Probability and Random, Processes, B.5 Ergodicity and law of large numbers, B.6 Central limit theorem, B.7 Convexity, concavity and Jensen’s inequality, B.8 Lagrange multipliers tech. & KKT conditions
Appendix A:Overview on Suprema and Limits
A.1 Supremum and maximum
A.2 Infimum and minimum
A.3 Boundedness and suprema operations
A.4 Sequences and their limits
Appendix B:Overview in Probability and Random
Processes
B.5 Ergodicity and law of large numbers
B.6 Central limit theorem
B.7 Convexity, concavity and Jensen’s inequality
B.8 Lagrange multipliers tech. & KKT conditions
Chapter 2：Information Measures for Discrete Systems
2.1.1 Self-information
2.1.2 Entropy
2.1.3 Properties of entropy
2.1.4 Joint entropy and conditional entropy
2.1.5 Properties of joint and conditional entropy
2.2 Mutual information
2.2.1 Properties of m
Chapter 2：Information Measures for Discrete System
2.7 Convexity/concavity of information measures
2.8 Fundamentals of hypothesis testing
2.9 R´enyi’s information measures
Chapter 3：Lossless Data Compression
3.3 Variable-Length Code for Lossless Data Comp.
3.3.1 Non-singular Codes and Uniquely Decodable Codes
3.3.2 Prefix or Instantaneous Codes
