#Information-Theory
2 posts in tag “Information-Theory”.
Shannon's Information Theory from First Principles: Entropy, Channel Capacity, and the Fundamental Limits of Communication
2025-03-05Build Shannon's information theory from the ground up: entropy as a measure of uncertainty, source coding theorem, channel capacity, and the noisy-channel coding theorem that established the theoretical limits of reliable communication.
Landauer's Principle and the Thermodynamics of Computation: Why Bits Have an Energy Floor
2025-02-22Explore the deep connection between thermodynamics and information: Landauer's principle that erasing a bit costs kT ln 2 in energy, the Maxwell's demon resolution, and the quest for reversible, energy-efficient computing.