Why the way we measure digital information is fundamentally tied to logarithms
In short: Information is not just about bytes and bits. At its core, the capacity to store data is defined by the number of possible states in a system, a principle formalized by Claude Shannon. This mathematical foundation reveals why we use different units, from the common bit to the theoretical nat.
The measurement of information relies on a fundamental principle observed by Ralph Hartley in 1928 and later formalized by Claude Shannon in 1945. They determined that the information a system can hold is proportional to the logarithm of the number of possible states within that system. Because changing the base of a logarithm simply involves multiplying by a fixed constant, the choice of base effectively defines the unit of measurement used.
In binary hardware, the most prevalent technology today, the base is two. This gives us the bit, which represents one of two possible values. Other units exist for different bases: a trit corresponds to base three, while a decimal digit or hartley corresponds to base ten. The nat, based on the natural logarithm base e, is particularly significant in information theory because natural logarithms offer superior mathematical convenience, even if they are rarely used to describe physical storage capacity.
Beyond these theoretical units, computing relies on conventional groupings. A byte, historically variable but now almost universally eight bits, serves as the standard base for storage. Larger quantities are described using prefixes, though this creates ambiguity. While metric prefixes like kilo traditionally denote powers of 1000, they are often used in computing to refer to powers of 1024. To resolve this, the International Electrotechnical Commission introduced binary prefixes to explicitly denote powers of 1024, aiming to standardize the language of data size.
Hardware architecture further dictates how we organize these units. Computers manipulate data in fixed-size chunks called words, the length of which depends on the CPU register size or memory fetch operations. These structures extend into memory management, where systems partition storage into cache lines and virtual memory pages. Understanding these hierarchies is essential for grasping how modern devices process everything from a single line of text to massive enterprise-grade storage arrays.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Units of information
To cite this card, link to https://intellectplus.co.uk/app/science-units-of-information.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
Related
More in Measurement · More in Science · Latest · A–Z · All topics