Entropy - Four Definitions, One Concept

This article examines the profound mathematical convergence of entropy across four distinct scientific traditions. Clausius operationally defined macroscopic entropy as "transformation content," tracking degraded energy capacity via heat flows. Boltzmann shifted the paradigm to microscopic counting, identifying entropy as a probabilistic measure of a macrostate's corresponding microstates, which requires coarse-graining. Gibbs generalized this into an ensemble average over arbitrary probability distributions, framing entropy as an expression of missing microscopic data. Shannon formalized it purely mathematically as a measure of informational uncertainty. While all four definitions match numerically, they spark an unresolved metaphysical debate: is entropy an objective physical property or a relational measure of observer ignorance? Ultimately, the Boltzmann constant serves as a historical conversion factor, while the underlying mathematical structure remains deeply tied to information loss, thermodynamics, and gravitational systems like black holes.

Rui Manuel de Almeida Pinheiro

Mainframe Analyst. Prompt Engineering. Content Engineering. Framework Design.

April 23, 2026

Domain 5 — Thermodynamics & Statistical Mechanics

5.6 — Entropy — Clausius, Boltzmann, Gibbs, Shannon — Four Definitions, One Concept

The problem

Entropy is the most important concept in theoretical physics that nobody agrees on.

Not in the sense of a controversy — the mathematics is settled, the predictions are confirmed, the applications are ubiquitous. In the sense that four distinct definitions exist, developed by four distinct intellectual traditions, using four distinct mathematical frameworks, for four distinct purposes — and they all produce the same formula. Whether this convergence reveals a deep unity in nature or merely a remarkable coincidence of mathematical structure is a question that remains genuinely open.

This section examines each definition on its own terms, then asks the hard question: are they the same thing, or do they merely agree numerically?

Definition I — Clausius (1865): Entropy as transformation content

Rudolf Clausius constructed entropy from the macroscopic behaviour of heat engines. As established in 5.4, he noticed that for any reversible cyclic process:

∮ δQ_rev/T = 0

The vanishing of this integral means δQ_rev/T is an exact differential — the differential of a state function. He called that state function entropy S:

dS = δQ_rev/T

And he showed that for any real — irreversible — process:

dS ≥ δQ/T

with equality only for the reversible idealisation.

The Clausius entropy is operationally defined. It is measurable — in principle — by carefully measuring heat flows and temperatures along reversible paths. It makes no reference to molecules, probabilities, or information. It is a purely macroscopic object derived from two empirical facts: that heat engines have an efficiency ceiling, and that this ceiling depends only on reservoir temperatures.

Its meaning, in Clausius's own terms, is the "transformation content" of a system — a measure of how much of its energy has been irreversibly degraded, how much of its capacity for useful work has been permanently lost. High entropy means the energy is in a form from which no further work can be extracted. Low entropy means gradients remain, work is still extractable, transformation is still possible.

The limitation of the Clausius definition is precisely its strength: it is macroscopic. It tells you nothing about what entropy is at the level of molecules. It gives you the law without the mechanism. Clausius knew this. He did not pretend otherwise.

Definition II — Boltzmann (1877): Entropy as probability

Ludwig Boltzmann's definition is the most famous in physics, inscribed on his tombstone in Vienna:

S = k log W

W is the number of distinct microscopic configurations — microstates — consistent with the observed macroscopic state — the macrostate. k is Boltzmann's constant, 1.38 × 10⁻²³ J/K, the conversion factor between the temperature scale humans chose and the natural units of molecular energy.

The conceptual revolution is total. Entropy is no longer a property of heat flows. It is a property of counting. It measures how many ways the microscopic constituents of a system can be arranged while producing the same macroscopic appearance.

A gas compressed into one corner of a box has low W — there are relatively few ways to arrange the molecules all in one region. A gas filling the entire box has enormously high W — there are vastly more ways to distribute the molecules throughout the available volume. The gas expands because the expanded state is overwhelmingly more probable. Not because of any force. Because of counting.

Entropy increases because the universe moves toward states with more microscopic realisations. The Second Law is not a fundamental prohibition — it is an overwhelming probabilistic bias. The reverse process — gas spontaneously collecting in one corner — is not impossible. It is merely so improbable that it will not happen in the lifetime of the universe for any macroscopic sample.

The logarithm in Boltzmann's formula is not arbitrary. It ensures that entropy is additive — that the entropy of two independent systems is the sum of their individual entropies. If system A has W_A microstates and system B has W_B microstates, the combined system has W_A × W_B microstates. The logarithm converts multiplication to addition: log(W_A × W_B) = log W_A + log W_B. Entropy is extensive precisely because the logarithm makes it so.

Boltzmann's definition recovers Clausius's exactly — with the correct value of k — under appropriate conditions. This is one of the great achievements of nineteenth century physics: the macroscopic law derived from microscopic counting, with no adjustable parameters.

But the Boltzmann definition introduces something the Clausius definition does not contain: coarse-graining. W counts microstates consistent with the macrostate — but who defines the macrostate? The macrostate is defined by the macroscopic variables we choose to observe: pressure, volume, temperature, composition. Different choices of what to observe produce different partitions of microstate space and potentially different entropy values. The entropy depends, subtly but irreducibly, on the observer's choice of description.

This is not a flaw. It is a feature that becomes central in the information-theoretic interpretation. But it means the Boltzmann entropy is not purely objective in the way the Clausius entropy appeared to be. It is relational — defined relative to a macroscopic description.

Definition III — Gibbs (1902): Entropy as ensemble average

Josiah Willard Gibbs — working in almost complete intellectual isolation in New Haven, producing work that was not fully appreciated until decades after his death — constructed a more general and mathematically rigorous definition.

Gibbs considered not a single system but an ensemble: an infinite collection of imaginary copies of the system, distributed across all possible microstates according to a probability distribution p_i — the probability that the system is in microstate i. The Gibbs entropy is:

S = −k Σ p_i log p_i

where the sum runs over all microstates.

Several things to note immediately.

First, this reduces to Boltzmann in the microcanonical case. If the system is equally likely to be in any of W microstates — and zero probability elsewhere — then p_i = 1/W for each accessible microstate, and:

S = −k Σ (1/W) log(1/W) = −k · W · (1/W) log(1/W) = k log W

Boltzmann recovered exactly.

Second, Gibbs's definition is more general. It applies when the probabilities are not equal — when some microstates are more likely than others. In the canonical ensemble, where the system is in thermal contact with a heat bath at temperature T, the probabilities are Boltzmann-weighted: p_i = e^(−E_i/kT)/Z, and the Gibbs entropy produces the full thermodynamic entropy of the system including its thermal fluctuations.

Third — and this is the philosophically important point — the Gibbs entropy is defined on a probability distribution, not on a physical state. It is a property of our knowledge of the system, not of the system itself. If we knew exactly which microstate the system was in — complete microscopic knowledge — all probabilities except one would be zero, and S = 0 regardless of the system's macroscopic appearance.

This is deeply uncomfortable for a purely physical interpretation of entropy. It suggests that entropy is, at least in part, a measure of ignorance — of missing information about the microscopic state. The system does not have entropy the way it has energy. Entropy is a relationship between the system and a description.

Gibbs was aware of this. He was careful never to claim that his entropy was the "real" entropy of a system. He treated it as a calculational framework — enormously powerful, precisely correct, and epistemologically agnostic about ultimate ontology.

Definition IV — Shannon (1948): Entropy as information

Claude Shannon — working at Bell Laboratories on the mathematical theory of communication — asked a precise engineering question: how much information is contained in a message? Or equivalently: how much uncertainty is resolved when a message is received?

He showed that the unique function satisfying reasonable axiomatic requirements — continuity, maximality for uniform distributions, additivity for independent sources — is:

H = −Σ p_i log p_i

where p_i is the probability of symbol i in the message source, and the sum runs over all possible symbols. H is measured in bits when the logarithm is base 2, in nats when base e.

Shannon called this the entropy of the source. The name was suggested to him by John von Neumann, who told him: "You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it gives you a great philosophical priority. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage."

Von Neumann was half joking. He was entirely right.

The mathematical identity between Shannon's H and Gibbs's S — up to the factor of k and the choice of logarithm base — is exact. Not approximate. Not analogous. Identical in structure.

H measures the average uncertainty in a probability distribution. A uniform distribution — all outcomes equally likely — has maximum H. A distribution concentrated entirely on one outcome — a certain event — has H = 0. Between these extremes, H measures how spread out, how uncertain, how information-rich the distribution is.

Shannon's entropy is operationally defined without any reference to physics. It applies to any probability distribution: horse races, language, genetics, financial markets, quantum states. It is a mathematical object that happens to coincide exactly with the physical entropy when applied to the probability distributions of thermodynamic systems.

The convergence — one formula, four faces

Set them side by side:

Clausius: dS = δQ_rev/T — macroscopic, operational, physics of heat flows.

Boltzmann: S = k log W — microscopic, combinatorial, physics of counting.

Gibbs: S = −k Σ p_i log p_i — ensemble, probabilistic, physics of distributions.

Shannon: H = −Σ p_i log p_i — informational, axiomatic, mathematics of uncertainty.

The last three share the same functional form. The first is derivable from the third under appropriate physical assumptions. The convergence is not coincidental — it reflects something structural. But what, exactly?

Are they the same concept?

This is the hard question. Two positions exist, both defensible, neither fully satisfying.

Position A: Yes, they are the same concept.

Entropy is missing information — the number of binary questions that would need to be answered to specify the microstate exactly, given knowledge of the macrostate. Clausius entropy, Boltzmann entropy, Gibbs entropy, and Shannon entropy are all measures of the same thing: the gap between what is known and what is the case at the microscopic level.

The Second Law, on this view, is not a law of physics. It is a law of inference. Entropy increases because macroscopic descriptions systematically coarse-grain over microscopic details, and the coarse-graining loses information irreversibly. The arrow of time is the arrow of information loss, not a fundamental feature of physical dynamics.

The strongest advocate of this position was Edwin Jaynes, who built a programme of statistical mechanics on the maximum entropy principle: the correct probability distribution to assign to a system is the one that maximises Shannon entropy subject to the known constraints. This is not an approximation — it is the epistemically correct assignment given limited knowledge. Jaynes recovered all of Gibbs's statistical mechanics from this principle, treating entropy as fundamentally informational from the start.

The appeal of this position is its unification: one concept, one formula, one interpretation, covering physics, information theory, and epistemology simultaneously.

Position B: No, they share a formula but not a concept.

Physical entropy is objective — it corresponds to real physical degrees of freedom, real energy distributions, real macroscopic constraints. Shannon entropy is subjective — it depends on what an agent knows, what questions are being asked, what alphabet is being used. The numerical coincidence is a mathematical fact about logarithms and probability distributions, not evidence of conceptual identity.

On this view, Maxwell's Demon — which will be examined in 5.11 — does not resolve the tension between information and physics by identifying them. It illustrates how they interact: the Demon's information must be encoded physically, and erasure of that physical encoding has thermodynamic costs. The interaction is real, but information and entropy remain distinct concepts that happen to be quantitatively related.

The strongest advocate of this position would note that Shannon himself was agnostic about the physical significance of his formula. He was solving an engineering problem. The coincidence with Boltzmann was suggestive, not conclusive.

The honest verdict

The four definitions agree numerically wherever they are all applicable. The formula is the same. The predictions are identical. No experiment can distinguish them.

The question of whether they are the same concept is therefore not an empirical question. It is a question about what kind of thing entropy is — whether it belongs to the physical world or to the relationship between an observer and the physical world. It is, in the precise sense of the word, a metaphysical question.

Both positions are internally consistent. Both have been defended by physicists of the highest calibre. The question has not been resolved and may not be resolvable by physics alone.

What can be said without philosophical commitment: the mathematical structure of entropy — the logarithm of probability — is not an accident. It is the unique functional form that satisfies additivity, continuity, and maximality simultaneously. Nature, information theory, and statistical mechanics all converge on it because it is the correct measure of spread in a probability distribution. Whatever entropy fundamentally is, it lives in this mathematical structure. And that structure is richer, more universal, and more deeply connected to the fabric of inference and physics than any single interpretation captures.

The Boltzmann constant — the conversion factor

The factor k = 1.38 × 10⁻²³ J/K separating physical entropy from information entropy is not a fundamental constant in the way c or ℏ are. It is a conversion factor — an artefact of the historical accident that humans chose the Kelvin scale before understanding the microscopic basis of temperature.

In natural units — where k = 1 — temperature is measured in joules, entropy is dimensionless, and the distinction between physical and informational entropy dissolves into a pure number. Many theoretical physicists work in these units. The factor of k is a reminder that thermodynamics was built before statistical mechanics, not a signal of deep ontological separation.

Entropy and the three vertical axes

The information axis receives its most direct treatment here. Shannon entropy and Boltzmann entropy share a formula. Landauer's principle — that information erasure has a thermodynamic cost of kT ln 2 per bit — establishes that information processing and thermodynamic entropy production are not merely analogous but operationally linked. This will be the central argument of 5.11.

The Second Law axis is the entropy axis. Everything said here about entropy increase, irreversibility, and the arrow of time is the content of 5.4 seen now from its microscopic foundation. The Clausius entropy and the Boltzmann entropy agree — but the agreement requires the Stosszahlansatz, the molecular chaos assumption, whose status is the content of 5.10.

The gravity axis makes its first decisive appearance: the Bekenstein-Hawking entropy of a black hole — S = A/4lₚ² — is not proportional to the volume of the black hole but to its surface area. This is the holographic principle in its most concrete form: the maximum entropy of a region of space is proportional to the area of its boundary, not its volume. The implication is that the degrees of freedom of gravity are fundamentally lower-dimensional than those of ordinary matter — that the Boltzmann counting that works for gases fails for gravitational systems in a way that is deep, unresolved, and central to the programme of quantum gravity.

Threads carried forward

The Gibbs entropy — S = −k Σ p_i log p_i — is the natural object for the ensemble formalism of 5.9. The microcanonical, canonical, and grand canonical ensembles each correspond to a different constraint on the probability distribution, and the Gibbs entropy computed in each gives the appropriate thermodynamic potential.

The Boltzmann entropy — S = k log W — is the natural object for the H-theorem of 5.10, where Boltzmann's attempt to derive entropy increase from mechanics runs into the Loschmidt paradox. The coarse-graining implicit in defining W is where the time asymmetry is smuggled in, and 5.10 is where that smuggling is examined without mercy.

The Shannon entropy is the natural object for Maxwell's Demon in 5.11, where the informational and physical definitions of entropy are brought into direct operational contact and their relationship is established not philosophically but through the physics of measurement and erasure.

And the deepest thread: entropy as a measure of missing information, of the gap between macroscopic description and microscopic reality, is the foundation of the breakdown section 5.14 — where black hole thermodynamics forces the question of whether information is ever truly lost, and where the holographic principle suggests that the counting of microstates in gravitational systems works by rules that no current theory fully understands.