Members' shelf
All 9 nodes on Naming & Classification are for members only.
Pay with the email you sign in with here, so the Pass is linked to this account.
Members' shelf
All 16 nodes on System Protection & Threat Modeling are for members only.
Pay with the email you sign in with here, so the Pass is linked to this account.
Information and uncertainty
1. What happened
Shannon's founding paper of 1948 opened by doing something that still reads as counterintuitive: it took meaning out of the problem[1]Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423; 27(4), 623–656. DOI 10.1002/j.1538-7305.1948.tb01338.x. Bibliographic details checked against the official record. Claims matched against the full text. Provenance double-checked across models. The introduction sets the question of meaning explicitly outside the engineering problem: the semantic aspects of communication are irrelevant to it.. The original is blunt about it. The semantic aspects of communication are irrelevant to the engineering problem.
That is not evasion, it is a boundary. To make information a quantity that can be computed, transmitted and compared, you first have to give up asking what it says. The book of the following year, The Mathematical Theory of Communication, carried Weaver's introduction, which split communication into technical, semantic and effectiveness levels and granted that the theory is on its face applicable only to the first; he argued at the same time that it is helpful and suggestive for the other two[3]Shannon, C. E. & Weaver, W. (1949). The Mathematical Theory of Communication. Urbana: University of Illinois Press. Bibliographic details checked against the publisher's record. Claims matched against the full text. Provenance double-checked across models. Weaver's introduction divides the problems of communication into technical, semantic and effectiveness levels. He grants that the mathematical theory is on its face applicable only to the first, and argues at the same time that it is helpful and suggestive for the other two..
With meaning taken out, what is left to measure? The ruling out of possibilities. Picture a situation not yet resolved, with a set of possible outcomes. When one of them comes true the rest are ruled out, and the amount ruled out is the information the event carries.
That definition has an immediately testable consequence: the same sentence carries different amounts of information in different situations. To someone who already knows it will rain tomorrow, it will rain tomorrow carries almost none. To someone whose forecasts all said clear skies, the same sentence carries a great deal. What changed is not the sentence. It is what the sentence rules out.
One more step separates two levels. How much a single event rules out is the information of that event. How hard the whole situation is to guess on average is the entropy of that system. The two get run together often, and they answer different questions: one asks how much this resolution told me, the other asks how uncertain I was before it. The next two pages take them one at a time.
The logarithmic structure is older. Hartley's 1928 paper connected quantity of message to the number of distinguishable states and laid the ground for the later formalisation[2]Hartley, R. V. L. (1928). Transmission of Information. Bell System Technical Journal, 7(3), 535–563. DOI 10.1002/j.1538-7305.1928.tb01236.x. Bibliographic details checked against the official record. Claims matched against the full text. Builds the prototype of the logarithmic structure of information from a count of distinguishable states. The paper states that it first assumes there is no external interference, and it contains no capacity in terms of noise power; attributing the latter to Hartley is a citation error (see this site's page on channel and capacity).. Note that it handles the noiseless case; attributing capacity under noise to it is a citation error that arose later.
2. Why this matters to you
You use the word information every day, in its everyday sense: more information means said more, said in more detail, said more professionally. This theory says that is a different thing, and the difference changes how you judge what is in front of you.
A three-thousand-word set of meeting minutes whose entire content you could have predicted in advance carries information close to zero, whatever its word count. A model output can read as well organised and fully furnished with terminology and still rule out almost nothing, because all it did was rearrange your own question.
So the question is not whether this content is rich enough. It is what did it rule out that I previously took to be possible. The first question has no answer. The second one does.
The boundary has a harder corollary: a high quantity of information does not mean correct. The more unexpected an assertion, the more possibilities it rules out and the higher its information, while whether it is true belongs entirely to another ruler.
When a model gives a bold and specific judgement, that really is a high-information output under this theory, and that fact by itself is no reason to believe it. Keeping the two rulers apart is the first price of using this language, and the first thing it gives you.
Counter-examples and failure boundaries
The first boundary, and the most important one: this definition is no philosophical claim about what information is. It is a trade made to make quantification possible. Giving up semantics buys computability, and the price is that the theory stays permanently silent about what a sentence says. Using it to argue that meaning does not matter reads an engineering boundary as an ontological verdict.
The second boundary is where the probabilities come from. How much was ruled out presupposes that you already have a space of possibilities and a set of probabilities over it. In practice that set is often your own expectation rather than an objective frequency, so the same event carries different information for different people with no way to arbitrate between them. That is a condition of use rather than a defect: settle whose probabilities these are, and the number you compute means something.
The third is the most common misuse in practice: taking it as a ruler for content quality. Quantity of information is orthogonal to correctness and orthogonal to importance. A system can produce high-information output that is entirely wrong, and under this theory that is no contradiction. Judging right and wrong, or weight, takes other instruments.
Lab cases Written only where there is evidence
No field record for this node yet.
- A common account says that high information means valuable content. The two are independent axes. A completely wrong sentence can carry a great deal of information, and a deeply meaningful one can carry almost none. The theory measures the first and says nothing about the second. #
- A common account says that Hartley proposed the capacity formula for information. What Hartley gave in 1928 is a count of distinguishable states in the noiseless case, not capacity under noise power. The latter belongs to Shannon's 1948 paper. The error does circulate, and this site handles it on a separate page. #
Records how far each claim was checked, not a guarantee that it is true.
How it was checked
The structure of the sourcing on this page is stated first, because it differs from the other pages on this shelf.
The framework statement here, that information is the reduction of uncertainty and is orthogonal to meaning, fell in a June 2026 round of cross-model checking into the class judged settled textbook canon. Both models swept this concept group, and the second model's deep checking concentrated on the sources flagged as contested in that round, without returning item by item to this framework statement.
Verification of that statement is therefore at the level of the discipline, not a sentence-by-sentence check against primaries. We say so plainly, because in the same round the search stage did produce two sourcing errors, both of them caught by the deep-checking stage.
The specific sources cited here rest on stronger ground. The three sourcing items for Shannon's 1948 paper, journal, volume and issue, and DOI, together with the 1949 book The Mathematical Theory of Communication as an anchor, were confirmed by the cross-model deep check in that round[1]Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423; 27(4), 623–656. DOI 10.1002/j.1538-7305.1948.tb01338.x. Bibliographic details checked against the official record. Claims matched against the full text. Provenance double-checked across models. The introduction sets the question of meaning explicitly outside the engineering problem: the semantic aspects of communication are irrelevant to it.[3]Shannon, C. E. & Weaver, W. (1949). The Mathematical Theory of Communication. Urbana: University of Illinois Press. Bibliographic details checked against the publisher's record. Claims matched against the full text. Provenance double-checked across models. Weaver's introduction divides the problems of communication into technical, semantic and effectiveness levels. He grants that the mathematical theory is on its face applicable only to the first, and argues at the same time that it is helpful and suggestive for the other two.. The source of Hartley's 1928 paper and its restriction of scope were confirmed at depth as well: it gives a count of distinguishable states in the noiseless case, not capacity under noise[2]Hartley, R. V. L. (1928). Transmission of Information. Bell System Technical Journal, 7(3), 535–563. DOI 10.1002/j.1538-7305.1928.tb01236.x. Bibliographic details checked against the official record. Claims matched against the full text. Builds the prototype of the logarithmic structure of information from a count of distinguishable states. The paper states that it first assumes there is no external interference, and it contains no capacity in terms of noise power; attributing the latter to Hartley is a citation error (see this site's page on channel and capacity).. This site has since re-read three of the sources against their text: the sentence in Shannon's 1948 introduction that sets semantics aside, the passage in Hartley's 1928 paper that first assumes there is no external interference, and Weaver's three levels together with what he says about the theory's reach[1]Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423; 27(4), 623–656. DOI 10.1002/j.1538-7305.1948.tb01338.x. Bibliographic details checked against the official record. Claims matched against the full text. Provenance double-checked across models. The introduction sets the question of meaning explicitly outside the engineering problem: the semantic aspects of communication are irrelevant to it.[2]Hartley, R. V. L. (1928). Transmission of Information. Bell System Technical Journal, 7(3), 535–563. DOI 10.1002/j.1538-7305.1928.tb01236.x. Bibliographic details checked against the official record. Claims matched against the full text. Builds the prototype of the logarithmic structure of information from a count of distinguishable states. The paper states that it first assumes there is no external interference, and it contains no capacity in terms of noise power; attributing the latter to Hartley is a citation error (see this site's page on channel and capacity).[3]Shannon, C. E. & Weaver, W. (1949). The Mathematical Theory of Communication. Urbana: University of Illinois Press. Bibliographic details checked against the publisher's record. Claims matched against the full text. Provenance double-checked across models. Weaver's introduction divides the problems of communication into technical, semantic and effectiveness levels. He grants that the mathematical theory is on its face applicable only to the first, and argues at the same time that it is helpful and suggestive for the other two..
One residual item. This page treats the cybernetics line as a parallel development of the same period, which is a general-history statement. We have not separately checked its sources, so the body makes no specific attribution and it is not listed in the references.
How to read the source notes “Bibliographic details checked” means author, year, title and venue were confirmed. It does not mean the source's claims were checked. “Matched against the full text” means the claim was compared with the text of the source itself. Definitions as of 2026-09-20.
- [1] Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423; 27(4), 623–656. ↩
- [2] Hartley, R. V. L. (1928). Transmission of Information. Bell System Technical Journal, 7(3), 535–563. ↩
- [3] Shannon, C. E. & Weaver, W. (1949). The Mathematical Theory of Communication. University of Illinois Press. ↩
Tell me when a new shelf opens
Tell me when a new shelf opens
Shelf notices are sent only when a new shelf opens or when the reading rules change.
Tell the editor privately: a correction, a counter-example or where it applies
Submissions are read only by the editor, and nothing is published before they are handled. Once a note is handled, a summary and response written by the editor are published. No sign-up is needed, and the nickname may be left blank.
We have your note