Trigpoint
Collections › Information Theory › Coding and compression
“Compression means deleting some of it”: wrong.

Coding and compression

Coding and Compression · Shannon (1948) / Fano (1949) / Huffman (1952)
Abstracts checked ✓ Lab case ✓ Information theory · representation and compression Read first: Entropy Read first: Self-information Tell the editor privately ▸
In three sentences Compression is not deleting content. It is representing the same content in a shorter way. The compressible part is called redundancy: the part the receiving end could already have worked out. Removing it loses nothing. And there is a floor to this. The average length of lossless compression can never fall below the entropy.

↳ Seen a counter-example?

Coding and compression Shannon (1948) · Fano (1949) · Huffman (1952) Both roads get shorter; only one loses nothing Original form has redundancy Right road: remove redundancy find the predictable receiver can recover it common → short rare → long codes Stops at entropy a floor you cannot cross Wrong turn: delete content cut what looks minor qualifiers go first shorter too, but because information was thrown away the reader gets a cleaner version that is easier to misuse One question tells the two apart can the reader recover the sentence I deleted from what is left? if yes it was redundancy, if no it was information The flow is conceptual: no compression ratios, proportions or measured numbers. The floor binds lossless compression only; once distortion is allowed, another set of conditions sets the limit. Figure: trigpoint.pub
Figure 1 · The right road and the wrong turn. The right road recognises regularity and carries the same content in a shorter representation, with the entropy floor as its end point. The wrong turn deletes content outright: it does get shorter, but it gets shorter because information was thrown away rather than because redundancy was removed. View full size ▸

1. What happened

Two words that get used interchangeably should be separated first. Coding turns content into some system of symbols and need not make it shorter. Compression is one optimisation goal for coding: exploit the regularity in the content to represent its predictable parts more briefly. The key word is predictable. Writing at length what the receiving end could already guess is pure waste, and that waste is what redundancy means.

From this follows an allocation principle: short codes for what occurs often, long codes for what is rare. Two classic implementations of that principle arrived in 1949 and 1952. The first is a top-down heuristic; the second proved that a bottom-up construction guarantees the minimum average code length[2]Fano, R. M. (1949). The Transmission of Information (Technical Report No. 65). Research Laboratory of Electronics, Massachusetts Institute of Technology. Bibliographic details checked against the publisher's record. Claims matched against the full text. Where the Shannon-Fano coding method was published: messages are ordered by probability and then split again and again into two groups of as nearly equal probability as possible.[3]Huffman, D. A. (1952). A Method for the Construction of Minimum-Redundancy Codes. Proceedings of the IRE, 40(9), 1098–1101. DOI 10.1109/JRPROC.1952.273898. Bibliographic details checked against the official record. Consistent with the abstract. The bottom-up construction of an optimal prefix code, which guarantees the minimum average code length.. The difference between them is not efficiency alone but guarantee: one is usually good, the other is provably optimal.

Whatever method is used, there is one line that cannot be crossed. Theorem 9 of Shannon's 1948 paper fixes the lower bound on average code length for lossless compression, and that bound is the entropy[1]Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423; 27(4), 623–656. Bibliographic details checked against the official record. Claims matched against the full text. Provenance double-checked across models. Theorem 9 in section 9, the fundamental theorem for a noiseless channel: the entropy of a source sets the lowest average rate needed to send it over a noiseless channel, which can be approached arbitrarily closely and never beaten. That is the lower bound on average code length in lossless compression..

The line has two practical implications, both worth keeping. First, the ceiling on how far you can compress is set by the probability structure of the data and not by your technique. Second, as you approach the line, each further step stops removing redundancy and starts discarding information.

Lossless and lossy part company here. Lossless requires complete reconstruction and is bound by the entropy floor. Lossy permits distortion in exchange for shorter length, and it has an entirely different theoretical limit, which is the subject of Rate-Distortion Theory.

Attribution where due: The lower bound on lossless compression belongs to Shannon (1948), sourced to theorem 9 in section 9 of that paper[1]Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423; 27(4), 623–656. Bibliographic details checked against the official record. Claims matched against the full text. Provenance double-checked across models. Theorem 9 in section 9, the fundamental theorem for a noiseless channel: the entropy of a source sets the lowest average rate needed to send it over a noiseless channel, which can be approached arbitrarily closely and never beaten. That is the lower bound on average code length in lossless compression.. The top-down heuristic coding method is conventionally named for Shannon and Fano together, and its formal place of publication is Fano's technical report of 1949[2]Fano, R. M. (1949). The Transmission of Information (Technical Report No. 65). Research Laboratory of Electronics, Massachusetts Institute of Technology. Bibliographic details checked against the publisher's record. Claims matched against the full text. Where the Shannon-Fano coding method was published: messages are ordered by probability and then split again and again into two groups of as nearly equal probability as possible.. The construction that guarantees minimum average code length belongs to Huffman (1952)[3]Huffman, D. A. (1952). A Method for the Construction of Minimum-Redundancy Codes. Proceedings of the IRE, 40(9), 1098–1101. DOI 10.1109/JRPROC.1952.273898. Bibliographic details checked against the official record. Consistent with the abstract. The bottom-up construction of an optimal prefix code, which guarantees the minimum average code length.. The standing of the two results differs: one is a heuristic, the other is provably optimal, and neither name should be used for the other.

2. Why this matters to you

Most people, when they summarise, are cutting rather than compressing. One question separates the two: can the reader reconstruct the sentence I deleted from what remains? If yes, it was redundancy and the cut was right. If no, it was information, and all you did was make the document shorter.

That test changes how you write handovers and explanations at once. What should usually go is not the detail but the groundwork the reader's existing knowledge already covers: background introductions, restatements of general principles, repetitions of premises.

And what should least go often looks exactly like what could go, namely the sentences carrying conditions. Unless, depends on, only if, must: these words carry causation and boundaries, they cannot be reconstructed from the other sentences, and cutting them leaves the reader with a version that looks cleaner and is easier to misuse. Rate-Distortion Theory puts this more precisely.

The second corollary concerns the decoding end. Compression presupposes that both sides share the same regularities. If the receiving end does not hold them, your short code is a cipher to them. So the same content needs different compression ratios for different audiences, which is not a courtesy but a direct consequence of the theory. A briefing for an old hand and an explanation for a newcomer cannot be one document, and not because the newcomer needs more information. They are missing the codebook that made your compression work.

The third corollary is the most counterintuitive: sometimes not being able to compress further is good news. It means there is little redundancy left in the material and what remains cannot be derived. What to do then is to accept the length, not keep shaving.

Verification layer · Members
The guided layer of this node is open to all readers.
Verification details and failure boundaries are for members.
US$8.25 a month on average · or subscribe monthly ▸

Lab cases 2 first-hand cases from our studio

Open previewField record · 2026-08

Before deleting a duplicate, check piece by piece that it can really be recovered

Situation: two rules in our knowledge base deal with compression. The first is to link rather than rewrite: a fact is written in one place, and every other place that needs it links across. The second is three-layer density: one piece of content carries a one-line summary, a three-second summary and the expanded main line, which are three grains of the same content rather than three copies.

On 16 August 2026 the person running the studio proposed keeping explanation and tutorials apart in the knowledge base, so we first read every file’s contents to see where the types were actually mixed. The knowledge nodes were left whole: three-layer density already separates looking something up from understanding it inside one piece of content, and 751 nodes stayed as they were. The real mixing was in the documents describing working procedures. At the top of one of them, notes on each version repeated the version table at the end of the same file.

Decision point: there were at least four workable ways to clear the duplicate. One, delete the version notes at the top. Two, delete the version table at the end and keep the top. Three, keep both and add a rule that the two must stay in step. Four, check each note at the top against the table first, then compress the top into one line pointing to the table.

What we did and what happened: we took the fourth. The inventory reported 15 notes at the top; the actual count was 12 version notes plus one line describing the base version. The check found that the version then in use had no row in the table. Its history existed only at the top, so deleting the duplicate as reported would have deleted that version’s only record.

We added the row first, then compressed the top into one line; a script confirmed that each of the 10 rule notes below the top was kept. The pointer line also states that from then on each new version is recorded only as a table row. As of 25 October the document has been revised many times since, no version notes have grown back at the top, and the version history has moved to a separate file in the same folder.

Take-away: before deleting content that looks duplicated, check piece by piece that each part can be recovered from what remains. What can be recovered is redundancy; what cannot is information, and it has to be moved across before anything is deleted.

Trigpoint PassField record · 2026-04

The bill for double entry: a role table in several places, a title changed, the old copy still read

In April 2026 our table of roles was written in several documents. Adding one role meant writing the change into three places separately; two days later, two of the role tables no longer matched. There were at least four workable responses at that point, and they differ in how many places the next change has to remember, and in whether the copy that gets missed will be taken as true.

Trigpoint Pass content · includes applicable criteria

Read the member edition ▸

Full access to the lab cases. Trigpoint Pass ▸

Common misreadings

Page verification: Abstracts checked ✓ · Content last revised 2026-11-14

  • A common account says compression means deleting the unimportant parts. Lossless compression deletes no content at all: it only switches to a shorter representation, and the original can be reconstructed in full. What deletes things is lossy compression, which is a different theory with its own page on this site. Mixing the two up leads people to think every compression has to pay a price in distortion. #
  • A common account says the compression ratio depends on how good the algorithm is. The algorithm decides how far you are from the limit, but the limit itself is set by the probability structure of the data. Most of the difference in compression ratio when one algorithm runs on different data comes from the data, not the algorithm. #
References

How to read the source notes “Bibliographic details checked” means author, year, title and venue were confirmed. It does not mean the source's claims were checked. “Consistent with the abstract” means the claim agrees with the paper's abstract. It does not include sentence-by-sentence checking against the full text. “Matched against the full text” means the claim was compared with the text of the source itself. Definitions as of 2026-09-20.

  1. [1] Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27(3), 379–423; 27(4), 623–656. ↩
  2. [2] Fano, R. M. (1949). The Transmission of Information. MIT RLE Technical Report No. 65. ↩
  3. [3] Huffman, D. A. (1952). A Method for the Construction of Minimum-Redundancy Codes. Proceedings of the IRE, 40(9), 1098–1101. ↩

↳ Is a citation wrong?

Tell me when a new shelf opens

Tell me when a new shelf opens

Shelf notices are sent only when a new shelf opens or when the reading rules change.

Trial

Tell the editor privately: a correction, a counter-example or where it applies

Submissions are read only by the editor, and nothing is published before they are handled. Once a note is handled, a summary and response written by the editor are published. No sign-up is needed, and the nickname may be left blank.

What are you submitting?
Set a public name to be credited by name ▸
Submit anonymously, without signing in

Signing in lets you follow the status on your page. What you have written is kept.

You are signed in: this note is linked to your account, visible only to you and the editor, so you can follow its status on your page. The text is never published. Full privacy policy ▸

Submit anonymously instead, not linked to your account ▸

No email is collected. Your original text and nickname are read only by the editor and are never published; the site publishes only a summary the editor writes. The human check is provided by Cloudflare Turnstile. To limit abuse, the system keeps a hash of your IP address, not the address itself. Your original text is stored with our cloud provider and can be deleted on request. Full privacy policy ▸