Media Compression · Study deck
Media Compression Mechanics: Audio, Image, and Video
Audio masking, image transforms, and video prediction save bits through different mechanisms with different errors.
Broker Bex is your guide for this deck.
After studying this chapter
Every saving needs a mechanism and an error
Compression needs an explanation of both its saving and its effect on the receiver.
- A masking threshold can make selected audio error less audible.The encoder needs signal-dependent noise allocation because equal total noise power does not establish equal audibility.
- An image transform can concentrate energy into a few coefficients.Smooth blocks need mainly DC and low-frequency terms, while lettering and sharp edges require more high-frequency detail.
- Quantization and coefficient order have different jobs.Rounding can create zeros; JPEG zig-zag ordering exposes those zeros to run-length and entropy coding without changing their values.
- Motion prediction can reduce the information left for residual coding.The encoder must compare motion information plus residual cost with an intra-coded alternative when the reference predicts poorly.
Major section
Compact representation and acceptable loss are different savings
Lossless packing and perceptually acceptable distortion solve different representation problems.
- Statistical redundancy can be removed without changing the reconstructed information.Repeated or predictable source values can have a more compact representation while preserving the same decoded result.
- Perceptual irrelevancy permits changes that the intended receiver may not notice.Lossy coding needs evidence from the signal, listening or viewing task, playback path, and encoder settings.
- Lossy media coding can use both kinds of saving.A transform and quantization stage can discard selected detail before entropy coding packs the remaining values compactly.
- An acceptable error depends on the actual receiver and task.A change that is hard to hear or see in one tested setting is not automatically irrelevant in every use.
Major section
Masking follows the signal across time and frequency
A moving detection threshold gives the encoder a reason to place error selectively.
- The quiet threshold is different at different frequencies.Typical hearing is more sensitive through part of the mid-frequency range than at very low or very high frequencies.
- A strong component can raise the nearby detection threshold.Energy below that raised masking threshold remains physically present even when the listener is less likely to notice the component.
- A perceptual encoder can allocate error beneath the changing threshold.Time-frequency energy estimates guide where quantization noise may be less audible and therefore need fewer coding bits.
- Masking is more specific than deleting all high frequencies.The allowed error depends on the signal’s local spectral and temporal behaviour instead of one fixed frequency-removal rule.
Major section
Noise placement matters as much as total power
The arrangement of noise can change audibility even when total power stays equal.
- Equal noise power can produce different listening results.Broad white noise may be obvious while signal-shaped noise with the same nominal relationship is difficult to hear.
- Noise allocation needs both frequency and timing information.A masker’s influence spreads unevenly across frequency and time, so one overall power measurement cannot describe the local threshold.
- Less audible distortion is still a changed signal.Masking supports lossy coding rather than a promise that the decoder reproduces every original sample exactly.
- The playback path remains part of the evidence.The listener, signal, encoder settings, and listening conditions determine whether the predicted hidden error stays acceptable.
Major section
The masking demonstration separates level from audibility
The four described conditions show why noise level alone cannot settle audibility.
- The original condition is the reference without added noise.The A/B/C/D comparison needs that baseline to judge what the added components change for the listener.
- White noise at -13 dB is described as clearly audible.That condition prevents treating a single noise-to-signal value as a universal threshold for inaudibility.
- Shaped noise can be difficult to hear at the same nominal relationship.Its placement against the signal’s masking threshold explains why the level alone cannot predict the listening result.
- The isolated difference can reveal physically present hidden energy.The chapter provides no licensed listening assets, so its workbench is a spectrum prelude rather than a reproduced masking experiment.
Activity 1 · Match
✎ Match the four masking conditions
I want you to separate the reference, noise level, noise placement, and isolated difference.
Match A, B, C, and D to original signal, original plus white noise, original plus signal-shaped noise, and difference alone. Explain why the same nominal noise relationship can produce different audibility.
3 minutes · Pen and paper · Answer: Activity 1
Major section
A constant block concentrates into DC
A smooth image can need far fewer strong transform coefficients than original samples.
- The 8 by 8 DCT can describe spatial-frequency combinations.Each coefficient is a description of a horizontal and vertical frequency pair rather than the brightness of one original pixel.
- The DC coefficient is the scaled average level of the block.The other 63 AC coefficients describe changes across the block, including the faster patterns needed for texture and sharp edges.
- A constant block has only one non-zero coefficient before quantization.For equal level-shifted samples a, the chapter gives C(0,0)=8a while every AC coefficient is zero.
- Smooth real regions can concentrate energy near the low-frequency corner.A sky or wall may need fewer strong AC terms than lettering, edges, or sharply textured content.
Major section
Quantization creates zeros and zig-zag exposes runs
The JPEG pipeline separates lossy coefficient rounding from compact representation of the result.
- Quantization can divide coefficients by position-dependent steps and round the results.Larger steps can turn small high-frequency coefficients into zeros, introducing loss before the later packing stages.
- The zig-zag sequence can begin at DC and cross low-frequency diagonals.The scan reaches the highest-frequency corner last, placing likely surviving coefficients ahead of the high-frequency zero tail.
- Reordering alone cannot change coefficients or create compression savings.The scan exposes zero runs already created by quantization so the following coding stages can exploit that structure.
- Run-length and entropy coding can represent the zero tail compactly.The saving depends on the actual quantized block; sharp detail may leave more non-zero coefficients than a smooth region.
Major section
Motion information identifies a reference prediction
The reference supplies a prediction while the residual preserves its unexplained difference.
- A bounded search can select a low-distortion reference candidate.Sum of absolute differences compares the target with displaced reference samples to find the candidate used for prediction.
- The selected displacement is the motion vector.The decoder can apply that vector to its reference frame to construct the same predicted block.
- The residual is the target minus its prediction.The chapter writes E=T-P, preserving the information that the shifted reference block failed to explain.
- A good prediction can leave smaller and less structured residual values.The encoder still needs motion information and a transformed, quantized, entropy-coded residual before transport can carry the result.
Major section
Five small residuals make a promising prediction
A sparse small-valued residual is a useful candidate for economical coding.
- The worked residual has five non-zero samples among sixteen positions.Every non-zero magnitude is one, so the prediction leaves a sparse, small-valued block for the transform stage.
- The transform can describe what the prediction missed.Motion compensation changes the transform input to the residual instead of replacing transform and quantization.
- The decoder needs both prediction and decoded residual information.The decoded target is the prediction plus the residual, rather than merely a copy of an imperfect earlier block.
- Sparse residuals are promising inputs rather than guaranteed mode winners.The encoder must count the motion information and coded residual before deciding whether prediction costs less than an intra block.
Activity 2 · Work it out
✎ Count the residual before choosing a mode
I need the residual count before I believe this prediction saves bits.
Copy these residual rows: 0,0,1,0; 0,-1,1,0; 0,0,0,0; 1,0,0,-1. Count non-zero samples and their magnitudes. Explain why a scene cut can reverse the prediction advantage.
4 minutes · Pen and paper · Answer: Activity 2
Major section
Mode costs and rate examples need honest boundaries
A coding advantage must survive the actual mode comparison and the evidence behind the reported rates.
- A scene cut can make an intra block cheaper than prediction.A poor reference leaves a large residual whose cost, plus motion information, may exceed a self-contained coded block.
- The lecture’s rate ranges are examples rather than current codec targets.Wideband audio is listed at 224 kbps raw and 8-12 kbps compressed, with results dependent on content and settings.
- The raw-video comparison can illustrate scale without promising a universal result.The 720p example is 530 Mbps raw versus 0.768-1.536 Mbps compressed, described as about 345-690 times reduction.
- The still-image row cannot support an honest compression ratio.The printed 30-300 MB compressed range is larger than the 3.14 MB raw image, so no honest compression ratio follows.
Major section
Audio, images, and video control different errors
The audio, image, and video mechanisms each have a distinct limit on acceptable loss.
- Masking can hide selected audio error below a signal-dependent threshold.An exceeded threshold can make distortion audible, so noise placement needs the intended listening conditions.
- The DCT can concentrate image energy before quantization discards detail.JPEG ordering can expose the resulting zeros to compact packing without being the stage that creates lossy rounding.
- Video prediction can reduce the information that residual coding must describe.A poor reference can instead make the combined vector and residual more expensive than an intra alternative.
- A defensible compression result needs content, settings, and valid units.The chapter’s rejected still-image range shows why an internally inconsistent comparison cannot become a measured saving.
Deck summary
The receiver must still be able to use the result
The intended receiver and measured coding path define whether the saving is useful.
- An inaudible-looking error still needs a relevant listening test.A signal-dependent masking model guides distortion allocation but cannot make every listener and playback path equivalent.
- Image compression needs energy concentration, lossy rounding, and compact packing.DCT coefficients, quantization, zig-zag order, and entropy coding have distinct roles in the final representation.
- An earlier frame is useful only when its prediction earns a saving.The mode decision must compare the full motion-information and residual cost with the intra-coded alternative.
- A claimed reduction needs measurements with consistent quantities and units.Actual images, encoder settings, and decoder paths provide a sounder record than silently repairing the rejected still-image source range.
Retrieval practice
Recall check 1 of 3
Broker Bex says: answer from memory, then check your reasoning.
Q1Two noise signals have the same total power. White noise is clearly audible over a music clip, while signal-shaped noise is difficult to hear. Which explanation matches psychoacoustic masking?
Show answer
Answer: A A perceptual coder allocates distortion where a signal-dependent model predicts that the listener is least likely to detect it.
Retrieval practice
Recall check 2 of 3
Broker Bex says: answer from memory, then check your reasoning.
Q2After quantization, an 8 by 8 image block has several non-zero low-frequency DCT coefficients and many high-frequency zeros. Why does JPEG use a zig-zag scan?
Show answer
Answer: A DCT compacts energy, quantization removes small coefficients, and zig-zag ordering makes the resulting zero runs easy to represent.
Retrieval practice
Recall check 3 of 3
Broker Bex says: answer from memory, then check your reasoning.
Q3A scene cut makes the best motion-compensated prediction differ greatly from the target block. What should the encoder conclude?
Show answer
Answer: A Motion compensation is a rate-distortion choice.
Print reference
Answers
Answer key.
- A · A perceptual coder allocates distortion where a signal-dependent model predicts that the listener is least likely to detect it.
- A · DCT compacts energy, quantization removes small coefficients, and zig-zag ordering makes the resulting zero runs easy to represent.
- A · Motion compensation is a rate-distortion choice.
Print reference
Activity 1 answer
Model answer.
Match: A is original; B adds white noise at -13 dB; C shapes noise against the signal-dependent masking threshold; D isolates the difference. Time-frequency placement affects audibility, so equal nominal noise level does not imply equal listening results.
Print reference
Activity 2 answer
Model answer.
Work it out: Five of sixteen samples are non-zero and every magnitude is one. This is a sparse transform input. A scene cut may create many large residuals, making motion-vector plus residual cost exceed an intra-coded block; compare actual mode costs.