Multimedia Commnication Unit 4 VTU

Audio and video compression
Raghudathesh G P
Asst Professor
MULTIMEDIA COMMUNICATION
(VTU) - 10EC841
UNIT 4:
Audio and video compression: audio compression, video compression, video compression
principles, video compression standards: H.261, H.263, P1.323, MPEG 1, MPEG 2, Other
coding formats for text, speech, image and video.
6 Hours
TEXT BOOK:
1. Multimedia Communications: Applications, Networks, Protocols and Standards, Fred

Halsall, Pearson Education, Asia, Second Indian reprint 2002.
the
sh
REFERENCE BOOKS:
1. Multimedia Fundamentals: Vol 1 - Media Coding and Content Processing, Ralf

Steinmetz, Klara Narstedt, Pearson Education, 2004.
Special Thanks To:
ud
a
1. Ramisuniverse for providing free e-material.

2. Students :
BY:
Ra
gh
RAGHUDATHESH G P
Asst Prof
ECE Dept, GMIT
Davangere 577004
Cell: +917411459249
Mail: [email protected]
Quotes from Great People:

End of Education is character.
No one is so old as those who have outlived enthusiasm.
As water collected in a tank gets pure by filtration, so accumulated wealth is preserved by being
employed in charity.
No one is useless in this world who lightens the burdens of another.
ECE Dept, GMIT
[email protected]
Page No - 1
Raghudathesh G P
Asst Professor
Audio & Video Compression

Introduction:
Both audio and most video signals are continuously varying analog signals.
After digitization, both comprise a continuous stream of digital values, each value
representing the amplitude of a sample of the particular analog signal taken at repetitive
time intervals.
Thus, the compression algorithms associated with digitized audio and video are different
from those of text and image.
Audio Basics:
Digitization of audio signals is done using the digitization process known as pulse code
modulation (PCM).
PCM involves sampling the (analog) audio signal/waveform at a minimum rate which is
twice that of the maximum frequency component that makes up the signal.
If the (frequency) bandwidth of the communications channel to be used is less than that
of the signal, then the sampling rate is determined by the bandwidth of the
communication channel then is known as a bandlimited signal.
In the case of a speech signal, the maximum frequency component is 10 kHz and hence
the minimum sampling rate is 20 ksps.
Ex.: Sampling rate and number of bits per sample

Sampling rate Audio = 20ksps.
Music = 40ksps.
Number of bits per sample Audio = 16 bits.

Speech = 12 bits.
Stereophonic signal has 2 channels, hence, two signals must be digitized.

Due to the above reason, this produces a bit rate of 240 kbps for a speech signal and 1.28
Mbps for a stereophonic music/general audio signal, the latter being the rate used with
compact discs. In practice, however, in most multimedia applications involving audio, the
bandwidth of the communication channels that are available dictate rates that are less
than these.
It can be achieved in one of two ways:
1. The audio signal is sampled at a lower- rate (if necessary, with fewer bits per
sample).
2. Compression algorithm is used.
The first method is relatively simple to implement, using a lower sampling rate.
Ra
gh
ud
a
the
sh
ECE Dept, GMIT
[email protected]
Page No - 2
Raghudathesh G P
Asst Professor
Disadvantage:
1. The quality of the decoded signal is reduced due to the loss of the higher
frequency components from the original signal.
2. The use of fewer bits per sample results in the introduction of higher levels of
quantization noise.
Thus often, a compression algorithm is used and its advantages are as follows:
This achieves a comparable perceptual quality (as perceived by the ear) to that
obtained with a higher sampling rate but with a reduced bandwidth requirement.
Differential pulse code modulation (DPCM):
Differential pulse code modulation (DPCM) is a derivative of standard PCM.
Exploits the fact that, for most audio signals, the range of the differences in amplitude
between successive samples of the audio waveform is less than the range of the actual
sample amplitudes. Hence, if only the digitized difference signal is used to encode the
waveform then fewer bits are required than for a comparable PCM signal with the same
sampling rate.
A DPCM encoder and decoder are shown in Figure (a) and a simplified timing
diagramofr the encoder is shown in Figure (b).
Ra
gh
ud
a
the
sh
ECE Dept, GMIT
[email protected]
Page No - 3
Raghudathesh G P
Asst Professor
Operation:
The previous digitized sample of the analog input signal is held in the register (R) a
temporary storage facility and the difference signal (DPCM) is computed by
subtracting the current contents of the register (Ro) from the new digitized sample
output by the ADC (PCM).
The value in the register is then updated by adding to the current register contents the
computed difference signal output by the subtractor prior to its transmission.
The decoder operates by simply adding the received difference signal (DPCM) to the
previously computed signal held in the register (PCM).
Typical savings with DPCM, are limited to just 1 bit, which, for a standard PCM
voice signal, example, reduces the bit rate requirement from 64kbps to 56 kbps.
Observations:
For the circuit shown in Figure (a) we see that, the output of the ADC is used directly
and hence the accuracy of each computed difference signal also known as the residual
(signal) is determined by the accuracy of the previous signal/value held in the
register.
As, all ADC operations produce quantization errors and hence a string of, say,
the
sh
positive errors, will have a cumulative effect on the accuracy of the value that is held
The above method is implemented by predicting the previous signal by using not only the
estimate of the current signal, but also varying proportions of a number of the
immediately preceding estimated signals.
gh
ud
a
in the register.
Thus, with a basic DPCM scheme, the previous value held in the register is only an
approximation.
Due to above reason a more sophisticated technique has been developed for estimating
and is known as predicting.
Ra
The proportions used are determined by what is known as predictor coefficients and the
principle is shown in the figure below.
ECE Dept, GMIT
[email protected]
Page No - 4
Raghudathesh G P
Asst Professor
the
sh
In this scheme, the difference signal is computed by subtracting varying proportions of

the last three predicted values from the current digitized value output by the ADC.
Ex.: If the three predictor coefficients have the values C, = 0.5 and C2 = C3 = 0.25, then
the contents of register R1 would be shifted right by 1 bit (thereby multiplying its
contents by 0.5) and registers R2 and R3 by 2 bits (multiplying their contents by 0.25).
The three shifted values are then-added together and the resulting sum subtracted from
the current digitized value output by the ADC (PCM).
The current contents of register R1 are then transferred to register R2 and that of register
R2 to register R3.
Ra
gh
ud
a
The new predicted value is then loaded into register R1 ready for the next sample to be
processed.
The decoder operates in a similar way by adding the same proportions of the last three
computed PCM signals to the received DPCM signal.
By using this approach, a similar performance level to standard PCM is obtained by using
only 6 bits for the difference signal which produces a bit rate of 32 kbps.
ECE Dept, GMIT
[email protected]
Page No - 5
Raghudathesh G P
Asst Professor
DPCM ENCODING AND DECODING
DPCM = PCM - R0
R1 = R0 + DPCM
Where,
DPCM- Differential Pulse Code Modulation
PCM - Pulse Code Modulation
R0 - Current contents of register R
R1 New/updated contents
Consider a signal with voltage levels +5V to -5V (a dynamic range of 10V). The signal sequence is
as given below:
5
4
1
-2
-5
-3
0
3
5
3
the
sh
ENCODING
The input sequence is encoded using the DPCM encoder as follows:
ud
a
Step 1: PCM= 3, R0= 0

Subtractor Output= DPCM= PCM-R0
DPCM= 3-0= 3
Adder Output= R1= R0+ DPCM
R1= 0+3= 3
Ra
gh
Step 2: PCM= 5, R0= 3

DPCM= 5-3= 2
R1= 3+2= 5
Step 3: PCM= 4, R0= 5
DPCM= 4-5= -1
R1= 5-1= 4
Step 4: PCM= 1, R0= 4

DPCM= 1-4= -3
ECE Dept, GMIT
[email protected]
Page No - 6
Raghudathesh G P
Asst Professor

R1= 4-3= 1
the
sh
Step 6: PCM= -5, R0= -2

DPCM= -5-(-2) = -3
R1= -3-2= -5
Step 5: PCM= -2, R0= 1

DPCM= -2-1= -3
R1= 1-3= -2
ud
a
Step 7: PCM= -3, R0= -5

DPCM= -3-(-5) = 2
R1= -5+2= -3
Ra
gh
Step 8: PCM= 0, R0= -3

DPCM= 0-(-3) = 3
R1= -3+3= 0
Step 9: PCM= 3, R0= 0
DPCM= 3-0= 3
R1= 0+3= 3
Step 10: PCM= 5, R0= 3
DPCM= 5-3= 2
ECE Dept, GMIT
[email protected]
Page No - 7
Raghudathesh G P
Asst Professor
R1= 3+2= 5
Therefore, the encoded sequence is,
3
2
-1
-3
-3
-3
The encoded sequence is decoded using the DPCM decoder as follows:

Encoded sequence: 3
2
-1
-3
-3
-3
2
3
DECODING
P
G
the
sh
Step 1: DPCM= 3, R0= 0

Adder Output (PCM) = R1= DPCM+R0
R1= 3+0= 3

R1= 2+3= 5
ud
a
Step 3: DPCM= -1, R0= 5

R1= -1+5= 4
gh
Step 4: DPCM= -3, R0= 4

R1= -3+4= 1
Ra
Step 5: DPCM= -3, R0= 1

R1= -3+1= -2
Step 6: DPCM= -3, R0= -2
R1= -3-2= -5
Step 7: DPCM= 2, R0= -5
R1= 2-5= -3
ECE Dept, GMIT
[email protected]
Page No - 8
Raghudathesh G P
Asst Professor
Step 8: DPCM= 3, R0= -3

R1= 3-3= 0
R1= 3+0= 3
Step 10: DPCM= 2, R0= 3

R1= 2+3= 5
Therefore, the decoded sequence is,
5
Adaptive differential PCM:
-2
-5
-3
the
sh
Definition: using varying number of bits for the difference signal depending on its
amplitude; that is, using fewer bits to encode (and hence transmit) smaller difference
values than for larger values.
Advantages: Saving in bandwidth and improving in audio quality.
Is an international standard for this is defined in ITU-T Recommendation G.721.

This is based on the same principle as DPCM except an eight-order predictor is used and
the number of bits used to quantize each difference value is varied.
Variants:
1. 6 bits producing 32 kbps to obtain a better quality output than with third-order
DPCM, or
2. 5 bits producing 16 kbps if lower bandwidth is more important.
Ra
gh
ud
a
A second ADPCM standard, which is a derivative of G.721, is defined in ITU-T

Recommendation G.722.
This provides better sound quality than the G.721 standard at the expense of added
complexity.
To achieve this, it uses an added technique known as subband coding.
The input speech bandwidth is extended to be from 50 Hz through to 7 kHz compared

with 3.4 kHz for a standard PCM system and hence the wider bandwidth produces a
higher-fidelity speech signal.
ECE Dept, GMIT
[email protected]
Page No - 9
Raghudathesh G P
Asst Professor
This is particularly important in conferencing applications, in order to enable the

members of the conference to discriminate between the different voices of the members
present.
The general principle of the scheme is shown in Figure below
To allow for the higher signal bandwidth, prior to sampling the audio input signal, it is
first passed through two filters:
1. One which passes only signal frequencies in the range 50 Hz through to 3.5 kHz,
and
2. The other only frequencies in the range 3.5 kHz through to 7 kHz.
By doing above process, the input (speech) signal is effectively divided into two separate
equal bandwidth signals, the first known as the lower subband signal and the second the
upper subband signal.
Each subband is then sampled and encoded independently using ADPCM, the sampling
rate of the upper subband signal being 16ksps to allow for the presence of the higher
frequency components in this subband.
ud
a
gh
Ra
the
sh
The use of two subbands has the advantage that different bit rates can be used for each
subband.
The frequency components that are present in the lower subband signal have a higher
perceptual importance than those in the higher subband.
The operating bit rate can be 64, 56, or 48 kbps.
With a bit rate of 64 kbps, the lower subband is ADPCM encoded at 48 kbps and the
upper subband at 16 kbps.
ECE Dept, GMIT
[email protected]
Page No - 10
Raghudathesh G P
Asst Professor
The two bitstreams are then multiplexed (merged) together to produce the transmitted
(64kbps) signal in such a way that the decoder in the receiver is able to divide them back
again into two separate streams for decoding.
A third standard based on ADPCM is also available. This is defined in ITU-T

Recommendation G.726. This also uses subband coding but with a speech bandwidth of
3.4 kHz. The operating bit rate can be 40, 32. 24 or 16 kbps.
Adaptive predictive coding:

Higher levels of compression at higher levels of complexity can be obtained by making
the predictor coefficients adaptive. This is the principle of adaptive predictive coding
(APC).
Here the predictor coefficients continuously change.
Practically, the optimum set of predictor coefficients continuously vary since they are a
function of the characteristics of the audio signal being digitized.
To exploit this property, the input speech signal is divided into fixed time segments and,
for each segment, the currently prevailing characteristics are determined.
The optimum sets of coefficients are then computed and these are used to predict more
accurately the previous signal.
the
sh
This type of compression can reduce the bandwidth requirements to 8 kbps while still
obtaining an acceptable perceived quality.
ud
a
Linear predictive coding:
All the previous algorithms we have considered are based on sampling the time-varying
speech waveform and then either sending the quantized samples directly (PCM) or
sending the quantized difference signal (DPCM and its derivatives).
But with the advent of inexpensive digital signal processing circuits, an alternative
approach has become possible which involves the source simply analyzing the audio
waveform to determine a selection of the perceptual features it contains. These are
then quantized and sent and the destination uses them, together with a sound synthesizer,
to regenerate a sound that is perceptually comparable with the source audio signal.
Ra
gh
The above method is the basis of the linear predictive coding (LPC) technique and,
although with this the generated sound normally speech can often sound synthetic, very
high levels of compression (and hence low bit rates) can be achieved.
The key to this approach is to identify the set of perceptual features to be used. In terms
of speech, the three features which determine the perception of a signal by the ear are its:
ECE Dept, GMIT
[email protected]
Page No - 11
Raghudathesh G P
Asst Professor
Pitch: this is closely related to the frequency of the signal and is important
because the ear is more sensitive to frequencies in the range 2-5 kHz than to
frequencies that are higher or lower than these;
Period: this is the duration of the signal;
Loudness: this is determined by the amount of energy in the signal.
The origins of the sound are important. These are known as vocal tract
excitation parameters and classified as:
Voiced sounds: these are generated through the vocal chords and
examples include the sounds relating to the letters m, v, and I;
Unvoiced sounds: with these the vocal chords are open and example,
include the sounds relating to the letters f and s.
When the above features have been obtained from the source waveform, it is possibly to
use them, together with a suitable model of the vocal tract, to generate a, synthesized
version of the original speech signal.
The basic features of an LPC encoder/decoder are shown in Figure below.
Ra
gh
ud
a
the
sh
ECE Dept, GMIT
[email protected]
Page No - 12
Asst Professor
Operation:
The input speech waveform is first sampled and quantized at a defined rate.
A block of digitized samples known as a segment is then analyzed to determine the
various perceptual parameters of the speech that it contains.
The speech signal generated by the vocal tract model in the decoder is a function of
the present output of the speech synthesizer as determined by the current set of model
coefficients plus a linear combination of the previous set of model coefficients.
Thus, the vocal tract model used is adaptive and, the encoder determines and sends a
new set of coefficients for each quantized segment.
Hence, the output of the encoder is a string of frames, one for each segment.
Raghudathesh G P
Each frame contains fields for pitch and loudness the period is determined by the
sampling rate being used a notification of whether the signal is voiced or unvoiced, and a
new set of computed model coefficients.
Some LPC encoders use up to ten sets of previous model coefficients to predict the
output sound (LPC-10) and use bit rates as low as 2.4 kbps or even 1.2 kbps.
However, the generated sound at these rates is often very synthetic and hence LPC coders
are used primarily in military application in which bandwidth is all-important.
the
sh
ud
a
Code-excited LPC:
Synthesizers used in most LPC decoders are based on a very basic model of the vocal
tract.
A more sophisticated version of this, known as a code-excited linear prediction (CEIP)
model, is also used.
Ex.: enhanced excitation (LPC) models.
Used primarily for applications in which the amount of bandwidth available is limited but
the perceived quality of the speech must be of an acceptable standard for use in various
multimedia applications.
Ra
gh
In this model for encoding purposes, just a limited set of segments is used, each known as
a waveform template. A precomputed set of templates are held by the encoder and
decoder in what is known as a template codebook.
Each of the individual digitized samples that make up a particular template in the
codebook are differentially encoded
Each codeword that is sent selects a particular template from the codebook whose
difference values best match those quantized by the encoder.
Hence there is continuity from one set of samples to another, as a result, an improvement
in sound quality is obtained.
ECE Dept, GMIT
[email protected]
Page No - 13
Raghudathesh G P
Asst Professor
There are four international standards available they are ITU-T Recommendations G.728,
729, 729(A), and 723.1 all of which give a good perceived quality at low bit rates.
Every coders of this type have a delay associated with them which is incurred while each
block of digitized samples is analyzed by the encoder and the speech is reconstructed at
the decoder. The combined delay value is known as the coder's processing delay.
Also before the speech samples can be analyzed, it is necessary to buffer in memory the
block of samples. The time to accumulate the block of samples is known as the
algorithmic delay and, in some CELP coders, this is extended to include samples from
the next successive block, a technique known as lookahead.
These delays occur in the coders are in addition to the end-to-end transmission delay over
the network. Nevertheless, the combined delay value of a coder is an important parameter
as it often determines the suitability of the coder for a specific application.
Example:
For a conventional telephony application, a low-delay coder is required since a
large delay can impede the flow of a conversation.
For an interactive application that involves the output of speech stored in a file, a
delay of several seconds before the speech starts to be output is often acceptable
and hence the coder's delay is less important.
Other parameters of the coder that are considered are the complexity of the coding
algorithm and the perceived quality of the output speech.
In general, a compromise has to be reached between a coder's speech quality and its
delay/complexity.
Application: Both LPC and CELP are used primarily in telephone application for the
compression of speech application.
gh
ud
a
the
sh
Perceptual Coding:
Perceptual, encoders are designed for the compression of general audio associated with a
digital television broadcast.
They use a known as a psychoacoustic model, its role is to exploit a number of the
limitations of the human ear.
Here sampled segments of the source audio waveform are analyzed as with CELP-based
coders but only those features that are perceptible to the ear are transmitted.
Ex.: the human ear is sensitive to signals in the range 15Hz through to 20kHz, the level of
sensitivity to each signal is non-linear; that is, the ear is more sensitive to some signals
than other.
Ra
ECE Dept, GMIT
[email protected]
Page No - 14
Raghudathesh G P
Asst Professor
When multiple signals are present a strong signal may reduce the level of sensitivity of
the ear to other signals which are near to it in frequency, this effect is known as
frequency masking.
When the ear hears a loud sound, it takes a short but finite time before it can hear a
quieter sound, an effect known as temporal masking.
A psychoacoustic model is used to identify those signals that are influenced by both these
effects. These are then eliminated from the transmitted signals and, in so doing; this
reduces the amount of information to be transmitted.
Application: range of audio compression algorithm.
Sensitivity of the ear:
For the ear, its dynamic range is the ratio of the loudest sound it can hear to the quietest
sound and is in the region of 96dB.
But, the sensitivity of the ear varies with the frequency of the signal and, assuming just a
single frequency signal is present at any one time, the perception threshold of the ear that
is, its minimum level of sensitivity as a function of frequency is shown in Figure below
The ear is most sensitive to signals in the range 2-5 kHz and hence signals within this
band are the quietest the ear is sensitive to.
The vertical axis, indicates the amplitude level of all the other signal frequencies relative
to this level - measured in dB - that are required for them to be heard. Thus in the figure,
although the two signals A and B have the same relative amplitude, signal A would be
heard - that is, it is above the hearing threshold - while signal B would not.
Ra
gh
ud
a
the
sh
ECE Dept, GMIT
[email protected]
Page No - 15
Raghudathesh G P
Asst Professor
the
sh
Frequency masking:
When an audio sound consists of multiple signal frequency, the sensitivity of the ear
changes and varies with the relative amplitude of the signals. The curve showed below
shows how the sensitivity of the ear changes in the vicinity of a loud signal.
In the above curve signal B is larger in amplitude than signal A, this causes the basic
sensitivity curve of the ear to be distorted in the region of signal B.
Due to the above reason, signal A will no longer be heard even though on its own, it is
above the hearing threshold of the ear for a signal of that frequency.
This is the origin of the term frequency masking and, in practice, the masking effect also
varies with frequency as we show in Figure below
Ra
gh
ud
a
Different curves show the masking effect of a selection of different frequency signals - 1,
4, and 8 kHz - and, the width of the masking curves - that is, the range of frequencies that
are affected increase with increasing frequency. The width of each curve at a particular
signal level is known as the critical bandwidth for that frequency.
ECE Dept, GMIT
[email protected]
Page No - 16
Raghudathesh G P
Asst Professor
Experiments have shown that, for frequencies less than 500 Hz, the critical bandwidth
remains constant at about 100 Hz. For frequencies greater than 500 Hz, however, the
critical bandwidth increases (approximately) linearly in multiples of 100Hz.
Ex.:
For signal of 1 kHz (2 x 500 Hz), the critical bandwidth is 200 (2 x 100) Hz
For signal of 5 kHz (10 x 500 Hz), the critical bandwidth is 1000 (10 x 100) Hz
Thus if the magnitude of the frequency components that make up an audio sound can be
determined, then it is possible to determine those frequencies that can be and not to be
transmitted.
Temporal masking:
the
sh
Problem: Given a frequency of 5000 Hz, what is the next highest (Integer) frequency signal that
is distinguishable by the human ear assuming the latter signal is of a substantially lower
amplitude?
Solution: 5,000 Hz (10 x 500) critical bandwith is 10 x 100 Hz = 1000Hz
Assuming the given frequency is in the middle of the critical bandwidth, thus, range of band is
4500 5500 Hz
So next highest audible frequency is 5501 Hz
Definition: It is the amount of taken by human ear to hear a quite sound after hearing a
loud sound.
Temporal masking or non-simultaneous masking occurs when a sudden stimulus sound

makes inaudible other sounds which are present immediately preceding or following the
stimulus.
The general effect of temporal masking is as shown in the graph below
Ra
gh
ud
a
From the figure above we see that after a loud sound ceases it takes a short time (1/10 th of
millisecond) for the signal amplitude to decay.
During decay time, signals whose amplitude are less than the decay envelope will not be
heard and thus are need not to be transmitted.
ECE Dept, GMIT
[email protected]
Page No - 17
Raghudathesh G P
Asst Professor
Conclusion: in order to exploit temporal masking, it is necessary to process the input

audio waveform over a time period that is comparable with that associated with temporal
masking.
MPEG audio Coders:

The Motion Pictures Expert Group (MPEG) was formed by the ISO to formulate a set of
standards relating to a range of multimedia applications that involve the use of video
with sound.
The coders associated with the audio compression part of these standards are known as
MPEG audio coders and a number of these use perceptual coding.
All the signal processing operations associated with a perceptual coder carried out
digitally and a schematic diagram of a basic encoder and decoder is shown in figure
below.
Ra
gh
ud
a
the
sh
ECE Dept, GMIT
[email protected]
Page No - 18
Raghudathesh G P
Asst Professor
The time-varying audio input signal is first sampled and quantized using PCM, the
sampling rate and number of bits per sample being determined the specific application.
The bandwidth that is available for transmission is divided into a number of frequency
subbands using a bank of analysis filters which, because of their role, are also known
as critical-band filters.
Each frequency subband is of equal width and, essentially, the bank of filters maps each
set of 32 (time-related) PCM samples into an equivalent set of 32 frequency samples, one
per subband. Hence each is known as a subband sample and indicates the magnitude of
each of the 32 frequency components that are present in a segment of the audio input
signal of a time duration equal 32 PCM samples.
Ex.: assuming 32 subbands and a sampling rate of 32 ksps thus, a maximum signal
frequency of 16 kHz and each subband has bandwidth of 500 Hz.
In a basic encoder, the time duration of each sampled segment of the audio input signal is
equal to the time to accumulate 12 successive sets 32 PCM and hence subband samples;
that is, a time duration equal to (12 x 32) PCM samples.
In addition to filtering the input samples into separate frequency subbands, the analysis
filter bank also determines the maximum amplitude the 12 subband samples in each
subband.
Each of the known as the scaling factor for the subband and these are passed both to the
psychoacoustic model and together with the set of frequency samples in each subband, to
the corresponding quantizer block.
The processing associated with both frequency and temporal masking is carried out by
the psychoacoustic model which is performed concurrently with the filtering and analysis
operations.
ud
a
gh
the
sh
The 12 sets of 32 PCM samples are first transformed into an equivalent set of frequency
components using a mathematical technique known as the discrete Fourier transform.
Using the known hearing thresholds and masking properties of each subband, the model
determines the various masking effects of this set of signals.
The output of the model is a set of what are known as signal-mask ratios (SMRs) and
indicate those frequency components whose amplitude below the related audible
threshold. In addition, the set of scaling factors used to determine the quantization
accuracy - and hence bit allocations - be used for each of the audible components.
The above method is done so that those frequency components that are in regions of
highest sensitivity can be quantize with more accuracy and hence less quantization noise
than those in regions where the ear is less sensitive.
In a basic encoder, all the frequency components in a sampled segment are encoded and
these are carried in frame the format of which is shown in figure below.
Ra
ECE Dept, GMIT
[email protected]
Page No - 19
Raghudathesh G P
Asst Professor
The header contains information such as the sampling frequency that has been used.
The quantization is performed in two stages using a form of companding.
The peak amplitude level in each subband i.e., the scaling factor is first quantized using 6
bits giving 1 of 64 levels and a further 4 bits are then used to quantize the 12 frequency
components in the subband relative to this level. Collectively this is known as the
subband sample (SBS) format and, in this way, all the information necessary for
decoding is carried within each frame.
In the decoder, after the magnitude of each set of 32 subband samples have been
determined by the dequantizers, these are passed to the synthesis filter bank. The latter
then produces the corresponding set of PCM samples which are decoded to produce the
time-varying analog output segment.
The ancillary data field at the end of a frame is optional and is used, for example, to carry
additional coded samples associated with, say, the surround-sound that is present with
some digital video broadcasts.
The use in the encoder of different scaling factors for each subband means that the
frequency components in the different subbands have varying levels of quantization noise
associated with them.
The bank of synthesis filters in the decoder, however, limits the level of quantization
noise in each subband to the same band of frequencies as the set of frequency
components in that subband. As a result, the effect of quantization noise is reduced since
the signal-to-noise ratio in each subband is increased by the larger amplitude of the signal
frequency components in each subband masking the reduced level of quantization noise
that is-present.
Ra
gh
ud
a
the
sh
From the figure we see that, the psychoacoustic model is not required in the decoder and,
as a result, it is less complex than the encoder. This is a particularly desirable feature in
audio and video broadcast applications as the cost of the decoder is an important factor.
Also, it means that different psychoacoustic models can be used or, if bandwidth is
plentiful, none at all.
An international standard based on this approach is defined in ISO Recominenclation
11172-3.
ECE Dept, GMIT
[email protected]
Page No - 20
Raghudathesh G P
Asst Professor
There are three levels processing associated with this known as:
layer 1,
layer 2, and
layer 3.
Layer 1 is the basic mode and the other two have increasing levels of processing
associated with them which, in turn, produce a corresponding increase in the level of
compression for the same perceived audio quality.
Ex., layer 1 does not include temporal masking but this is present in layers 2 and 3.
The performance of the three layers and examples of their corresponding application
domains are summarized in Table below
Digital audio
cassette
Digital audio &
video broadcast
CD quality audio
over low bit rate
channel
2
3
Compressed bit
rate
32-448 kbps
Quality
Application
HI-fi quality of 192 kbps per

channel
Near CD quality at 128
kbps/channel
CD quality at 7-64 kbps /
channel
the
sh
Layer
32-192 kbps
65 kbps
Example
IO delay
20 ms
40 ms
60 ms
The encoders associated with each of the three layers obtain increasing levels of
compression and perception quality.
The encoder and decoder delay figures are determined by the PCM sampling rate used
and the corresponding frame size.
Ex., with layer 1, the sampling rates used are:

32 ksps for use with broadcast communications equipment
44.1 ksps for use with CD-quality audio equipment
48 ksps for use with professional sound equipment.
These produce corresponding frame durations of 12, 8.7, and 8 milliseconds with 384
samples per frame.
Ra
gh
ud
a
The actual input-to-output delay, however, can be as much as two to three times these
values owing to additional processing delays in the encoder and decoder.
Layer 2 is identical to a standard known as MUSICAM and indeed was based on this.
The format of each layer 2 frame is similar to that of layer 1 except that each frame
contains three sets of 384 samples and hence is of a time duration equal to 1152 samples.
The bit rate shown in Table are all for a single audio channel. In practice, four alternative
forms of audio have been identified for multimedia applications:
ECE Dept, GMIT
[email protected]
Page No - 21
Asst Professor
1. monophonic,
2. dual monophonic,
3. two-channel (also known as disjoint) stereo, and
4. single-channel joint stereo.
The latter is the digitized version of the composite stereo sound signal and hence exploits
the redundancy that is present between the two channels. So the bandwidth required for
audio may be the figures shown in the table or monophonic and joint stereo or double the
values shown for dual monophonic and two-channel stereo.
Since the three layers require increasing levels of complexity (and hence cost) to achieve
a particular perceived quality, the choice of layer and hit rate is often a compromise
between the desired perceived quality and the available bit rate.
Raghudathesh G P
the
sh
Dolby audio coders:

The psychoacoustic models associated with the various MPEG coders control the
quantization accuracy of each subband sample by computing and allocating the number
of bits to be used to quantize each sample.
As the quantization accuracy that is used for each sample in a subband may vary from
one set of subband samples to the next, the bit allocation information that is used to
quantize the samples in each subband is sent with the actual quantized samples.
The above information is then used by the decoder to dequantized the set of subband
samples in the frame. This mode of operation of perceptual coder is known, therefore, as
the forward adaptive bit allocation mode, a simplified schematic diagram showing this
operational mode is given in figure (a) below
Ra
gh
ud
a
ECE Dept, GMIT
[email protected]
Page No - 22
Raghudathesh G P
Asst Professor
Advantage of MPEG Coders: psychoacoustic model is required only in the encoder.

Disadvantage of MPEG Coders: a significant portion of each encoded frame contains
bit allocation information which, in turn, leads to a relatively inefficient use of the
available bit rate.
A variation of this approach is to use a fixed bit allocation strategy for each subband
which is then used by both the encoder and decoder. The principle of operation of this
mode is shown in figure(b) below.
Typically, the bit allocations that are selected for each subband are determined by the
known sensitivity characteristics of the ear and the use of fixed allocations means that
this information need not be sent in the frame. This approach is used in a standard known
as Dolby AC-1, the acronym "AC" meaning acoustic coder.
Dolby AC was designed for use in satellites to relay FM radio programs and the
sound associated with television programs. It uses a low-complexity psychoacoustic
model with 40 subbands at a sampling rate of 32Itsps and proportionately more at 44.1
and 48 ksps. A typical compressed bit rate is 512 kbps for two channel stereo.
Ra
gh
ud
a
the
sh
A second variation, which allows the bit allocations per subband to be adaptive while at
the same time minimizing the overheads in the encoder bit-stream, is for the decoder also
to contain a copy of the psychoacoustic model.
Above is then used by the decoder to compute the same or very similar bit allocations
that the psychoacoustic model in the encoder has used to quantize each set of subband
samples.
In order for the psychoacoustic model in the decoder to carry out its own computation of
the bit allocations, it is necessary for it to have a copy of the subband samples.
ECE Dept, GMIT
[email protected]
Page No - 23
Raghudathesh G P
Asst Professor
Hence with this operational mode, instead of each frame containing bit allocation
information in addition to the set of quantized samples it contains the encoded frequency
coefficients that are present in the sampled waveform segment.
The above method is known as the encoded spectral envelope and this mode of
operation, the backward adaptive bit allocation mode. Figure(c) below illustrates the
principle of operation of both the encoder and decoder.
This approach is used in the Dolby AC-2 standard which is utilized in many applications
including the compression associated with the audio of a number of PC sound cards.
Advantage of this method: these produce audio of hi-fi quality at a bit rate of 256 kbps.
Disadvantage: For broadcast applications, the psychoacoustic model is required in the

decoder, the model in the encoder cannot be modified without changing all decoders.
To meet the above requirement, a third variation has been developed that uses both
backward and forward bit allocation principles. This is known as the hybrid
backward/forward adaptive bit allocation mode and is illustrated in figure (b) below.
Ra
gh
ud
a
the
sh
ECE Dept, GMIT
[email protected]
Page No - 24
Raghudathesh G P
Asst Professor
As we can deduce from part (c) of the figure, with the backward bit allocation method on
its own, since the psychoacoustic model uses the encoded spectral envelope, the
quantization accuracy of the subband samples is affected by the quantization noise
introduced by the spectral encoder.
In case of hybrid scheme, although, a backward adaptive bit allocation scheme is used as
in AC-2 - using PMB - an additional psychoacoustic model - PMF - is used to compute the
difference between the bit allocations computed by PMB and those that are computed by
PMF using the forward-adaptive bit allocation scheme.
Above information is then used by PMB to improve the quantization accuracy of the set
of subband samples.
The modification information is also sent in the encoded frame and is used by the PMB in
the decoder to improve the dequantization accuracy.
In addition, should it be required to modify the operational parameters of the PMB in the
encoder and decoder(s), then information can be sent also with the computed difference
information.
As we can see from the figure, the PMF must compute two sets of quantization
information for each set of subband samples and hence is relatively complex. However,
since this is not required in the decoder, this is not an issue.
The hybrid approach is used in the Dolby AC-3 standard which has been defined for use
in a similar range of applications as the MPEG audio standards including the audio
associated with advanced television (ATV).
Ex.: HDTV standard in North America and, in this application, the acoustic
quality of both the MPEG and Dolby audio coders were found to be comparable.
The sampling rate can be 32, 44.1, or 48 ksps depending on the bandwidth of the
source audio signal.
Each encoded block contains 512 sub-band samples. However, in order to obtain
continuity from one block to the next, the last 256 subband samples in the previous block
are repeated to become the first 256 samples in the next block and hence each block
contains only 256 new samples.
Ra
gh
ud
a
the
sh
Assuming a PCM sampling rate of 32 ksps, although each block of samples is of 8ms
duration 256/32 the duration of each encoder block is 16 ms.
The audio signal bandwidth at this sampling rate is 15 kHz and hence each subband has a
bandwidth of 62.5 Hz, that is, 15k/256. The (stereo) bit rate is, typically, 192 kbps.
ECE Dept, GMIT
[email protected]
Page No - 25
Raghudathesh G P
Asst Professor
Video Compression:
Video compression principles:
With respect to compression, video is simply a sequence of digitized pictures, can also
referred to as moving pictures and the terms "frame and "picture" are used
interchangeably.
One approach to compressing a video source is to apply the JPEG algorithm to each
frame independently. This approach is known as moving JPEG or MJPEG.
The drawback of JPEG is that its typical compression ratios obtainable with JPEG are
between 10:1 and 20:1, neither of which is large enough on its own to produce the
compression ratios needed with the video.
In practice, in addition to the spatial redundancy present in each frame, considerable

redundancy is often present between a set of frames since, in general, only a small
portion of each frame is involved with any motion that is taking place.
Ex.:
1. the movement of a person's lips or eyes in a video telephony application and
2. a person or vehicle moving across the screen in a movie.
the
sh
In the latter case, a typical scene in a movie has a minimum duration of about 3 seconds,
assuming a frame refresh rate of 60 frames per second, each scene is composed of a
minimum of 180 frames.
Thus by sending only information relating to those segments of each frame that have
movement associated with them, considerable additional savings in bandwidth can be
made by exploiting the temporal differences that exist between many of the frames.
The technique that is used to exploit the high correlation between successive frames is to
predict the content of many of the frames. This is based on a combination of a
preceding and in some instances a succeeding frame.
Instead of sending the source video as a set of individually compressed frames, just a
selection is sent in this form and, for the remaining frames, only the differences between
the actual frame contents and the predicted frame contents are sent.
The accuracy of the prediction operation is determined by how well any movement
between successive frames is estimated. This operation is known as motion estimation.
Ra
gh
ud
a
As the estimation process is not exact, additional information must also be sent to
indicate any small differences between the predicted and actual positions of the moving
segments involved. This is known as motion compensation.
Frame types:
ECE Dept, GMIT
[email protected]
Page No - 26
Raghudathesh G P
Asst Professor
There are two basic types of compressed frame:

1. Frames that are encoded independently and,
2. Frames that are predicted.
The first is known as Intra coded frames or I-frames.
In practice, there are two types of predicted frames:

1. Predictive or P-frames and
2. Bidirectional or B-frames
Because of the way they are derived, the above frames are also known as intercoded or
interpolation frames.
A typical sequence of frames involving just I- and P-frames are shown in Figure below
A sequence involving all three frame types is shown in Figure below
Ra
gh
ud
a
the
sh
I Frames:
I-frames are encoded without reference to any other frames.

Each frame is treated as a separate (digitized) picture and the Y, C b and Cr matrices are
encoded independently using the JPEG algorithm except that the quantization threshold
values that are used are the same for all DCT coefficients.
Thus, the level of compression obtained with I-frames are relatively small.
In principle, therefore, it would appear to be best if these were limited to, say, the first
frame relating to a new scene in a movie.
In practice, however, the compression algorithm is independent of the contents of frames

and hence has no knowledge of the start and end of scenes.
ECE Dept, GMIT
[email protected]
Page No - 27
Raghudathesh G P
Asst Professor
I-frames must be present in the output stream at regular intervals in order to allow for the
possibility of the contents of an encoded I-frame being corrupted during transmission. If
an I-frame was corrupted then, in the case of a movie since the predicted frames are
based on the contents of an I-frame, a complete scene would be lost which, of course,
would be totally unacceptable.
Therefore, I-frames are inserted into the output stream relatively frequently.
The number of frames/pictures between successive I-frames known as a group of
pictures or GOP.
It is represented by symbol N and typical values for N are from 3 through to 12.
P Frames:
The encoding of a P-frame is relative to the contents of either a preceding I-frame or a

preceding P-frame.
P-frames are encoded using a combination of motion estimation and motion

compensation and hence significantly higher levels of compression can be obtained with
them.
The number of P-frame between each successive pair of I-frames is limited since any
errors present the first P-frame will be propagated to the next.
the
sh
The number of frame between a P-frame and the immediately preceding I- or P-frame is
called the prediction span. It is given the symbol M and typical values range from 1,
through to 3, as shown in Figures above.
B Frames:
ud
a
Motion estimation involves comparing small segments of two consecutive frames for
differences and, should a difference be detected, a search is carried out to determine to
which neighboring segment the original segment has moved.
In order to minimize the time for each search, the search region is limited to just a few
neighboring segments.
Applications such as video telephony, the amount of movement between consecutive

frames is relatively small and hence this approach works well.
In applications that involve very fast moving objects, however, it is possible for a
segment to have moved outside of the search region. To allow for this possibility, in
applications such as movies, in addition to P-frames, second type of prediction frame
is used. These are the B-frames and, as can see in Figure above, their contents are
predicted using search regions in both past and future frames.
In addition to allowing for occasional moving objects, this also provides better motion
estimation when an object moves in front of or behind another object.
Ra
gh
ECE Dept, GMIT
[email protected]
Page No - 28
Raghudathesh G P
Asst Professor
Comparison between P Frames and B Frames:

In case for P-frames their contents are encoded by considering the contents of the current
(uncoded) frame relative to the contents of immediately preceding (uncoded) frame.
In the case of B-frames, three (uncoded) frame contents are involved:

1. the immediately preceding I- or P-frame
2. the current frame being encoded, and
3. the immediately succeeding I- or P-frame.
This results in an increase in the encoding (and decoding) delay which is equal to the
time to wait for the next I- or P-frame in the sequence.
B-frames provide the highest level of compression and, because they are not involved in
the coding of other frames, they do not propagate errors.
Decoding Operation:
I Frames:
Information relating to I-frames can be decoded immediately it is received in
order to recreate the original frame.
P Frames:
The received information is first decoded and the resulting information is then
used, together with the decoded contents of the preceding I- or P-frame, to derive
the decoded frame contents.
B Frames:
The received information is first decoded and the resulting information is then
used, together with both the immediately preceding I- or P-frame contents and the
immediately succeeding P- or I-frame contents, to derive the decoded frame
contents.
In order to minimize the time required to decode each B-frame, the order of encoding
(and transmission) of the (encoded) frames is changed so that both the pre-ceding and
succeeding I- or P-frames are available when the B-frame is received.
Ex.: Consider a uncoded frame sequence given as:

IBBPBBPBBI..
Then the reordered sequence would be:

IPBBPBBIBBPBB.
Thus, with B-frames, the decoded contents of both the immediately preceding I- or Pframe and the immediately succeeding P- or I-frame are available when the B-frame is
received.
Ra
gh
ud
a
the
sh
ECE Dept, GMIT
[email protected]
Page No - 29
Raghudathesh G P
Asst Professor
PB Frames:
This does not refer to a new frame type as such but rather the way two neighboring Pand B-frames are encoded as if they were a single frame.
The general principle is as shown in Figure below
This is done in order to increase the frame rate without significantly increasing the
resulting bit rate required.
the
sh
D Frames:
It has been defined for use in specific application like movie/video-on-demand
applications.
With this type of application, a user (at home) can select and watch a particular
movie/video which is stored in a remote server connected to a network.
The selection operation is performed by means of a set-top box and, as with a VCR, the
user may wish to rewind or fast-forward through the movie. Clearly, this requires the
compressed video to be decompressed at much higher speeds.
Hence to support above function, the encoded video stream also contains D-frames which
are inserted at regular intervals throughout the stream.
Ra
gh
ud
a
These are highly compressed frames and are ignored during the decoding of P- and Bframes.
In case of DCT compression algorithm, the DC coefficient associated with each 8 x 8

block of pixels both for the luminance and the two chrominance signals is the mean of all
the values in the related block.
Hence, by using only the encoded DC coefficients of each block of pixels in the
periodically inserted D-frames, a low-resolution sequence of frames is provided each of
which can be decoded at the higher speeds that are expected with the rewind and fastforward operations.
ECE Dept, GMIT
[email protected]
Page No - 30
Raghudathesh G P
Asst Professor
Motion estimation and Compensation:

The encoded contents of both P- and B-frames are predicted by estimating any motion
that has taken place between the frame being encoded and the preceding I- or P-frame
and, in the case of B-frames, the succeeding P- or I-frame.
The various steps that are involved in encoding each P-frame are shown in Figure
below.
Ra
gh
ud
a
the
sh
In figure (a) above, the digitized contents of the Y matrix associated with each frame are
first divided into a two-dimensional matrix of 16 x16 pixels known as a macroblock.
Ex, the 4:1:1 digitization format is assumed and hence the related C b and Cc matrices in
the macro-block are both 8 x 8 pixels.
ECE Dept, GMIT
[email protected]
Page No - 31
Raghudathesh G P
Asst Professor
For identification purposes, each macroblock has an address associated with it and, since
the block size used for the DCT operation is also 8 x 8 pixels, a macroblock comprises
four DCT blocks for luminance and one each for the two chrominance signals.
To encode a P-frame, the contents of each macroblock in the frame -known as the target
frame are compared on a pixel-by-pixel basis with the contents of the corresponding
macroblock in the preceding -I or P-frame. The latter is known as the reference frame.
If a close match is found, then only the address of the macroblock is encoded. If a match
is not found, the search is extended to cover an area around the macroblock in the
reference frame. Typically, this comprises a number of macroblocks as shown in Figure
(b) above.
In practice, the various standards do not specify either the extent of the search area or a
specific search strategy and instead specify only how the results of the search are to be
encoded.
Normally, only the contents of the Y matrix are used in the search, and a match is said to
be found if the mean of the absolute errors in all the pixel positions in the difference
macroblock is less than a given threshold.
Hence, using a particular strategy, all the possible macroblocks in the selected search area
in the reference frame are searched for a match and, if a close match is found, two
parameters are encoded. The first is known as the motion vector and indicates the (x,y)
offset of the macro-block being encoded and the location of the block of pixels in the
reference frame which produces the (close) match.
The search - and hence offset - can be either on macroblock boundaries or, as in the
figure, on pixel boundaries. The motion vector is then said to be single-pixel resolution.
The second parameter is known as the prediction error and comprises three matrices (one
each for Y, Cb and C.) each of which contains the difference values (in all the pixel
locations) between those in the target macroblock and the set of pixels in the search area
that produced the close match.
Ra
gh
ud
a
the
sh
Since the physical area of coverage of a macroblock is small, the motion vectors can be
relatively large values. Also, most moving objects are normally much larger than a single
macroblock. Hence, when an object moves, multiple macroblocks are affected in a
similar way.
The motion vectors, therefore, are encoded using differential encoding (DE) and the
resulting codewords are then Huffman encoded.
The three difference matrices, however, are encoded using the same steps as for I-frames:
DCT, quantization, entropy encoding. Finally, if a match cannot be found - for example if
the moving object has moved out of the extended search area - the macroblock is encoded
independently in the same way as the macroblocks in an I-frame.
ECE Dept, GMIT
[email protected]
Page No - 32
Raghudathesh G P
Asst Professor
Encoding B-frame:
Any motion is estimated with reference to both the immediate preceding I- or P-frame
and the immediately succeeding P- or 1-frame.
The general scheme is shown in Figure below.
Ra
gh
ud
a
the
sh
The motion vector and difference matrices are computed using first the preceding frame
as the reference and then the succeeding frame as the reference.
A third motion vector and set of difference matrices are then computed using the target
and the mean of the two other predicted sets of values.
The set with the lowest set of difference matrices is then chosen and these are encoded in
the same way as for P-frames. The motion vector is then said to be in a resolution of a
fraction of a pixel.
ECE Dept, GMIT
[email protected]
Page No - 33
Raghudathesh G P
Asst Professor
Ex, half-pixel resolution.
Implementation issues:
A schematic diagram showing the essential units associated with the encoding of Iframes are given in the Figure above.
From the figure above we can deduce that, the encoding procedure used for the
macroblocks that make up an I-frame is the same as that used in the JPEG standard to
encode each 8 x 8 block of pixels.
The procedure involves each macroblock being encoded using the three steps:
1. Forward DCT
2. Quantization, and
3. Entropy encoding.
Hence, assuming four blocks for luminance and two for chrominance, each macroblock
would require six 8 x 8 pixel blocks to be encoded.
Ra
gh
ud
a
the
sh
A schematic diagram showing the essential units associated with the encoding of Pframes are given in the Figure above.
In the case of P-frames, the encoding of each macroblock is dependent on the output of
the motion estimation unit which, in turn, depends on the contents of the macroblock
being encoded and the contents of the macro-block in the search area of the reference
frame that produces the closest match to that being encoded.
ECE Dept, GMIT
[email protected]
Page No - 34
Raghudathesh G P
Asst Professor
There are three possibilities:

1. If the two contents are the same, only the address of the macroblock in the
reference frame is encoded.
2. If the two contents are very close, both the motion vector and the difference
matrices associated with the macroblock in the reference frame are encoded.
3. If no close match is found, then the target macroblock is encoded in the same way
as a macroblock in an I-frame.
As we can see in Figure (b). In order to carry out its role, the motion estimation unit
containing the search logic, utilizes a copy of the (uncoded) reference frame.
Above is obtained by taking the computed difference values between the frame currently
being compressed (the target frame) and the current reference frame and decompressing
them using the dequantize (DQ) plus inverse DCT (IDCT) blocks.
After the complete target frame has been compressed, the related set of difference values
are used to update the current reference frame contents ready to encode the next (target)
frame.
The same procedure is followed for encoding B-frames except both the preceding
(reference) frame and the succeeding frame to the target frame are involved.
A schematic diagram showing the essential units associated with the encoding of Bframes are given in the Figure below.
Ra
gh
ud
a
the
sh
An example format of the encoded bitstream output by the encoder is given in the figure
below
ECE Dept, GMIT
[email protected]
Page No - 35
Raghudathesh G P
Asst Professor
Thus, for each macroblock, it is necessary to identify the type of encoding that has been
used. This is the role of the formatter and a typical format that is used to encode the
macroblocks in each frame is shown in part (d) of the figure.
The type field indicates the type of frame being encoded P-, or B and the address identify
the location of the macroblock in the frame. The quantization value is the threshold value
that has been used to quantize all the DCT coefficients in the macroblock and motion
vector is the encoded vector if one is present.
The blocks present, indicates which of the six 8 x 8 pixel blocks that make up the
macroblock are present if any, and, for those present, the JPEG-encoded DCT
coefficients for each block.
Thus, the amount of information output by the encoder varies and depends on the
complexity of the source video. Hence a basic video encoder of the type shown in the
figures above generates an encoded bitstream that has a variable bit rate.
Decoding:
The decoding of the received bitstream is simpler (and hence faster) than the
encoding operation as the time-consuming motion estimation processing is not
required.
Since the encoded bitstream is received and decoded, each new frame is assembled a
macroblock at a time.
Decoding the macroblocks of an I-frame is the same as decoding the blocks in a
JPEG-encoded image.
To decode a P-frame, the decoder retains a copy of the immediately preceding
(decoded) I- or P- frame in a buffer and uses this, together with the received encoded
information relating to each macroblock, to build the Y, Cb, and Cr matrices for the
new frame in a second buffer.
Ra
gh
ud
a
the
sh
ECE Dept, GMIT
[email protected]
Page No - 36
Raghudathesh G P
Asst Professor
With uncoded macroblocks, the macroblock's address is used to locate the

corresponding macroblock in the previous frame and its contents are then transferred
into the second buffer unchanged.
With fully encoded macroblocks, these are decoded directly and the contents
transferred to the buffer.
With macroblocks containing a motion vector and a set of difference matrices, then
these are used, together with the related set of matrices in the first buffer, to
determine the new values for the macroblock in the second buffer.
The decoding of B-frames are similar, except three buffers are used:
One containing the decoded contents of the preceding I- or P-frame,
For the decoded contents of the succeeding P- or I-frame, and
To hold the frame being assembled.
Performance:
The compression ratio for I-frames and hence all intracoded frames is similar to that
obtained with JPEG and, for video frames, typically is in the region of 10:1 through 20:1
depending on the complexity of the frame contents.
The compression ratio of both P- and B- frames is higher and depends on the search
method used. However, typical figures are in the region of 20:1 through 30:1 for Pframes and 30:1 through 50:1 for B-frames.
ud
a
the
sh
H.261:
Ra
The H.261 video compression standard has been defined by the ITU-T for the provision
of video telephony and videoconferencing services over an integrated-services digital
network (ISDN).
Thus the standard assumes that the network offers transmission channels of multiples of
64 kbps.
gh
The standard is also known, therefore, as p x 64 where p can be 1 through 30.

The digitization format used is either the common intermediate format (CIF) or the
quarter CIF (QCIF). Normally, the CIF is used for videoconferencing and the QCIF for
video telephony.
Each frame is divided into macroblocks of 16 x 16 pixels for compression, the horizontal
resolution is reduced from 360 to 352 pixels to produce an integral number of 22
macroblocks.
Hence, since both formats use subsampling of the two chrominance signals at half the
rate used for the luminance signal, the spatial resolution of each format is:
CIF: Y= 352 x 288, Cb = Cr = 176 x 144
ECE Dept, GMIT
[email protected]
Page No - 37
Raghudathesh G P
Asst Professor
QCIF: Y= 176 x 144, Cb = Cr = 88 x 72

Progressive (non-interlaced) scanning is used with a frame refresh rate 30 fps for the
CIF and either 15 or 7.5 fps for the QCIF.
Only I- and P- frames are used in H.261 with three P-frames between pair of I-frames.
The encoding of each of the six 8 x 8 pixel blocks that make up each macroblock in both
I- and P-frames 4 blocks for Y and one each for Cb and Cr- is carried out.
The format of each encoded macroblock is shown in outline in Figure below.
For each macroblock:1. Address us for identification purposes

2. The type field indicates whether the macroblock has been encoded independently
intracoded or with reference to a macroblock in a preceding frame intercoded.
3. The Quantization value is the threshold value that has been used to quantize all
the DCT coefficients in the macroblock.
4. Motion vector is the encoded vector if one present.
5. The coded block patient indicates which of the six 8 x 8 pixel bloc that rake up
the macroblock are present if any, and, for those present, the JPEG-encoded DCT
coefficients are given in each block.
Ra
gh
ud
a
the
sh
The format of each complete frame is shown in the figure below
ECE Dept, GMIT
[email protected]
Page No - 38
Raghudathesh G P
Asst Professor
The picture start code indicate the start of each new (encoded) video frame/picture.
The temporal reference field which is a time stamp to enable the decoder to synchronize
each video block with an associated audio block containing the same time stamp.
The picture type indicates whether the frame is an I- or P-frame.

Although the encoding operation is carried out on individual macroblocks, a larger data
structure known as a group of (macro) blocks (GOB) is also defined. This is a matrix of
11 x 3 macroblocks, the size of which has been chosen that both the CIF and QCIF
comprise an integral number of GOBs 12 in the case of the CIF (2 x 6) and 3 in the case
of the QCIF (1 x 3) which allows interworking between the two formats.
At the head of each GOB is a unique start code which is chosen so that no valid sequence
of variable-length codewords from the table of codewords used in the entropy encoding
stage can produce the same code.
In the event of a transmission error affecting a GOB, the decoder simply searches the
received bitstream for this code which signals the start of the next GOB. For this reason
the start code is also known resynchronization marker.
the
sh
each GOB has a group number associated with it which allows for a string of GOBs to be
missing from a particular frame. This may be necessary, for example, if the amount of
(compressed) information to be transmitted is temporarily greater than the bandwidth the
transmission channel.
With motion estimation, the amount of information produced during the compression
operation varies. However, since the transmission bandwidth that is available with target
applications of the H.261 standard is fixed 64 kbps or multiples of this in order to
optimize the use of this bandwidth, it is necessary to convert the variable bit rate
produced by the basic encoder into a constant bit rate.
This is achieved by first passing the encoded bitstream output by the encoder through a
first-in, first-out (FIFO) buffer prior to it being transmitted and at then providing a
feedback path from this to the quantizer unit within the encoder.
Ra
gh
ud
a
The role of FIFO buffer is shown in figure below.
ECE Dept, GMIT
[email protected]
Page No - 39
Raghudathesh G P
Asst Professor
Role of FIFO Buffer:

The output bit rate produced by the encoder is determined by the quantization
threshold values that are used; the higher the threshold the lower the accuracy and
hence the lower is the output bit rate.
Hence, since the same compression technique is used for macroblocks in video
encoders.
It is possible to obtain a constant output bit rate from the encode by dynamically
varying the quantization threshold used. This is the role of the FIFO buffer.
FIFO Buffer:
As the name implies, the order of the output from a FIFO buffer is the same as
that on input.
However, since the output rate from the buffer is a constant determined by the
(constant) bit rate of the transmission channel the input rate temporarily exceeds
the output rate then the buffer will start to fill.
Conversely, if the input rate falls below the output rate then the buffer contents
will decrease.
In order to exploit this property, two threshold level is derided:
The low threshold and
The high threshold.
The amount of information in the buffer is continuously monitored and, should
the contents fall below the low threshold, then the quantization threshold is
reduced thereby increasing the output rate from the encoder.
Conversely, should the contents increase beyond the high threshold, then the quantization
threshold is increased in order to reduce the output rate from the encoder.
Normally, the control procedure operates at the GOB level rather than at the macroblock
level.
gh
Ra
ud
a
the
sh
Hence, should the high threshold be reached, first the quantization value associated with
the GOB is increased and, if this is not sufficient, GOBs are dropped until the overload
subsides.
Of course, any adjustments to the quantization threshold values that are made must be
made also to those used in the matching dequantizer.
In addition, the standard also allows complete frames to be missing in order to match the
frame rate to the level of transmission bandwidth that is available.
The general Implementation Schematic of H.261 encoder is:
ECE Dept, GMIT
[email protected]
Page No - 40
Raghudathesh G P
Asst Professor
Additional Questions:
the
sh
Ra
gh
ud
a
1. List and explain psychological phenomena that are exploited in audio compression. [04]
2012
2. What are the fundamental differences between MPEG and Dolby audio compression
algorithms? Give an advantage and a disadvantage of Dolby with respect to MPEG audio
compression. [06] 2012
3.
ECE Dept, GMIT
[email protected]
Page No - 41

Multimedia Commnication Unit 4 VTU

Uploaded by

Copyright:

Available Formats

Multimedia Commnication Unit 4 VTU

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Multimedia Commnication Unit 4 VTU

Uploaded by

Copyright:

Available Formats

Audio and video compression

1. Multimedia Communications: Applications, Networks, Protocols and Standards, Fred

1. Multimedia Fundamentals: Vol 1 - Media Coding and Content Processing, Ralf

1. Ramisuniverse for providing free e-material.

Quotes from Great People:

ECE Dept, GMIT

Audio and video compression

Audio & Video Compression

Ex.: Sampling rate and number of bits per sample

Number of bits per sample Audio = 16 bits.

Stereophonic signal has 2 channels, hence, two signals must be digitized.

ECE Dept, GMIT

Audio and video compression

Differential pulse code modulation (DPCM):

Differential pulse code modulation (DPCM) is a derivative of standard PCM.

ECE Dept, GMIT

Audio and video compression

ECE Dept, GMIT

Audio and video compression

In this scheme, the difference signal is computed by subtracting varying proportions of

ECE Dept, GMIT

Audio and video compression

DPCM ENCODING AND DECODING

Step 1: PCM= 3, R0= 0

Step 2: PCM= 5, R0= 3

Step 4: PCM= 1, R0= 4

ECE Dept, GMIT

Audio and video compression

Adder Output= R1= R0+ DPCM

Step 6: PCM= -5, R0= -2

Step 5: PCM= -2, R0= 1

Step 7: PCM= -3, R0= -5

Step 8: PCM= 0, R0= -3

ECE Dept, GMIT

Audio and video compression

The encoded sequence is decoded using the DPCM decoder as follows:

Step 1: DPCM= 3, R0= 0

Step 2: DPCM= 2, R0= 3

Step 3: DPCM= -1, R0= 5

Step 4: DPCM= -3, R0= 4

Step 5: DPCM= -3, R0= 1

ECE Dept, GMIT

Audio and video compression

Step 8: DPCM= 3, R0= -3

Step 10: DPCM= 2, R0= 3

Adaptive differential PCM:

Advantages: Saving in bandwidth and improving in audio quality.

Is an international standard for this is defined in ITU-T Recommendation G.721.

A second ADPCM standard, which is a derivative of G.721, is defined in ITU-T

The input speech bandwidth is extended to be from 50 Hz through to 7 kHz compared

ECE Dept, GMIT

Audio and video compression

This is particularly important in conferencing applications, in order to enable the

The general principle of the scheme is shown in Figure below

The operating bit rate can be 64, 56, or 48 kbps.

ECE Dept, GMIT

Audio and video compression

A third standard based on ADPCM is also available. This is defined in ITU-T