SIF - ADVANCED VIDEO CODEC

BASED ON HUMAN VISION MODEL

EXECUTIVE SUMMARY
CODEC DESCRIPTION
SIF is a new generation video codec,codec which is using psycho-
psycho-
visual model close to the processes of the human eye as the
base for effective image encoding, allowing to get better
subjective quality with lower file size.
size
GOAL
SIF is designed to be royalty-
royalty-free codec, which does not infringe
MPEG LA patents, and to be more efficient than H.264 codec.
Projected SIF 2 is targeting the efficiency of H.265 and HEVC
codecs.

EFFICIENCY
Current version of SIF is encoding videos with subjectively the
same quality as H.264 with lower bitrate and minimal output
lag (1 shot buffer).To reach efficiency in objective rankings
(e.g. PSNR or SSIM)
SSIM codec needs routine optimization by a lot
of man hours of work. Now the encoded video is lower by 2 –
2,5 times in size than encoded by H.264 due to radically new
technology.

AREAS OF USE
At the first stage:
stage
 Internet TV (video streaming, video conferences, etc), om-line
games;
 Industrial and commercial TV and entertainment
 Remote control, real-time monitoring. Surveillance
At the second stage:
stage: SIF-
SIF-Codec will become new standard of video
coding and will be used in all devices and in all streaming
platforms.
VIDEO AND INTERNET

• Internet traffic is
growing at a rate 24%
per year

• The share of video in

internet traffic is more
than 80%

• 78% of people watch Source: Cisco Report «Zettabyte Era: Trends and Analytics»
at least 1 video per day
VIDEO CODECS AND VIDEO TRAFFIC

Film a video Edit Watch

Sensors capture video stream and Specified programs edit the initial TV or monitor decodes the
record it to memory card file and encode it into a small container and plays the video
container stream to the viewer
We get a large file with maximum
video quality. Then the container can be sent
online or burnt to DVD

CODEC = COMPRESSION
CO + DECOMPRESSION
DEC
Each video from the internet is processed by a codec
COMMERCIAL VIDEO CODECS MARKET

Alliance for Open Media MPEG LA Group

Announced the creation of video codec free for It has a patent pool (about 500 patents), covering
commercial usage in 2016 all video encoding and decoding algorithms known
to the market.
However, it has not released a working version
yet. The actual monopolist in the commercial market
THE COST OF A LICENSE FOR A COMMERCIAL VIDEO CODEC
THEORETICAL LIMIT OF INFO COMPRESSION IS ALMOST REACHED

MPEG LA codec has almost reached the theoretical limit of

compression. Further increase in the complexity of codecs
gives less and less benefit in compression efficiency.
Efficiency of
compression
WHAT’S NEXT? How else can we improve
2.9
codecs?

WE SHALL OPTIMIZE PSYCHO-

PSYCHO-
2.4
VISUAL MODEL
And SIF codec knows all about it:
• SIF codec algorithm has been initially oriented to the 1.9
psycho-visual model, thus it will correlate with it ideally
• unlike the process of DCT used by MPEG LA, SIF
codec uses prediction, segmentation and interpolation
which are optimally coordinated with the model of the 1.4
human eye

0.9
1 2 3 4 5 6
Complexity of codecs
CODEC ALGORITHM

The goal of royalty-free codec does not allow to use in-

in-loop MPEG LA CODEC CONCEPTUAL SCHEME
filter patented by MPEG LA,
LA so the author came up with a
SIF-transformation algorithm, which bases on Laplace
+
-+
pyramid method.
method Video partitioning & Scanning &
DCT transform Quantization
SIF codec offers hybrid algorithm of:
• Adaptive predictors Inverse scanning &
Dequantization
• Soft segmentation Intra/inter
estimation Intra prediction
• Adaptive quantization +
• Adaptive interpolation
Motion
estimation
Motion
compensation
Loop
deblocking
+ Inverse DCT
transform

filter
Soft segmentation does not combine different segments
into complex objects, but uses adaptive predictors to
eliminate correlations between segments.
CODEC ALGORITHM
The key element of SIF transformation is a Resampling
pattern = segment optimized for specific piece of image +
set of specific methods which are necessary for optimal

-+
obtaining of pattern elements (sub-samples), and optimal Video
+ Tree processing &
Quantization
interpolation of pixel values within a segment. Forward SIF
transform Inverse tree &
Recovery is carried out using all data surrounding the Dequantization
segment, which allows to use effective methods of two- two- Intra/inter Adaptive
dimensional adaptive interpolation,
interpolation and get rid of block estimation Psycho-visual prediction
model +
artifacts and aliasing artifacts.

The method for image optimal interpolation, divided into a

Motion
estimation
Motion
compensation
Inverse SIF
transform
+ Restore of signal
structure

set of segments, is described in the patent 8,374,446.

8
PSYCHO-VISUAL MODEL
DESCRIPTION
The inspiration for the algorithm was taken from the human eye
processes,
processes which allowed to get diametrically different method of
video encoding. The amount of information transmitted visually to the
brain is higher than its capacity, so an eye does transformation.
transformation The
idea is to leave unattended the information which is not vitally
important in that moment and to concentrate on what is needed.
needed
Example: a hunter does not need the detailed image of every tree in a
forest if it is looking for a moving object, so he will see the prey in
higher detail.
The codec uses the same algorithm, called psycho- psycho-visual model,
which allows to initially transform the image close to the eye and only
then use masking functions.

Existing psycho-visual metrics require a bigger amount of calculations

and there is no possibility to implement them effectively in codecs with
low lag. The psycho-visual model which we have is integrated directly
into the core of the codec and works in real-time without necessity to
analyze encoded video ahead.

There are three basic functions for the psycho-visual model:

 Brightness
 Space
 Time (for motion)
These functions allow to make shadows more detailed, static sciences
with more accurate representation and dynamic with better movement
render and much more.
As SIF-transformation is designed to work as the eye does, the
psycho-visual model is calculated by the results of SIF-transform and
is almost free in terms of computational powers.
powers DCT can also
implement psycho-visual model, but it requires far more resources to
calculate it.
The current version of codec is not good at objective PSNR ranking,
although the subjective tests show comparable or even better quality.
quality
The reason for this is that MPEG LA codecs are optimized for objective
ratings, while SIF is created for better visuals.
visuals The codec also needs
routine optimization of the code and additional functions, which will
make it comparable to MPEG LA codes in PSNR rankings.
In a more advanced rating SSIM,
SSIM which considers subjective quality,
codec shows better results.
results Any new metric, that would consider
subjective quality more thoroughly, would allow to achieve even better
results.
results
Based on the results of visual testing done by the Computer Graphics
and Multimedia Laboratory of Moscow University (MSU), the results of
visual codec comparison are approximately 1.5 times better than the
results of SSIM metrics on the same compositions.
The outdated version of SIF codec was tested at that time. The current
version should show even better results.
results.
The comparison was made with best-in-class codecs, which included
psycho-visual optimizations.
While being compared, all other codecs worked in the encoding mode
with high lag. SIF-codec working in the mode with low lag has the best
effectiveness of psycho-visual model in its class.
It means, the psycho-
psycho-visual model
mode implemented in SIF codec far
exceeds the best products available on the market.
ENCODING BY MEANS OF GENERALIZED EMBEDDED TREE

Each node of the generalized embedded tree may

contain not one, but several (two and more) splitting
options,
options which are encoded by several bits
Splitting options may contain links to daughter nodes
and/or predefined set of leaves with predefined pattern
types and/or combination of links to daughter nodes and
one and/or several leaves and/or extra data used during
encoding and decoding

Daughter nodes may be of different types, containing

different sets of splitting options. A parent node selects a
daughter node while selecting a splitting option Root node

Tree leaves may also be of different predefined types

and contain data about type and form of the pattern,
type of pattern predictors, quantization and
interpolation parameters, etc.
While encoding by pattern groups, a leaf may contain
quantization parameters, predictor types, interpolation
parameters and other data, defined for the whole
pattern group.
Adaptive predictors
Encoding starts with a root node. If the root node does
not contain splitting options acceptable for this very
part of the picture, then transition to one or several
daughter nodes takes place and the splitting options
procedure repeats.
GENERALIZED EMBEDDED TREE

The transmission of data of the generalized tree to the

output stream and its restoration in the decoder is carried
out by means of rules unified for the encoder and decoder
and pre-
pre-defined for bypassing nodes and leaves of the tree

While selecting a splitting option, parameters of the psycho-

visual model, determined for this node, can be used, as well
as objective characteristics - the level of distortion
introduced by a specific splitting option

The exact boundaries of the objects in the image are

interpolated from the values of the pattern samples using
SIF inverse. The smaller the size of the pattern will be
broken sections of the image on the boundaries of objects,
the more accurately the shape of the boundaries will be
transmitted, and the bigger bits the compressed image will
occupy
SIF CODEC ADVANTAGES
SIF transformation does not create artifacts in the form of
blocks on the decoded image. This allows to exclude the
loop filter from the codec.
codec

Predictors are not calculated from the restored image, as

MPEG codecs do. SIF codec works with the intermediate
result of the SIF inverse,
inverse i.e. after the procedure of
geometric reconstruction of the signal.
The calculation of predictors is combined with entropy
decoding and is structurally separated from other decoding
operations.

The remaining methods used in the codec can be efficiently

distributed to thousands of threads and run on modern
GPUs or DSPs. The entropy decoding and predictor
module has a low computational complexity and can be
performed on general-purpose processors.
Decoded data in the form of two-dimensional matrices are
transmitted for further parallel processing into the following
modules, which perform the basic operations of decoding in
a parallel mode.

All this leads to:

• saving of amplifying power
• compatibility with multi-stream decoding
• reduction of Aliasing effect
EFFICIENCY

SIZE

Codec algorithm allows to gain higher efficiency in file size

by using psycho-visual model. The developer expects to
get 40-
40-50%
50% advantage over H.265 when the code is
optimized and finalized.

VALUE

Codec is free from MPEG LA patents,

patents so this allows to gain
sufficient economy on licensing payments with no
compromise on quality.
quality

OBJECTIVE METRICS

The theory states that there is a particular threshold in

objective compressing capabilities,
capabilities that standard methods
can achieve. This means that MPEG LA codecs will soon
reach the boundary in objective metrics,
metrics which will be hard
to improve.
Optimized SIF codec will reach practically the same score
in PSNR, while it will deliver better subjective quality for the
viewer. This means that the psycho-
psycho-visual model allows to
overcome the threshold and achieve higher quality.
quality
CODE

PARALLEL COMPUTATION
Codec has multi-
multi-thread support and is easily parallelized on
CPU’s and GPU’s, allowing to get faster encoding and
decoding even on mobile devices. Contrary to existing
codecs, SIF codec allows intraframe parallelization.
parallelization
Parallelization also allows to increase efficiency when
processing powers of new processors increase.

Entropy
Loop
compression

Parallelized CPU
on GPU (hard to parallelize)

HARDWARE REQUIREMENTS
Codec does not require any specific hardware and can be
implemented on any computers or mobile devices.

LOW LAG
Psycho-visual model allows to have 1 frame buffer rate,
rate
which means no lag for decoding and can be used for
effective online broadcasts.
IDEAL SCENARIO
Broadcast television.
television MPEG LA is partly or fully replaced by SIF codec

REALISTIC SCENARIO
Аналитика

 Computer game platforms

Video streaming  Cyber sport tournaments
service  Live broadcasts (twitch.tv, you tube,
livestream, etc.)
Информация
 Telemedicine о гостях
Video conference  Personal and group video conferences Minimal entry cost.
Internet TV  Remote cassation and supervisory legal Main focus
procedures
 Situation centers
 Distance learning Целевые
рассылки
Online cinemas

Relatively low entry cost.

Industrial TV and surveillance
Achievable market

Requires appropriate
Remote control with the minimal lag requirement, real-
real-time monitoring hardware and codec
optimization. Hardly
achievable market
IP PROTECTION

PATENTS

Patent #8,374,446 «Method of encoding digital signals

(variants), method of decoding digital signals (variants),
device for implementation thereof (variants), and system for
image transmission via limited throughput communication
channels (variants)» released by USPTO
Patent #: US 15/644,279: Invention name: Method of
encoding of data stream, method of decoding of data
stream, and devices for implementation of said methods
SIF 2 protection via several additional patents applications,
including unique entropy engine, protected by separate
patent application

1. MOTION COMPENSATION IN SYSTEM AND METHOD

FOR PROCESSING DIGITAL SIGNALS BASED ON
COMPRESSION OF HIERARCHICAL PYRAMID
2. IMPROVED ENCODING AND DECODING OF DIGITAL
SIGNALS BASED ON COMPRESSION OF
HIERARCHICAL PYRAMID
LINKS TO WATCH VIDEO

Кодек SIF1 1.70 Star Wars Sucker Punch

Fan Beasts Rogue One Inuyashiki

CONTACT DETAILS

ADDRESS

SIF Codec LLC

4241 Jutland Dr., Suite 2015
San Diego, CA, USA, 92117

PHONE
+1 (858) 4059128

FAX

+1 (858) 4059128
WEB
www.sifcodec.com

E-MAILS

Vadim Asadov asadov@neurok.

[email protected]