Scribd Logo
Upload

Welcome to Scribd!

  • 120 views

    Independent Component Analysis For Time Series Separation

    Uploaded by

    Pravin Bhole
    Independent Component Analysis (ICA) is a technique for separating mixtures of signals into their underlying independent source signals. ICA can be used to separate audio mixtures, medical data, financial data, and more. The goal is to estimate an unknown mixing matrix A and recover the independent source signals s using only the observed mixture signals x, where x=As. ICA relies on the fact that a sum of independent signals tends toward a Gaussian, while the individual independent signals themselves are typically non-Gaussian. Measures of non-Gaussianity are used to estimate the mixing matrix and recover the independent components. Data preprocessing such as centering, whitening, and dimensionality reduction are typically applied before performing ICA.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPT, PDF, TXT or read online from Scribd

    Independent Component Analysis For Time Series Separation

    Uploaded by

    Pravin Bhole
    120 views13 pages
    Independent Component Analysis (ICA) is a technique for separating mixtures of signals into their underlying independent source signals. ICA can be used to separate audio mixtures, medical data, financial data, and more. The goal is to estimate an unknown mixing matrix A and recover the independent source signals s using only the observed mixture signals x, where x=As. ICA relies on the fact that a sum of independent signals tends toward a Gaussian, while the individual independent signals themselves are typically non-Gaussian. Measures of non-Gaussianity are used to estimate the mixing matrix and recover the independent components. Data preprocessing such as centering, whitening, and dimensionality reduction are typically applied before performing ICA.

    Original Description:

    Presentation on Independent Component AnalysisFor Time Series Separation

    Original Title

    ICA.ppt

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    PPT, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Independent Component Analysis (ICA) is a technique for separating mixtures of signals into their underlying independent source signals. ICA can be used to separate audio mixtures, medical data, financial data, and more. The goal is to estimate an unknown mixing matrix A and recover the independent source signals s using only the observed mixture signals x, where x=As. ICA relies on the fact that a sum of independent signals tends toward a Gaussian, while the individual independent signals themselves are typically non-Gaussian. Measures of non-Gaussianity are used to estimate the mixing matrix and recover the independent components. Data preprocessing such as centering, whitening, and dimensionality reduction are typically applied before performing ICA.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPT, PDF, TXT or read online from Scribd
    Download as ppt, pdf, or txt
    120 views13 pages

    Independent Component Analysis For Time Series Separation

    Uploaded by

    Pravin Bhole
    Independent Component Analysis (ICA) is a technique for separating mixtures of signals into their underlying independent source signals. ICA can be used to separate audio mixtures, medical data, financial data, and more. The goal is to estimate an unknown mixing matrix A and recover the independent source signals s using only the observed mixture signals x, where x=As. ICA relies on the fact that a sum of independent signals tends toward a Gaussian, while the individual independent signals themselves are typically non-Gaussian. Measures of non-Gaussianity are used to estimate the mixing matrix and recover the independent components. Data preprocessing such as centering, whitening, and dimensionality reduction are typically applied before performing ICA.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPT, PDF, TXT or read online from Scribd
    Download as ppt, pdf, or txt
    You are on page 1of 13

    Independent Component

    Analysis
    For Time Series Separation
    ICA
    Blind Signal Separation (BSS) or Independent Component Analysis (ICA) is the
    identification & separation of mixtures of sources with little prior
    information.
    Applications include:

    Audio Processing
    Medical data
    Finance
    Array processing (beamforming)
    Coding
    and most applications where Factor Analysis and PCA is currently used.
    While PCA seeks directions that represents data best in a |x
    0
    - x|
    2
    sense,
    ICA seeks such directions that are most independent from each other.
    We will concentrate on Time Series separation of Multiple Targets
    The simple Cocktail Party Problem
    Sources
    Observations
    s
    1

    s
    2

    x
    1

    x
    2

    Mixing matrix A
    x = As
    n sources, m=n observations
    Motivation
    Two Independent Sources
    Mixture at two Mics
    a
    IJ
    ... Depend on the distances of the microphones from the speakers
    2 22 1 21 2
    2 12 1 11 1
    ) (
    ) (
    s a s a t x
    s a s a t x
    + =
    + =
    Motivation
    Get the Independent Signals out of the Mixture
    ICA Model (Noise Free)
    Use statistical latent variables system
    Random variable s
    k
    instead of time signal
    x
    j
    = a
    j1
    s
    1
    + a
    j2
    s
    2
    + .. + a
    jn
    s
    n
    , for all j
    x = As
    ICs s are latent variables & are unknown AND Mixing matrix A is
    also unknown
    Task: estimate A and s using only the observeable random vector x
    Lets assume that no. of ICs = no of observable mixtures
    and A is square and invertible
    So after estimating A, we can compute W=A
    -1
    and hence
    s = Wx = A
    -1
    x

    Illustration
    2 ICs with distribution:




    Zero mean and variance equal to 1

    Mixing matrix A is



    The edges of the parallelogram are in the
    direction of the cols of A
    So if we can Est joint pdf of x
    1
    & x
    2
    and then
    locating the edges, we can Est A.
    |
    |
    .
    |

    \
    |
    =
    1 2
    3 2
    A

    s
    =
    otherwise
    s if
    s p
    i
    i
    0
    3 | |
    3 2
    1
    ) (
    Restrictions
    s
    i
    are statistically independent
    p(s
    1
    ,s
    2
    ) = p(s
    1
    )p(s
    2
    )
    Nongaussian distributions
    The joint density of unit
    variance s
    1
    & s
    2
    is symmetric.
    So it doesnt contain any
    information about the
    directions of the cols of the
    mixing matrix A. So A cannt
    be estimated.
    If only one IC is gaussian, the
    estimation is still possible.

    |
    |
    .
    |

    \
    |
    +
    =
    2
    exp
    2
    1
    ) , (
    2
    2
    2
    1
    2 1
    x x
    x x p
    t
    Ambiguities
    Cant determine the variances (energies)
    of the ICs
    Both s & A are unknowns, any scalar multiple in one of the
    sources can always be cancelled by dividing the corresponding
    col of A by it.
    Fix magnitudes of ICs assuming unit variance: E{s
    i
    2
    } = 1
    Only ambiguity of sign remains
    Cant determine the order of the ICs
    Terms can be freely changed, because both s and A are
    unknown. So we can call any IC as the first one.
    ICA Principal (Non-Gaussian is Independent)
    Key to estimating A is non-gaussianity
    The distribution of a sum of independent random variables tends toward a Gaussian
    distribution. (By CLT)





    f(s
    1
    ) f(s
    2
    ) f(x
    1
    ) = f(s
    1
    +s
    2
    )
    Where w is one of the rows of matrix W.


    y is a linear combination of s
    i
    , with weights given by z
    i
    .
    Since sum of two indep r.v. is more gaussian than individual r.v., so z
    T
    s is more
    gaussian than either of s
    i
    . AND becomes least gaussian when its equal to one of s
    i
    .
    So we could take w as a vector which maximizes the non-gaussianity of w
    T
    x.
    Such a w would correspond to a z with only one non zero comp. So we get back the s
    i.
    s z As w x w y
    T T T
    = = =
    Measures of Non-Gaussianity
    We need to have a quantitative measure of non-gaussianity for ICA
    Estimation.
    Kurtotis : gauss=0 (sensitive to outliers)

    Entropy : gauss=largest

    Neg-entropy : gauss = 0 (difficult to estimate)

    Approximations



    where v is a standard gaussian random variable and :


    2 2 4
    }) { ( 3 } { ) ( y E y E y kurt =
    }
    = dy y f y f y H ) ( log ) ( ) (
    ) ( ) ( ) ( y H y H y J
    gauss
    =
    { }
    2
    2
    2
    ) (
    48
    1
    12
    1
    ) ( y kurt y E y J + =
    { } { } | |
    2
    ) ( ) ( ) ( v G E y G E y J ~
    ) 2 / . exp( ) (
    ) . cosh( log
    1
    ) (
    2
    u a y G
    y a
    a
    y G
    =
    =
    Data Centering & Whitening
    Centering
    x = x E{x}
    But this doesnt mean that ICA cannt estimate the mean, but it just simplifies the
    Alg.
    ICs are also zero mean because of:
    E{s} = WE{x}
    After ICA, add W.E{x} to zero mean ICs
    Whitening
    We transform the xs linearly so that the x
    ~
    are white. Its done by EVD.
    x
    ~
    = (ED
    -1/2
    E
    T
    )x = ED
    -1/2
    E
    T
    Ax = A
    ~
    s
    where E{xx
    ~
    } = EDE
    T
    So we have to Estimate Orthonormal Matrix A
    ~

    An orthonormal matrix has n(n-1)/2 degrees of freedom. So for large dim A we
    have to est only half as much parameters. This greatly simplifies ICA.
    Reducing dim of data (choosing dominant Eig) while doing whitening also
    help.
    Noisy ICA Model
    x = As + n
    A ... mxn mixing matrix
    s ... n-dimensional vector of ICs
    n ... m-dimensional random noise vector
    Same assumptions as for noise-free model, if we use measures of
    nongaussianity which are immune to gaussian noise.
    So gaussian moments are used as contrast functions. i.e.


    however, in pre-whitening the effect of noise must be taken in to account:
    x
    ~
    = (E{xx
    T
    } - )
    -1/2
    x
    x
    ~
    = Bs + n
    ~
    .

    { } { } | |
    ( ) ) 2 / exp( 2 / 1 ) (
    ) ( ) ( ) (
    2 2
    2
    c x c y G
    v G E y G E y J
    =
    ~
    t

    You might also like