Scribd Logo
Upload

Welcome to Scribd!

  • 23 views

    Eco Survey 2017-18 Summary

    Uploaded by

    Sarojkumar Bhosle
    This document contains 7 problems related to information theory and coding. Problem 1 asks to identify countable and uncountable sample spaces in various cases. Problem 2 defines probability as a set function and lists probability axioms. Problem 3 asks to compute the joint distribution of random variables U and V defined based on the outcomes of two fair dice throws. Problems 4-7 present additional questions related to concepts like discrete memoryless channels, entropy, conditional probability, and relationships between random variables.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Eco Survey 2017-18 Summary

    Uploaded by

    Sarojkumar Bhosle
    23 views1 page
    This document contains 7 problems related to information theory and coding. Problem 1 asks to identify countable and uncountable sample spaces in various cases. Problem 2 defines probability as a set function and lists probability axioms. Problem 3 asks to compute the joint distribution of random variables U and V defined based on the outcomes of two fair dice throws. Problems 4-7 present additional questions related to concepts like discrete memoryless channels, entropy, conditional probability, and relationships between random variables.

    Original Description:

    Problem set

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document contains 7 problems related to information theory and coding. Problem 1 asks to identify countable and uncountable sample spaces in various cases. Problem 2 defines probability as a set function and lists probability axioms. Problem 3 asks to compute the joint distribution of random variables U and V defined based on the outcomes of two fair dice throws. Problems 4-7 present additional questions related to concepts like discrete memoryless channels, entropy, conditional probability, and relationships between random variables.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    23 views1 page

    Eco Survey 2017-18 Summary

    Uploaded by

    Sarojkumar Bhosle
    This document contains 7 problems related to information theory and coding. Problem 1 asks to identify countable and uncountable sample spaces in various cases. Problem 2 defines probability as a set function and lists probability axioms. Problem 3 asks to compute the joint distribution of random variables U and V defined based on the outcomes of two fair dice throws. Problems 4-7 present additional questions related to concepts like discrete memoryless channels, entropy, conditional probability, and relationships between random variables.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 1

    SGGSIE&T, Nanded

    Department of E&TC
    Information Theory and Coding- Problem Set 1
    ([email protected])

    1. Identify in each of the following cases whether the 4. Consider a Discrete Memoryless Channel (DMC)
    sample space is countable or uncountable. Justify your with input X and output Y . The input takes
    answer. values from the alphabate of equiprobable symbols
    {x1 , x2 } and the output takes values in the alphabet
    a. Number of electrons moving to conduction band
    {y1 , y2 , y3 , y4 }. The transition probability matrix for
    from valence band in a given time interval due
    the channel is shown below
    to thermal or optical excitation
     
    b. The amount of charge that flows into earth due 0.1 0.2 0.3 0.4
    P =
    to lightning 0.5 0.3 0.1 0.1

    c. Tossing of infinite number of coins where, P (i, j) = Pr(Y = yj |X = xi ). Given that the
    symbol received is y1 what is the probability that the
    d. The waiting time in queue before getting service
    symbol transmitted was x1 ? Compute the mutual in-
    e. The amount of water in a dam at any given time formation I(y1 , x1 ). Determine entropies of X and Y .

    2. Probability is a set function P(·) which assigns, to


    5. A coin with P(H) = p is tossed until the first
    any event (set) A, a number P(A) called probability
    head occurs. Let X denotes the number of coin flips
    of A and which satisfies the following axioms
    required. Find the entropy H(X) in bits. The follow-
    1. P(A) ≥ 0 ing expressions may be useful:

    2. P(Ω) = 1 ∞
    X 1
    ∞
    X r
    n
    r = , nrn =
    3. If A ∩ B = Φ, then P(A ∪ B) = P(A) + P(B) 1−r (1 − r)2
    n=0 n=0

    Prove that the conditional probability defined as


    6. Let X be a random variable taking discrete values
    P(A ∩ M )
    P(A|M ) = and Y is defined as Y = 2X . Derive the relationship
    P(M )
    between H(X) and H(Y ).
    is indeed the probability.

    7. For random variables X and Y define ρ(X, Y ) =


    3. Two fair dice are thrown and their outcomes are
    H(X|Y ) + H(Y |X). show that ρ(X, Y ) = 0, if there is
    recorded as X and Y respectively. The random vari-
    a bijection from X to Y . (Bijection is a mapping that
    ables U and V are defined as
    is both one-to-one (an injection) and onto (a surjec-
    U = max(X, Y ) tion), i.e. a function which relates each member of a
    V = min(X, Y ) set S (the domain) to a separate and distinct member
    of another set T (the range), where each member in T
    Compute the joint distribution of U and V . also has a corresponding member in S.)

    You might also like