Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Information theory and source coding-application

  1. Nov 12, 2011 #1
    So I am enrolled into this course.

    And we are learning probability and statistics and all that good stuff.

    But one thing bothers me VERY much.

    I lie, a lot of things bother me about this course but here is the first one:

    Transformation of random variables.

    So far we have been only doing that pure, raw unattractive math. And I cannot swallow the point of that.

    Says here in the book, you have a voltage in input and you get something else out...

    But still I cannot wrap my head around it.

    How does this apply to communications!?!??! This transformation of random variable?!

    LTI systems I understand. But we get precise inputs and precise outputs. What role does a random variable take in my learning?!
  2. jcsd
  3. Nov 12, 2011 #2


    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    A study of random variables will teach you how to model noise, which one of the most important (and deep) concepts in communication theory.

    - Warren
  4. Nov 12, 2011 #3
    Please go on. I am dying over here.
  5. Nov 12, 2011 #4


    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Well, noise is a random process, and random variables provide a formal, analytical means of analyzing noise and its effects on a system. This alone is incredibly important.

    Random variables can also be used to model higher-level concepts, like the arrival time of packets on a communications network (Poisson random variables). They can be used to model the behavior of simple receivers, inter-symbol interference, and all kinds of other phenomena.

    Random variables are one of the most important mathematical tools in the study of signal processing and communications. They're pretty fun, too, once you understand them. Keep at it; it's worth it.

    - Warren
  6. Nov 12, 2011 #5

    "Once you understand them". Now theres an interesting idea :rofl:

    I usually have trouble understanding math. Why? Because mainly, in materials that I have, we only get the definition. What is random variable(strict mathematical definition) and no examples.

    Took me a week to truly understand (using loosely "truly" here) random variables. First I thought random variables map a number to a probability. Then after that week, I saw that I was waaaaaaay off.

    Transformation of random variable, is something I find so so so hard to swallow. Why? Because we only get strict mathematical definition of it and barely any examples. We got one on recitation, and I couldn't conclude everything from that.

    Do you suggest that I try using Matlab for this? Will I only get lost?

    My main tool for understanding random processes was Matlab. And, I am self taught. We learnt very few in recitation. They focus us on math so much...
  7. Nov 12, 2011 #6


    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Bassalisk, I feel your pain. It sounds like you do not have a particularly good teacher. Take a look at this Wikipedia page on RVs; it's a good place to start.


    Also, here's an interesting handout from a political science class at Stanford. It explains random variables in the context of social sciences. I think the examples are particularly concrete and easy to grasp.


    Feel free to ask any specific questions you might have on RVs, and I'll do my best to answer.

    - Warren
  8. Nov 12, 2011 #7
    My teacher is good if you like math :D. He explained it us as a mathematician and not engineer. Why? Because he is mathematician and he has PhD in math.

    Thank you for your resource, I will definitely check that stanford out. wiki gave me the base intuition in the start.

    I won't hesitate to ask any questions. Thank you very much kind sir.
    Currently we are trying to make me understand this thread:


    EDIT: wrong thread. this one is mine
    Last edited: Nov 12, 2011
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook