Inverse Transformation Method and Random Variables

Date: 02/03/2009 at 12:00:41
From: Peter
Subject: Generate random number using Inverse Transformation Method

Hi,

I am looking for guidance on the proof of the Inverse Transformation
Method to simulate a random variable having a continuous distribution.

This is taken from the book "A First Course in Probability", sixth
edition by Sheldon Ross, page 558, Proposition 2.1, as shown below:

Let U be a uniform (0, 1) random variable. For any continuous
distribution function F, if we define the random variable Y by:

Y = F_inv(U) then the random variable Y has distribution function F.
[F_inv(x) is defined to equal that value for which F(y) = x).]

Proof:

#1# F_Y(a) = P{Y <= a}
#2# = P{F_inv(U) <= a}  (sub in def for Y)

#3# Now since F(x) is a monotone function, it follows that F_inv(U) 
<= a if and only if U < F(a). Hence, from statement #2# we see that:

#4# F_Y(a) = P{U <= F(a)}
#5# = F(a)

#6# it follows from Proposition 2.1 that we can simulate a random
variable X having a continuous distribution function F by generating
random number U and then setting X = F_inv(U).


Am I correct in thinking that in statement #3# that F is applied to
each side of F_inv(U) <= a and monotone means that F is never 
negative?

In statements #4# and #5# it is shown that F_Y(a) = F(a) but what is
F(a) and why does it not have a subscript?

Any help would be appreciated,

Peter

Date: 03/03/2009 at 21:50:23
From: Doctor George
Subject: Re: Generate random number using Inverse Transformation Method

Hi Peter,

Thanks for writing to Doctor Math.

Let me write this a little differently, and hopefully it will be clearer.

Let U be uniform (0,1) and let

       -1
  Z = F  (U)
       Y

We want to show that Z has the same density as Y.

  F (z) = P[Z <= z]
   Z
             -1
        = P[F  (U) <= z]
             Y

Since F (y) is monotonic it has an inverse, which we have denoted 
       Y

above.  (Negative values for the inverse are not a problem.)  So,
applying F (y) gives us the following.
          Y

  F (z) = P[U <= F (z)]
    Z             Y

        = F (F (z))
           U  Y

Now differentiate with respect to z using the chain rule to obtain the
density function.

  f (z) = f (F (z)) * f (z)
   Z       U  Y        Y

        = f (u) * f (z)
           U       Y

        = f (z)
           Y

Does that make sense? Write again if you need more help.

- Doctor George, The Math Forum
  http://mathforum.org/dr.math/ 

Date: 03/04/2009 at 06:05:43
From: Peter
Subject: Generate random number using Inverse Transformation Method

Hi Dr. George,

Thank you for your help, it was not easy typing all that out
especially with getting all the subscripts and super scripts to line up.

Your answer has given me great insight but unfortunately it has
generated some questions on my part that I would be obliged if you
would address them.

Firstly you let:

       -1
  Z = F  (U)
       Y

Does this imply that:
      
  F  (z) = U
   Y

The same way as Let F(x) = 2x = u

      -1
Then F  (U) = u/2 = x

so plugging and chugging starting with x = 5

  F(5) = 10 = u

      -1
Then F  (10) = 10/2 = 5 = x

This is presumably how you convert from:

  f (z) = f (F (z)) * f (z)
   Z       U  Y        Y

to:
        = f (u) * f (z)
           U       Y

The next part is to do with converting:

        = f (u) * f (z)
           U       Y
To :
        = f (z)
           Y

Does that follow from the definition of the density for the uniform
f(x) = 1/(b - a) where the interval is (a, b) since U is defined: U be
uniform (0,1)

then f(x) = 1/(1 - 0) = 1

Next, you wrote: 

  Since F (y) is monotonic it has an inverse, which we have denoted 
       Y

  above. (Negative values for the inverse are not a problem.).....
  What do you mean by  (Negative values for the inverse are not a
  problem.)?

Why are they not a problem?  What type of problem could they be?

Many thanks again for your help,

Peter

Date: 03/04/2009 at 08:47:59
From: Doctor George
Subject: Re: Generate random number using Inverse Transformation Method

Hi Peter,

You are following everything very well in regard to the 
transformations and densities.

In regard to the negative values, I was reflecting on your question
about whether "monotone means that F is never negative."  It is true
that F is never negative, but the point of F being monotonic is that
it is always increasing, not that it is never negative.

The inverse of F is also monotonic, but can have negative values, as
with the standard normal distribution.  Sorry for the confusion on
this point.

A discrete illustration might be helpful.  Consider constructing a
histogram with very narrow bins over the probability density function.
Now stack these histogram bars on top of each other.  Next select a
uniformly random location along the concatenated bar.  Then select the
center coordinate of the histogram bin that corresponds to the segment
that contains the random location.  The limit of this process as the
histogram bins get narrower is what the continuous case produces.

Let me know if anything is still unclear.

- Doctor George, The Math Forum
  http://mathforum.org/dr.math/