Limits of Limits and Patterns in Higher Derivatives

Date: 11/11/2005 at 16:04:47
From: Phillip
Subject: Limit of a Limit - Higher derivatives

Hi,

I had a few questions about higher derivatives and the limits of 
limits.

Starting with the general funtion f(x) which is cont. for all reals 
the first derivative is f'(x) = Lim(h->0)((f(x+h) - f(x))/h).

the second derivative would be 

f"(x) = Lim(i->0)((f'(x+i) - f'(x))/i) 

using (i) in place of (h) and replacing f' with the first derivative 
you would get 

f"(x) = Lim(i->0)((Lim(h->0)((f(x+h+i) - f(x+i))/h) -
        Lim(h->0)((f(x+h) - f(x))/h))/i)

and then doing some simplifying I got

f"(x) = Lim(i->0)(Lim(h->0)((f(x+h+i) - f(x+i) - f(x+h) - f(x))/h*i))

Now here is one of my questions.  Can you combine the two limits into 
one statement by making h=i and get? 

f"(x) = Lim(h->0)((f(x+2h) - 2f(x+h) + f(x))/h^2)

I tried a few simple functions and arrived at the correct derivative. 
Using the above method I arrived at the following statement for the 
3rd derivative.

f'''(x) = Lim(h->0)((f(x+3h) - 3f(x+2h) + 3f(x+h) - f(x))/h^3)

My other question has to deal with the third and higher derivatives. 
They seem to follow a pattern that is similar to the pattern formed 
when you expand a exponent EX (x-1)^3 = x^3-3x^2+3x-1 notice that 
1 -3  3 -1 are the same in both the derivative and the polynomal.  I
tried this with the 4th derivative and got 1 -4 6 -4 1.  How come the 
derivative function follows the same pattern when there is no mult. 
involved (that I know of)?

Thank you for you time.

-Phillip

Date: 11/11/2005 at 18:15:58
From: Doctor Vogler
Subject: Re: Limit of a Limit - Higher derivatives

Hi Phillip,

Thanks for writing to Dr. Math.  You have good questions.  I'll 
respond as they come up.

>Starting with the general function f(x) which is cont. for all reals 
>the first derivative is f'(x) = Lim(h->0)((f(x+h) - f(x))/h).
>
>the second derivative would be 
>
>f"(x) = Lim(i->0)((f'(x+i) - f'(x))/i) 
>
>using (i) in place of (h) and replacing f' with the first derivative 
>you would get 
>
>f"(x) = Lim(i->0)((Lim(h->0)((f(x+h+i) - f(x+i))/h) -
>        Lim(h->0)((f(x+h) - f(x))/h))/i)

This is precisely correct.

>and then doing some simplifying I got
>
>f"(x) = Lim(i->0)(Lim(h->0)((f(x+h+i) - f(x+i) - f(x+h) - f(x))/h*i))

Here we need to stop a moment and look more closely at what you did. 
You had two limits as h->0 inside the limit as i->0, and you combined
them.  That is, you assumed

  lim(h->0) g1(h) - lim(h->0) g2(h) = lim(h->0) g1(h) - g2(h).

In fact, this is usually true, which is why all of the derivatives you
checked worked.  There is a theorem which says that if the first two
limits (on the left) exist, then the limit on the right also exists
and the equation is satisfied.  But the caution that you need to be
concerned with is:  It can happen that the two limits on the left do
not exist, but the limit on the right does.

In other words, you can do this simplification if you already know
that your function is differentiable, but if you start with a function
that is _not_ differentiable, you might still get a valid limit after
you do this simplification, which would lead you to believe that the
second derivative exists when really it doesn't.

>Now here is one of my questions.  Can you combine the two limits into 
>one statement by making h=i and get? 
>
>f"(x) = Lim(h->0)((f(x+2h) - 2f(x+h) + f(x))/h^2)

That's a very good question.  The answer is similar to my previous
comment, but a little more complicated.  The best way to think about
this is as a limit of a two-variable function.  (The variables are the
limit variables i and h, not x, because x is constant with respect to
the limit.)  You want the limit of

  g(h,i)

as h and i approach zero.  Consider g as a function that takes values
at different points of the h-i plane, except you don't know about its
values on the two axes.  When you take the limit as h->0, you first
approximate the values on the i axis by the values nearby on
horizontal lines, and then when you take the limit as i->0, you use
those new values on the i axis to get the limit at the origin.  If you
do things in the other order, it is possible that you get a different
value.  For example, try the limit of

  i/(i+h)

in both ways, and also in your simplification with i=h.  Another way
to take a limit of such a function is to say that _all_ (i,h) pairs
sufficiently close to the origin approach the same value.  This is a
stronger, more stable version of a limit of a two-variable function,
but it requires the function to be defined everywhere outside of the
origin, which yours is not (since you are dividing by i*h).  You want
to change the limit by taking a single path along the line i=h.  In
any case, there are conditions on a function that guarantee that all
the different types of limits give the same result, but this requires
that the function is defined everywhere outside of the origin. 
Nevertheless, you have a special type of limit, so you might be able
to prove something as strong as:  If the second derivative exists,
then it will be given by your formula.  On the other hand, there might
be certain kinds of pathological functions that have a second
derivative, but this formula gives the wrong value.  (I expect that
the third derivative would not exist in this case.)

>I tried a few simple functions and arrived at the correct derivative. 
>Using the above method I arrived at the following statement for the 
>3rd derivative.
>
>f'''(x) = Lim(h->0)((f(x+3h) - 3f(x+2h) + 3f(x+h) - f(x))/h^3)
>
>My other question has to deal with the third and higher derivatives. 
>They seem to follow a pattern that is simlar to the pattern formed 
>when you expand a exponent EX (x-1)^3 = x^3-3x^2+3x-1 notice that
>1 -3 3 -1 are the same in both the derivative and the polynomal. I
>tried this with the 4th derivative and got 1 -4 6 -4 1. How come the 
>derivative function follows the same pattern when there is no mult. 
>involved (that I know of)?

Yes, the binomial coefficients show up all over the place, don't they?
They also show up in Leibniz' Rule for the n'th derivative of a
product f(x)*g(x).  You might also notice that your numerator looks
rather like the formula for the "n'th difference" if you learn
something about finite differences, generally used in studying sequences.

If you have any questions about this or need more help, please write
back and show me what you have been able to do, and I will try to
offer further suggestions.

- Doctor Vogler, The Math Forum
  http://mathforum.org/dr.math/ 

Date: 11/11/2005 at 23:33:51
From: Phillip
Subject: Limit of a Limit - Higher derivatives

Hi again,

Thank you for responding to my question.  It was very helfpul.  I do
have one question, though.  What is a pathological function?  Thank
you again for your time.

-Phillip

Date: 11/12/2005 at 09:59:54
From: Doctor Vogler
Subject: Re: Limit of a Limit - Higher derivatives

Hi Phillip,

Pathology is the study of things going wrong.  It's a term used in
medicine (i.e. doctors), and I knew people who studied "speech
pathology" in college.  So a pathological function is a function that
was carefully designed to provide a counterexample to a conjectured
theorem that is usually true; i.e. a function designed to make things
go wrong.

- Doctor Vogler, The Math Forum
  http://mathforum.org/dr.math/