Fitting Equation to Data with Regression Techniques

Date: 04/20/2007 at 07:36:47
From: Daniel
Subject: Curvilinear or non-linear regression

I have a set of values that I'm trying to find the best fit of.  I've
used linear regression with little success.  I'll list some of the
values below.

 x    y
--- -----
 7   .438
10  1.091
12  1.111
13  1.625
14  1.680
17  2.519
48  2.800
52  3.500
55  3.850

My problem is that I've only dealt with linear regression.  I don't 
know how to go about any version of non-linear regression.  Can you 
step me through how I need to approach that?

Date: 04/20/2007 at 08:40:24
From: Doctor George
Subject: Re: Curvilinear or non-linear regression

Hi Daniel,

Thanks for writing to Doctor Math.

Let's say you want to fit a second order polynomial to the data.  The
general form would be this.

  y = ax^2 + bx + c

For each (x,y) point you get an equation in a, b and c.  In matrix
form it looks like the following.
  _    _     _             _ _   _
  | y1 |     |x1^2   x1   1| | a |
  | y2 |     |x2^2   x2   1| | b |
  | y3 |     |x3^2   x3   1| | c |
  | .  |  =  | .     .    .| -   -
  | .  |     | .     .    .|
  | .  |     | .     .    .|
  | yn |     |xn^2   xn   1|
  -    -     -             -

Now you need the least squares solution to find a, b and c.

Does that make sense?  Write again if you need more help.

- Doctor George, The Math Forum
  http://mathforum.org/dr.math/ 

Date: 04/20/2007 at 12:18:20
From: Daniel
Subject: Curvilinear or non-linear regression

Thanks for the reply.  My last calculus class was 12 years ago.  I
feel like a freshman in college all over again!

Does the matrix shown mean that I need to solve for each set of 
variables then use normal linear regression?  How do I come up with 
a, b, and c?

Date: 04/20/2007 at 14:03:21
From: Doctor George
Subject: Re: Curvilinear or non-linear regression

Hi Daniel,

This is actually a linear algebra problem.  Substitute your (x,y)
pairs into the matrices.

  _    _     _             _ _   _
  | y1 |     |x1^2   x1   1| | a |
  | y2 |     |x2^2   x2   1| | b |
  | y3 |     |x3^2   x3   1| | c |
  | .  |  =  | .     .    .| -   -
  | .  |     | .     .    .|
  | .  |     | .     .    .|
  | yn |     |xn^2   xn   1|
  -    -     -             -

Now I will give the vectors and the matrix the names b', A and x'.
With these names we can write

  b' = A x'

or

  A x' = b'

We want to find the vector x' that is the least squares solution to
this overdetermined system of equations.  Check the index of a linear
algebra book and you should find more information.

The vector x' gives you the a, b, and c for the best fit second order
regression polynomial.

Write again if you need more help.

- Doctor George, The Math Forum
  http://mathforum.org/dr.math/