Why Transpose a Matrix?

Date: 03/23/2008 at 16:00:47
From: Rob
Subject: Why Transpose a Matrix

Why is a transpose of a matrix needed?  I know how to transpose a
matrix, I just don't know why I have to do it.  I have read through my
Adjustment computations book by Ghilani and Wolf as well as consulted
online help with out finding any answers.

Date: 03/24/2008 at 12:08:33
From: Doctor Fenton
Subject: Re: Why Transpose a Matrix

Hi Rob,

Thanks for writing to Dr. Math.  The principal value of the transpose
arises in connection with scalar or dot products.  If 

  u = <u_1,u_2,...,u_n>  and  v = <v-1,v_2,...v_n>

are vectors in R^n, then the dot product of u and v, u.v, is 
defined by

  u.v = u-1*v_1 + u_2*v_2 + ... + u_n*v_n .

If A is an n x n matrix, then direct computation shows that

  (Au).v = u.(A^tv)   and   u.(Av) = (A^tu).v .

That is, if you have a dot product of two vectors, with a matrix A
applied to one of them, you can "move" the matrix to the other vector
if you transpose it.

That has many consequences, one of which is the following.  Note that
the magnitude ||v|| of a vector is related to the dot product by

  ||v||^2 = v.v .

Suppose that A is an n x n matrix which preserves the dot product, so 
that for all vectors u and v,

  (Au).(Av) = u.v  .

Then taking u = v, we see that

  ||Av||^2 = ||v||2 ,

so that  ||Av|| = ||v||, which means that A preserves distance.  In
addition, A preserves angles, since

  (Au).(Av) = ||Au|| ||Av|| cos(@1) ,

where @1 is the angle between Av and Av,

and

  u.v = ||u|| ||v|| cos(@2)  where @2 is the angle between u and v.

Then since ||Au|| = ||u|| and  ||Av|| = ||v||, we have that

  cos(@1) = cos(@2),  (and @1 and @2 are in Quadrants 1 or 2)

so @1 = @2, and A preserves angles (the angle between Au and Av is
the same as the angle between u and v).

Also, if (Au,Av) = u.v for all u and v, then  

  (A^tAu).v = u.v   for all u and v,

so

  ((A^tA-I)u).v  = 0  for all v, 

and therefore A^tAu = 0 for all u, since the only vector whose dot 
product with every vector is 0 is the zero vector.  This means that 
A^tA = I, that A is invertible, and A^(-1) = A^t.

You also use this property to show that the eigenvectors belonging
to different eigenvalues of a symmetric matrix in an inner product 
space must be orthogonal.

A side benefit is that you can represent vectors with matrices.  For 
example, if we write vectors as column matrices, then the dot product 
becomes a matrix operation:

  u.v = (u^t)v  .

If you have any questions, please write back, and I will try to
explain further.

- Doctor Fenton, The Math Forum
  http://mathforum.org/dr.math/