Simple Salvation from the Scary Squared Sums: Matrices As Another Method of Computing

--Simple Salvation from the Scary Squared Sums: Matrices as Another Method of Computing Linear Least Squares.

Anybody who has taken introductory statistics knows that estimating lines of least squares without a calculator is one of the most time-consuming and difficult-to-memorize endeavors in the curriculum. Just from looking at the formula, it can be difficult to understand how and why it works. However, with a small amount of knowledge in the area of matrix algebra, a newer and easier-to-understand process can be utilized to just as accurately nail the line of best fit! If no calculators are available, it’s indispensable!

Regression shows up everywhere in the real world! You can’t afford to forget how to calculate linear relationships! Join us Wednesday, December 13, in Walker Hall for this unforgettable presentation!

Least Squares Linear Regression

What is Least Squares Linear Regression?

Least Squares LinearRegression is, put simply, the process by which one finds the line that comes closest to going through as many points on a graph as possible. The most obvious applications of this can be found any time data is ever collected as you to find a trend between observations as a variable, usually time, increases. It could be a plot of how long the semester has dragged on to how late you get to class, or your average grades on assignments as class goes on, or perhaps the number of points scored per football game as the season keeps going. As long as you have a way to dictate a score versus progress, you can use least squares linear regression to detect the most likely trend (if there is one at all) between the two.

How Does One Calculate the Line of Least Squares?

The line that best fits the data – called, not surprisingly, the Line of Best Fit or the Line of Least Squares – can be calculated by one of two general methods. The first one most students are generally exposed to uses a calculus-based definition.

First: Add together all of the ordinates (X-values). This is called SX. Do the same for the Y-values.

Second: Square all of the X-values, and then add them together. This is SXX. Do the same for the Y-values.

Third: Multiply each X-value by its corresponding Y-value, and then add all of those values together. This is SXY.

Fourth: Remember that n is the number of observations.

From there, the line is of the form Y=mX+ b where:

m=(nSXY –SXSY)/(nSXX – SXSX) and

b=(SY – mSX)/n.

If there is any difficulty with understanding this notation, please proceed to the

demonstration area, where a problem is worked out using this method.

The alternate method is to use matrix algebra.

First: Create an nx2 matrix “A” where the values of X are listed in order vertically in the first column and the second column is filled with values of 1.

Second: Create the transpose of that matrix “AT”. In other words, create a 2xn matrix where the values of X are listed horizontally in the first row and the second row is filled with values of 1.

Third: Perform the function ATA to get a 2x2 matrix.

Fourth: Create an nx1 column vector “B” composed of all the values of Y in order.

Fifth: Perform the function ATB to get a 2x1 matrix.

Sixth: Invert ATA to create (ATA)-1.

Seventh: Perform the function (ATA)-1B to get a 2x1 matrix. The first value in this matrix is the slope or m value; the second is the Y-intercept or b value.

Believe it or not, when it’s explained, it’s surprisingly easy to keep track of. The exact method of how this works is explained more in the demonstration area where it is simply easier to write the matrix algebra symbols. There, you may decide for yourself which method is easier to use.