The fourth subspace:
Column space (span of columns, Ax for all possible x)
Row space (span of rows, yTA for all possible y)
Null space (solutions of Ax=0, all vectors orthogonal to the rows, all dependencies among columns)
Left null space (solutions of yTA=0, all vectors orthogonal to the columns, all dependencies among the rows, nullspace of AT).
B =
1 -2 4 -3 -1 0
2 3 1 8 2 3
1 1 1 3 2 0
3 -1 7 1 1 1
2 1 -2 2 0 1
5 5 0 13 0 8
> rref(B)
ans =
1.0000 0 0 0.2000 0 0.2000
0 1.0000 0 2.4000 0 1.4000
0 0 1.0000 0.4000 0 0.4000
0 0 0 0 1.0000 -1.0000
0 0 0 0 0 0
0 0 0 0 0 0
Rank(B)=4=dim row space = dim col space
Basis of col space:
B(:,[1 2 3 5])
ans =
1 -2 4 -1
2 3 1 2
1 1 1 2
3 -1 7 1
2 1 -2 0
5 5 0 0
Basis of row space:
1.0000 0 0 0.2000 0 0.2000
0 1.0000 0 2.4000 0 1.4000
0 0 1.0000 0.4000 0 0.4000
0 0 0 0 1.0000 -1.0000
Null space basis (worked out by hand from above):
[ -.2 -2.4 -.4 1 0 0]
[ -.2 -1.4 -.4 0 1 1]
> null(B,'r') %let matlab do it
ans =
-0.2000 -0.2000
-2.4000 -1.4000
-0.4000 -0.4000
1.0000 0
0 1.0000
0 1.0000
Left null space basis
> null(B','r')
ans =
-1.4444 -0.4444
-0.3333 -2.3333
-0.8889 2.1111
1.0000 0
0 -1.0000
0 1.0000
> [B(:,[1 2 3 5]) null(B','r')]
ans =
1.0000 -2.0000 4.0000 -1.0000 -1.4444 -0.4444
2.0000 3.0000 1.0000 2.0000 -0.3333 -2.3333
1.0000 1.0000 1.0000 2.0000 -0.8889 2.1111
3.0000 -1.0000 7.0000 1.0000 1.0000 0
2.0000 1.0000 -2.0000 0 0 -1.0000
5.0000 5.0000 0 0 0 1.0000
First four columns are basis of column space
Last two columns are basis of left null space
Together they are a basis for R6
Characterization of column space: Ax=b has a solution (b is in the column space) if and only if yTb=0 for any y in left null space.
Definition of inverse: If A is square, we define the inverse of A (if it exists) as the matrix A-1 that satisfies AA-1=I and A-1A=I.
Theorem: A-1 exists (A has an inverse) if and only if A~I (or the rank of A is n or dim col space=n , Ax=0 only if x=0 )
How to calculate A-1
Think of solving for B that satisfies AB=I – this is, column by column in B, a system of linear equations, i.e. A*(col j of B)=(col j of I) and we solve all these equations at once by putting all the columns of I into a super-augmented matrix: [A | I] . Then we reduce as usual, and assuming that A~I we would obtain
[A | I] ~ [I | A-1]
because each column on the right is one of the columns of B that we seek.
(If you discover that the rank of A is less than n, then A-1 doesn’t exist, e.g. note that if A-1 exists then A(A-1b)=b for any b, which shows that Ax=b always has a solution. If the rank of A is less than n, this can’t be true and so A-1 can’t exist)
4 subspaces in one reduction: We already know how to get a basis of the column space, row space and null space of the matrix A from the reduced row echelon form of A. Now consider the reduction
[A | I] ~ [R | B] where R is the (reduced) row echelon form of A. R will have m-r rows of zeros at the bottom, where r is the rank of A. Then the bottom m-r rows of B are a basis for the left null space. Here’s the reason: Every row operation can be carried out by a left-multiplication so [A | I] ~ [R | B] means that C[A | I]=[R | B] for some C. But then we must have CA=R and CI=B; from the latter we see that C=B so that BA=R. Now the bottom m-r rows of R are equal to the corresponding m-r rows of B times A and the result is zero for each such row. Thus each of the bottom m-r rows of B is in the left null space. The dimension of the left null space is m-r and those rows of B are linearly independent because B is row equivalent to I. Thus those rows are a basis of the left null space.
Here is our previous example of the matrix B of rank 4. We find the left null space using the procedure outlined above.
> [B eye(6)]
ans =
1 -2 4 -3 -1 0 1 0 0 0 0 0
2 3 1 8 2 3 0 1 0 0 0 0
1 1 1 3 2 0 0 0 1 0 0 0
3 -1 7 1 1 1 0 0 0 1 0 0
2 1 -2 2 0 1 0 0 0 0 1 0
5 5 0 13 0 8 0 0 0 0 0 1
> rref([B eye(6)])
ans =
1 0 0 0.2000 0 0.2000 0 0 -0.0690 0.1379 0.4483 -0.0483
0 1 0 2.4000 0 1.4000 0 0 0.0690 -0.1379 -0.4483 0.2483
0 0 1 0.4000 0 0.4000 0 0 -0.0345 0.0690 -0.2759 0.0759
0 0 0 0 1 -1 0 0 0.5172 -0.0345 0.1379 -0.1379
0 0 0 0 0 0 1 0 0.8621 -0.7241 -0.1034 0.1034
0 0 0 0 0 0 0 1 -1.0690 0.1379 0.4483 -0.4483
The rank of B is 4, the last two rows in the reduced form of B are zeroes. The basis for the left null space is then the last two rows in the right-hand side of the augmented matrix, namely
[1 0 0.8621 -0.7241 -0.1034 0.1034]
[0 1 -1.0690 0.1379 0.4483 -0.4483]
Of course, it’s not clear that this is any easier than simply finding the null space of the transpose of the matrix in question, which is the more direct approach.