- Intro to Matrices and Vectors -
- Matrix Operations: Addition and Subtraction -
- Associative and Distributive Properties
- Associative -
- Identity: A special matrix -
- Matrix Operation: Inverses
- Matrix Operation: Transpose
- Previous Section
- Next Section
If you have a background in basic Vectors and Matrices - including Matrix Addition, Subtraction, Multiplication, Inverse and Transpose -, you can skip this lesson, but otherwise this should give you a good primer.
Intro to Matrices and Vectors -
A matrix is essentially a collection of numbers. It is physically represented as a rectangle of numbers.
A matrix is not exactly just a list of numbers, in that their position also matters. For example,
A vector, on the other hand, is just an ordered list of numbers - it is only a column or row. It is often represented this way:
Like with Matrices (plural for Matrix), the following statement is true:
Matrix Operations: Addition and Subtraction -
When you add two matrices, you produce another matrix with the same dimensions/shape where each element is the sum of the corresponding elements. You'll see what that means in a second, but an important note is that when adding two matrices, the two must be of the same shape. Here's an example,
1 + 0 = 1 2 + 9 = 11 0 + 3 = 3 3 + 2 = 5
As you can see, the elements in the same position are each added up individually.
On the other hand, look at this attempted addition:
Since these two matrices have different shapes, there's no meaning to adding the corresponding terms, so you can't add these kinds of matrices.
Subtraction is just the opposite as Addition, so instead of adding the individual elements and reconstructing a matrix, you just need to subtract them.
Vector Operations: Dot Product
A dot product is one of the simplest mathematical operations. Let us take two vectors, A and BA =
The dot product of A and B, denoted as:
Means that you multiply the corresponding elements. This is much like how you would add these two vectors (Matrices with only one row, or only one column), except you multiply the individual terms instead of adding. After doing this, you must sum up these products.
Try this problem as well:
Matrix Operations: Multiplication
Matrix Multiplication is a little complicated, but you will eventually develop an intuition for it.
NOTE: The 157 in the image should be a 139
Let's denote the first matrix by X and the second matrix by Y for the time being. This has nothing to do with the meaning of those variables in ML, so don't get confused with that.
When we multiply X and Y, we essentially take the dot product of the first row of X with the first column of Y, and put that in the first place. Then, we take the dot product of the first row of X with the second column of Y, and put that to the right of the previous value. We will keep doing this for all the columns of Y with the first row of X. This entire process is then repeated with all the rows of X, until the new matrix is filled out.
We will try some more Matrix Multiplications in the Quiz, and through a program, but now, take a look at the dimensions of X, Y and the final matrix.
The dimensions of a matrix are essentially the numbers of rows and columns it has. For example, the following matrix:
has two rows, and 3 columns, denoted as a 2*3 matrix.
has three rows and 2 columns, denoted as a 3*2 matrix.
Matrix multiplication has specifications with which two matrices can be multiplied, but these are somewhat different from those required by addition and subtraction. If you take a look at Example Two from above(reproduced below), the two matrices being multiplied don't have the same shape/dimension - they definitely could not have been added. And yet, we have successfully multiplied them......
The criteria for matrices to be multiplied is that the number of columns in X matches the number of rows in Y. In Example 1, both the first and second matrices were square - meaning that they had the same number of rows and columns - and they were both 22, so this criteria was automatically met. However, in the case of Example 2, the first matrix was 23 and the second 3*2. This too meets the criteria.
The dimensions of the resultant matrix are also determined by those of the original two. The number of rows of the first matrix carries over to the resultant matrix, while the number of columns of the second matrix carries over to the resultant matrix. In Example 2, we see that the first matrix is 23 and the second is 32, and the resultant matrix is 2*2.
Associative and Distributive Properties
Matrix multiplication has two main properties: It is associative and distributive.
In any case, you will not have to do too much with these.
Try out this one:
Identity: A special matrix -
The identity matrix is a square matrix. The square may have any number of rows/columns, but the principle is the same. An identity matrix plays the part of the number 1 in the matrix world.By multiplying any number by 1, you get the same number: by multiplying a matrix by the identity matrix, you get the same matrix back. The identity matrix consists of one diagonal line, running from the top-left to the bottom-right, of 1s, where the rest of the matrix is filled with zeroes. The identity matrix looks like this:
And so on.
Matrix Operation: Inverses
We'll not get into the actual dynamics of calculating a matrix inverse, as they can vary based on dimension, but the important point is that multiplying by a matrix's inverse is equivalent, essentially, to dividing by it, since you cannot directly do Matrix Division.
Given a matrix A, the inverse, denoted:
is such that
where I is the identity matrix.
Matrix Operation: Transpose
The transpose operation effectively turns a matrix over its diagonal. It flips its dimensions. For example, the transpose of A is denoted:
If A is:
While imagining some kind of rotation may be a good idea, the most practical way to take the transpose of a matrix is as follows:
If a particular element a belongs at row r and column c of matrix A, then it belongs at row c and column r of its transpose. Try making that mapping between the two matrices shown above.
What's the matrix A, if this is true:
And that's all we'll need! Congratulations - you just went through a grueling math review and you deserve a break. In case you felt like this was a little too difficult, or you didn't grasp some of the more specific concepts like Matrix Multiplication, don't worry. We'll not only have more exercises that will let you practice to proficiency, this math is not fully required to do the ML we'll have in mind. The better you understand these ideas, the more you'll be able to understand why we're doing what we're doing in the next lessons, but even if not, you will not be at a significant disadvantage!
1. (Dot Product) 41
2. (Matrix Multiplication)