Intro to Matrices and VectorsMatrix Operations: Addition and SubtractionVector Operations: Dot ProductMatrix Operations: MultiplicationExample 1:Example 2:Associative and Distributive PropertiesAssociativeDistributiveThe Identity MatrixMatrix Operation: InversesMatrix Operation: Transpose
If you have a background in basic Vectors and Matrices - including Matrix Addition, Subtraction, Multiplication, Inverse and Transpose -, you can skip this lesson, but otherwise this should give you a good primer.
Intro to Matrices and Vectors
A matrix is essentially a collection of numbers. It is physically represented as a rectangle of numbers.
A matrix is not exactly just a collection of numbers, in that their position also matters. For example,
A vector, on the other hand, is a one dimensional matrix, meaning that it can be represented by a single column/row. Here's what a vector could look like:
Like with matrices (plural for matrix), the following statement is true:
Matrix Operations: Addition and Subtraction
When you add two matrices, you produce another matrix with the same dimensions/shape where each element is the sum of the corresponding elements. You'll see what that means in a second, but an important note is that when adding two matrices, the two must be of the same shape. Here's an example:
As you can see, all we need to do is add together elements that are in the same position.
But what about addition as shown below?
Since these two matrices have different shapes, there's no way we can add together corresponding terms, so they cannot be summed.
Subtraction is just the opposite as addition, so instead of adding the individual elements and reconstructing a matrix, you just need to subtract them.
Vector Operations: Dot Product
A dot product is one of the simplest mathematical operations. Let's take two vectors, and .
The dot product of and , denoted as , means that you multiply the corresponding elements. This is much like how you would add these two vectors (Matrices with only one row, or only one column), except you multiply the individual terms instead of adding. After doing this, you sum up these products.
Try this problem as well:
Matrix Operations: Multiplication
Matrix multiplication is a little complicated, but you will eventually develop an intuition for it.
Example 1:
Example 2:
Let's denote the first matrix by X and the second matrix by Y for the time being. This has nothing to do with the meaning of those variables in ML, so don't get confused with that.
When we multiply X and Y, we essentially take the dot product of the first row of X with the first column of Y, and put that in the first place. Then, we take the dot product of the first row of X with the second column of Y, and put that to the right of the previous value. We will keep doing this for all the columns of Y with the first row of X. This entire process is then repeated with all the rows of X, until the new matrix is filled out.
We will try some more matrix multiplications in the quiz, and through a program, but now, take a look at the dimensions of X, Y and the final matrix.
The dimensions of a matrix are essentially the numbers of rows and columns it has.
For example, the following matrix
has 2 rows, and 3 columns, denoted as a matrix. This matrix
has three rows and 2 columns, denoted as a matrix.
Matrix multiplication has specifications with which two matrices can be multiplied, but these are somewhat different from those required by addition and subtraction. If you take a look at example two from above (reproduced below), the two matrices being multiplied don't have the same shape/dimension — they definitely could not have been added. And yet, we have successfully multiplied them.
The criteria for matrices to be multiplied is that the number of columns in matches the number of rows in . In example 1, both the first and second matrices were square - meaning that they had the same number of rows and columns - and they were both , so this criteria was automatically met. However, in the case of Example 2, was and was . This too meets the criteria.
The dimensions of the resultant matrix are also determined by those of the original two. The number of rows of the first matrix carries over to the resultant matrix, while the number of columns of the second matrix carries over to the resultant matrix. In Example 2, we see that the first matrix is and the second is , and the resultant matrix is .
Associative and Distributive Properties
Matrix multiplication has two main properties. It is both associative and distributive.
Associative
Distributive
In any case, you will not have work too much with these.
Try out this multiplication problem:
The Identity Matrix
The identity matrix is a square matrix. The square may have any number of rows/columns, but the principle is the same. An identity matrix plays the part of the number 1 in the matrix world. By multiplying any number by 1, you get the same number; by multiplying a matrix by the identity matrix, you get the same matrix back. The identity matrix consists of a diagonal line of 1s, running from the top-left to the bottom-right, and the rest of the matrix is filled with zeroes. The identity matrix looks like this:
For a matrix
For a matrix
For a matrix
And so on.
Matrix Operation: Inverses
We won't get into the actual dynamics of calculating a matrix inverse, as they can vary based on dimension, but the important point is that multiplying a matrix by a matrix's inverse is essentially equivalent to dividing by that matrix, since you cannot directly do matrix division.
Given a matrix , the inverse, denoted by , is defined such that
where is the identity matrix.
Matrix Operation: Transpose
The transpose operation effectively turns a matrix over its diagonal, flipping its dimensions. We can denote the transpose of a matrix with a symbol. For example, the transpose of a matrix would be denoted by .
If
then
While imagining transposition as kind of rotation might seem like a good idea, the most practical way to take the transpose of a matrix is as follows:
If a particular element a belongs at row and column of matrix , then it belongs at row and column of . Try making that mapping between the two matrices shown above.
What would matrix be, if the following is true?
And that's all we'll need! Congratulations — you just went through a grueling math review and you deserve a break. In case you felt like this was a little too difficult, or you didn't grasp some of the more specific concepts like matrix multiplication, don't worry. We won't have more exercises that will let you practice to proficiency, as this math is not fully required to do the ML we'll have in mind. The better you understand these ideas, the more you'll be able to understand why we're doing what we're doing in the next lessons, but even if you aren't too familiar with them, you will not be at a significant disadvantage!
Answer Key:
- (Dot Product)
- (Matrix Multiplication)
- (Matrix Transposition)
Copyright © 2021 Code 4 Tomorrow. All rights reserved.
The code in this course is licensed under the MIT License.
If you would like to use content from any of our courses, you must obtain our explicit written permission and provide credit. Please contact classes@code4tomorrow.org for inquiries.