Mathematics for Data Science - 2

Professor. Sarang S. Sane

Department of Mathematics
Indian Institute of Technology, Madras
Lecture No. 27
Introduction to Vector Spaces

Hello, and welcome to the Maths 2 component of the online B.Sc. course on data science. In today's
video we are going to look at vector spaces. So, we will basically see in abstraction things that we
have seen in the past two weeks, wherein we saw vectors in ℝ𝑛 and we looked at various algebraic
properties. And we saw that these vectors are related to data because often data comes in a table
and we are interested in particular rows and columns of that table and some algebraic
manipulations on such data.

So, we are going to study vector spaces. This is slightly abstract as compared to previous videos.
But we will see that once we study this, much of what we have done in the previous videos can be
generalized to a very vast degree and it will come up when we do vector calculus later on some
principles of this.

So, what are vector spaces? Broadly speaking, vector spaces are precisely those places where
vectors live. So, we have talked about vectors, but we have not talked about what, where these
vectors are coming from, because they always came from ℝ𝑛 . So, in this video, I am going to try
and explain that there is a general notion of a vector space and then we use these concepts to glean
some information out of vectors which will come in handy later on. So, let us begin.
(Refer Slide Time: 01:36)

So, let us recall first what are vectors in ℝ𝑛 ? So, we have defined vectors in ℝ𝑛 as n-tuples of real
numbers, so x1, x2, xn, where each x is a real number. So, now suppose we have two vectors
𝑥1 , 𝑥2 , 𝑥𝑛 and 𝑦1 , 𝑦2 , 𝑦𝑛 in ℝ𝑛 and we have a scalar which means a real number c. So, we can define
addition of these vectors. So, addition is remember coordinate wise. What that means is, if you do
x1, x2, xn + y1, y2, yn, you get x1 + y2 in the first place, x2 + y2 in the second place, and so on up till
xn + yn in the nth place.

Similarly, we have defined scalar multiplication. So, this meant you to take this scalar c, this is a
real number, and multiply to, let us say, 𝑥1 , 𝑥2 , 𝑥𝑛 , and what we get is coordinate wise
multiplication, which means you get c× 𝑥1 , c × 𝑥2 and then c× 𝑥𝑛 at the end.
(Refer Slide Time: 02:36)

So, what are the properties of addition and scalar multiplication? So, suppose we have vectors v,
w and v’in ℝ𝑛 , and suppose a and b are scalars, so we are going to bring out certain properties of
vectors. These are going to be very easy and you might wonder why we are doing this. But in the
subsequent five minutes from now, we are use these same properties and they will be the guiding
principles for what is a vector space.

So, one of the properties we know about vectors is that if you take two vectors then if you add
them in a particular way, 𝑣 + 𝑤, then you could add them in the opposite way, meaning w + v
and both these give you the same answer. So, 𝑣 + 𝑤 is 𝑤 + 𝑣. Similarly, if you take three
vectors, you could add them. And then you have to first ask, should I add the first two first and
then the third to the result or should I add the last two, the second and third first and then add that
to the first. And the answer is it does not matter, because they are clearly the same. This is easy
checking in ℝ𝑛 .

And then we have something called the 0 vector and the 0 vector satisfies that, 0 vector is the one
where all the entries are 0, so 0, 0, 0 n times. And it satisfies that if you add that to any vector from
either side meaning v + 0 or 0 + V, then you get back the vector v. Then we have a vector called -
v, which is the, so v is 𝑣1 , 𝑣2 , 𝑣𝑛 the components are 𝑣1 , 𝑣2 , 𝑣𝑛 and then - v has components
−𝑣1 , −𝑣2 , −𝑣1 . And we can clearly see that if you do 𝑣 + − 𝑣, then you get 0. And similarly, if
you do - v + v, we get 0.
And then we have this very nice property that if you take 1, which is the scalar number 1, a real
number 1 and multiply to v, well you get just the v. So, v, 1× v is v. And then if you take these two
scalars, a and b, and then if you take a× b, that is a new real number and then multiply that to v,
then that is the same as first multiplying b to v and then multiplying a to the result of what we got.

And then you have 𝑎 × 𝑣 + 𝑤, which is 𝑎 × 𝑣 + 𝑎 × 𝑤. So, a distributes over addition. So,
scalar multiplication distributes over addition. And then finally, we have, if you take a + b× v, so
you take these two real numbers, you add them, and then multiply that to v, that is the same as first
doing scalar multiplication of a and v, then b and v and then adding the two that gives you the
same thing.

So, these are clearly obvious properties in ℝ𝑛 which you can see or you can do the algebra and
check that they are the same, that they are satisfied. So, we are going to use these to define a vector
space. So, a vector space is a set with two operations, like addition and scalar multiplication. So,
they are, in fact, called addition and scalar multiplication. And it has the properties 1 through 8.
So, I am going to make a formal definition now.

(Refer Slide Time: 06:14)

So, a vector space V over R, so why over R? What this means is that the numbers we are going to
use for scalar multiplication are real numbers. So, actually, you could use other kinds of numbers.
But for this course, the real numbers are good enough. So, a vector space V over R is a set along
with two functions, which I am going to denote by +. So, this + is going to correspond to what was
addition in ℝ𝑛 and dot, dot was going to, is going to correspond to what the scalar multiplication
in ℝ𝑛 .

So, this dot is from R cross V to V. That means if you take a scalar and V and an element of V,
you get back an element of V. And similarly, if you take two elements of V, you get V. So, what
is written here is that is for each pair of elements 𝑣1 and 𝑣2 in V, there is a unique element 𝑣1 + 𝑣2
in V. And for each constant, that is a scalar in R and vector v in V there is a unique element c dot
v in V. That is what we mean by, we have these two kinds of functions.

So, you should use, the intuition here is that + corresponds to addition and dot corresponds to
scalar multiplication. So, it is standard to suppress the dot and to write c× v instead of c dot with
v. So, this is exactly what we did in the ℝ𝑛 . We did not, we do not keep using multiplied by. We
keep, we suppress the multiplied by because we consider it as obvious. So, similarly, here, we are
going to suppress the dot and we are going to write just c× v for scalar multiplication. So, the
functions + and dot are required to satisfy the following rules and this is exactly the set of rules 1
through 8.

(Refer Slide Time: 08:09)

So, what are the rules? If you take two vectors 𝑣1 and 𝑣2 , so what are vectors? Vectors are elements
of this set capital V, our vector space capital V. So, if you take two vectors 𝑣1 and 𝑣2 , 𝑣1 + 𝑣2 is
𝑣2 + 𝑣1, you take three vectors 𝑣1 , 𝑣2 and 𝑣3 , then you can first add v1 and v2 and then add the
result to v3 or you can first add v2 and v3 and add the result v1 and the outcome of both these
processes is the same. So, this is, in other words you say that the operation +, the function + is
associative. That is a mathematical terminology for this.

There exists an element v in V, 0 in V which, we are going to denote it also by 0. And what does
it satisfy, it satisfies that v + 0 is v. So, when you add any vector to 0, then you get back the same
vector. And property 1 here, axiom one, ensures that v + 0 is the same as 0 + v. So, you could have
also have written this as v + 0 is 0 + v is v, which is how you will find it in some books.

So, for every element V, there exists an element 𝑣‘, such that v + v’is 0. So, this v’is like - v. This
v’is playing the role of - v. So, for each element v in V, 1× v is v. What is this 1? This 1 is the 1 in
the real numbers. And what is this 1× v? This is scalar multiplication. So, we are scalar multiplying
1 and v and the output is just v.

And then again, if you have two elements a and b, that is two real numbers a and b, and you choose
to multiply them to a vector v, we can do it in two possible ways. We could first multiply a and b,
that is multiplication in real numbers and then multiply the product, by that I mean scalar multiply
the product to v, that is the left hand side of this expression or we could do b× v, which is scalar
multiplication of v and b, and then, so that is again a vector, and then we take a and that new vector
b× v and do scalar multiplication and we get a× bv. So, both these operations, both the ways of
doing this yield the same result. That is what this is saying.

And then finally, well, not finally, we have one more to go. So, for each element a in R, so that
means each real number, if you take two vectors v1 and v2 and you do a× v1 + v2, then that is the
same as doing a× v1 + a× v2. So, it distributes over addition. Scalar multiplication distributes over
addition.

And then, if you have two real numbers a and b and you have a vector v, then you can get a + b×
v or you can do 𝑎 × 𝑣 + 𝑏 × 𝑣 and the result is the same. This is exactly the 1 to 8 that we saw
two slides back or maybe in the previous, two slides back, which were properties of the vectors in
ℝ𝑛 . And we are going, we are saying that any set along with these two operations, these two
functions + and not, which satisfies those 8 conditions is a vector space.

Now, of course, one, so obviously an example of a vector space is ℝ𝑛 with the usual addition and
scalar multiplication, but there are other examples.
(Refer Slide Time: 12:12)

For example, we could take the matrices. So, we have seen matrices in the last week or maybe last
two weeks. So, you take m by n matrices with real entries. So, we are going to denote that set by
m subscript m cross n. We have done this last week. So, this is the set of all matrices, m by n
matrices with real numbers, so m rows and n columns in this any particular matrix.

So, now, remember that when we dealt with matrices, we define addition of matrices and we define
scalar multiplication of matrices. This is from the second video maybe. So, how did we do that, I
will quickly recall. So, if you do A + B and so I have to, if I have to tell you what is A + B, I have
to tell you what is the ijth entry of A + B, that is what specifies a matrix. So, the ijth entry of A +
B is you take the ijth entry of A and add that to the ijth entry of B. This is how you define the sum
A + B.

And similarly, if you take the scalar multiplication that means I have to tell you what is c× A where
c is a real number and A is a matrix then what you do is you take c and multiply it to each entry of
A that is scalar multiplication. So, the ijth entry in particular is c× Aij. So, we have addition and
scalar multiplication and we actually have checked along the way in that video for matrices some
of those I stated and some of those we actually did.

We checked those axioms or we at least stated those axioms. So, the conditions required for this
to have to a vector, to be, for the matrices to be a vector space have been stated or checked already.
And if not, they are anyway easy to check. So, I would encourage you, I need to check them again
if you have forgotten. So, that makes the m by n matrices with real entries a vector space.

(Refer Slide Time: 14:33)

Another example is if you take solutions of a homogeneous system. So, we have seen systems of
linear equations and we have seen what is the homogeneous system. So, homogeneous system
looks like Ax equals 0, and then solutions are exactly those vectors x such that this is satisfied.

So, let us look at the set. So, consider a set of solutions V. So, we are going to write V as or denote
by V, the set of solutions of a homogeneous system Ax equals 0, and A is a matrix, let us say of
size m by n. So, this means it has m linear equations in n unknowns or n variables. So, x is consists
of the coordinates x1, x2, xn.

So, now if we have two vectors, v and w, well, I am calling them vectors, because we know that v
and w are already, I mean, this is a set of solutions. So, this is already a subset of ℝ𝑛 . So, the set
V is a subset of ℝ𝑛 . So, let us keep that in mind. So, when I say vectors, I mean vectors in ℝ𝑛 . So,
we have two vectors in ℝ𝑛 , which are actually in this subset v. So, both of these are solutions of
this linear system, of this homogeneous system.

So, then I can look at v + w. And I can do 𝐴 × 𝑣 + 𝑤. And then we have seen that how matrix
products work. That tells us that A× v + w is 𝐴 × 𝑣 + 𝐴 × 𝑤, but both v and w are coming from
capital V, which is the set of solutions. That means A× v is 0 and A× w is 0. That means this is 0
+ 0, which gives us 0. So, what does the net result? The net result is that v + w is in V.
So, similarly, if you do, if you take a scalar c and multiply it to a vector little v, such that little v is
in capital V, meaning little v is a solution of this homogeneous system, then I can do A×, c× v. So,
c× v remember is scalar multiplication. But, while we have seen this before, you can actually take
the c out, because I mean, so you can check this if you are unsure.

So, basically, you can, if you want, you can write this as c× the identity matrix of size n by n, and
then you can move to the other side of A and write it as c× identity matrix of size m by m. That is
one way. And there are, you could just directly do this by checking out the multiplication. So, the
c comes out and then that gives us c× A× v.

But remember, v is a solution of this system, which means that A× v is 0, that means you get c×
0, which is 0. So, what is the net result? The net result is that c× v belongs to V. So, what this
implies is that addition and scalar multiplication in ℝ𝑛 restricts to the solution set. So, I will qualify
this. So, from here, first of all, what we can say is that v + w belongs to V. And from here, what
we can see is that c× v belongs to V. And I am saying that once we know this, we know it is a
vector space.

Now, you are probably wondering why am I saying this, because I need to check the axioms. We,
there is a set of eight conditions and I have to check those. Well, so the point is, this is, remember,
a subset of ℝ𝑛 and we already know that for ℝ𝑛 all those axioms hold. So, for every subset also
those axioms will hold. The only axiom that we have to extra check maybe is that, well, not the
axiom, but what we have to really check is that when we do addition then we could in principle
get a vector which is not in V. So, we have seen here that that does not happen.

If we add two vectors in V, we get a vector in V. And similarly, if you use scalar multiply you get
a vector in V. And that is the only thing that is required. Once we have done this, we know that
this is a vector space, because all the other axioms are already satisfied since V is a subset of ℝ𝑛 .
So, this is an example of what is called a subspace of a vector space.

So, basically, it is a subset along with these properties that if you take two things, two vectors in
that subset, the sum is in the subset and if you take a vector in the subset and you do scalar
multiplication, then it belongs to the subset. So, automatically, this subset will inherit the property
of being a vector space. So, this is an example.

(Refer Slide Time: 20:04)

Let us look at a couple of non-examples. So, suppose I do addition and multiplication in ℝ2 as
follows. This is some strange addition and multiplication. So, you do 𝑥1 , 𝑥2 + 𝑦1 , 𝑦2 is 𝑥1 + 𝑦1, so
first coordinate there is addition. And in the second coordinate there is subtraction, so you do 𝑥2 −
𝑦2 . So, this is the definition of the +. And then c× 𝑥2 , 𝑥2 is c× 𝑥,c× 𝑥2 . So, the scalar multiplication
is the usual scalar multiplication. So, some things fail what fails. So, check that 1, 2 and 8 fail.

So, first of all let us just to get a hang, feel for this, let us check that 1 fails. So, 1 fails, why does
1 fail, because if I do 𝑥1 , 𝑥2 + 𝑦1 , 𝑦2 , well, that is written up here, that is 𝑥1 + 𝑦1 , 𝑥2 − 𝑦2 . And if I
do 𝑦1 , 𝑦2 + 𝑥1 , 𝑥2 , then I get 𝑦1 + 𝑥, which is not a problem. But the next term is going to give me
y2 - x2 and these are not the same as you can see.

For example, if you do, let us say 0, 0 + 1, 1, and 1, 1 + 0, 0, the answer here is 1,- 1, the answer
here is 1,1. They are not the same. So, these two are not the same. So, the first axiom side that v +
w is w + v, and that fails. You can check similarly that the second axiom fails and you can check
similarly that the eighth axiom fails. So, I will leave that to you.

(Refer Slide Time: 22:08)

So, let us do another example. So, this time let us define it as 𝑥1 , 𝑥2 + 𝑦1 , 𝑦2 is 𝑥1 + 𝑦1 ,0 and c× 𝑥1 ,
𝑥2 ,is c× 𝑥1 ,0. And this time, I maybe want to do the checking. So, what fails, check that the third,
fourth and fifth axioms fail. I will not, let me leave this to you as an exercise. This is not hard to
do. But it is important to do this because that way you will sort of develop an idea for what is a
vector space. And in particular, how defining these operations is really at the heart of what is a
vector space, addition and scalar multiplication.

Let us quickly recall what we have done thus far in this video. We have defined what is a vector
space which we saw was basically the abstraction of the properties of ℝ𝑛 . We saw the examples
of vector spaces apart from ℝ𝑛 , for example, matrices, and very importantly, the example of the
solution set of a homogeneous system of linear equations. That is really what we are going to, that
you should remember and keep in mind as we go ahead.
(Refer Slide Time: 23:30)

So, thank you.