Definition

Vectors are ordered arrays of single numbers and are an example of 1st-order tensor. Vectors are members of objects known as vector spaces. A vector space can be thought of as the entire collection of all possible vectors of a particular length (or dimension). The three-dimensional real-valued vector space, denoted by �3 is often used to represent our real-world notion of three-dimensional space mathematically.

More formally a vector space is an �-dimensional Cartesian product of a set with itself, along with proper definitions on how to add vectors and multiply them with scalar values. If all of the scalars in a vector are real-valued then the notation �∈�� states that the (boldface lowercase) vector value � is a member of the �-dimensional vector space of real numbers, ��.

Sometimes it is necessary to identify the components of a vector explicitly. The �th scalar element of a vector is written as ��. Notice that this is non-bold lowercase since the element is a scalar. An �-dimensional vector itself can be explicitly written using the following notation:

�=[�1�2⋮��]

Given that scalars exist to represent values why are vectors necessary? One of the primary use cases for vectors is to represent physical quantities that have both a magnitude and a direction. Scalars are only capable of representing magnitudes.

For instance scalars and vectors encode the difference between the speed of a car and its velocity. The velocity contains not only its speed but also its direction of travel. It is not difficult to imagine many more physical quantities that possess similar characteristics such as gravitational and electromagnetic forces or wind velocity.

In machine learning vectors often represent feature vectors, with their individual components specifying how important a particular feature is. Such features could include relative importance of words in a text document, the intensity of a set of pixels in a two-dimensional image or historical price values for a cross-section of financial instruments.