Properties

Matrices are rectangular arrays consisting of numbers and are an example of 2nd-order tensors. If � and � are positive integers, that is �,�∈� then the �×� matrix contains �� numbers, with � rows and � columns.

If all of the scalars in a matrix are real-valued then a matrix is denoted with uppercase boldface letters, such as �∈��×�. That is the matrix lives in a �×�-dimensional real-valued vector space. Hence matrices are really vectors that are just written in a two-dimensional table-like manner.

Its components are now identified by two indices � and �. � represents the index to the matrix row, while � represents the index to the matrix column. Each component of � is identified by ���.

The full �×� matrix can be written as:

�=[�11�12�13…�1��21�22�23…�2��31�32�33…�3�⋮⋮⋮⋱⋮��1��2��3…���]

It is often useful to abbreviate the full matrix component display into the following expression:

�=[���]��

Where ��� is referred to as the (�,�)-element of the matrix �. The subscript of �×� can be dropped if the dimension of the matrix is clear from the context.

Note that a column vector is a size �×1 matrix, since it has � rows and 1 column. Unless otherwise specified all vectors will be considered to be column vectors.

Matrices represent a type of function known as a linear map. Based on rules that will be outlined in subsequent articles, it is possible to define multiplication operations between matrices or between matrices and vectors. Such operations are immensely important across the physical sciences, quantitative finance, computer science and machine learning.

Matrices can encode geometric operations such as rotation, reflection and transformation. Thus if a collection of vectors represents the vertices of a three-dimensional geometric model in Computer Aided Design software then multiplying these vectors individually by a pre-defined rotation matrix will output new vectors that represent the locations of the rotated vertices. This is the basis of modern 3D computer graphics.

In deep learning neural network weights are stored as matrices, while feature inputs are stored as vectors. Formulating the problem in terms of linear algebra allows compact handling of these computations. By casting the problem in terms of tensors and utilising the machinery of linear algebra, rapid training times on modern GPU hardware can be obtained.