1. Definition of a matrix.
The question of what a matrix is, precisely, is one I had for a long time as a high school student. It took many tries to get a straight answer, because people tend to conflate "matrix" with "linear transformation". The two are closely related, but NOT the same thing. So let me start with the fully rigorous definition of a matrix:
An $m$ by $n$ matrix is a function of two variables, the first of which has domain $\{1,2,\dots,m\}$ and the second of which has domain $\{1,2,\dots,n\}$.
This is the formal definition of matrices, but it's not how we usually think about them. We have a special notation for matrices--the "box of numbers" you are familiar with, where the value of the function at $(1,1)$ is put in the top left corner, the value at $(2,1)$ is put just below it, etc. We usually think of the matrix as just this box, and forget that it is a function. However, sometimes you need to remember that a matrix has a more formal definition, like when implementing matrices on a computer (most programming languages have matrices built into them).
2. What matrices represent.
Matrices can represent different things in different contexts, but there is one application that is most common. The most common application is linear transformations (a.k.a. linear maps), but before I get into that, let me briefly mention some other applications:
- Matrices can be used to store data. For example, images on a computer are often stored as a matrix, where the matrix's value at $(i,j)$ is the intensity of light on the camera pixel that is $i^{th}$ from the top and $j^{th}$ from the left.
- Matrices can be used as computational tools. For example, one way to compute the Fibonacci numbers is from powers of the matrix
$$M = \begin{bmatrix}
1 & 1 \\
1 & 0 \\
\end{bmatrix}$$
It turns out that $(M^k)_{11}$ is the $k^{th}$ Fibonacci number.
- Matrices can be used to encode some mathematical structure. I'm going to be sort of hand-wavy about this, but an example of what I have in mind is an adjacency matrix for a graph or network, which tells you which nodes are connected to which.
So the point is that a matrix can be used for lots of things. However, one usage prevails as most common, and that is representing linear transformations. The prevalence of this usage is why people often conflate the two concepts. A linear transformation is a function $f$ of vectors which has the following properties:
- $f(x+y) = f(x) + f(y)$ for any vectors $x$ and $y$.
- $f(ax) = af(x)$ for any vector $x$ and any scalar $a$.
These properties are what it takes to ensure that the function $f$ has "no curvature". So it's like a straight line, but possibly in higher dimensions.
The relationship between matrices and linear transformations comes from the fact that a linear transformation is completely specified by the values it takes on a basis for its domain. (I presume you know what a basis is.) To see how this works, suppose we have a linear transformation $f$ which has domain $V$ and range $W$, where $V$ is a vector space with basis $v_1,v_2,\dots, v_n$ and $W$ is a vector space with basis $w_1,w_2,\dots,w_m$. Then there is a matrix $M$ representing $f$ with respect to these bases, which has as element $(i,j)$ the coefficient of $w_i$ when you express $f(v_j)$ as a sum of basis elements in $W$.
The reason that this is a good idea is that if you have some miscellaneous vector $x = a_1 v_1 + a_2 v_2 + \cdots + a_n v_n \in V$, then if you represent $x$ as a column vector $[a_1,a_2,\dots,a_n]^T$ and $f$ as its matrix $M$, then the value $f(x)$ is given by the matrix product of $M$ and $[a_1,a_2,\dots,a_n]^T$. So the matrix $M$ completely encodes the linear transformation $f$, and matrix multiplication tells you how to decode it, i.e. how to use the matrix to get values of $f$.
3. Geometrical intuition.
In my opinion, the most important theorem for getting intuition for matrices and linear transformations is the singular value decomposition theorem. This says that any linear transformation can be written as a sequence of three simple transformations: a rotation, a stretching, and another rotation. Note that the stretching operation can stretch by different amounts in different orthogonal directions. This tells you that all linear transformations are some combination of rotation and stretching.
Other properties of matrices often have direct geometric interpretation, too. For example, the determinant tells you how a linear transformation changes volumes. By the singular value decomposition, a linear transformation turns a cube into some sort of stretched and rotated parallelogram. The determinant is the ratio of the volume of the resulting parallelogram to that of the cube you started with.
Not all properties of a matrix can be easily associated with familiar geometric concepts, though. I don't know of a good geometric picture for the trace, for instance. That doesn't mean that the trace is any less useful or easy to work with, though!
4. Other properties.
Almost all of the "properties" and "operations" for matrices come from properties of linear maps and theorems about them. For example, the standard multiplication of matrices is designed specifically to give the values of linear maps as explained above. This is NOT the only type of multiplication that can be defined on matrices, and in fact there are other types of multiplication for matrices (for example, the Hadamard product and the Kronecker product). These other types of multiplication are sometimes useful, but generally not as useful as regular matrix multiplication, so people often don't know (or care) about them.
5. TL;DR
The moral of the story is that you can use matrices for whatever you want (and they are indeed used in many different ways), but the way that most people use them most of the time is to represent linear maps, and the standard definitions and "properties" of matrices reflect this bias. The study of linear maps goes by the name "linear algebra", and a textbook on this subject is a good place to start if you want to learn more about matrices. (Depending on your background, you may find some good reference suggestions here: link.)
