Rank (matrix)
A mathematical property defining the maximum number of linearly independent column or row vectors within a matrix, representing its true dimensionality.
Think of It Like This
Like measuring the true size of a team by counting only the people who bring unique skills, ignoring those who just perfectly copy someone else.
If a massive matrix has a very low rank, it means it contains an enormous amount of redundant information. This insight is the mathematical foundation for techniques like LoRA and Singular Value Decomposition (SVD). By identifying the true low-rank structure, engineers can drastically compress neural network weights with negligible loss in accuracy.