What is Singular Value Decomposition (SVD)?

Let A be an mxn rectangular matrix. Using Singular Value Decomposition (SVD), we can decompose the matrix A in the following way:

Here, U is an mxm matrix. S is an mxn matrix and VT is an nxn matrix.

To calculate the U, S, and VT matrices, we need to find out the eigen values and eigen vectors of AAT and ATA. The columns of the matrix U are the eigenvectors of AAT. And the columns of the matrix V are the eigenvectors of ATA. And the diagonal entries of S are the square roots of the eigenvalues of AAT and ATA. These diagonal entries of S are also called the singular values. And, if A is a real matrix, then U, S, and V matrices are also real.

Singular Value Decomposition (SVD) using Python

We can use the following Python code to perform Singular Value Decomposition (SVD) using Python.

import numpy

A = numpy.array([[1, 2], [2, 1], [0, 1]])

U, s, V_transpose = numpy.linalg.svd(A)
S = numpy.zeros((3, 2), numpy.float64)


numpy.fill_diagonal(S, s)
print("Matrix U: \n", U)
print("Matrix S: \n", S)
print("Matrix V_transpose: \n", V_transpose)

A_calculated = U.dot(S).dot(V_transpose)
if numpy.allclose(A_calculated, A):
    print("SVD is successful")
else:
    print("SVD is not successful")

Here, A is a 3×2 matrix. We are using numpy.linalg.svd() function to perform Singular Value Decomposition (SVD) of A. The function returns the matrices U and VT and the diagonal elements of S.

As S can be a rectangular matrix, we are first creating a null matrix and then, filling the diagonal of the null matrix with the returned diagonal elements s.

We are then calculating the matrix USVT and using the function numpy.allclose() to check the calculated matrix is the same as A element-wise.

The output of the given program will be:

Matrix U: 
 [[-0.70002658  0.47354883 -0.53452248]
 [-0.67156256 -0.69106812  0.26726124]
 [-0.2428302   0.54605526  0.80178373]]
Matrix S: 
 [[3.08725264 0.        ]
 [0.         1.21196994]
 [0.         0.        ]]
Matrix V_transpose: 
 [[-0.66180256 -0.74967818]
 [-0.74967818  0.66180256]]
SVD is successful