CODEWITHSUNDEEP
matrixjulia
aberdysh
aberdysh

Reputation: 1664

Definition of matrix-vector division operator of Julia

I stumbled upon something, which I consider very strange.

As an example consider the code

A = reshape(1:6, 3,2)
A/[1 1]

which gives

3×1 Array{Float64,2}:
 2.5
 3.5
 4.5

As I understand, in general such division gives the weighted average of columns, where each weight is inversely proportional to the corresponding element of the vector.

So my question is, why is it defined such way?

What is the mathematical justification of this definition?

Upvotes: 4

Views: 634

Answers (2)

Chris Rackauckas
Chris Rackauckas

Reputation: 19132

Let A be an NxN matrix and b be a Nx1 column vector. Then \ solves Ax=b, and / solves xA=b.

As Stefan mentions, this is extended to underdetermined cases as the least squares solution. This is done via the QR or SVD decompositions. See the details on these algorithms to see why this is the case. Hint: the linear form of the OLS estimator can actually be written as the solution to matrix decompositions, so it's the same thing.

Now you might ask, how does it actually solve it? That's a complicated question. Essentially, it uses a matrix factorization. But which matrix factorization is used is dependent on the matrix type. The reason for this is because Gaussian elimination is O(n^3), and so treating the problem generally is usually not good. But whenever you can specialize, you can get speedups. So essentially \ (and /, which transposes and calls \) check for a bunch of special types and pick a factorization or other algorithm (LU, QR, SVD, Cholesky, etc.) based on the matrix type. The flow chart from MATLAB explains this very well. There's a lot of details here, and it gets even more details when the matrix is sparse. Also IterativeSolvers.jl should be mentioned because it's another set of algorithms for solving Ax=b.

Most applied math problems reduce down to linear algebra, with solving Ax=b being one of the most important and difficult problems, which is why there is tons of research on the subject. In fact, you can probably say that the vast majority of the field of numerical linear algebra is devoted to finding fast methods for solving Ax=b on specific matrix types. \ essentially puts all of the direct (non-iterative) methods into one convenient operator.

Upvotes: 6

StefanKarpinski
StefanKarpinski

Reputation: 33259

It's the minimum error solution to |A - v*[1 1]|₂ – which, being overconstrained, has no exact solution in general (i.e. value v such that the norm is precisely zero). The behavior of / and \ is heavily overloaded, solving both under and overconstrained systems by a variety of techniques and heuristics. Whether this kind of overloading is a good idea or not is debatable, but it's what people have come to expect from these operations in Matlab and Octave, and it's often quite convenient to have so much functionality available in a single operator.

Upvotes: 7

Related Questions