CODEWITHSUNDEEP
algorithmaverage
wajs
wajs

Reputation: 309

Algorithm to evaluate best weights for weighted average

I have a data set of the form:

[9.1 5.6 7.4] => 8.5, [4.1 4.4 5.2] => 4.9, ... , x => y(x)

So x is a real vector of three elements and y is a scalar function.

I'm assuming a weighted average model of this data:

y(x) = (a * x[0] + b * x[1] + c * x[2]) / (a+b+c) + E(x)

where E is an unknown random error term.

I need an algorithm to find a,b,c, that minimizes total sum square error:

error = sum over all x of { E(x)^2 }

for a given data set.

Upvotes: 2

Views: 3453

Answers (1)

Gene
Gene

Reputation: 47020

Assume that the weights are normalized to sum to 1 (which happily is without loss of generality), then we can re-cast the problem with c = 1 - a - b, so we are actually solving for a and b.

With this we can write

error(a,b) = sum over all x { a x[0] + b x[1] + (1 - a - b) x[2] - y(x) }^2

Now it's just a question of taking the partial derivatives d_error/da and d_error/db and setting them to zero to find the minimum.

With some fiddling, you get a system of two equations in a and b.

C(X[0],X[0],X[2]) a + C(X[0],X[1],X[2]) b = C(X[0],Y,X[2])
C(X[1],X[0],X[2]) a + C(X[1],X[1],X[2]) b = C(X[1],Y,X[2])

The meaning of X[i] is the vector of all i'th components from the dataset x values.

The meaning of Y is the vector of all y(x) values.

The coefficient function C has the following meaning:

C(p, q, r) = sum over i { p[i] ( q[i] - r[i] ) }

I'll omit how to solve the 2x2 system unless this is a problem.

If we plug in the two-element data set you gave, we should get precise coefficients because you can always approximate two points perfectly with a line. So for example the first equation coefficients are:

C(X[0],X[0],X[2]) =  9.1(9.1 - 7.4) + 4.1(4.1 - 5.2) = 10.96
C(X[0],X[1],X[2]) = -19.66
C(X[0],Y,X[2]) = 8.78

Similarly for the second equation: 4.68 -13.6 4.84

Solving the 2x2 system produces: a = 0.42515, b = -0.20958. Therefore c = 0.78443.

Note that in this problem a negative coefficient results. There is nothing to guarantee they'll be positive, though "real" data sets may produce this result.

Indeed if you compute weighted averages with these coefficients, they are 8.5 and 4.9.

For fun I also tried this data set:

X[0]        X[1]        X[2]        Y
0.018056028 9.70442075  9.368093544 6.360312244
8.138752835 5.181373099 3.824747424 5.423581239
6.296398214 4.74405298  9.837741509 7.714662742
5.177385358 1.241610571 5.028388255 4.491743107
4.251033792 8.261317658 7.415111851 6.430957844
4.720645386 1.0721718   2.187147908 2.815078796
1.941872069 1.108191586 6.24591771  3.994268819
4.220448549 9.931055481 4.435085917 5.233711923
9.398867623 2.799376317 7.982096264 7.612485261
4.971020963 1.578519218 0.462459906 2.248086465

I generated the Y values with 1/3 x[0] + 1/6 x[1] + 1/2 x[2] + E where E is a random number in [-0.1..+0.1]. If the algorithm is working correctly we'd expect to get roughly a = 1/3 and b = 1/6 from this result. Indeed we get a = .3472 and b = .1845.

OP has now said that his actual data are larger than 3-vectors. This method generalizes without much trouble. If the vectors are of length n, then you get an n-1 x n-1 system to solve.

Upvotes: 4

Related Questions