Facemashing the Proposals

The Social Network was pretty misleading. The Elo rating system algorithm used in Facemash has been around for years, and is not the best predictive algorithm in existence. It basically performs a weighted average of results, which, iteratively, it updates scores based on performance vs. expectation.

Consider 2 people, A and B, with a current rating Ra and Rb. Then at each update step, you update each rating as follows:

Ra' = Ra + K(Sa - Ea)

where Ra' is the new score, K is just a scaling factor for change sensitivity, and Sa are the actual "points" made in the competition, and Ea are the expected number of points. Put simply, if we have Ra currently scoring better than Rb, then in a binary point system like Facemash would have, Ea = 1. So if Ra wins, its score doesn't change. If it loses, so that Sa is 0 or -1, then Ra is reduced by K or 2K. Same thing for Rb.

The way we could implement the same kind of thing in class is simply to rank the worst proposal with a "0", and add one to every higher rated proposal, so the highest gets a 27 or 28, depending if you're allowed to rank your own proposal. We don't even need to average -- we could just take the raw scores at the end.

Some tests between the systems indicate that the overall scores will be different, but the order of ranking remains the same. I might not be testing it thoroughly enough. I was using http://elo.divergentinformatics.com/ to test the elo system.

We might try a blocky distribution, where the 29 proposals get put into 5 equally-rated bins, and then add all those scores together. Ties for values that would have simply been "close" the other way can then be resolved by a popular vote. This stops a proposal from losing only by a narrow margin.