On ranking merit: applying the page-rank algorithm to the electoral process Robert Spekkens Perimeter Institute for Theoretical Physics October 19, 2010 WICI seminar

The pagerank algorithm provides a way of ranking the members of a community by merit using the aggregate opinions of the community and without any prior ranking Call this the merit-rank algorithm This should be considered a module to be incorporated into broader systems for collective decision-making Ex: Appointment of the most meritorious members of a community to a particular set of offices â€˘ the most trustworthy to decision-makers â€˘ the most fair to jurors â€˘ the most expert to policy-makers

Outline • • • • • •

Shortcomings of current schemes How the merit-rank algorithm works Case study: Google’s search engine Case study: Citation networks Criticisms and Possible failure modes Beyond pagerank

Two schemes for identifying merit and their shortcomings By authority

- Requires a prior notion of who is best qualified to judge merit - Susceptible to corruption - doesnâ€™t scale well - Each authority has a short horizon of deep familiarity

By majority vote

-Popular opinion may be less reliable than that of a better-qualified minority (the pitfalls of rule by referendum) - Each voter has a short horizon of deep familiarity

Merit-rank can hope to avoid some of these shortcomings

How the merit-rank algorithm works

Pagerank as a slogan Important webpages are those that are linked to by other important webpages

Merit-rank as a slogan Meritorious individuals are those who are judged to have merit by other meritorious individuals

What kinds of merit will the algorithm work for? â€˘ Auto-indicating merit: An individual having merit is better able to assess merit in others Or equivalently, â€˘ Merit that is transitive: If Alice esteems Bob, then she would also esteem those who are esteemed by Bob.

From a majority vote system to the merit-rank algorithm

Ranking a slate of candidates

Ranking the entire community

One vote per person

One unit of voting power per person

Either: - split equally among targets - split arbitrarily among targets

1

1 1 1

1

1

1

1

1

1

1

1

1 1 1

1

1

1

1

1

1

1

0

0 0.33 0

0

0

3.83

4.0

0.33

0.5

1.0

Beyond majority vote: adding recursion Primitive version of merit-rank algorithm Iterate the calculation of individual ranks At step 0, everyone has equal merit-rank Alice’s merit-rank at step k = Bob’s merit-rank at step k-1 £ Fraction of Bob’s vote cast for Alice + Charlie’s merit-rank at step k-1 £ Fraction of Charlie’s vote cast for Alice +… If the calculation converges, final ranking = merit-rank

Merit-rank at step 0 1

1 1 1

1

1

1

1

1

1

1

Merit-rank at step 0 1

1 1 1

1

1

1

1

1

1

1

Merit-rank at step 1 0

0 0.33 0

0

0

3.83

4.0

0.33

0.5

1.0

Merit-rank at step 1 0

0 0.33 0

0

0

3.83

4.0

0.33

0.5

1.0

Merit-rank at step 2 0

0 1.33 0

0

0

2.66

0.17

1.33

0.33

3.83

Merit-rank at step 2 0

0 1.33 0

0

0

2.66

0.17

1.33

0.33

3.83

Merit-rank at step 3 0

0 0.06 0

0

0

0.67

0.06

5.22

0.67

2.66

Problems with primitive version People who earn but do not cast any votes are sinks for merit-rank 0

0

0.06 0

0

5.22

0.67

0.06

2.66

0.67

0

Solâ€™n: Uniformly distribute their vote

Problems with primitive version People who earn but cast no votes other than to themselves & Groups who earn but cast no votes other than to their own membership are sinks for merit-rank 0

0

0

0

0.67

0.06

0.06

5.22

2.66

0.67

0

Solâ€™n: Uniformly distribute a fraction of their vote

Problems with primitive version People who earn no votes are left with no voting power after the first step 0

0

0.06 0

0

5.22

0.67

0.06

2.66

0.67

0

Solâ€™n: Uniformly distribute a fraction of every vote

The merit-rank algorithm “taxing votes for the common good” Fraction X of vote uniformly distributed Fraction 1-X of vote distributed at voter’s discretion (uniform if unspecified) Standard choice: X=0.15

Merit-rank at step 0 1

1 1 1

1

1

1

1

1

1

1

Merit-rank at step 0 1

1 1 1

1

1

1

1

1

1

1

0

0 0.33 0

0

0

3.83

4.0

0.33

0.5

1.0

Merit-rank at step 1 0

0

0.15

+ 0.85

0.33 0

c

3.83

4.0

0 0.33

0

0

=

0

0.33 0

3.83

4.0

0 0.33

0

0.5

1.0

0.5

1.0

Merit-rank at step 1

Merit-rank at step 1

Merit-rank at step 2

0.15

+ 0.85 c

= …………………

final merit-rank

The algorithm always converges to a unique solution

Compare Weighted in-degree

final merit-rank

0

0

0.33 0

0

0

3.83

4.0

0.33

0.5

1.0

Case study: Googleâ€™s search engine

Sergey Brin and Lawrence Page (1998), "The anatomy of a largeâ€“scale hypertextual Web search engine," at http://www-db.stanford.edu/~backrub/google.html Webpages vote for one another by linking to one another Every webpage has a unit of voting power which is divided equally among the webpages to which it links

The random surfer picture with probability 0.85: follows a random link from the webpage she is currently on; or with probability 0.15: â€œteleportsâ€? to a completely random webpage. The long-time probability of ending up on a given webpage converges to a fixed value = the pagerank of that webpage

Googleâ€™s dominance over other search engines is perhaps the strongest recommendation of pagerank

Case study: Citation networks

Applying pagerank to a citation network P. Chen, H. Xie, S. Maslov, S. Redner, “Finding Scientific Gems with Google,” J.Informet. 1, 8-15 (2007) 353,268 nodes = all publications in the Physical Review family of journals from 1893–2003 3,110,839 links = all citations to Physical Review articles from other Physical Review articles A value X>0 is required to prevent all votes to sink to the oldest papers. Chosen value: X=0.5.

Strong correlation between # of citations and pagerank

However, outliers constitute exceptional papers

Benefits of merit-rank • Identifies a set of individuals that are more exceptional than the set that majority vote would identify • Completely democratic yet gives more weight to the opinions of the best qualified • Plays to our strengths by permitting us to assess only those we know well

Criticisms and Possible failure modes

Unrecognized merit Not necessarily a problem ďƒ The algorithm actually ranks people by their degree of vetted merit

Disenfranchisement

Proportional representation See: Xie, Yan and Maslov, â€œOptimal ranking in networks with community structureâ€?, arXiv:physics/0510107

Merit-ranking of groups If the algorithm works for individuals, it should work for groups

Voting based on ideology rather than on merit It is not necessarily a failure of the algorithm if an individual chooses to judge merit primarily in terms of ideology The network may partition into ideologically homogeneous groups Still, we have proportional representation for different ideologies

The celebrity failure mode A large imbalance in degree of recognition can trump considerations of merit Note: Merit-rank fares better than majority vote (consider the difficulty of gaming google) Possible fix: A weighting factor in proportion to the depth of a relationship

Are the relevant kinds of merit really auto-indicating? - the trustworthy can be naive - Experts can fall prey to groupthink Response: Unreliability of assessments of merit increases in proportion to superficiality of the relationship Possible fix: A weighting factor in proportion to the depth of a relationship

Few will understand the algorithm How can a community that doesnâ€™t understand an algorithm ever come to endorse it? Answer: Trust based on past performance )

No secret ballot The algorithm needs to know how everyone voted But how can one trust the institution that calculates the outcome without making public all of the information and thereby opening the door to bribery and coercion? Possible Fix: A cryptographic scheme

Beyond pagerank

The HITS algorithm – Hubs and Authorities Jon Kleinberg, "Authoritative sources in a hyperlinked environment“ Journal of the ACM 46 (5): 604–632 (1999).

Recall: Pagerank as a slogan Important webpages are those that are linked to by other important webpages HITS algorithm as a slogan Hubs are webpages that link to authorities, authorities are webpages that are linked to by hubs

The HITS algorithm – Hubs and Authorities The HITS algorithm returns two numbers for a webpage: • Authority value = the value of the content of the page • Hub value = the value of its links to other pages. Start with authority value = in-degree Hub value = out-degree Weight in-degree by Hub values Weight out-degree by Authority values Iterate.

Using a HITS-like algorithm to rank expertise Suppose some nodes have only incoming links (pure authorities) and others only outgoing links (pure hubs) Let Pure Hubs be the experts Let Pure Authorities be the beliefs of the experts Experts are the people who have the right beliefs. The right beliefs are the ones believed by the experts. Such a scheme can overcome the problem of unrecognized merit

Outlook Consider more complicated schemes (ex: negative votes, combination of HITS and pagerank, etc.)

Numerical tests of the various failure modes Find a good forum for a real-world trial - online gaming community? - facebook applications?