Large experimental studies comparing 4 different voting systems in France 2007

The French Presidential election of 2007 was conducted using plurality voting (21-22 April) with a subsequent top-2-runoff round (5-6 May). There were 12 candidates on ballot. Meanwhile, three different scientific studies of alternative voting systems were underway, with real voters at official polling places in France – these studies were all conducted by academics with government funding and cooperation.

The result is an unprecedently high-quality, head-to-head, fair comparison of these different voting systems in action in real life.

#ranked #ballots
   1      30
   2      67
   3     163
   4      95
   5      78
   6      37
   7      17
   8       9
   9       3
  10       9
  11      15
  12     370
total unspoiled  893 
mean #ranked 7.3

#approved #ballots  (# in 2002)
   0       120      36
   1       736     287
   2       905     569
   3       673     783
   4       264     492
   5        75     258
   6        23      94
   7        13      40
   8         1      16
   9         1       6
  10         1       1
 >10         1       5
total      2813    2587
mean#approved 2.33*  3.15

Notes on the table

The rank-order-ballot voters were told the voting system would be instant runoff voting. Hence, attempts to examine other rank-order voting systems such as Coombs elimination are (somewhat) less justified. The score-ballot voters were told the voting system would be median-based (Balinski & Laraki's "majority judgment") with no-score counting as zero. Hence, attempts to examine other score-based voting systems such as average-based range voting with no-score ignored, are (somewhat) less justified. However, median-based, average-based with no-score=zero, and average-based with no-score ignored, all returned the same finish order of their top 8 candidates. There were no Condorcet cycles hence there was an unambiguous finish order deducible from the rank-order ballots

with each finisher successively preferred over the next, pairwise, by a voter majority. This order was quite similar (although not exactly the same) as

but quite different from

The fact that Coombs and pseudo-Borda did not return ridiculous results is because these French rank-order voters evidently did not strategically/dishonestly demote frontrunners to last or nearly-last place. (At least, the fraction who did, was small.) But that kind of "strategic burial" behavior is very common (80-95%) in Australian rank-order voting, which would have been enough to make both Coombs and Borda yield completely ridiculous results. This difference between Australian and French voters is presumably due to either

1. The fact full ranking is compulsory in most of Australia,
2. The prevalence of party "how to vote" cards offering rank-order-advice from the Australian parties,
3. The greater experience of the Australians with rank-order voting.

It would have been ridiculous to try to use the Borda voting system because only 370 (38.5%) of the 960 rank-order ballots ranked all 12 candidates – Borda would have had an insane ≈60% spoilage rate. However, one can still deduce "pseudo-Borda" scores for each candidate by using the theorem that a candidate's Borda score is the sum of her pairwise vote-totals. If we employ the pairwise-table below and assume that the voters who did not express "A</>B" opinions would have (if forced) expressed those opinions in the same ratios as the voters who did express an opinion about A vs. B, then we get the pseudo-Borda score for each candidate by summing her row-numbers.

Total approval was 232.5%, i.e. the average ballot approved 2.33 out of the 12 candidates. This figure comes from table 15.15 page 382 of Baujard & Igersheim's Handbook paper, and is based on an "extrapolation to all France" of their data correcting for sampling and participation biases. In 2007 the most common number to approve was 2 (done by 32.2% of all ballots), but in the preceding 2002 election it was 3 (by 30.3%). Note that even the approval-winner (Bayrou) got below 50% approval. (And in the preceding 2002 election, Chirac got the most approval, with 36.7%, which was even further below 50%.) In the ranked-ballot election, 41% of valid ballots ranked all 12 candidates (the most common type); and the second most common type were ballots ranking 3 candidates (18.2% of valid ballots); on average ballots ranked 7.3 candidates.

Baujard & Igersheim also asked their voters various questions about voting systems, and about 40% of them answered those questions. Of those respondents, 83.5% said "yes" to "does the principle of approval voting seem clear to you?" while 89.2% said yes for same question about range(0,1,2) voting. 75.1% opined that approval voting could be used for official (presidential, legislative & other) elections, versus 87.9% for range(0,1,2). Asked for opinions, 295 voters "sung the praises" of Approval and Range(0,1,2) – appreciating more expressivity and simplicity – versus 75 who disliked them.

This is among the first clear evidence (over 6σ confidence on both!) that score voting [more precisely, range(0,1,2)] actually is both more popular and comprehensible than approval voting!! It also is clear evidence that score and approval voting are "adoptible" and apparently would win by a massive margin if given as an option on a referendum. Unfortunately, these conclusions are attackable because only 40% of the pollees answered these optional questions and hence there may have been self-selection bias.

In contrast, an October 2010 nationwide professional telephone poll of 1202 random Australian adults found that they prefer plain-plurality voting versus the preferential (instant runoff) system they presently use to elect their House. If forced to choose one, they'd choose to abandon IRV – the poll's result was 57% to 37% (with 5% don't know/refuse). Also Britain conducted a binding referendum on 5 May 2011 asking voters to decide whether the UK should switch from plurality to IRV voting, resulting in a massive landslide victory (68% to 32% of the 19.3 million votes) for "stay with plurality." Another big referendum defeat for IRV was on 12 May 2009 in British Columbia, Canada, where "switch to IRV" got only 39.09% of the 1.65 million votes.

Britain: Turnout=42%; the total score was IRV=6152607(32.1%), plurality=13013123(67.9%), spoiled ballots=113292(0.59%).
British Columbia: Turnout=55%; the total score was IRV=623420(39.1%), plurality=971350(60.9%), spoiled ballots=56369(3.4%).
Australian poll question 9 wording: Currently, elections for the Federal House of Representatives, or lower house, use a preferential voting system. This is where voters indicate an order of preferences for all candidates, and these preferences are taken into account when deciding which candidate wins. (PAUSE). An alternative system would be "first past the post", where voters only vote for one candidate and the candidate with the most votes wins. Would you personally prefer...? READ OUT [PROG NOTE: - SINGLE RESPONSE - RANDOMISE 1-2, THEN 3 LAST]

1. A preferential system
2. A first past the post system
3. (DO NOT READ) Neither \ don't know

There also were nationwide pre-election polls by a number of polling agencies agreeing Bayrou would have defeated every rival in a head to head "Bayrou versus X" election:

According to a nationwide IPSOS poll ending 21 April, Bayrou would have beaten Sarkozy in a head-to-head (i.e. potential second-round) election by 52.5% to 47.5%, and this conclusion was true no matter what time the poll was taken from early March until late April. According to a CSA poll ending 20 April, a head-to-head Sarkozy v. Royal matchup – which in fact happened and Sarkozy won – would be a tossup (50%-50%), while the IFOP poll (also ending 20 April) gave a 51-49 edge to Sarkozy. The IFOP poll also indicated Bayrou would have beaten Sarkozy 55-45 head to head, and Bayrou would have beaten Royal 58-42 head-to-head. At least 9 other polls also examined a possible Bayrou-Sarkozy runoff and all also concluded Bayrou would win it.

IRV paradoxes that occurred in this election:

1. Royal was a "spoiler." That is, with IRV as the voting method the winner was Sarkozy, but if Royal had dropped out of the race, then the IRV winner would have been Bayrou.
2. Was this election "nonmonotonic"? Almost certainly. According to Farvaque, Jayet & Ragot's table 3, the final three candidates in IRV were Bayrou, Royal, and Sarkozy. Bayrou was eliminated with 27.4% of the (still active) ballots versus Royal's 32% and Sarkozy's 40.5%. Then, in the final IRV round, Sarkozy beat Royal 54.15% to 45.85%. Note that Royal got a 13.85% boost (from 32% to 45.85%) due to vote-transfers from Bayrou, whereas Sarkozy received a slightly smaller transfer of 13.65%. This small advantage in transfers was not enough for Royal to win.
       Assume ≥4.61% of the total number of voters ranked the "big three" in the order Royal>Sarkozy>Bayrou. (Because 4.61 is a small percentage, this assumption seems extremely likely to be true; but I cannot be certain of it without access to Farvaque, Jayet & Ragot's raw data.) Now considr the effect of such a 4.61% switching their vote from Royal to Sarkozy. In that case, Royal would have been eliminated. The final round would then have been Bayrou versus Sarkozy. According to the head-to-head pairwise data table, Bayrou would have beaten Sarkozy in this final round 52.0-48.0 (and those promotions of Sarkozy to top in those 4.61% would not have changed this at all). That's non-monotonicity: if about 1.8 million "Royal>Sakrozy>Bayrou" voters had switched their vote from Royal to the winner Sarkozy, that would have made Sarkozy lose!
3. Another paradox IRV suffered in this election was a no-show paradox. Suppose that 1.8 million Royal>Bayrou>Sarkozy voters – who, we reiterate, ranked Sarkozy dead last – had simply "not shown up" i.e. had chosen not to thus-vote against Sarkozy. That would have made Bayrou win. We repeat: the act of voting Sarkozy dead last, made him win.
       Well, not quite. Actually, this paradox does not quite work as stated, but it does work in combination with the above non-monotonicity paradox, e.g. erase 1.4 million Royal>Bayrou>Sarkozy voters ("no show") plus make 0.4 million Royal>Sarkozy>Bayrou voters switch to Sarkozy>Royal>Bayrou ("non-monotonicity") – that combination paradoxically causes Sarkozy to lose.
   

   Again, this conclusion is attackable since our argument only works if ≥1.8 million Royal>Bayrou>Sarkozy voters actually existed. They probably did, but I reiterate am not 100% sure they did since I do not have access to Farvaque, Jayet & Ragot's raw data.

In this IRV election, approximately 4% of the IRV voters with unspoiled ballots failed to affect the final Sarkozy vs. Royal round (because, e.g, they had failed to rank either) – plus 7.0% of the ballots were spoiled, so in all about 11% of the IRV voters failed to have any impact on the final Sarkozy vs Royal round. That considerably exceeds the Sarkozy-vs-Royal victory margin. In the official election (2-round plurality voting) however, the spoilage rates were well below the S-vs-R victory margin. This means that the official 2-round noninstant runoff, in this case, had a greater claim to legitimacy than an instant runoff would have had (even though they both delivered the same winner, Sarkozy).

Conclusions

These studies are very exciting. They look like they show the following:

1. Score voting is adoptable and will win if put to a referendum.
2. Score{0,1,2} voting is more clear and more popular than both approval and instant runoff voting.
3. The studies said approval, score{0,1,2}, median-based score{0,1,2,3,4,5}, average-based score{0,1,2,3,4,5}, Coombs, pseudo-Borda, and Condorcet – and pre-election head-to-head polls also indicated – that Bayrou was the candidate France wanted, and would have defeated every rival head-to-head. But the instant-runoff, non-instant top-2-runoff, and plain plurality election systems would have elected (and did elect) Sarkozy. This was a failure of democracy which France in retrospect (based on later polls and elections) appears to have regretted.
4. Rank-order ballots exhibit the largest (7.0%) spoilage rates (which would be an enormous 60% if used for Borda). Approval-style ballots exhibit the smallest (0.8%) spoilage rates. Score voting ballots have 1.1% spoilage rate, while the official spoilage rates using plurality-style ballots were 1.44 and 4.20%.

Journal/Book references

Antoinette Baujard & Herrade Igersheim: Framed-field experiment on Approval voting and Evaluation voting. Some teachings to reform the French Presidential Electoral System, in B. Grofman, A. Laurent and B. Dolez (eds), In situ and Laboratory Experiments on Electoral Law Reform: French Presidential Elections, Heidelberg: Springer, 2010.

Antoinette Baujard & Herrade Igersheim: Framed field experiments on approval voting, lessons from the 2002 and 2007 French presidential elections, ch.15 (pp.357-395) in Handbook on Approval Voting (ed. J.Laslier & M.Remzi Sanver) Springer 2010.

Michel Balinski & Rida Laraki: A theory of measuring, electing, and ranking, Proc. Nat'l Acad. Sci. USA 104,21 (22 May 2007) 8720-8725.

Etienne Farvaque, Hubert Jayet, Lionel Ragot: Quel mode de scrutin pour quel vainqueur? Une experience sur le vote preferentiel transferable, Revue d'Economie Politique 119,2 (March-April 2009) 221-246.

Etienne Farvaque, Hubert Jayet, Lionel Ragot: A 'winner' under any voting rule? An experiment on the single transferable vote, English-language pdf, Sorbonne "Documents de travail du Centre d'Economie" #09067.

Messed-up Elections thru history

Range & other voting systems: elections over last ≈50 years

Return to main page