In 2015 and 2016, the Hugo Awards suffered a serious crisis. At nomination stage, several hundred activists collectively voted for slates of candidates proposed by various individuals. The slates completely dominated several categories, and included a number of candidates which were not to the wider taste of fandom, with the result that a record five No Awards were made in 2015 and another two in 2016. A number of reforms were passed in 2015 and ratified in 2016 to make it more difficult for entire ballot categories to be dominated in this way in future. This post is about the effects of those reforms and of a couple more that have been put forward to the WSFS Business Meeting this year.
(There were a number of other issues, of course, and I was one of many people who were pretty vocal about them at the time. But this post simply addresses the mechanics of preventing a small minority of voters from completely capturing the Hugo ballot.)
Just to make it crystal clear: I am the administrator of this year's Hugo Awards, but I write here in a personal capacity only and I have exactly as much right to decide these matters as any other attending member of this year's Worldcon.
5 and 6
Two key reforms to the nominations process came into effect in 2017. The first was simply called "5 and 6". Previously voters had nominated up to five candidates in each Hugo category, and the final ballot consisted of the top five after all votes were tallied (or more if there was a tie for fifth place, or fewer if the last one or two failed to get more than 5% of the total vote). From 2017 on, voters still nominate up to five works, but it is the top six who form the final ballot. The result is that this year's Hugo ballot is the longest ever, with 108 finalists in 18 categories. (There are 17 normally, but we added one as explained previously.)
While it is straightforward to get a couple of hundred supporters to vote for a five-item slate and take five places, it's much more difficult to marshall those supporters to take all six places if they can only vote for five. (It's not impossible, as the case of Eastern Rumelia in 1879 shows, but it is much more difficult.) Since nobody tried that this year, the proposition remains untested.
The visible effect has been that 20% more people were delighted to discover that they or their works were Hugo finalists, and conscientious voters had 20% more to read in the usual categories. My sense is that finalists and voters are happy enough with those consequences. The 5 and 6 provision is on this year's agenda as amendment C.12, because the Business Meeting has the power to suspend it for the following year. I don't see any reason to do so.
EPH
The second and much bigger reform brought in for the 2017 Hugos is the E Pluribus Hugo tallying system for the nominations phase. Using this system, the value of each voter's nomination ballot in a given category is split between the candidates. If you nominate five, your vote is worth 0.2 to each of them; if you nominate four, it's worth 0.25. The points are tallied for all candidates, and those with the fewest removed, and the value of their supporters' votes recalculated, until only six remain. (So if you nominate five candidates, and you are the only person who has nominated one of them, it will be removed from the calculation and the other four will get 0.25 of your vote rather than 0.2.)
I did a lot of calculation of the effect of EPH on votes as they had been cast in previous years, and found that it had two principal effects. First, it does indeed make it much more difficult for a slate to get all of the places on a particular ballot. Let's take one egregious case, the Best Novella ballot in 2015, where the slates got all five places (and voters chose No Award). Under EPH, 84 nominating votes would have got the slates a place on the ballot, but they would have needed 191 to get two places, 310 to get three places, 416 to get four places, and 621 to get all five places (and 750 to get all six if the 5 and 6 rule had been in play and they had perfect discipline). In fact the most popular slate candidates in that category had more than 310 votes, but less than 416, so three out of five ballot places in this category would have been taken by the slates even with EPH. That would still have meant two non-slate candidates (three if the 5 and 6 rule had been in place); EPH would have opened up the ballot considerably, and No Award would not have won. (Editing note: I recalculated the numbers here, having got it wrong first time.)
For whatever reason, we did not have the same attack on the Hugo process by slates this year as we had in 2015 and 2016. There was no visible attempt to capture entire categories. I tend to think that this change of behaviour was in part due to the prospect that EPH would achieve what it set out to achieve - to drastically increase the opportunity cost of stacking entire sections of the Hugo ballot. Of course, there will have been other factors as well, but I'm not going to speculate about that here.
Second, however, EPH makes it comparatively much easier for a single candidate to get onto the final ballot with "bullet" votes, ie voters nominating just that candidate and no others. This is because most finalists are supported by a broad coalition of voters who may have supported other finalists as well - so in general you expect to finish with a point total of about 60%-65% of your number of votes (I make the average this year 64.6%). Therefore a candidate whose voters have supported them alone, and nobody else, starts with a major advantage. In fact the one attempt at a slate this year followed this strategy, with a list of single candidates (in a couple of cases two) for various categories, some of which were successful and some of which were not.
Is this what we want? I would argue that it is. I think it's a fine thing for enthusiastic fans whose own particular corner of fandom is a bit obscure to face a lower threshold for putting their pet cause on the Hugo ballot. They will then of course be subject to the judgement of other fans once the ballot is published, and if their choice is not to general taste it won't win (and if it's way out of whack with general taste, or if they have behaved like assholes in the process of getting it on the ballot, it may even be beaten by No Award). However, in my view the Hugo process is best served by a diverse ballot, and this somewhat unintended consequence is a plus for EPH, which is on the Business Meeting agenda as item C.5 in case anyone wishes to suspend it for next year.
EPH+
This is a minor wrinkle to EPH. Rather than your vote being divided by 2 if you nominate two candidates, by 3 if you nominate three, by 4 if you nominate four and by 5 if you nominate five, the divisors are 1, 3, 5, 7 and 9. From the election nerd point of view this is the difference between d'Hondt and Sainte-Laguë, and the latter is generally preferable among connoisseurs. (For Americans it's Thomas Jefferson vs Daniel Webster.) The practical effect is that it will make it a little easier for a small organised group to get one candidate on the final ballot in a given category, but much more difficult for them to get two. I think this is preferable, and anyway I'll take Sainte-Laguë over d'Hondt any day.
To be specific, if EPH+ had applied to the 2015 Best Novella ballot, as mentioned earlier. the slates would still have needed 84 nominating votes to be sure of a single spot, but would have needed 286 votes for a second slot, 516 for a third, 729 to take four out of five and 1126 to take all five slots. In real life, 125 disciplined voters would have been enough to completely capture the category (in fact they had more like 300 in two different factions of about 150).
It's amendment C.6 on the Business Meeting agenda and I will vote for it.
Three Stage Voting
I was one of a number of people last year who put our names to another proposal, Three Stage Voting, which is on the agenda as amendment C.4. This would introduce an extra voting stage. After nominations are made, and the top fifteen candidates identified, Hugo voters (members of that year's Worldcon) would vote on whether or not to accept the top fifteen as acceptable finalists. Voters would choose "Reject"; "Accept; or "Abstain". Those rejected by at least 60% of the combined total of “Accept” and “Reject” votes, if and only if the number of “Reject” votes is at least the higher of 600 or 20% of the number of eligible voters, would be barred from the final ballot, which would have the top five (or six) vote-getters from the nominations tally minus any that were rejected by the new process. This was passed by the 2016 Business Meeting and must now be ratified by the 2017 Business meeting to come into effect for next year.
I signed this last year, but I no longer support it, for the following reasons.
First, it was not clear a year ago that EPH would pass, or that it would work to deter slates if it did pass. Self-evidently, it did pass; and I think the evidence is very clear that the slate problems of the last two years have receded, and also that EPH provides a sufficent bulwark against anyone trying the same trick in future years (and EPH+ even more so). I do not think that the reputation of the Hugos is served by constant large-scale revision of the rules, especially against a threat that seems to have gone away.
Second, I think it's become clear that fans in general feel that Hugo ballot categories with manifestly unsuitable candidates are a disappointing but not unacceptable part of the process. In previous years I took the view that I would personally No-Award not only an all-slate ballot category, but also a ballot category with only one non-slate nominee, on the basis that there is in that case no real contest. It's clear that the Hugo voters as a whole disagreed with me. My sense is that while fannish disapproval of slates runs very deep, it is not so strong as to require an extra step in the process to eliminate slate candidates from the final ballots.
Third, I don't really like what this would do to the politics of the process. As things stand, if the nominations cycle delivers us some wildly inappropriate and unpopular finalists (as it has always done), voters can (and do) rank them below No Award at voting stage. The introduction of a new qualifying stage opens up the way for people to actively campaign on whether or not a given long-list candidate is worthy of the attention and respect of fandom. Fundamentally, this is not a very nice thing to do, and it might easily be turned onto other minorities than its intended targets. We never were in the business of determining that anyone is a wrongfan having wrongfun, and we should not start now.
Fourth, purely on the technicalities, we have no evidence whatsoever that 60% of the combined total of “Accept” and “Reject” votes is the right threshold, or that the higher of 600 or 20% of the number of eligible voters is the right threshold. These are numbers pulled out of thin air. If we get it wrong, we'll be stuck in another two-year cycle of revisions to correct a flawed proposal which we will have hardwired into the constitution.
My withdrawal of my support doesn't make any difference to the presence of the Three-Stage Voting proposal on the agenda - having been passed last year, it's now the property of the Business Meeting as a whole. But I will vote against it and I hope that others will too.
December is good enough
My name is on another constitutional amendment, C.2, which moves the deadline for people to register in order to nominate for the Hugos from 31 January of the year of the relevant Worldcon to the end of the previous year. The merging of voter data from three conventions is the single most tedious task associated with Hugo administration, and this will enable the process to get off to a clean start. (Incidentally, 31 January is the only date specified in the constitution, for anything.) It's a minor tweak but I hope people will support it.
Two Years Are Enough
Last for today, another proposed amendment, C.1, would remove nominating rights from the members of the following year's Worldcon (though members of the previous year's Worldcon would be unaffected). I'm not convinced by this. True, the current system does impose an extra burden on the administrator; but really we're talking about a few hundred extra voters to include in a process that already involves many thousands. I like the broadening out of nominating rights to the future, and it's not clear to me that the system is being exploited (or even capable of being exploited). So I'm not inclined to support this one.
A few more technical resolutions are in front of the WSFS Business Meeting, but that's quite enough for now.
(There were a number of other issues, of course, and I was one of many people who were pretty vocal about them at the time. But this post simply addresses the mechanics of preventing a small minority of voters from completely capturing the Hugo ballot.)
Just to make it crystal clear: I am the administrator of this year's Hugo Awards, but I write here in a personal capacity only and I have exactly as much right to decide these matters as any other attending member of this year's Worldcon.
5 and 6
Two key reforms to the nominations process came into effect in 2017. The first was simply called "5 and 6". Previously voters had nominated up to five candidates in each Hugo category, and the final ballot consisted of the top five after all votes were tallied (or more if there was a tie for fifth place, or fewer if the last one or two failed to get more than 5% of the total vote). From 2017 on, voters still nominate up to five works, but it is the top six who form the final ballot. The result is that this year's Hugo ballot is the longest ever, with 108 finalists in 18 categories. (There are 17 normally, but we added one as explained previously.)
While it is straightforward to get a couple of hundred supporters to vote for a five-item slate and take five places, it's much more difficult to marshall those supporters to take all six places if they can only vote for five. (It's not impossible, as the case of Eastern Rumelia in 1879 shows, but it is much more difficult.) Since nobody tried that this year, the proposition remains untested.
The visible effect has been that 20% more people were delighted to discover that they or their works were Hugo finalists, and conscientious voters had 20% more to read in the usual categories. My sense is that finalists and voters are happy enough with those consequences. The 5 and 6 provision is on this year's agenda as amendment C.12, because the Business Meeting has the power to suspend it for the following year. I don't see any reason to do so.
EPH
The second and much bigger reform brought in for the 2017 Hugos is the E Pluribus Hugo tallying system for the nominations phase. Using this system, the value of each voter's nomination ballot in a given category is split between the candidates. If you nominate five, your vote is worth 0.2 to each of them; if you nominate four, it's worth 0.25. The points are tallied for all candidates, and those with the fewest removed, and the value of their supporters' votes recalculated, until only six remain. (So if you nominate five candidates, and you are the only person who has nominated one of them, it will be removed from the calculation and the other four will get 0.25 of your vote rather than 0.2.)
I did a lot of calculation of the effect of EPH on votes as they had been cast in previous years, and found that it had two principal effects. First, it does indeed make it much more difficult for a slate to get all of the places on a particular ballot. Let's take one egregious case, the Best Novella ballot in 2015, where the slates got all five places (and voters chose No Award). Under EPH, 84 nominating votes would have got the slates a place on the ballot, but they would have needed 191 to get two places, 310 to get three places, 416 to get four places, and 621 to get all five places (and 750 to get all six if the 5 and 6 rule had been in play and they had perfect discipline). In fact the most popular slate candidates in that category had more than 310 votes, but less than 416, so three out of five ballot places in this category would have been taken by the slates even with EPH. That would still have meant two non-slate candidates (three if the 5 and 6 rule had been in place); EPH would have opened up the ballot considerably, and No Award would not have won. (Editing note: I recalculated the numbers here, having got it wrong first time.)
For whatever reason, we did not have the same attack on the Hugo process by slates this year as we had in 2015 and 2016. There was no visible attempt to capture entire categories. I tend to think that this change of behaviour was in part due to the prospect that EPH would achieve what it set out to achieve - to drastically increase the opportunity cost of stacking entire sections of the Hugo ballot. Of course, there will have been other factors as well, but I'm not going to speculate about that here.
Second, however, EPH makes it comparatively much easier for a single candidate to get onto the final ballot with "bullet" votes, ie voters nominating just that candidate and no others. This is because most finalists are supported by a broad coalition of voters who may have supported other finalists as well - so in general you expect to finish with a point total of about 60%-65% of your number of votes (I make the average this year 64.6%). Therefore a candidate whose voters have supported them alone, and nobody else, starts with a major advantage. In fact the one attempt at a slate this year followed this strategy, with a list of single candidates (in a couple of cases two) for various categories, some of which were successful and some of which were not.
Is this what we want? I would argue that it is. I think it's a fine thing for enthusiastic fans whose own particular corner of fandom is a bit obscure to face a lower threshold for putting their pet cause on the Hugo ballot. They will then of course be subject to the judgement of other fans once the ballot is published, and if their choice is not to general taste it won't win (and if it's way out of whack with general taste, or if they have behaved like assholes in the process of getting it on the ballot, it may even be beaten by No Award). However, in my view the Hugo process is best served by a diverse ballot, and this somewhat unintended consequence is a plus for EPH, which is on the Business Meeting agenda as item C.5 in case anyone wishes to suspend it for next year.
EPH+
This is a minor wrinkle to EPH. Rather than your vote being divided by 2 if you nominate two candidates, by 3 if you nominate three, by 4 if you nominate four and by 5 if you nominate five, the divisors are 1, 3, 5, 7 and 9. From the election nerd point of view this is the difference between d'Hondt and Sainte-Laguë, and the latter is generally preferable among connoisseurs. (For Americans it's Thomas Jefferson vs Daniel Webster.) The practical effect is that it will make it a little easier for a small organised group to get one candidate on the final ballot in a given category, but much more difficult for them to get two. I think this is preferable, and anyway I'll take Sainte-Laguë over d'Hondt any day.
To be specific, if EPH+ had applied to the 2015 Best Novella ballot, as mentioned earlier. the slates would still have needed 84 nominating votes to be sure of a single spot, but would have needed 286 votes for a second slot, 516 for a third, 729 to take four out of five and 1126 to take all five slots. In real life, 125 disciplined voters would have been enough to completely capture the category (in fact they had more like 300 in two different factions of about 150).
It's amendment C.6 on the Business Meeting agenda and I will vote for it.
Three Stage Voting
I was one of a number of people last year who put our names to another proposal, Three Stage Voting, which is on the agenda as amendment C.4. This would introduce an extra voting stage. After nominations are made, and the top fifteen candidates identified, Hugo voters (members of that year's Worldcon) would vote on whether or not to accept the top fifteen as acceptable finalists. Voters would choose "Reject"; "Accept; or "Abstain". Those rejected by at least 60% of the combined total of “Accept” and “Reject” votes, if and only if the number of “Reject” votes is at least the higher of 600 or 20% of the number of eligible voters, would be barred from the final ballot, which would have the top five (or six) vote-getters from the nominations tally minus any that were rejected by the new process. This was passed by the 2016 Business Meeting and must now be ratified by the 2017 Business meeting to come into effect for next year.
I signed this last year, but I no longer support it, for the following reasons.
First, it was not clear a year ago that EPH would pass, or that it would work to deter slates if it did pass. Self-evidently, it did pass; and I think the evidence is very clear that the slate problems of the last two years have receded, and also that EPH provides a sufficent bulwark against anyone trying the same trick in future years (and EPH+ even more so). I do not think that the reputation of the Hugos is served by constant large-scale revision of the rules, especially against a threat that seems to have gone away.
Second, I think it's become clear that fans in general feel that Hugo ballot categories with manifestly unsuitable candidates are a disappointing but not unacceptable part of the process. In previous years I took the view that I would personally No-Award not only an all-slate ballot category, but also a ballot category with only one non-slate nominee, on the basis that there is in that case no real contest. It's clear that the Hugo voters as a whole disagreed with me. My sense is that while fannish disapproval of slates runs very deep, it is not so strong as to require an extra step in the process to eliminate slate candidates from the final ballots.
Third, I don't really like what this would do to the politics of the process. As things stand, if the nominations cycle delivers us some wildly inappropriate and unpopular finalists (as it has always done), voters can (and do) rank them below No Award at voting stage. The introduction of a new qualifying stage opens up the way for people to actively campaign on whether or not a given long-list candidate is worthy of the attention and respect of fandom. Fundamentally, this is not a very nice thing to do, and it might easily be turned onto other minorities than its intended targets. We never were in the business of determining that anyone is a wrongfan having wrongfun, and we should not start now.
Fourth, purely on the technicalities, we have no evidence whatsoever that 60% of the combined total of “Accept” and “Reject” votes is the right threshold, or that the higher of 600 or 20% of the number of eligible voters is the right threshold. These are numbers pulled out of thin air. If we get it wrong, we'll be stuck in another two-year cycle of revisions to correct a flawed proposal which we will have hardwired into the constitution.
My withdrawal of my support doesn't make any difference to the presence of the Three-Stage Voting proposal on the agenda - having been passed last year, it's now the property of the Business Meeting as a whole. But I will vote against it and I hope that others will too.
December is good enough
My name is on another constitutional amendment, C.2, which moves the deadline for people to register in order to nominate for the Hugos from 31 January of the year of the relevant Worldcon to the end of the previous year. The merging of voter data from three conventions is the single most tedious task associated with Hugo administration, and this will enable the process to get off to a clean start. (Incidentally, 31 January is the only date specified in the constitution, for anything.) It's a minor tweak but I hope people will support it.
Two Years Are Enough
Last for today, another proposed amendment, C.1, would remove nominating rights from the members of the following year's Worldcon (though members of the previous year's Worldcon would be unaffected). I'm not convinced by this. True, the current system does impose an extra burden on the administrator; but really we're talking about a few hundred extra voters to include in a process that already involves many thousands. I like the broadening out of nominating rights to the future, and it's not clear to me that the system is being exploited (or even capable of being exploited). So I'm not inclined to support this one.
A few more technical resolutions are in front of the WSFS Business Meeting, but that's quite enough for now.
Comments
- 25 Jan 2023, 13:24O tempora! O mores!
- 24 Jan 2023, 10:34Hello! Your entry got to top-25 of the most popular entries in LiveJournal!
Learn more about LiveJournal Ratings in FAQ. - 8 Dec 2022, 12:44UK mailboxes aren't waterproof?! That seems like an odd design.
- 29 Oct 2022, 16:28Now I know that "psephologist" is a word.
- 9 Sep 2022, 11:19That would make things less awkward.
Designed by 
yoksel
Comments
(Note that because I'm a co-sponsor of 3SV, I don't have to be impartial about it, and will recuse myself from presiding over its ratification debate.)
To me the key point is that the immediate threat has receded, and we should not be making further drastic changes to fix last year's problem.
I share your concerns about 3SV. It solves the problem it's meant to, but it's also a huge change to the voting process, that can't help but change the feel and tenor of the award. And, it either leaves us less time with the shortlist, or less time to nominate -- and for myself, that's a serious, serious problem.
I mean, you *can* abstain from all the ones you haven't encountered, but there's still a strong impetus to read and rank some/all of the materials, especially the ones you suspect of being utter shit brigaded onto the ballot by literal neo-nazis whose main goal is sabotage.
(A hypothetical rule I'd actually like to see the results of, from someone with access to raw data: "any ballot including Ted Beale, works by Ted Beale, or anything from Ted Beale's 'Swastika Press' is a priori deemed to be in bad faith and all of its nominations are excluded from the counting process". Given THAT as a rule, I wonder what the final ballots would have looked like? I understand there's ALL KINDS of reasons that doesn't work as a general rule and would be a bad rule to pass, but I'm still curious to see what just trashing all the nominations from the vandals acting in bad faith would do to the results.)
Yes, for me this is key. The conscientious voter's interests are pretty important here.
I'm afraid I won't comment on your second point. Others have done the calculations for previous years, and will no doubt do so this year as well; I won't.
And I don't blame you in the slightest for not indulging my curiousity and OF COURSE you can't let raw nomination data out to internet randos. But I'm still curious to see that, even though I likely never will.
Edited at 2017-07-24 09:51 pm (UTC)
I've heard at least one person say that he quit taking the Hugo finalists seriously because of works like that appearing on it. He could understand us having a miss the first time someone did it, but he couldn't believe we let it happen two years in a row. He wasn't sympathetic when I explained the rules to him.
The nice thing about 3SV is that in most years, voters can just skip that phase. There's no need to read anything in the 15-item list other than the titles, unless someone raises serious issues.
Well, I have to say that there have often been some pretty poor works appearing on the Hugo final ballot, as I have recorded here and elsewhere over the years. I guess it's nice if more people are looking at the fine print than previously.
But 3SV makes this publicity problem more likely to recur, rather than less. It will be much easier to get a work into the top fifteen than the top six - roughly a third of the number of votes are needed. As soon as you have a long-list that members are voting on, the works and other candidates on it are "up for a Hugo" in the public mind.
I am baffled by the notion that we can somehow keep a long-list quiet. The fact that there is a controversial work on the "Hugo list" will get attention, and 3SV roughly makes it roughly three times easier for malefactors to bring that about.
When people look back at the awards, in general "Hugo Winner" gets most kudos and "Hugo Finalist" gets next most. It's hard to see someone being able to promote "making the long list and then being disqualified" as a success in marketing terms. NOW I'm not saying it's impossible because if you deal in "Alternative Facts" as Beale does, then he isn't bound by objective truth or reality*. But then again there's nothing we can do to stop someone presenting any perspective they like to their own readers, so there's not much point worrying about how we stop it. The best we can do is ensure that the language we use (e.g. "qualifier" or "candidate" rather than "semi-finalist" for the long list) and the way we publicise things ourselves, tries to make it harder for people to abuse the situation.
(As an aside: Kevin Standlee can attest to the number of people who, before we called them "Hugo Finalists", used to say online that they were a Hugo Nominee because a friend had told them they'd nominated them for the Award, i.e. they'd got one nomination... and it wasn't disingenuous. People unfamiliar with the Awards might assume it was juried or that on some other basis they really were a nominee).
But I get it that there are people who would not feel comfortable voting like that. Much as I'd love to be able to convince those people to see it my way, I don't think I or anyone else can. So I guess that if I don't want to make people feel guilty for not reading nearly 3 times as many works as they already do, I should not support 3SV.
Of course, I have a vested interest in EPH/EPH+ succeeding and continuing to be used. It's the first voting method I designed to actually be used in large-scale practice (though I hope not the last; I'd love to see GOLD voting used in British Columbia or the US, or 3-2-1 voting for single-winner elections anywhere, or 3RD for city councils). So from a purely self-interested perspective, this gives me another reason to oppose 3SV; without 3SV, EPH(+) is the only thing standing in the way of slate voting, and so is more likely to "stick".
Still, if I could get others to see 3SV as I do, I'd support it. But I can't, so I don't.
(This comment was originally marked as spam, when I included wiki dot electorama dot com links defining the various voting methods I mentioned. So I took out those links, but you can search that wiki if you're interested.)
Edited at 2017-07-26 08:38 pm (UTC)