Log in

No account? Create an account

Previous Entry | Next Entry

E Pluribus Hugo, revisited

I've spent more spare time than is healthy over the last few days musing on the proposed new system for counting Hugo nominations, designated E Pluribus Hugo (henceforth EPH) by its designers (to whom detailed observations should be directed here). I am in sympathy with its intent, which is to prevent any group - whoever that group may be - from absolutely excluding nominees from having the chance to be considered for the Hugo Award. I think that the proposal as it currently sits achieves that aim, but at a cost of making it too easy for a group which is otherwise utterly unconnected with Hugo voters to get a single work onto the ballot by "bullet votes" (ie votes for their candidate[s] and no other). I explore this problem below, using data from the 1984 Hugo nomination ballots, and propose a partial solution, which is to use square roots as divisors when weighting nomination votes.


I'm tremendously grateful to Paul Evans for providing me with the 1984 data he described here. Having spent a couple of evenings crunching figures, I now feel huge sympathy and admiration for the Hugo administrators trying to make sense of the variant titles and spelling submitted by voters. Administering what are essentially thousands of write-in ballots is not exactly straightforward, and I am not sure that I would have the patience to do so in an RL setting myself. Not surprisingly, my tallies vary a bit from Paul's. He has taken more time over it, so his numbers are probably right.

I've picked three different ballot categories from 1984 to analyse mainly because they were relatively easy to process, with less name and category confusion than some of the other options would have presented.

First, Best Fan Writer - in some ways the easiest, because fewest nominations were submitted in this category (481 nominations of 174 candidates by 181 voters). My figures differ from Paul Evans' totals - he found a couple more votes for some of the candidates than I did - but it doesn't make a lot of difference to the story. This is a case where EPH clearly works, and a slate candidate with bullet votes would have had to get enough support to win under the old system as well.

The top seven candidates by nominations were:

56 Mike Glyer
36 Richard Geis
33 Dave Langford
28 Arthur D. Hlavaty
18 Teresa Nielsen Hayden
12 Ted White
10 Claire Anderson

A nice big gap between the fifth and sixth placed candidates, and indeed between the fourth and fifth. Under EPH, the points for each candidate on the final ballot would have been:

Mike Glyer (56 nominations): 28 + 21/2 + 4/3 + 3/4 = 40.583
Richard Geis (36 nominations): 16 + 13/2 + 4/3 + 3/4 = 24.583
Arthur D. Hlavaty (28 nominations): 14 + 7/2 + 4/3 + 3/4 = 19.583
Dave Langford (33 nominations): 6 + 20/2 + 4/3 + 3/ = 18.083
Teresa Nielsen Hayden (18 nominations): 7 + 9/2 + 2/3 = 12.167

To knock Teresa Nielsen Hayden off the ballot, a single slate candidate would have needed to also have 18 nominations which were bullet votes (or at least shared less with other candidates than hers), which would have beaten her 12.167 EPH points. I must say I'm not completely content with this; precisely because TNH had more voters who also supported other popular candidates, she is in a disadvantage in a head-to-head against a more sectarian candidate. But I think any variation of a single divisible vote system delivers this result, and what it basically means is that the last place may go to a representative of a small minority. The question is, how small?

I looked at Best Non-Fiction next because there was a more even spread of candidates at the lower end. 206 voters made nominated 86 different works here. The top 9 were:

45 Dream Makers, volume ii
43 The High Kings
29 The Fantastic Art of Rowena
26 Staying Alive: A Writer's Guide
19 The Encyclopedia of Science Fiction and Fantasy, vol iii
16 Worlds Beyond: the Art of Chesley Bonestell
11 Amber Dreams
11 The SF Book of Lists
11 Uranian Worlds

If we apply the EPH system, it doesn't change much in ranking:

Dream Makers, volume ii (45 nominations): 18 + 13/2 + 9/3 + 4/4 + 1/5 = 28.7 points
The High Kings (43 nominations): 17 + 14/2 + 8/3 + 3/4 + 1/5 = 27.617 points
The Fantastic Art of Rowena (29 nominations): 12 + 8/2 + 4/3 + 4/4 + 1/5 = 18.533 points
Staying Alive: A Writer's Guide (26 nominations): 9 + 6/2 + 8/3 + 2/4 + 1/5 = 15.367 points
The Encyclopedia of Science Fiction and Fantasy, vol iii (19 nominations): 4 + 7/2 + 4/3 + 3/4 + 1/5 = 9.783 points

But because the successful candidates shared a relatively large number of supporters, a single slated candidate with 16 bullet votes would have got onto the ballot, and would have knocked off the actual winner, vol 3 of The Encyclopedia of Science Fiction and Fantasy.

41 slate voters, ie fewer than voted for either of the two front-runners in the category, who voted for two candidates, and only those two, could have knocked out Staying Alive out as well.

The most striking results I've found are for the 1984 John W. Campbell Award, which had a very lopsided vote distribution - one front runner far ahead of the field, and a number of candidates jostling around the edge of the ballot. 247 voters made 519 nominations here. The top 15 candidates, with their total nominations, were:

121 R.A. MacAvoy
19 Joseph H. Delaney
19 Joel Rosenberg
18 Timothy Zahn
18 Sheri S. Tepper
17 Lisa Goldstein
17 Warren Norwood
15 Barbara Hambly
12 Robin Wayne Bailey
12 Dan Simmons
10 P.C. Hodgell
10 Kim Stanley Robinson
10 Lucius Shepard
9 David Brin
9 John De Chancie

Some of these were not exactly new writers in 1984. Timothy Zahn had first published in 1980 and I guess was excluded from the final ballot for that reason. That brought in both Lisa Goldstein and Warren Norwood, tied on 17 nominations.

If we apply the EPH points system, the final six (Zahn having been excluded) end up in the following slightly different ranking:
R.A. MacAvoy (121 nominations): 89 + 23/2 + 9/3 = 103.5 points
Joel Rosenberg (19 nominations): 19 points
Warren Norwood (17 nominations): 9 + 7/3 + 1/4 = 11.583 points
Lisa Goldstein (17 nominations): 7 + 3/2 + 6/3 + 1/4 = 10.75 points
Joseph H. Delaney (19 nominations): 5 + 6/2 + 7/3 + 1/4 = 10.583 points
Sheri S. Tepper (18 nominations): 2 + 14/2 + 1/3 + 1/4 = 9.583 points

Sherri S Tepper would have been eliminated at this stage, although she had more nominations than two other surviving candidates (Warren Norwood and Lisa Goldstein). 165 of the original 247 nominating ballots are still in play. (Wow, MacAvoy made a pretty big splash, didn't she!)

After Sherri S Tepper's votes are redistributed, the point scores for what would have been the final ballot under EPH are as follows:

R.A. Macavoy (121 nominations): 103 + 10/2 + 8/3 = 110.667 points
Joel Rosenberg (19 nominations): 19 points
Warren Norwood (17 nominations): 9 + 1/2 + 7/3 = 11.833 points
Lisa Goldstein (17 nominations): 7 + 3/2 + 7/3 = 10.833 points
Joseph H. Delaney (19 nominations): 5 + 6/2 + 8/3 = 10.667 points

163 of the original 247 ballot papers would have remained in play.

A slate candidate with 11 nominators, none of whom supported any of the other surviving candidates, would have been ahead of Goldstein and Delany on points and would have made it to the final ballot (Goldstein would have lost due to having fewer nominations than Delaney). Such a candidate would have had fewer nominations than five excluded candidates - Lisa Goldstein, Sherri S Tepper, Barbara Hambly, Robin Wayne Bailey and Dan Simmons. These seems to me very unsatisfactory.

It's not irrelevant to note that Joel Rosenberg had 19 bullet votes at this stage (a few of whom had voted for other less popular candidates as well), and that these included ten voters with consecutive membership numbers who cast nominating votes identically for him in this category and for a novel called The Sleeping Dragon and a short story called "The Emigrant". You'll never guess who those works were by.

25 slate voters would have been able to get two works onto the shortlist here (if you eliminate Delaney, Norwood ends up with 12.5 points in third place). Again, that seems to me to be, simply, too low.

Conclusion and recommendation

I hope it's fairly clear that while EPH does, as advertised, make it very difficult for a small set of voters to dominate entire ballot categories, as has happened this year, it also actually lowers the barrier to a small detached group getting their first candidate onto the list. Of course, minorities should not have insurmountable barriers placed in front of them, but for my taste, EPH as presently constructed goes too far the other way. A slate candidate which had fewer nominations than ten more popular candidates could still have got onto the John W Campbell Award ballot. A slate candidate could have knocked the eventual RL winner off the Best Non-Fiction Work ballot despite getting fewer nominations. I don't think that's quite right.

My modest proposal is that the divisor for calculating points should not be the number of candidates supported by a voter, but the square root of that number. Square roots have a venerable place in political calculus, particularly in the apportionment of seats, both in the U.S. Congress and for enlightened commentators on the European Union. Basically, instead of weighting your vote by 0.5 if you still have two candidates left in the race, 0.333 if you have three, and 0.25 if you have four, your vote gets weighted instead as 0.707, 0.577 and 0.5. Looking at my test cases above, the new weighted points would be:

Best Fan Writer:
Mike Glyer5640.58346.659
Richard Geis3624.58329.002
Dave Langford2819.58322.759
Arthur D. Hlavaty3318.08323.952
Teresa Nielsen Hayden1812.16714.519

A slate candidate would still need 18 bullet votes to displace Teresa Nielsen Hayden from the ballot; no change.

Best Related Work:
Dream Makers, v24528.734.836
The High Kings4327.61733.466
The Fantastic Art of Rowena2918.53322.413
Staying Alive2615.36719.309
The Encyclopedia of Science Fiction and Fantasy, v3199.78313.206

A slate candidate would now need 19 bullet votes, rather than 16, to displace the Encyclopedia. That seems to me an improvement.

John W. Campbell Award:

Before we get to considering the slate, there's that matter of the fifth and sixth places to resolve. The top six now look like this:
R.A. MacAvoy121103.5110.460
Joel Rosenberg191919
Joseph H. Delaney 1910.58313.784
Warren Norwood 1711.58313.541
Lisa Goldstein 1710.7513.085
Sheri S. Tepper 189.58312.977

Lisa Goldstein would drop off due to having fewer nominations than Sheri S. Tepper, and the final ballot would look like this:
R.A. MacAvoy121105.833111.987
Joel Rosenberg191919
Joseph H. Delaney 1911.33314.380
Warren Norwood 1712.16714.008
Sheri S. Tepper 189.66713.054

A slate candidate would now need 15 bullet votes, rather than a mere 11, to get on the ballot. That would be the same number of votes as the candidate who just missed in 1984, Barbara Hambly. For me, that's a more equitable outcome.

So, basically, EPH can be improved by using square roots as divisors. That is all.



( 26 comments — Leave a comment )
Jun. 24th, 2015 01:28 am (UTC)
Thinking it through, I can see another problem with the proposal that's more psychological. The big problem is the small number of eligible people that nominate, making any small organised group more capable of getting there under any system (I recall some fuss about someone, Mira Grant? having a bunch of dedicated fans or something).

18 nominations being enough to get onto the ballot (and I know it's gone up, but still) is a bit poor given attendance, etc.

But some of the people I've spoken to online that could nominate don't because 'they don't know enough about the field' or similar, they're not wide enough read. My response is it's supposed to be wisdom of crowds, nominate the best eligible you encountered and let the mass numbers sort it out.

But if there are people already not nominating because they feel they don't know enough, a change that will make their single nomination worth substantially more than the nominations of someone far better read might put them off even more (I disagree with their reticence, but I can at least understand it, and I do think this could make things worse).

I think the biggest individual thing that needs to happen is for Worldcon to actively push for and solicit nominations from everyone eligible every year and word the solicitation in such a way that it gets people over the psychological hump of them not knowing enough, etc.
Jun. 24th, 2015 06:42 am (UTC)
You're thinking of the 2013 awards, when Mira Grant/Seanan McGuire received five nominations in four categories, including three of the four for prose fiction.
Jun. 24th, 2015 01:08 pm (UTC)
EPH is not superior to wellknown alternatives
Frankly, it seems to me, though of course I could be wrong, that STV would have solved the condundra both of excessive votes for a single work (surplus) and works with widespread secondary support. Seeing as how STV or a variant thereof is used widely throughout the world, including in the final round of Hugo voting, it is also a lot easier to understand for many people and a lot easier to defend as just being the "right way" to do things.

I know the people who did EPH are allergic to requiring voters to rank preferences at this stage, but seeing as how it is required (not optional!) at the next stage, I have trouble understanding why. I mean, I read the words but they do not make sense - one has to rank the works to vote anyway.

As it stands, for this and other potential flaws I do not support EPH. Though I do agree with our esteemed host that using the SQRT does improve it! Dr. Plokta's original suggestion (include all works above a certain percentage of the highest plurarity) is superior IMO, as well as easier to understand. Though it does need to be qualified with (at least the top 5) and (at most the top 10) I think to keep the numbers under control in an exceptional situation. It is a shame that this proposal is not even under consideration.

However, since I cannot attend this years (or actually most years) Worldcon my reservations are somewhat moot.

Arioch (7). Apparently I cannot change the "who am I" part of a comment here.
Jun. 24th, 2015 04:10 pm (UTC)
Re: EPH is not superior to wellknown alternatives
I had a good long look at the possibility STV, of which I am a huge fan in general (see my previous post). It's completely impossible to use it to generate a five-member ballot from write-in nominations. And I mean completely.

Basically the number of candidates who get a single first preference, and therefore face a first-round elimination, is usually at least a sixth (ie a quota) of the first preference votes. A lot of those are then simply lost to the process because they have no second preference. The numbers are small and the results look arbitrary and are difficult to defend even for the most diehard STV purist.

one has to rank the works to vote anyway

No, not really, apart from the binary ranking of whether you are nominating a particular work or not.
Jun. 24th, 2015 09:27 am (UTC)
Have you checked how your proposed change would impact how well E Pluribus Hugo would protect against slates?

At least for me, a campaign to get a specific author or work onto the ballot is far less problematic than a whole slate.

(In a way, this is an intrinsic problem. A well-read and discerning nominator is likely to fill entire categories, but is then indistinguishable from a slate voter from a mechanic standpoint.)
Jun. 24th, 2015 09:55 am (UTC)
It's a fair question. I haven't run the numbers, but my gut instinct is that my proposed modification imposes a heavier penalty on getting the first slated candidate in than EPH, but is lighter on subsequent slate additions. EPH is very penal indeed to later slate additions, so this may not be saying much...
Jun. 24th, 2015 12:13 pm (UTC)
I think it is a critical question. Right now EPH lets a slate get one or sometimes two slots. My gut feel is that dividing by the square root would change that "sometimes two" to "sometimes four" slots. At which point we're just about back to this year.
Jun. 24th, 2015 01:38 pm (UTC)
EPH should punish multi-voters *HOW* much
If they have that much support, in relative terms, that means the rest of us fandom is a completely useless bunch of lazy toads. Gero gero.

Let us be clear: with SDV (EPH) the reduction in value of ones vote depends on how many popular candidates one votes for (since the unpopular ones get eliminated).

This is not democratic SDV is not, and is not designed to be, democratic. STV is designed to be democratic, as in the value of the vote after electing one candidate is precisely the proportion of the vote that was not needed to elect that candidate.

Whereas with SDV, if there happens to be, say, 2 or 3 really great works that any right-thinking individual will vote for 2 of in a year, their vote is reduced by the same amount if the votes are 1000-A 1000-B 50-C 50-D 40-E as as for 100-A 100-B 50-C 50-D 40-E. Having the popular votes suck up voting potential like that almost *demands* tactical voting, the very antithesis of the "wisdom of crowds" conjecture.

At least using SQRT reduces the negative knock-on effect of popularity. IMO this much adjustment is absolutely needed to reduce the impact of *bullet* voting.

So-called "slate" voting being just a bunch of people with similar opinions, can certainly be indistinguishable in practice from voting for popular nominees. Which indeed, SDV as written penalises just as if it were a slate vote.

So you say that we are "just back to this year". That might certainly be considered to be unfortunate, though some might view the amazing toxic internet flamewars as being the most unfortunate happening of the year.

However, from here I think you have your arithmetic awry. In *theory* a slate would get more of a sway under the SQRT rule than the existing EPH proposal, however in *practice* the SQRT rule would benefit equally as many if not more nonslate voters who happened also to vote for some popular authors! So not actually.

Jun. 24th, 2015 04:20 pm (UTC)
Re: EPH should punish multi-voters *HOW* much
I personally think there is merit in the argument that if two works teetering on the edge of elimination from the ballot have the same number of nominations, the one whose voters are getting more satisfaction elsewhere, in that their voters have also preferred other popular works works, should perhaps defer to the one whose voters have no other chance of getting a work they like on the list. I haven't seen that argument being made by EPH's proponents as yet, so I'm not sure if they are in agreement.

I replied to the STV point above. It really is functionally impossible in this case. It's also designed to represent different factions effectively, whereas one of the goals of EPH (with which I agree) is precisely to discourage the forming of factions in the first place.
Jun. 25th, 2015 12:58 pm (UTC)
Re: EPH should punish multi-voters *HOW* much
It doesn't "punish" "multi-voters."

It *does* mean that people are not penalized for nominating their honest favorites even though a minority are block-nominating a list of acceptable second-bests for the purpose of expressing US Culture War resentments, (or for any other purpose.)
Jun. 25th, 2015 05:28 pm (UTC)
To be honest, I'm not going to trust anyone's instinct here (unless they're a math PhD with a special interest in election theory).

I'm not saying your proposal is bad, but only that I can't say if it will have the desired outcome, and that it might possibly run counter to the very need for E Pluribus Hugo in the first place.
Jun. 26th, 2015 02:11 am (UTC)
If gaming the awards is ok, see no reason to change the rules
There is a simple way to make it essentially impossible to game the nominations: two rounds of nominations. First round you take the top 20, and then you have a second round of voting on those 20 items, taking the top 5. A small dedicated group can get you to the second round, but only actual quality gets you on the final ballot.

No special math, no punishing people for having five works they want to nominate, no punishing people for liking popular works.

You can even use the "wisdom of the crowds" to do your eligibility checking during the second round of voting, so the total workload for the Hugo Administrators drops.

The only "down" side is that people who've been gaming the system for many years will lose their ability to do so.

247 voters with 519 total votes, and you get on the ballot with 17 votes? That's nuts.

Two rounds of nominating. Because changing one game-able for a different, more complicated, but just as gameable system, isn't an improvement.

I guess what it comes down to is this: is the Hugo Award nomination process supposed to be about finding the best? Or is it supposed to be able handing out Participation Awards like it's the Third grade?
Jun. 26th, 2015 07:37 am (UTC)
Re: If gaming the awards is ok, see no reason to change the rules
Your premises that "gaming the awards is ok" and "people have been gaming the system for many years" are completely wrong.

This proposal is also being discussed at http://nielsenhayden.com/makinglight/archives/016283.html#4178502 - I'll just note that I agree that it imposes a pretty heavy burden on administrators, and is also in fact just as vulnerable as the current system to gaming.
Jonah Thomas
Jun. 26th, 2015 10:47 am (UTC)
Re: If gaming the awards is ok, see no reason to change the rules
"There is a simple way to make it essentially impossible to game the nominations: two rounds of nominations."

Something like that would be a giant improvement. The group at TNH made a big attempt toward that, and then some people who appeared to be important at the Worldcon business meeting showed up and said they were utterly opposed to the idea. They were so opposed that the whole thing got dropped and the proposal was abandoned.

I'm not completely sure I understand the reasoning, but here's how I reconstruct it:

Slates are bad. What we usually do is not bad. The difference is that people who join slates collude with each other, they agree together about what to nominate. Therefore it's wrong for nominators to make any agreements. They should each make their choices completely independently, with no outside forces that influence them. We will get a lot of chaotic variation, nothing will get a lot of nominations, random things can happen, and that's how we like it! Other voting gets campaigning and advertising, bandwagon effects, politics. The Hugos are pure and good, because good fans do nothing to reach a consensus.

There was a proposal to have the voting website show, on a continuing basis, what has been nominated and how many votes each nominee has. That was utterly rejected -- it influences voters.

A runoff was rejected.

There was a proposal to have something less than a runoff. Maybe halfway through the nominations, publish a list of the top 15 without listing their votes. This was utterly rejected -- if voters see which is popular that will influence their vote.

There was a proposal to have the voting website provide a list of what has been nominated already. Then if you see that your choice has already been nominated you can nominate *the same thing* instead of typing in your own choice and requiring the administrators to decide they're the same. This was scornfully rejected -- if voters see what has already been nominated, that influences their choice.

Looking back, I wish a runoff proposal had been made. The worst that could happen is it would be rejected. And it would help with slates more than any voting system can help after-the-fact of the chaotic voting.
Jun. 27th, 2015 04:15 am (UTC)
Re: If gaming the awards is ok, see no reason to change the rules
I didn't follow that discussion, but an obvious problem with all three of those proposals is that they massively increase the administrative burden on the volunteers who run the process.

It is also not obvious that any of them addresses the immediate problem, which is the ability of slates to game the system.
Jun. 28th, 2015 12:41 am (UTC)
Re: If gaming the awards is ok, see no reason to change the rules
1. It greatly decreases the workload on the Administrators:

All they do is take the top 20 from round one, and put them out there.

Because they put out the top 20, without ordering, there's no "we have to validate everything before we say anything" that there is now. Because it's not "here's 5, oops we have to drop one, here's a new five, and everyone knows which one had the fewest nominations".

You're an author, your work was nominated in two categories? You can contact the Admins and tell them which one you want to be in, they drop you from the other, and the voters all know where to nominate you if they like your story.

You're one of the nominees, and you see that another nominee that isn't eligible? Send off an anonymous note to the Admins.

When you have 20 to pick and chose from, let the crowds do most of the digging, and only have to do "due diligence" on the final 5, most of which has probably already been done for you, and posted on various web sites.

2: It completely ends the problem of < 20% of the voters deciding all the nominees. There were 200 - 300 Sad Puppies / Rabid Puppies voters. There were ~2000 total voters. Those Puppy voters can get 5, 6, maybe even 7 items in the top 20. That leaves 13 - 15 spots for non-Puppy nominations. Unless the Puppies have picked items that the vast majority will like, those picks will be drowned out in the second round.

3: What it also ends is 40 people being able to vote together and get one or two nominees in the ballot in the low vote categories. And apparently that feature was too much of a "bug" for the people at Making Light.
Jun. 28th, 2015 06:59 am (UTC)
Re: If gaming the awards is ok, see no reason to change the rules
1. It greatly decreases the workload on the Administrators:

I have to say that this incredible comment makes me wonder how closely you have observed the Hugo process in the past. I encourage you to dive into the 1984 figures to see just how difficult it is to identify the top 5, let alone the top 20, vote-gathering candidates in any category, and the amount of effort that goes into administering each of the two phases of the current system. As someone pointed out to you on Making Light,
Any work that gets left off the longlist that should be on it, is damaged. Any tiny oversight could mean that the rightful last-place winner would lose. If they accept an obligation to get absolutely everything completely perfect twice in the same time they used to do it once, that's a big thing. It's more than twice as much work, because for example contacting 15-20 authors to see if they want to withdraw is harder than contacting 5. Plus the votes might change while they're making the calls.

Before imputing bad faith on the part of those who disagree with you, please familiarise yourself with the practicalities of what you are proposing. Those administrative problems are very real and you lose credibility by hand-waving them away.

As for the rest, you presume a preparedness for collective action among fandom that we have not really seen in previous years, and may not even see this year; we'll see what the results are in August. Any "solution" that relies on a change in fannish behaviour as part of its working seems to me over-optimistic.
Jun. 28th, 2015 08:05 pm (UTC)
Re: If gaming the awards is ok, see no reason to change the rules
1: Administrator workload: There were 3 mail in nominating ballots this year. Compared to the workload of writing and validating the program to do EPH, writing something that can pick out the top 20 from emailed in ballots is child's play.

2: If something is left off the long form ballot it was, at best number 21 in nominations. It didn't even rise to the state of being a meaningless blip in the current nomination process. Their moral claim on the time of the Administrators is essentially null. The point of extending it to 20 is that any problems at the margins are with works that are their by charity, not by "right".

In short, the major failure is the assumption that, with 20 nominees on the long form, the administrators need to get everything "completely perfect". As I said in my post, they don't.

3AS for "implying bad faith without proof": I'll quote from kjn from just up the thread here: "At least for me, a campaign to get a specific author or work onto the ballot is far less problematic than a whole slate."

I'm not "implying" bad faith, I'm reading what people are saying and responding to it. There are only two differences between what the Puppies did this year, and what others have done in previous years (see the 19 Joel Rosenberg "bullet voters" from 1984 for, again, an example from this very post) are
1: The Puppies did everything out in the open
2: The Puppies were more successful than anyone else has been.

I don't find either of those characteristics reprehensible. If what the Puppies did was "gaming the system", then so is every other campaign, done on the sly or not. You want to make it impossible for anyone to game the system? Great! Two rounds of voting will do that.

You want to block what the Puppies did, but allow all the other forms of campaigning? Well, that's what EPH and all its variants do. If you favor that over dual round, I'm going to judge you by your actions, and their perfectly predictable results. Esp. given the large number of people who've stated they have a problem with the Puppies, but not all the other campaigning
Jun. 29th, 2015 06:09 pm (UTC)
Re: If gaming the awards is ok, see no reason to change the rules
writing something that can pick out the top 20 from emailed in ballots is child's play

You really have no idea what the emailed ballots actually look like, do you?

It requires a vast amount of work to check through variant names, mis-categorised nominations and marginal cases, which if your proposal were implemented would have to be on the list because of the *rules*, not because of charity. If you want to publish the top 20, in some cases you may be down to a ten-way tie for fifteenth place with a dozen votes each. There is a reason why weeks elapse between the nominations deadline and the publication of the final ballots, and the reason is that even the present system requires many hours of dedicated effort from a small number of volunteers. And you hand-wave that away as "child's play"!

the major failure is the assumption that, with 20 nominees on the long form, the administrators need to get everything "completely perfect". As I said in my post, they don't.

Yes they do, and fandom rightly demands that they do, and they rightly demand it of themselves. Once you set a cutoff, any cutoff, it needs to be implemented fairly and correctly and it needs to be seen to be implemented fairly and correctly. That's professionalism and commitment. Maybe you don't want that from the award administrators, but I think you are in a minority if so.

given the large number of people who've stated they have a problem with the Puppies, but not all the other campaigning

Yep; that's why people are taking measures to prevent any group ever again from taking all the places on the ballot with only 15% of nominations, a problem which your proposal would do nothing to fix, while creating a huge amount of extra work all round. The situation this year was indeed unprecedented and not at all comparable to previous cases of campaigning, which the system proved able to absorb without any need to amend the rules.
Jun. 30th, 2015 04:23 am (UTC)
Re: If gaming the awards is ok, see no reason to change the rules
1: Hmm, so you've gone from accusing me of unjustifiably accusing people of "bad faith", to claiming it's perfectly reasonable to want to block slates, but preserve the ability of ~40 people to get something on the ballot? Good to know.

2: If it's unfair for 200+ people to knock five people out of Nomination slots they otherwise would have had, it's at least as unfair for 40 people to knock one or two people out Nomination slots they would have had, and deny all their fans a chance to vote for them.

3: We're talking about changing the rules here. Which means we can change all the rules. No? Including making rules changes that explicitly leave a lot of the verification for the second round, which means getting a lot of crowd-sourcing of it during the voting for the second round.

4: You have to check the variant names regardless, to see if that particular variant adds to someone relevant. So no, the workload is not significantly increased.

5: The point of having a cutoff at 20 for the long form is that, even with 3 full slates having enough support to get their full slates on the ballot, that still leaves rooms for 5 non-slate nominees on the long form, to crush in the second round. If it gets to the point where you have 60%+ voting for slates, you're out of luck, no matter what rules you get.

In a real design, as opposed to the ballpark discussion we're having here, I'd probably have something along the lines of:
A: Minimum of 10 nominees in each category
B: Any nominees above 10 must have had at least 4% of the voters in the category nominate them (note that's "of the voters" not "of the votes").
C: If there's a tie that would take you to more than 25 nominees in a category, all are dropped unless doing so would take you under 15 nominees, then all are kept.
D: If there are works with > 4% of the voters' support beyond the top 20, and works in the top 20 get DQed / pulled by the author, none of the works get moved up until at least 5 spots have opened up.

All that's open for discussion, debate, etc.

My major, not willing to change opinion on, points are these:

1: The real travesty is having a situation where, with 2000 total voters, all it takes in many categories is having 40 people in order to become a "Hugo Nominee". In fact, this year it took 23 votes to get nominated for Fan Artist (and top got 48), it took 60 for graphic story. For the rest of the categories this year, getting on the ballot in the weakest categories took more nominations than were needed to get on the ballot in 11 of the 17 categories last year. Those bars are too low.

2: If you're going to change the rules, you should change them to block ALL "gaming" of the system.

There is a workable, but the people screaming the most loudly about the Puppies are also the people most opposed to a fix that would get rid of all the gaming. Which tells me that their objection is not that the Hugos were "gamed", but that they are upset that the Hugos were gamed by someone else.

Am I going to try an get this in front of the Business Meeting? No. It's out there, anyone who really cares about the Hugo Awards being gamed is capable of picking it up and running with it. If no one does pick it up and run with it, well, that tells me where the outrage is really directed.
Jun. 29th, 2015 06:14 pm (UTC)
Re: If gaming the awards is ok, see no reason to change the rules
Incidentally, if you are a member of Sasquan, you and one other member can propose this to the Business Meeting if you get it in by 6 August, and then actually have it discussed and voted on by people who can decide, rather than by people like me who can't.
Jun. 24th, 2015 04:54 pm (UTC)
Even fewer bullets are required than you think
Teresa Nielsen Hayden (18 nominations): 7 + 9/2 + 2/3 = 12.167

Then only 13 bullet votes are needed. That would give "Candidate X" 13 points

The Encyclopedia of Science Fiction and Fantasy, vol iii (19 nominations): 4 + 7/2 + 4/3 + 3/4 + 1/5 = 9.783 points

Then only 10 bullet votes are needed.

Or am I missing something?
Jun. 24th, 2015 10:57 pm (UTC)
Re: Even fewer bullets are required than you think
You are possibly missing that it's not just selection points that matter. It's also total nominations. Suppose the bottom of the list is TNH with 18 nominations and 12.167 selection points, and Bulletman with 13 nominations and 13 selection points. So we compare Bulletman and TNH -- and it's TNH who survives, because once the selection has taken place, the elimination is done by who has more total nominations.
Jun. 25th, 2015 12:11 am (UTC)
Re: Even fewer bullets are required than you think
I reread the proposal, you are correct. They compare the last 2 point getters' nomination totals.

Oops! Sorry about that, Chief.

Which makes the answer to their
1. Can you explain the system in plain language?"

Even more of a "no" that I thought it was.
Sep. 19th, 2015 12:28 am (UTC)
Been testing different methods on the 1984 data with a simple android app
it's available here

I added your suggestion as one of the methods to run on the data. The numbers seem to match up with yours where the original data matches, but my copy of the CSV appears to have more errors than yours.

Against simulated slates it does seem a bit more robust than straight EPH
Sep. 19th, 2015 09:00 am (UTC)
Re: Been testing different methods on the 1984 data with a simple android app
I tidied up the CSV by hand quite a lot.

In the end I'm no longer convinced by my own suggestion. I don't think it hits slates hard enough, and have changed my mind on the main argument about diversity of input - I now actually think it's better to have one candidate on the ballot with only a minority of supporters behind it than for a large consensus group whose choices already fill four slots to get a fifth.
( 26 comments — Leave a comment )

Latest Month

November 2019


Powered by LiveJournal.com
Designed by yoksel