Log in

No account? Create an account

Previous Entry | Next Entry

What EPH would have meant in practice

MidAmeriCon II, this year's WorldCon now in full swing in Kansas City, has published a retrospective count of the Hugo nominations of the last two years as they would have been if the proposed new EPH system had been implemented. This is to an extent a counterfactual exercise - particularly for last year, when a number of finalists withdrew precisely because of the success of the slates. But no such comparison can ever be perfect. The reported effect of EPH on the final ballot (I'm not looking at the long-lists here) would have been as follows:

2015 Best Novel: Lock In by John Scalzi would have been on the final ballot instead of The Dark Between The Stars by Kevin Anderson, assuming both finalists who withdrew IRL still withdrew.
2015 Best Novella: Two of the John C. Wright finalists, "The Plural of Helen of Troy" and "Pale Realms of Shade", would have been replaced by Patrick Rothfuss' "The Slow Regard of Silent Things" and Ken Liu's "The Regular". This was a category where No Award won IRL.
2015 Best Novelette: No change to the final ballot, because John C. Wright's story which was disqualified IRL would not have qualified under EPH.
2015 Best Short Story: assuming the same withdrawals, John C. Wright's "The Parliament of Beasts and Birds" and Steve Diamond's "A Single Samurai" would have been replaced by Ursula Vernon's [Nebula-winning] "Jackalope Wives" and Aliette de Bodard's "The Breath of War". This was a category where No Award won IRL.
2015 Best Related Work: Wisdom From My Internet by Michael Z. Williamson and Letters to Gardner by Lou Antonelli would have been replaced by Jo Walton's What Makes This Book So Great and the Mad Norwegian compilation Chicks Dig Gaming. This was a category where No Award won IRL.
2015 Best Graphic Story: Sex Criminals volume 1 would have been replaced by Schlock Mercenary: Broken Wind
2015 Best Editor, Short Form: Bryan Thomas Schmidt and Vox Day would have been replaced by John Joseph Adams and Neil Clarke. This was a category where No Award won IRL.
2015 Best Editor, Long Form: Vox Day would have been replaced by Liz Gorinsky. This was a category where No Award won IRL.
2015 Best Professional Artist: Kirk DouPonce would have been replaced by John Picacio.
2015 Best Semiprozine: Andromeda Spaceways Inflight Magazine would have been replaced by The Book Smugglers.
2015 Best Fanzine: No change.
2015 Best Fancast: No change.
2015 Best Fan Writer: Matthew David Surridge, who withdrew IRL, and also Dave Freer would not have qualified; they would have been replaced by Abigail Nussbaum.
2015 Best Fan Artist: No change
2015 Campbell Award: No change.

2014 Hugos: Most categories would have been unchanged by EPH. The exceptions are:
2014 Best Editor, Short Form: Sheila Williams would have been replaced by Matthew David Surridge
2014 Best Professional Artist: Galen Dara and Fiona Staples (the latter tied for fifth place in nominations IRL) would have been replaced by Joey Hi-Fi.
2014 Best Fancast: the three-way tie at fifth place for nominations IRL would have been resolved in favour of Tea and Jeopardy, with The SF Signal Podcast - which actually won - and The Writer And The Critic losing out.

1939 Retro Hugos: there would have been relatively more changes, due I think to the relatively lower number of nominators.
1939 Best Novella: "A Matter of Form" by H.L. Gold would have been replaced by "Tarzan and the Elephant Men" by Edgar Rice Burroughs.
1939 Best Short Story: "Hollerbochen's Dilemma" by Ray Bradbury would have been replaced by "An Experiment of the Dead" by Helen Simpson.
1939 Best Fan Writer: Harry Warner Jr would have been replaced by William F. Temple.

There is a lot to think about here. It's clear that EPH in 2015 would have given a more diverse ballot, and would have provided two non-slate candidates in four of the five categories where voters No Awarded the slates. But applied to a "normal year", we actually lose a finalist who won; and for the lower-intensity Retro Hugos, the edge effects get a bit unpredictable.

Dave McCarty, Hugo Admnistrator both this year and in 2014, comments in the paper: "The changes to the Ballot and Long list are not easily verified and for people reviewing the detailed results at the end the only way to check that the process is working correctly would require access to secret nomination data and significant time. The difficulty in verification means that to check any result requires time which is NOT available to award administrators when it is time to close the nominating and prepare for the Ballot announcement. These are significant hurdles for a process that is generally designed to be open and democratic."

I sympathise, and I also have a more basic concern, which is that EPH (which seemed to me like a good idea when it was first proposed) is fundamentally designed to address last year's problem. On Sunday we will find out what difference it would have made this year. But as Hugo administrator for Worldcon 75, my personal concern is next year, and I guess if I were at the Business Meeting, I would need to be convinced that it is the right answer at this time. Of course, if the Business Meeting does ratify it this weekend, I'll duly implement it regardless of my own views.

Edited to add: I've been reflecting on this a bit more, after being reminded over on File 770 that the only change EPH would have made in 1984 would have been to drop a woman (Sherri S Tepper) from the Campbell ballot.

The changes it would have made in 2014 would have been to drop a woman in favour of a man for Best Editor Short Form, to drop a woman and a man and bring in a different man for Best Professional Artist, and to drop both the actual winner and another podcast run by a woman and a man from Best Fancast.

Admittedly it's a small sample, but I don't really like what I am seeing of EPH's effects on the diversity of the ballot in "normal" years. However I have to concede that it would have replaced that embarrassingly bad Ray Bradbury story with one by Helen Simpson on the Retro Hugo ballot.


( 4 comments — Leave a comment )
Aug. 19th, 2016 10:10 pm (UTC)
I'm pretty sure that the diversity effect you're seeing is due to a small sample. I see no plausible mechanism by which EPH could systematically hurt gender diversity in normal years.
Aug. 20th, 2016 07:29 am (UTC)
Just because you don't see a plausible mechanism, whoever you are, doesn't mean that there isn't one.

When EPH was first proposed, I expressed the concern that port that the quirky candidate that got a lot of people's fourth or fifth nominating votes will lose out. I think that is what we are seeing here.
Aug. 24th, 2016 11:04 am (UTC)
The bad effects are partially because EPH makes ties much less likely, I think. So while I don't think it's great that the fancast nominees drop off, I don't think it's bad to end up with 5 nominees instead of 7.

Anyway, at this point we ratified it. I think you are now instructed by the business meeting to produce the nominees for next year's Hugos as though they had been run under EPH and the old system, and since the Hugos will be earlier in the con next year there should be ample time to compare the two options before the final business meeting and if necessary vote to suspend EPH for 2018.
Aug. 24th, 2016 11:56 am (UTC)
For the record, I was very much reassured by the data released on Sunday with regard to the 2016 and 1941 ballots. The 1941 ballot would have been barely changed (Hannes Bok would have dropped off the Best Professional Artist category, but I put him fifth anyway), and the 2016 ballot would have been vastly improved. So I'm more ready to accept that the changes that concerned me on the 2014 ballot are essentially edge effects. I shall reflect this in a full post once the full 1941 and corrected full 2016 statistics have been published.

And, as you rightly point out, I am now obliged to count next year's nominations both ways, though the EPH results are the ones that will count.
( 4 comments — Leave a comment )

Latest Month


Page Summary

Powered by LiveJournal.com
Designed by yoksel