?

Log in

No account? Create an account

Previous Entry | Next Entry

(Part 1 of 2)


(The 2017 Hugo Award)


In 2014 I was the division head for Promotions for Loncon 3, which turned out to be the second biggest Worldcon ever. It was immensely hard work, but also immensely rewarding. I often felt out of my depth, but the moments of triumph - like getting the front page of the Guardian - were all the sweeter for that.

I swore to myself (and to my wife) that I would not be involved with another Worldcon at that level of responsibility again any time soon - way too stressful and time-consuming. For the next year I stayed out of Worldcon politics - I quietly supported DC for 2017, but could see that Helsinki was making the running, and when they won I assumed that I would probably go, as it is not all that far from Belgium.

Then, rather to my surprise, I was approached by Michael Lee, the WSFS division head for Helsinki (who I knew from Loncon preparations), and his deputy Kate Secor, who asked if I would consider administering the 2017 Hugo Awards. (WSFS stands for the World Science Fiction Society, and the WSFS division is the group within Worldcon staff responsible for implementing its obligations under the WSFS Constitution, which are to award the Hugos and to choose the location of the next-but-one Worldcon.) It took me a few days to make up my mind - there was a bit of a feeling of I-am-not-worthy - but in the end I said yes. I have been interested in the Hugos since I was a teenager, and had vigorously defended them as an institution during the slate attack of 2015. It was the right time to invest in a thing that I love.

I also love Finland. I had been there only twice - my sister and I spent three weeks volunteering at Valamo monastery in the centre of the country in 1990, and I'd been back again for a conference in 2003 - but I've been a fan of the rugged, lyrical, vital, colourful music of Jean Sibelius since my teens, and a fan of Tove Jansson's Moomin books for much longer than that. I've also been lucky enough to work alongside some excellent Finnish politicians in my professional career - though I have also had one encounter best described as a learning experience.

The first crucial choice was to identify a deputy Hugo administrator. I felt that I needed someone who was a good organiser and well-networked in US fandom. I had met Colette Fozard in the run-up to Loncon; she is a professional conference organiser who had chaired fan cons of the same order of magnitude as Loncon had been, and had been in the leadership of the unsuccessful Orlando 2015 and DC 2017 worldcon bids. I sensed that she might be persuadable to get stuck into Helsinki. And so it proved, ultimately rather more than either of us anticipated.

At the very beginning, we picked the brains of John Lorentz, who had been the Hugo administrator for 2015 (for the fourth time), and subsequently we received much valuable counsel from former Hugo administrators Vince Docherty (2010 and 2011) and Dave McCarty (2014, 2016 and 2018), both also former Worldcon chairs, particularly with some very helpful conversations during 2016 at Eastercon (Manchester), Octocon (Dublin) and Smofcon (Chicago). Dave and I frequently disagreed, but the exchange was always worth having.

The wider context, of course, was that the Hugos were under serious attack. A thin-skinned right-wing writer who was an unsuccessful finalist for the Campbell Award in 2011 had successfully pushed several hand-picked candidates onto the 2014 Hugo ballot, where voters had generally given them short shrift. Two of the authors who he had thus promoted were then much more successful in mobilising right-wing voters, including the far-right, to get slates of their favoured candidates onto the ballot in 2015, dominating several ballot categories with their chosen finalists, some of whom were of abysmal quality.

2015's Hugo voters instead chose No Award five times, equal to the total number of No Awards in the entire previous history of the Hugos. If the slaters had not been so greedy as to dominate entire categories, if they had picked better work, or if they had not generally behaved badly and invoked an unacceptable political agenda, the wider fannish reaction (including mine) might not have been so passionate; but they chose to make all three of these choices, and indeed escalated their argument into the wider political fight of the culture wars, causing much spilling of electrons on both sides (again, some of it by me).

To prevent this from happening again, several rule changes were passed in 2015 for ratification in 2016 to take effect in 2017 (WSFS has a two-year amendment cycle). These included a new tallying algorithm for the nomination phase of the awards (referred to as E Pluribus Hugo, or EPH for short) which would make it much more difficult to dominate an entire ballot category through a minority of nominators, and the provision that the final ballot would have six finalists although voters would nominate fewer than six candidates (the original proposal was four, later amended to keep it at five).

On my birthday in 2016, I was sitting in a noisy Brussels pub with a former Lib Dem MP and had to keep excusing myself as the Hugo finalists were announced by MidAmeriCon II on Twitter. Only one of the previous year's slaters had been active this time, but again several ballot categories were dominated by candidates chosen by him. I was dismayed. Concerned that the EPH system as proposed might not be sufficient to protect the Hugos in future, I put my name to a proposal supporting an extra preliminary stage of voting to screen out troll nominees, and to another moving the qualifying date for nominating back to December 31 of the previous year rather than January 31.

Both of these were passed at the WSFS MidAmeriCon II business meeting in August and sent on to Helsinki for ratification. After the 2016 Hugos had been handed out (with No Award winning only twice rather than five times), Dave McCarty provided detailed voting statistics showing that EPH would have drastically reduced the number of slate finalists. The WSFS business meeting consequently ratified both EPH and the shift to six finalists (but up to five nominations per voter) in each category, these changes to take effect for my turn as administrator in 2017.

This concentrated our minds rather wonderfully on the need to test our software for processing nominations, newly and beautifully designed by Eemeli Aro, using the new rules. Kate Secor had the brilliant idea of inviting people to submit nominations on Hugo-eligible work for a past year, and use that to check the interface. We settled on 1980 as the target year, partly because the earliest year for which we have more or less complete nomination statistics, and also partly because several of us had particular favourites from that year which we hoped would stand the test of time. (In my case: Sandkings; City of Death; Alien.)

We had planned to launch what we called The 1980 Timewarp Project with a certain amount of pizzazz in mid-October, but one older and wiser voice advised a bit of discretion, and as we were in the course of processing that advice, an unrelated communication crisis hit Worldcon 75 and we came to the conclusion that we were not in a position to presume on the good nature of wider fandom by making it a major event. Still, we went ahead with a soft launch, mainly through Worldcon 75's own membership list and File 770, and gathered enough information about how our own system worked to be confident that we could use it in January for the real thing.

The results were encouraging, and we launched them at the Worldcon 75 November staff meeting, which was a bit fraught for other reasons but otherwise was a good introduction to the Helsinki team. Colette and I took time after the meeting had ended to go downtown and see former President Martti Ahtisaari, who told us anecdotes about Hillary Clinton (who had lost the American election the previous week) and what it's like to get the Nobel Peace Prize.


(Picture taken by Sami Lehdensuo)


Meanwhile, things were changing in the Worldcon 75 leadership; one of the consequences was that in December Colette moved from being my deputy to my boss, as Vice-Chair of the convention as a whole, covering a number of divisions including WSFS. That left me in need of a new deputy; it did not take me long to think of Kathryn Duval, who had been Loncon 3's very successful North American agent. I knew I could work well with her again, and so it proved. She took on most of Colette's role, except that I took on co-ordinating with the ceremony (of which more later) and she took on co-ordination with DevOps, since she actually wrangles programmers for a living. Like Colette, Kathryn is based in the eastern US time zone, six hours away from me, which I think is the maximum sustainable for the relationship to work.

Kathryn came on board just in time to assist with the opening of nominations. An essential first step was the process of merging our membership lists with those of the 2016 Worldcon, MAC2, and 2018's Worldcon 76 in San Jose. The latter element of the process was facilitated by the fact that San Jose's head of registration happened to be one Kathryn Duval. The biggest problem turned out to be getting our own data out of one of the Finnish online payment systems. I did all the merging by eye from Excel spreadsheets; it was mind-numbingly tedious, and had to be done again in early February to catch people who had joined in January.

Last-minute hitches with the software (specifically its treatment of people with shared email addresses) meant a week's delay in opening nominations. Once we had done so, Brent Smart (a Colette find) stepped up as main answerer of emails to the hugohelp@worldcon address - mostly this just meant re-sending people their login links, but we also identified problems with Firefox which were fixed for the main Hugo vote. We also received a number of eligibility queries at this point, which we generally declined to answer, in line with established practice.

One important innovation was to set up a Hugo Research Team, to check eligibility and to discreetly get contact addresses for the potential finalists in advance of nominations closing. Of course, no actual ruling on eligibility was made until we knew which the top six finalists were, but it was very helpful to hash out the potential issues and even more helpful to get contact details in advance. In early February, I fed the Research team the top 20 nominees in each category, as matters then stood, and updated them in early March and again in the last few days as late rising candidates emerged. Lynda Manning-Schwartz, another of Colette's suggestions (who I also knewfrom Loncon preparations), did most of the work, with some asistance from Ian Moore, an old friend of mine from Dublin.

Emmeli's software was a particular delight in dealing with what we have come to call "canonicalisation", the combination of different forms of the name of particular nominees (to pick an obvious example, "Doctor Who", "Dr. Who" and "Dr Who"). We were able to start this process from the first day nominations opened. Towards the end I found that checking the latest votes twice a day neatly filled my daily commuting time (which is over an hour, each way).


(How some voters nominated the eventual winner of the Hugo for Best Novel.)


The morning that nominations closed, I fired up the complete symphonies of Beethoven and Sibelius (including Beethoven's little-known Battle Symphony, and Sibelius' Kullervo which is a particular favourite of mine) and with that stirring background dealt with some last-minute queries. One very young voter had found to her dismay that her browser, on a shared parental computer, had not saved her votes; we re-opened her ballot so that she could make her nominations. Another voter contacted us to ask why she had not received a link; to my horror I realised that I had mistakenly excluded her as a duplicate voter back in January, and again we had to adjust the system to fit reality. The total number of nominating voters was 2,464, the second highest ever.

With Beethoven and Sibelius, I then powered through all the categories to see what we had got, paying particular attention to the closest results; Kathryn then cross-checked. Colette had to extract the six postal ballots (all from Americans) from her Maryland PO Box and input them. The next day was Sunday and I prepared emails (drafted a week or so beforehand in most cases) to as many finalists as possible, to go out first thing Monday morning - though the first finalist who I informed was the then executive producer of Doctor Who, who also happens to be my cousin.

The slaters had been much less active this year. One lot fell completely silent; the other proposed only one or two favoured nominees in a number of categories, and with rather fewer of his supporters participating, not many got onto the final ballot. EPH had in fact helped the slate in some categories where they had only one candidate, but the dominance of previous years could not have been repeated on the 2017 turnout. I felt sorry for those who were slated onto the ballot without their knowledge, and who accepted nomination in good faith and with enthusiasm.

There were a few other difficulties. Apart from my cousin, it proved very tedious to get in touch with most of the Dramatic Presentation finalists. I took a couple of days to make my mind up about the most difficult judgement call, the Best Graphic Story category where the Finnish web comic Stand Still. Stay Silent was very close, but I concluded that to be consistent the first volume of The Vision was the one that qualified. Some addresses turned out to be wrong; some finalists were just ignoring emails and had to be tracked down via their agents, or in one case through a work colleague. I had worded the Best Editor Long Form notification email poorly, which caused some confusion.

Mostly, however, finalists were (justly) delighted to be on the ballot. Also, one finalist declined (as he had said he would do) immediately; another confirmed that his publication was ineligible; the makers of Game of Thrones chose to decline for one of the three episodes that had enough votes to be on the ballot; we had a delicate conversation with a sculptor, nominated for Best Professional Artist, who as it turned out had no work which qualified him for the rather precise conditions of the category; the artist next in line to replace him in that category turned out to have had no professional work published at all in 2016. It took two weeks to be fairly certain of all the ballot places (and even then there were changes to come).

I have already written about how we decided on and did the video announcing the final ballot. Here it is sufficient to thank again Sanna Lopperi, Katarina Ihalainen and Charlotte Laihonen for making it happen, Jukka Halme and Sari Polvinen for accommodating me, and the seventeen other presenters for doing the announcements. It proved a bit tricky to synchronise all elements of the announcement, but the information got out there in the end.



My employers felt that we needed a video explaining the Hugos as well, and we shot a brief explanation for the perplexed in the basement of the Shaftesbury Avenue Forbidden Planet.



At Eastercon in Birmingham, Dave McCarty and I did a late night and very indiscreet two-handed panel on the Hugos, "interviewing" each other (basically arguing Hugo minutiæ in front of an audience). Fortunately for our reputations, nobody seems to have minuted the proceedings, apart from one photo commemorating a rare moment of agreement. (That's what Dave looks like when he is agreeing.)


(Picture taken by Colette H. Fozard)


Jo Van Ekeren emerged at a late stage to compile the Hugo Voter Packet. She did a tremendous job on her own with little more than moral support from me and the rest of the team. I should have anticipated the burden of work and shared it out between her and others, starting with myself. If Dublin 2019 decides to run Retro Hugos for 1944 I'll have two different teams working on the two packets.

In the course of compiling the Packet, it became clear that two of the Fan Artist finalists simply had no eligible work from 2016 and we had to disqualify them. One of these came through at quite a late date, due to linguistic difficulties in explaining what the problem was - go on, you tell me how to translate "semiprozine" into Korean? As a result of this, they were replaced by two new finalists, one of whom went on to win the award in this category.

For the first time, as far as I know, we had games/interactive fiction on the ballot - in the shape of two games set in the world of Max Gladstone's Craft Sequence. Steam keys for the games were provided as part of the Hugo packet, though it took us longer to get that up and running than I would have liked. At some point the Hugos are going to have to take games seriously (a proposed "Best Video Game" award in 2006 failed to attract enough nominations to be viable); let the record show that it started in 2017.

The counting of the actual ballot was much more straightforward than the nominations tally - only seven per category, and little ambiguity about voters' wishes. The one point that I picked up for future years is that voters need to be told that their vote stops being counted once a blank preference is reached, even if that is the first preference. There was one category where a late surge displaced an early leader, but otherwise the finalists who started ahead finished ahead. No final results were particularly close, though in one category the eventual winner escaped elimination by 3 votes in an earlier round. At 3,319 total votes (3,315 electronic, and four paper ballots sent to the PO Box in Maryland), it was the third highest ever. Speaking for myself, it was the first time I can remember submitting a full Hugo ballot, with a preference for all 108 finalists. 7 of my 18 first choices won, which is about my average.

This was also the point where I calculated the nomination results as they would have been under EPH+, a variant of EPH with different divisors which had been approved in Kansas City and was up for ratification in Helsinki. I realised it was not going to pass as soon as I saw that it would have added two more slate candidates to the ballot. Even without that consideration, the results were simply much more difficult to understand and explain. The legitimacy of any political system depends on voters seeing a relationship between their votes and the result, even if they do not get the result they want, and EPH+ would have failed to deliver that.

The week before Worldcon, I finalised my reports. In past years, I have used my blog to highlight the closest results; this time I included that information in the two official results sheets (for the final vote and nominations), which included some fairly mind-boggling nominations data in the Best Fan Artist category. As well as those two standard reports, I added a report on the detail of EPH, a comparison of EPH with the old system and EPH+, and a list of decisions we had made as Hugo Administrators - this last to improve the transparency of the process.

And then we were ready to go.

(End of Part 1)

Comments

( 7 comments — Leave a comment )
kjn
Aug. 17th, 2017 05:16 pm (UTC)
Thanks for this interesting insight into the sausage factory!

For myself, I finally managed to grok how EPH worked, and also have sufficient confidence in the math behind EPH+ (since the same theory is used in every Swedish election) that I'd have voted for it despite it having a result that some would consider ungood. But that analyses the result, and fails to analyse the process. Not that I had time to go to the business meeting anyway.
nojay
Aug. 18th, 2017 08:04 am (UTC)
Thanks
The first thanks is for your hard work in organising and managing a long-term complex disputatious process all the way from inception through to conclusion. The second thanks is for describing just how much effort is involved in something like the Hugos. From the outside it seems simple -- count the nominations, count the votes, hand out the rockets, job done. It ain't that easy...

I look forward to reading part 2. I would hope that any future Hugo administrator reads and inwardly digests your reporting of what you did and how you did it.
livejournal
Aug. 18th, 2017 08:31 am (UTC)
Hello! Your entry got to top-25 of the most popular entries in LiveJournal!
Learn more about LiveJournal Ratings in FAQ.
amazingstoriesm
Aug. 18th, 2017 02:58 pm (UTC)
nice
Excellent insiders report. Looking forward to more!
pwilkinson
Aug. 19th, 2017 02:52 pm (UTC)
Thanks for all the reports - very informative!
jvjr
Aug. 28th, 2017 09:30 pm (UTC)
Ad voting software
I join the thanks for both your work administering and this analysis.

BUT, am I really the only one who HATED the UX of the final voting? With the javascript interface, the easiest thing would have been just to click on the finalists in the preferred order and send the ballot in. But no, I had to explicitly interact with the ordering dropdown menu."Pikachu, I choose you AND now select you to be the first!" "Now you, Bulbasaur, shall be the SECOND!" Etc. etc.; the click and drag could be used only later for rearranging the previous order, but why would I do that?

I did not have a preference for all 108 by far but it took me a lot of extra time anyway. Did it really feel "straightforward" for you? :-(
nwhyte
Aug. 29th, 2017 01:36 pm (UTC)
Re: Ad voting software
Thanks. You're not the only person to make that point, and we'll bear it in mind for future iterations.
( 7 comments — Leave a comment )

Latest Month

April 2018
S M T W T F S
1234567
891011121314
15161718192021
22232425262728
2930     

Tags

Powered by LiveJournal.com
Designed by yoksel