Single points of failure

This the first of an expected three posts about Brown Puzzlehunt.

Content warnings: discussions of respiratory illness and fatigue.

The solutions for Brown Puzzlehunt went up today. For reference, the hunt ended (i.e. no longer accepted answer submissions) one month and four days ago.

In that time, both CMU Puzzle Hunt and Microsoft Puzzle Hunt started, finished, and had their solutions released. (Both great hunts!)

This is obviously an exceedingly long time. This blog post will discuss why that happened, and why it’s not going to happen again in the future.

Part 1: Brown Puzzle Club’s Structure

Brown Puzzlehunt was Brown Puzzle Club’s first hunt; as is the nature of these things, the organisational structure of the constructing team began very narrow. There are a lot of great reasons why you would do this; the main one being that setting up wide, expansive teams to handle different things is work. It’s also work that requires you to know what the different skillsets and capabilities of all the people on those teams are, in order to delegate different aspects of tasks efficiently. And, should you not have enough people on those teams with the required skillsets, you spend some time showing them the ropes.

All of this is a near-necessity for large, complex hunts. While Brown Puzzlehunt arguably qualifies as one of these, we actually found a great amount of success with a set of people who had full control over each of their respective areas. Those areas were:

• Coordinating with admin, setting up tech, booking rooms, filling out forms, printing materials, and generally getting approval for the various insanities of the hour
• Designing the website (full-stack), implementing interactive puzzle components, and coordinating/directing artists
• Leading puzzle construction (including physical puzzles), organising events, editing puzzles, getting puzzles testsolved, and uploading them to the site

That last one was mine! With the exception of interactive puzzles (which received extensive input from Orion Bloomfield), all the puzzles on the website were edited and postprodded by me, with some editing help from our external testsolving team.

Why would we do this? Well:

a) it’s fast

b) it prevents potential disputes

c) it’s easy (in the sense that training other people is hard)

And as long as nothing bad happens to any of those key people, your event will go smoothly. And it did! Nobody got sick, or injured, or swamped with academic work, or crushed by falling pianos, or covered in olive oil and stuck in a bathtub with smooth walls.

With the exception of half-a-dozen errata/bad clues (and one technical issue that was Orion’s domain), the puzzles were all post-prodded and edited correctly. That’s still half-a-dozen more things than I would like, but considering the timeframe and personnel, it’s a result we’re happy with for this hunt.

Since the ball was already in my court, it also fell to me to make sure that solutions went up for the hunt. Puzzle writers had written internal documents to help with hinting; these were often abbreviated, had mostly raw data, or were written to guide a hinter rather than a solver. (This is not good practice, but Brown Puzzlehunt was written at speed, for reasons that I will probably discuss at a future date.) They were also to a solution all written up in Google Docs or a pdf. It would take some work to read through each of these, correct for any errors, convert into the required narrative-ish walkthrough solution, and then upload to the site correctly. In some cases, extra information about the puzzle or commentary on specific aspects had to be added. In others, the entire solution had to be rewritten from scratch to adjust them to this new standard.

It’s not a ridiculous amount of work, all told (remember: we had just run a puzzlehunt, so our scale was slightly off), but it was not inconsiderable. Ballpark: it takes an hour for each solution, and there are thirty puzzles. A single dedicated person can get through that in about a week.

Part 2: The Kicker

(To head off this entire section; yes, I am completely fine and in good health. Don’t worry.)

In November of last year, I got sick with something called infectious mononucleosis. It’s a fairly common viral disease that is very annoying, but isn’t serious in and of itself. I’m squarely in the demographic that is most likely to get IM, and I’m the sort of person who gets mildly sick after taking flights on airplanes, so it’s not at all surprising that I got something like this eventually.

After about a month or so of bed rest and taking steroids, I was fine. Except…

Infectious mononucleosis has a lot of strange and poorly understood interactions with your immune system. (Indeed, it increases your risk profiles for a number of other immune diseases, like multiple sclerosis.) One of these interactions is that it tends to increase the severity of other infections (viral and bacterial) you get for many, many months after you’ve had it. So if you’re the kind of person who gets sick because you went on an airplane, your infrequent low-grade illnesses get both more frequent and more high-grade. The result over the last month has been a revolving door of bacterial and viral infections, respiratory symptoms, constant fatigue, and other very fun things that I won’t get into.

(In hindsight, I was incredibly lucky to not get sick in the months leading up to Brown Puzzlehunt. See the bit about pianos.)

I’m fine now! The fatigue stays with you (and jet-lag doesn’t help); but I know several people who suffer from genuine immunocompromisation or chronic pains, and my very easy constant-cold experience in no way chalks up to anything that they go through.

But the result was that I wasn’t able to work on solutions extensively for a long period of time. Much, much longer than I expected. And because I wasn’t able to work on solutions…

Part 3: Moving Forward

The term “single point of failure” is used in a lot of contexts, but you’re probably most used to seeing it relate to software or hardware systems. This Tom Scott video is a great example. Any single thing which can bring down the entire system is a huge problem. Everything breaks eventually.

But a single point of failure can also be a person. Any hierarchical structure which is designed such that only a single person is able to conduct a certain critical task will at a some point fail. Humans are squishy things which get vibe-checked by the universe.

The excellent news is that we are already starting to fix this. Some preliminary plans for next year’s Brown Puzzlehunt have been drawn up; and an essential part of this planning is expanding the number of people who can perform the three critical functions which make a hunt happen. We have time to train people, to test internally, and to prepare.

Addendum

If you’re interested in more stories about Brown Puzzlehunt or other things I make, keep an eye on this website. I plan to post various things over time.

There isn’t a working RSS feed for this site yet, but I’ll try to get one up over the next few days.