XKCD's "Nerd Sniping" is the epitome of rabbit holing

The geek's Achille's Heel

I recently decided practice my algorithms skills everyday by working through Skiena's Algorithm Design Manual and doing some Interview Street problems. To make it somewhat more interesting I decided to do all the exercises and problems in Haskell.

On my second day of practice, I felt the urge to find a better Haskell syntax highlighting extension for Vim. I managed to find one that was distributed in a package called a "Vimball". I spent 10 minutes trying to figure out how to install a Vimball and then another 10 customizing the plugin's settings.

Absolutely none of that helped me with my ultimate goal of learning about algorithms. By the end of the 30 minutes I allotted, I had 9 browser tabs open about useless trivia and I had not advanced a single page in my textbook.

The names change but the story remains the same. Designers find themselves studying fancy, new CSS3 effects when they should have been wire framing their checkout page. Hapless students find that they are on the Wikipedia page for Esperanto instead of writing notes on Norse mythology. Like Alice led into Wonderland by the White Rabbit, geeks too easily fall into "the rabbit hole".

The most insidious part of this "rabbit hole" phenomenon is how unaware its victims are. We are lured into thinking we are making progress. What if this new language and setup this new stuff could help program so much better? How could you do something without having explored every tool and technique at you disposal?

As geeks, our pursuit of perfection and perfect information distract us from the task at hand, but it turns out that it is not always a bad thing.

Your weakness is your strength

Going down a rabbit hole is certainly devastating to productivity on a micro scale. But it is also a valuable "skill" when it comes to solving hard problems. Richard Feynman was fond of saying:

You have to keep a dozen of your favorite problems constantly present in your mind, although by and large they will lay in a dormant state. Every time you hear or read a new trick or a new result, test it against each of your twelve problems to see whether it helps. Every once in a while there will be a hit, and people will say, “How did he do it? He must be a genius!”

If one brilliant researcher wasn't enough, Richard Hamming puts it yet another way in his essay You and Your Research:

I notice that if you have the door to your office closed, you get more work done today and tomorrow, and you are more productive than most. But 10 years later somehow you don't know quite know what problems are worth working on … He who works with the door open gets all kinds of interruptions, but he also occasionally gets clues as to what the world is and what might be important. … [T]here is a pretty good correlation between those who work with the doors open and those who ultimately do important things, although people who work with doors closed often work harder.

What both of them are saying is that producing brilliant work is heavily reliant on serendipity. Putting your nose to the grindstone will certainly get things done, but when you are working on cutting-edge problems with no predetermined path to success you derive inspiration through chance discoveries.

Both these men were probably relied on conversations with their brilliant colleagues to deliver them random insights. But they also had the advantage of working at the top of their games at Caltech [1] and Bell Labs, respectively. The common geek today relies on the Internet, especially community watering holes like HackerNews and Reddit, to keep abreast of "what the world is and what might be important".

The solution

We have concluded that random walks of knowledge-gathering keep us from getting things done day-to-day but can also be catalysts for amazing work. Falling down the rabbit hole seems to be an activity best done in moderation. How can you use it to your advantage without letting it damage you?

My solution is pretty low tech and simple. During my workday, I keep a text editor open on my desktop. In very large text, I write very specifically what I am supposed to be doing at that moment.

Every time I detect that I am wandering, I check back to the window and ask myself whether what I am doing right now will immediately benefit that tasks.

But then I don't close the tab or stop my tinkering immediately. Obviously it was fascinating enough that I started looking into it. So I make sure that I record it somewhere so I can go back to it at my leisure.

For long form text, I am a fan on Instapaper and I've heard great things about Pinboard.in. If it wasn't something on the Internet but something more abstract, like a new feature I discovered in a program, I make a note of it in my Trello board, in a list called "Wishlist".

Make some time to revisit these repositories of information. It is easy to just hoard web snippets and never use them to your advantage. So set aside some time everyday to update yourself on your findings. It does not have to be much; just 15 minutes will suffice.

With enough discipline and practice, you may find that you are more productive than ever and maybe people will even say “How did he do it? He must be a genius!”.

Errata: Feynman had a career at Caltech, not MIT. Thanks to mturmon for the correction.