Stack Rank All The Things

One weird trick to make better decisions

I have this management process that I want to tell you about. No no no, don’t run away yet. I am not crazy. The process is simple to explain:

  1. Make a list of the things
  2. Rank the things
    (preferably, by stacking them vertically)

Stack ranking has gotten a bad rep recently, thanks to how Microsoft used it. We’ll talk about that, but first let’s talk about the problem that we’re trying to solve.

Sometimes, teams have to make difficult and negative decisions. Stack ranking makes those conversations more productive and positive.

Suppose you’re having an argument about your new blogging platform (*cough*). Dave argues that we should work on better copy-paste tools. Dustin thinks this is not important enough to work on. There’s not a good place for this discussion to go, to move forward. We will get stuck on the basic roadblock, “I think this is important and you do not.” It can also be a hurtful discussion, in the way that “this feature is not important” quickly slides into “what you care about is not important.”

When you stack rank, that transforms the whole conversation. We’re no longer arguing about whether a project is important or not. We’re trying to figure out whether feature X is more important than feature Y. This is, in many ways, a much easier discussion to have. We can focus on the positives. Dustin isn’t stuck arguing the negative that copy-paste is a bad feature. Instead, he can agree that copy-paste is a great feature, but that there are other great features we could be working on.

More importantly, stack ranking makes the allocation problem an explicit one. By “the allocation problem,” I mean that you only have N people, and there are a fixed number of hours in the day. If we decide to work on one thing, then, most of the time, we have to working on something else.

It is surprisingly easy to fall into this trap, because many projects are reactionary. Users ask for interop with WordPerfect. “Let’s put 3 engineers on that!” Users want a native Kindle app. “Four engineers on that!” Soon you’ve allocated 200% of engineering. Again, it’s easy to argue “yes, let’s work on that.” The benefits are real and easy to visualize; the costs are vague and easy to hand-wave away. We see this dynamic often in politics: politicians say they will pay for benefits by “closing tax loopholes” without naming the actual loopholes they plan to cut.

Stack ranking solves this problem by setting features directly against each other. It forces you to name what you’re going to cut back on to pay for the new things you want to do.

Where Stack Ranking Can Go Wrong

When Ballmer announced his departure from Microsoft, the business and tech press had a fun couple of weeks bemoaning the culture problems at that company. “Stack ranking” got a big chunk of the blame.

A few journalists argued that stack ranking ruined Microsoft and scared their best engineers away to Google and others. This is an ironically hilarious argument. You can use stack ranking for anything, and when I worked at Google, I used stack ranking for everything. We stack ranked upcoming features. We stack ranked major projects. When considering people for promotion, we stack ranked them first. Most managers stack rank every person who works for them.

It can be useful, and amazing, and liberating.

But there are ways to stack rank that are stressful and backstabby. The contentious question is whether its benefits outweigh its sharp edges.

Ranking People

Obviously, ranking people can get awkward quickly. No one wants to see their name on a list where they’re ranked lower than someone else. And, most importantly, friends don’t want to rank each other, or see themselves ranked by their friends. Except maybe first graders at recess ranking besties.

One solution to this is to keep the process secret, and hope the results don’t leak out. This is the most obvious solution, and simple to implement, but fraught with political problems.

Another solution is to rank people based on second-hand information. So, for example, each full-timer writes up an assessment of each summer intern. Then a group of people rank the interns on the basis of those assessments. This feels much less slimy, because it diffuses responsibility, and the artifacts of the process are a lot more helpful. We have to explicitly document what each intern did, and then we’re comparing those write-ups rather than directly comparing people.

The last solution I know of is to use bucket ranks. Programmers often do this with bugs: P1, P2, P3, P4. You may not need a total ordering. A partial ordering by bucket may be enough. For example, if you’re deciding from a group of people to hire, you can sort them into 4 buckets: “definitely no,” “maybe no,” “maybe yes,” and “definitely yes.” This is less awkward. Then you stack rank things close to the borders if you need additional help to make decisions.

Ranking Distributions

When you start stack ranking things, it’s tempting to start setting targets for what you expect the distribution of ranking to be. Recall the Lake Wobegon motto: “where all the children are above average.” Sometimes stack ranking brings out the dark inverse: “all the people below the 10% line must be terrible, and should be fired.” Both these statements are logical fallacies, because they confuse absolute values with distribution curves. This, I suspect, is the source of the culture problems at Microsoft.

Let’s be clear: setting percentages can be really useful as a The real distribution curve is not going to perfectly match expectations. For example, we might say “we should check up on the bottom 10% and see if they need better mentoring.” They may or may not need help. But it’s a useful way to focus on who might be struggling.

Or, we might sit down and say “20% of our effort should be devoted to iOS apps.” The number makes its ranking explicit. It ensures that everyone is on the same page about how important iOS is. At the end of every month, we can look at how many people are working on iOS. If the percentage is less than 20%, we may need to seriously ask ourselves if iOS is as important to us as we thought, or if we need to make a bigger effort hiring iOS expertise.

I heard today that Microsoft is getting rid of stack ranking. I hope they find a process that works better for them.

But I believe it is possible to use stack ranking in a mindful way. We don’t have to be slaves to the process. It’s not an excuse for making dumb decisions, just because the stack rank told us to do so. It’s a tool for guiding decisions and clarifying our priorities.

Software Engineer. Trying new things @tilt_dev. Formerly @Medium, @Google. Yay Brooklyn.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store