Ranking a list of twenty things in your head is hard. Picking the better of two is easy. Head-to-head ranking builds on that difference: instead of sorting the whole list at once, you answer a series of "A or B?" questions and let a procedure turn those answers into an order.
This works for almost anything you can put on a list: feature ideas, job offers, baby names, vacation spots, films, candidates for a shortlist. What changes from one method to another is how many matchups you have to answer and what you get at the end. This guide compares the four common approaches, shows the numbers, and walks through doing it in a free tool.
Why Matchups Beat Ranking Everything at Once
When you look at a full list and try to order it directly, you are holding many items in working memory and making dozens of comparisons silently. The ones near the top and bottom usually land in the right place. The middle of the list tends to come out in whatever order you happened to read it.
A matchup asks for one judgment at a time. You only have to answer whether this one beats that one, which is a question most people can answer quickly and consistently. Repeat that enough times with a sensible pairing strategy and the full order falls out. The strategy matters a lot, because a careless one can ask you hundreds of questions you don't need.
Four Ways to Run Head-to-Head Matchups
1. Single-elimination bracket
The sports-tournament format. Items are paired off, winners advance, losers are out, and you repeat until one item is left. It takes exactly n − 1 matchups, since every matchup eliminates one item and all but one must be eliminated.
The catch: a bracket finds a winner, not a ranking. The item that loses in the first round to the eventual champion might be the second-best item on the list, but the bracket never finds out. If all you need is "pick one" (which of these five restaurants tonight?), a bracket is fast and fun. If you need positions 2 through 10 as well, it isn't enough.
2. Round-robin
Every item meets every other item exactly once. That's n(n − 1)/2 matchups. You end up with a complete record: each item's wins, losses and win rate, which tells you how far ahead the leader is and which items are nearly indistinguishable.
The cost grows quickly. Double the list and you roughly quadruple the work. Round-robin is a good fit for short lists (under about 8 items) or when you specifically want per-item statistics, not just an order.
3. Adaptive sorting (merge sort or binary insertion)
Here the tool chooses each matchup based on the answers you've already given, the same way a sorting algorithm on a computer does. Merge sort ranks small groups, then merges ranked groups by comparing only the items at the front of each. Binary insertion adds items one at a time and finds each one's place by halving the range of possible positions.
Both assume your preferences are transitive: if you said A beats B and B beats C, the tool won't ask you about A versus C. That inference is where the savings come from. The worst case for either is about n·⌈log₂ n⌉ − 2^⌈log₂ n⌉ + 1 matchups, and real sessions often need fewer. (Plain insertion without the halving trick can need as many as a full round-robin, so the "binary" part matters.)
You get a complete order. Win/loss counts are recorded along the way, but since each item only meets some of the others, they are a partial record rather than the full table a round-robin produces.
4. Elo or Glicko-style ratings
Borrowed from chess. Each item starts with a rating; after every matchup the winner gains points and the loser drops, with bigger swings when an underdog wins. Glicko adds a measure of how uncertain each rating is.
Ratings suit open-ended situations: many voters each doing a few random matchups, or a list that keeps growing. There's no natural end point, though. The ranking keeps shifting as votes come in, and with few matchups per item the order can depend on who happened to be paired with whom. For one person ranking a fixed list, a sorting approach gets to a stable answer with fewer questions.
How Many Matchups Each Method Needs
The table below uses the formulas above. The merge-sort figure is the worst case, so a real session can come in lower. Elo has no fixed count, so the example assumes each item plays 10 matchups (10 × n ÷ 2, since every matchup involves two items).
| Method | 8 items | 16 items | 32 items | What you get |
|---|---|---|---|---|
| Single-elimination bracket (n − 1) | 7 | 15 | 31 | A winner only |
| Merge sort / binary insertion (worst case) | 17 | 49 | 129 | A full order |
| Elo ratings (example: 10 matchups per item) | 40 | 80 | 160 | An approximate order that keeps updating |
| Round-robin n(n − 1)/2 | 28 | 120 | 496 | A full order plus complete win/loss records |
Two things stand out. First, at 8 items the difference between merge sort and round-robin is small (17 versus 28), so either is fine. Second, at 32 items the gap is huge: 129 matchups is a long but doable session, while 496 is not something anyone finishes with their attention intact. For a deeper look at the arithmetic, see how many pairwise comparisons you need.
Handling Ties, Inconsistency and Fatigue
Ties
Sometimes two items really are equal to you. Forcing a pick produces a coin flip that looks like a decision, so a tie option is useful. In PairwisePro, a tie counts as half a win for each item in the win-rate figure, and for ordering purposes the two items keep their current relative position. Use ties for genuine toss-ups, not as an escape from a hard call. If you find yourself pressing "tie" constantly, the items may be too vaguely worded to compare (see the fatigue tips below).
Inconsistency (A > B > C > A)
People aren't perfectly consistent. You might prefer A to B, B to C, and still pick C over A if asked directly. This is called a preference cycle, and different methods deal with it differently:
- Round-robin records the cycle in full. The three items end up with similar win counts, which is an honest signal that they're close.
- Adaptive sorting never asks the third question, because it infers the answer from the first two. The cycle can't happen in your answers, but a single careless click can still put one item a few places away from where it belongs.
- Elo absorbs the contradiction into the ratings, which pulls the three items closer together.
Practical advice: answer from a stable frame of reference. Before you start, decide what "better" means ("more valuable to customers this quarter", "where I'd rather live for five years") and keep that question in mind for every matchup. If you realize you clicked wrong, undo it immediately rather than hoping it averages out.
Fatigue
Judgment quality drops when you're tired, and late matchups get answered faster and more carelessly. A few habits help:
- Trim before you start. Remove items that are obviously out (blocked, duplicates, already decided). Every item you cut saves several matchups.
- Word items so they can be compared. "Improve onboarding" against "Fix checkout bug" is hard if neither says what it involves. Short, concrete labels make each matchup faster.
- Take a break around the halfway mark on long lists. In PairwisePro your progress is saved in the browser, so you can close the tab and resume later.
- Use the keyboard. Arrow keys or 1/2 to pick, T for a tie, Z to undo. It's faster than clicking and keeps your attention on the items.
Step by Step: Rank a List Head-to-Head with PairwisePro
PairwisePro's ranking tool is a free head-to-head ranking generator that runs in your browser. There's no account, and your list stays on your device.
- Add your items. Open /tool/ and enter one item per line, or load one of the example lists to try it out. You need at least two items; the limit is 100. The tool shows an estimate of how many matchups to expect before you start.
- Start ranking. The tool uses an adaptive merge-sort approach, so it only asks matchups whose answers it can't infer from earlier ones.
- Answer each matchup. Pick the item that better fits your criterion. Use "It's a tie" for genuine toss-ups and Undo if you misclick. A progress bar shows how far along you are.
- Read the results. You get the full ranked list with each item's wins, ties and win rate.
- Share or export. Copy a share link that encodes the results, copy the list as text, or export to CSV or JSON for a spreadsheet or doc.
If your decision is better described by explicit criteria and weights (price, quality, risk), a weighted decision matrix may fit better. Our guide to decision matrix vs pairwise comparison covers when to use which, and the pairwise comparison overview explains the method in more depth. If you're weighing different software for the job, see our comparison of pairwise ranking tools.
Quick Recap
- A bracket is fastest but only finds a winner.
- Round-robin gives the most complete data but grows quadratically; keep it for short lists.
- Adaptive sorting (merge sort, binary insertion) gives a full order with far fewer matchups and is the best default for one person ranking a fixed list.
- Elo-style ratings fit crowds and lists that change over time.
Have a list waiting? Rank it head-to-head now. It's free, and a 10-item list usually takes a few minutes.