Ranking question

When ranking is the right question

Ranking forces trade-offs, and that is its entire value. Ask people to rate six council priorities on a five-point importance scale and most will rate five of them "very important", which tells you nothing. Ask them to put the six in order and they have to decide what comes second, which is the decision you actually needed them to make.

Use it when the options genuinely compete for the same fixed resource — budget, time, one appointment slot, one policy. Do not use it when the options are independent, because forcing an order on things that do not compete invents a preference that does not exist.

Full ranking versus top-N

A full ranking asks for every option to be placed. A top-N asks only for the leading few — "drag your top three into order" — and leaves the rest unranked.

Top-N is usually the better question. People have clear views about their first and second choices and are effectively guessing by position seven, so a full ranking of ten items collects three real answers and seven pieces of noise, at the cost of a much longer interaction. NumoForms's ranking question supports both full and top-N.

A worked example

"Drag these into order, most important first. You can rank as few as three."

• More frequent bus services
• Safer cycle routes
• Cheaper parking in the town centre
• Better pavements and crossings
• Electric vehicle charging points

Two ways to read the result. Mean rank — average position across respondents, lower is better — is the usual headline, but it is a mean of ordinal data and it buries disagreement: an option ranked first by half your sample and last by the other half lands in the middle, looking unremarkable, when it is in fact the most contested thing on the list. So also look at first-place share and the full distribution of positions per option. If two options have similar means and wildly different spreads, that is your story.

Common mistakes

• Too many items. Above about seven, ranking becomes a sorting chore and people give up or ballpark it. Watch completion rate around ranking questions specifically.
• Options of different scale. "Fix the potholes" against "reform local government finance" is not a fair contest; respondents rank the concrete thing higher because they can picture it.
• Reading mean rank as importance. It is relative position only. Everything on the list might be trivial to your respondents, and the ranking will still produce a confident-looking order.
• Fixed option order. Items near the top of the list get ranked higher purely from position. Randomising the starting order spreads that effect out — see the note below.

Accessibility

Dragging is a pointer gesture, and a question that can only be answered by dragging excludes keyboard and switch users. WCAG 2.2 addresses this directly under Dragging Movements (2.5.7), which requires a single-pointer alternative to any drag operation. Treat a keyboard-operable path through a ranking question as a requirement rather than a nicety; the reasoning is set out on the accessibility page.

In NumoForms

Ranking is one of the 16 question types: respondents drag options into order, and the question can be configured for a full ranking or a top-N. Answer randomisation is available and is seeded per page load, so a respondent who goes back does not find the list reshuffled underneath them, and options marked fixed — "None of these", typically — stay put. Results and CSV export give you the positions per respondent so you can compute mean rank or first-place share yourself. See the question types page for the rest, and closed-ended questions for when a simple pick-one is the better tool.