INSIGHTS / B2B GEO RESEARCH

Can Candidate Order Change an AI Vendor Recommendation?

Yes. In controlled tests, changing the order of options can change an AI’s choice. Check a vendor comparison before treating its first choice as a firm preference. This does not mean that putting your brand first on a page will make a live AI service choose it. A list you supply and a search of the web are different tests.

6 min read

TL;DR

  • Save the order of vendors with each AI comparison. The list is part of the test input.
  • Models and tasks show different position preferences. There is no slot that always wins.
  • An answer can switch between two close options. A switch alone does not prove that the new choice is worse.
  • Try the same facts in more than one order, keep repeat runs and check the reasons against the brief. Do not select just a favorable screenshot.

Why the Candidate List Belongs in the Record

UC Irvine’s August 2026 research news post describes models changing choices when the same options are reordered. It raises a useful question for a B2B review. Did the facts change, or did only their order change?

For a generative engine optimization (GEO) report, save the full prompt when a team supplies vendors and asks AI to choose. The result concerns those supplied options. It does not show how often the service would find your brand on its own.

First Position Was Not Favored by Every Model

A University of Granada preprint, October 2026, tested product choices with 2,000 generated queries and 20 random list orders per query. In its five-option test at temperature 0, GPT-OSS 20B chose the first slot 25.38% of the time and the fifth 17.28%. LLaMA 4 Scout showed the reverse: 17.11% first and 20.99% fifth.

For GPT-OSS 120B, the shares were 23.18% first and 17.93% fifth. Each share averages over queries and random list orders, using valid responses. It is not a named vendor’s chance of winning. With five slots, the no-position-preference reference is 20%.

Figure 1. Position Preferences Varied Across ModelsSelection share (%) · five candidates, temperature 0
First positionFifth position
GPT-OSS 20BFirst position: 25.38 · Fifth position: 17.28
GPT-OSS 120BFirst position: 23.18 · Fifth position: 17.93
LLaMA 4 ScoutFirst position: 17.11 · Fifth position: 20.99
01530

Peláez-González et al., October 2026 v1, Table 5; p(i) × 100. Two of five positions shown. Fixed product-catalog prompt, 2,000 queries × 20 orders; parsing errors and refusals excluded. The 20% reference is not a brand success target. Study, method and Table 5.

An Order Effect Is Not Always a Worse Choice

Yin, Vardi and Choudhary’s August 2026 PNAS Nexus study compared résumés and paint colors. In three-option tests at temperature 0, GPT-4o-mini chose the middle option for Plain colors 100% of the time. Claude 3 Haiku chose the middle for Harsh colors 83% of the time. These are different quality tiers, not a model ranking.

The paper separates fragile preferences from robust choices. For a buyer, that raises a further question. Did the answer switch between close options, or did the new choice fail the brief? A switch and a worse choice need separate proof.

An Order Effect Is Not Always a Worse Choice
Direct researchReported resultBoundary for a B2B review
Granada, October 2026GPT-OSS 20B: first 25.38%, fifth 17.28%; five-candidate testMarginal position shares; supplied retail candidates, not open-web brand visibility
PNAS Nexus, August 2026Middle choice: 100% GPT-4o-mini/Plain; 83% Claude 3 Haiku/HarshDifferent color tiers; not a supplier fit or purchasing study

Figure 2. Evidence comparison. These are independent teams and different experiments. One uses catalog recommendations; the other studies preference structure. Neither establishes a B2B conversion effect or a dependable website ordering tactic.

The Granada study also notes that order can break ties among close options. Its measures of worse choices have further limits. Do not treat every changed answer as proof that an AI chose an inferior supplier.

A Completed Review of an Order-Sensitive Comparison

This example is fictional. The outputs below show how a review record can work; we did not run this AI test. A team compares calibration services for one gauge type in one region. The same approved facts are present for suppliers A, B and C. Current quotes and availability are not yet checked.

A Completed Review of an Order-Sensitive Comparison
Review fieldCompleted fictional recordDecision supported
Fixed task and evidenceSame gauge type, service region and supplier fact cards in both promptsKeep the comparison scope unchanged
Input changePrompt 1 orders A–B–C; prompt 2 orders C–B–A; whole cards move intactOrder changed; descriptions and instructions did not
Illustrative outputsFirst answer chooses A; second chooses C; both cite regional coverageA discrepancy to investigate, not proof of a better supplier
Next check and reportRepeat the original order as well as reordered lists; retain all usable answers and failuresDo not claim a stable preference from one pair; verify fit and quotes separately

Move each whole fact card. If you shorten one card or add a stronger claim, you change the facts as well as the order. Keep each supplier’s ID fixed. Avoid adding labels such as “best” to one version. Those edits would change the test.

Repeat runs with a fixed order help show normal answer variation. Runs with a changed order add a different check. A small test can reveal a gap worth a closer look. It cannot measure a general position effect. Report the saved cases and their limits.

A supplier can make its facts clear and easy to check on comparison pages. These studies do not justify hidden instructions or a promise to control source order. Our comparison-page guide covers evidence for buyers.

Keep this check separate from requirement fit. A stable answer can still miss a must-have. A changing answer can still name two options that both fit. Those are different findings.

AI Candidate Order: Common Questions

Does choosing the first option prove position bias?

No. The first option may be the best fit. Keep the facts and task fixed while changing the order, and account for repeat-run variation. One choice cannot separate these causes.

Should every vendor be recommended equally often?

Not by name. A better fit may deserve more recommendations. The 20% reference concerns slots after five options are put in random order. It is not a rule that all five vendors deserve equal preference.

Can reversing a list remove the problem?

A reversal can reveal a gap, but it is not a cure for every case. Other slots and normal answer variation can still matter. Match the check to the decision. Two outputs are not a statistical audit.

Does this mean a website should put its brand first?

No. These tests put the options inside prompts. A live service may find and arrange its own sources. The studies do not show that moving a website section changes how often a brand is recommended.

Bring the Prompt, Candidate Cards and Saved Answers

If a supplied-vendor comparison is used as proof of AI visibility, bring the full prompt, fact cards, list order and saved answers to SemShift’s AI visibility audit discussion. We can discuss what the test shows and what evidence would support a narrower claim that can be checked again.