You Cannot Check This Once

Open ChatGPT, ask it to recommend distributors for your category, and write down the four names it gives back. Do it again an hour later. You will probably get a different four. This is not a glitch, and it is not you doing it wrong.

SparkToro ran this experiment properly at the start of 2026, thousands of repeated prompts across ChatGPT, Claude, and Google's AI results. The same prompt returned the same list of brands less than 1% of the time. The same list in the same order showed up less than once in a thousand tries. Ask the same question a hundred times and you get close to a hundred different answers.

So a single check tells you almost nothing. You caught one roll of the dice. The number that means something is your appearance rate. Out of twenty runs of the same buyer question, how many name you at all. SparkToro found that this rate, measured across many runs, holds far steadier than where you rank in any one answer. Rank on a single pull is noise. Appearance rate across many pulls is signal. That distinction is the whole game here.

Ask Marketing for the Number

The reflex is to assume someone already has this handled. Marketing added schema. The agency sends a monthly report with the word visibility in it. A consultant ran an audit in the spring. The box feels checked.

So ask one question. What is our appearance rate this month, on the questions our buyers actually ask, and what was it last month. Most of the time you get a slide about AI readiness and a recommendation to publish more content. That is activity, not measurement. The agency is tracking whether your pages can be crawled. Nobody is tracking whether the AI names you when a manufacturer or distributor asks who to buy from.

Those are different things, and only the second one maps to revenue. The good news, and there is some good news in here, is that you do not need the agency to measure it. You need a list of questions and an hour a month.

Build It: The AI Visibility Audit

Setup is short. Create a Claude Project and call it AI Visibility Audit. The project does one job. It reads a raw AI answer you paste in and hands back one clean, structured row you can log. The running happens in the real tools your buyers use. The counting happens in the project. Start by writing the questions. Four kinds cover most of how buyers actually ask:

The category recommendation. Worded the way a buyer words it. "We are a 200-employee manufacturer in Ohio. We need a fastener distributor that can hold safety stock and ship same-day on emergency orders. Who should we look at?" This is the prompt that builds the shortlist.
The head-to-head. "Is [you] or [a known competitor] better for [use case]?" Run it, and also run a version that names neither of you, to see whether you surface on your own.
The criteria question. "What should I look for in a [category] supplier?" These return lists. You want your customer material to be the source those lists get built from.
The problem-first question. No vendor named, just the pain. "We keep running out of bearings on second shift and the line goes down. What are our options?" This is where most companies are completely invisible, and it is the most honest test of the four.

Run each question in the actual tool, logged out, so it does not personalize the answer to you. Paste the answer into the project. Get back one row. Run the same question again, paste again, another row. Twenty runs per question gives you an appearance rate that means something. Set the project's instruction once and do not fiddle with it mid-audit. Here is the instruction:

You are my AI visibility logger. I will paste in the raw answer an AI tool gave to one buyer question. Return exactly one plain-text row I can drop into a spreadsheet: QUESTION: [which question this was] RUN: [run number, e.g. 4 of 20] NAMED_US: [yes / no] (yes only if "Acme Fasteners" actually appears) OUR_POSITION: [1st, 2nd, 3rd, or "not named"] OUR_FRAMING: [the exact phrase used to describe us, or "none"] OTHER_VENDORS: [every other company named, in the order listed] CITED_SOURCES: [any links or sources shown, or "none"] Rules: - Read only what is in the pasted answer. Do not add companies from your own knowledge. Do not guess. - If a company name looks invented, flag it. Do not tidy it up. - Quote our framing word for word. Do not improve it.

How to Read the Output

Appearance rate is the headline number. Of all your runs of one question, the share that named you. Watch it over months, not days. Everything else is supporting detail.
The competitor set is who shows up next to you. If it is the same three names on most runs, that is who the AI thinks you compete with. Sometimes that list is a surprise, and the surprise is the useful part.
The cited source is what the AI leaned on. If it keeps pointing at one review site or one directory, that is where your effort belongs, not on your own homepage.
The framing is the sentence the AI uses to describe you. "A solid option for small orders" and "holds safety stock and ships same day" are not the same outcome. Track the adjective, not just the mention.
Spot check before you trust the totals. After your first handful of rows, pick two and read them against the actual answers you pasted. If the project logged you as named when you were not, or invented a competitor, you catch it now, before a wrong number goes into the log and turns into a wrong trend.

An Honest Take

Most of the AI-visibility industry is selling you the workout and never letting you near the scale. An agency will happily run a one-time audit, hand you a forty-page deck, and recommend a retainer. The one number that would tell you whether the retainer is working is the number they are slowest to give you. I think that is on purpose. A monthly appearance rate makes effort accountable, and accountable effort is uncomfortable for anyone selling effort by the month. So measure it yourself. It does not require their tooling, it takes an hour, and once you have your own number, every pitch you hear afterward gets easier to judge. The teams that own their own measurement quietly stop getting sold to. That is the real reason to build this, more than any single reading it produces.

What to Watch Out For

Personalization. Run the prompts logged into your own account and the tool already knows who you are, then tilts toward you. Run them logged out, or in a temporary chat. Otherwise you are measuring your own reflection.
Invented companies. AI makes up vendor names that sound right. Before you celebrate a competitor's absence or panic about a new rival, confirm the names are real companies that actually exist.
One platform is not the market. ChatGPT, Perplexity, Google's AI, and Copilot do not agree with each other. Pick the two your buyers actually use and measure both. Do not let ChatGPT stand in for all of them.
Phrasing drift. Small wording changes move the results. Lock your question list and reuse the exact text every month, or you are comparing two different questions and calling it a trend.
The number moves slowly. Last issue's story-extraction work will not show up next week. Citation influence takes weeks to settle, sometimes a couple of months. Measure monthly. Judge quarterly. Do not yank the strategy after one flat reading.

Make It Repeatable

Run the audit once and you have a baseline. That is already worth more than what most of your competitors have. Run it every month and you have the only thing that tells you whether the AI-visibility work is paying off. The baseline is the point of this issue. The trend is the point of running it again.

Set the cadence to once a month and put one person's name on it, not a committee. The run takes under an hour once the project is built, and most of that is the project doing the counting while you get coffee. If your plan does not include scheduled or automated runs, you can still do the whole thing by hand from the project. Paste the questions, log the rows yourself. Slower, same result. Do not let the lack of automation be the reason you skip it.

And this closes the loop on the last issue. The story-extraction project gives marketing the raw material. This audit tells you whether the material is doing its job. Without it, you are publishing customer stories into the void and hoping. With it, you can point at a number and say it went from 2 in 20 to 9 in 20 since we started. That is the sentence that keeps the program funded after the novelty wears off.

The Bottom Line

The AI answer now decides who makes the shortlist. You can feed it better material, and you should. But you cannot manage what you refuse to look at, and most companies are refusing to look. They hold strong opinions about their AI visibility and zero measurements of it.

Build the audit. Write the four questions your buyers actually ask. Run each one twenty times, logged out, and count how often you show up. That is your baseline. Run it again next month. The companies that win this are not the ones with the loudest AI strategy. They are the ones who can tell you their appearance rate, what it was last quarter, and which competitor keeps showing up in the answer they want.