How we collect data
How we collect data
How a question reaches the AI assistants, how many times, in what language and from what country, and what happens when an assistant doesn't answer.
Survelium asks your prompts to AI assistants, the same way a person would, and keeps every answer in full. All the numbers in the app are calculated from these answers.
A check
A check is one pass through all of a brand's active prompts:
answers per check = active prompts × AI models × runs per prompt
- The AI models are the same for all brands. See AI models.
- Runs per prompt depend on your plan (on the Billing page: "Sampling depth"). With 3 runs, the same prompt is asked 3 times to the same model, and each run is a separate answer. Assistants never answer the same way twice, so more runs give a more stable number.
Example: 10 active prompts, 7 models, 1 run per prompt = 70 answers per check.
How often
- The first check starts as soon as you create the brand.
- After that, checks run automatically, once a day, according to your plan's frequency (on the Billing page: "Check frequency").
- A new or reactivated prompt joins the next check; it doesn't run right away.
- An inactive brand (the "Active" switch in the brand settings) and a paused prompt are no longer checked. Their history stays.
The check history, with the status of each check, is in the brand settings, on the Check run history tab.
Language and country
You choose them for each brand (see Brands):
- Language: each prompt is asked with the requirement that the assistant answer in the brand's language. Write your prompts in the same language.
- Country: we send it as the location to the assistants that accept one — ChatGPT, Copilot, Google AI Mode and Google AI Overview. The others answer without a specific location, so their answers may be less local.
When an assistant doesn't answer
- A failed run is retried automatically a few times. If it still doesn't succeed, it stays failed and doesn't count in any number — it isn't counted as zero.
- An empty answer counts as a failure, with one exception: Google AI Overview. There, "Google showed no AI Overview" is a real observation (you don't appear in that search), so it counts.
- A check is failed only if all of its runs failed. If some succeeded, it is completed and the numbers are calculated from what was received.
- If an assistant is unavailable for a day, that day's numbers come only from the others. To see a single assistant, use the model filter.
When it doesn't run
A check is skipped, with the reason shown in Check run history, when:
- the trial is used up ("Trial quota reached");
- the subscription is no longer active ("Subscription inactive");
- the workspace has gone over its plan limit ("Over quota").
See Billing and plans.
Read next
- AI models — which assistants we ask and what data each one gives.
- How we read an answer — mentions, position, sources, competitors, sentiment.