Define what is supposed to stay comparable
Create a comparison plan before the next run. Keep exact buyer questions, market, language, observation surface, repetitions and scoring definitions aligned. Preserve model or product identifiers and relevant tool settings. Record any platform change that prevents a direct match.
Version the question set. If a new question is useful, add it to a new cohort rather than silently replacing a weak result in the old set. When a model version is unavailable, state that limit instead of claiming the environment was unchanged.
Separate completion from outcome
Synthetic method example, not client performance: the first run plans 24 observations, obtains 18 valid answers and records 6 brand mentions. The next run plans 24, obtains 20 valid answers and records 7 mentions.
- Coverage changes from 75.0% to 83.3%; that is improved completion, not automatically improved brand perception.
- The valid-answer rates differ by about 1.7 percentage points. The valid sets differ, so that descriptive change is not proof of a causal improvement.
- Also inspect the observation units valid in both runs, and disclose which questions or surfaces remain incomparable.
| Synthetic run | Planned | Valid | Missing | Mentions | Valid-answer mention rate |
|---|---|---|---|---|---|
| Baseline | 24 | 18 | 6 | 6 | 33.3% |
| Remeasurement | 24 | 20 | 4 | 7 | 35.0% |
Keep four different outcomes in separate records
Preserve first-touch, last non-direct touch and the customer’s self-reported source separately. Unknown is a useful value. An unobserved referral must not be filled in with a convenient AI attribution.
| Outcome | Evidence needed | What it does not prove |
|---|---|---|
| Brand visibility | Original answer and the recorded mention/citation rule | That someone visited or bought |
| Referral visit | A measured website session and observed source data | That the visitor became a qualified lead |
| Qualified inquiry | Contact and demand evidence reviewed in the CRM | That a contract was signed or paid |
| Confirmed receipt | Contract and manually confirmed bank-receipt evidence | That AI caused the payment, or that full revenue and profit accounting is complete |
Describe attribution at the strength the evidence permits
Document the work completed between runs, its publication receipt and the dates. Record concurrent changes such as a new product page, a trade event, a paid campaign or a model update. These may help interpret the pattern without proving which change caused it.
A controlled comparison may support a stronger inference than two before-and-after counts, but only when its design and limits are actually documented. Do not assign an invented “AI contribution percentage” to every new contract.
Google describes AI-feature traffic within its search reporting. That is useful context for a search-traffic report; it does not turn an API observation or an entire customer journey into measured search revenue.
Sources for this section: Google Search Central — AI features and your website
Make the renewal discussion about evidence and completed work
Present what was delivered, what was published, how many planned observations were valid, what changed and what remains uncertain. Link the interpretation back to the underlying evidence and versioned delivery.
Commercial completion and measurement interpretation are separate. In the existing manual workflow, payment confirmation, task completion and customer acceptance have their own conditions. A mention-rate change cannot replace those conditions or guarantee renewal.
- Keep the service fee, provider usage, research labor and other actual costs distinguishable where they are recorded.
- Mark missing cost or attribution data as missing; do not present a complete margin ledger before it exists.
- Propose the next period’s scope from unresolved buyer questions and evidence gaps, with a named review point.
A practical acceptance checklist
The current ecosystem has not completed production acceptance for automated comparable remeasurement, complete attribution, full costs or renewal operations. This guide explains the method and acceptance criteria; it is not a claim that those outcomes have already been delivered.
- The report identifies both protocol versions and the observation units that can be compared.
- All rates show valid and planned denominators; failures remain visible.
- Original answers, source checks and publication receipts can be inspected.
- Visits, inquiries, contracts and confirmed payments retain their separate evidence.
- Conclusions distinguish observation, hypothesis and any supported causal inference.