The short answerEquivalence is supported only when the complete paired confidence interval lies strictly inside the negligible-change band you justified before reviewing outcomes. An interval that touches or crosses either margin is inconclusive—it is not proof of a meaningful difference.
Different question, different testA non-significant McNemar result asks whether direction changed detectably. It cannot prove no meaningful change. This page asks whether the remaining uncertainty is narrow enough to fit inside a prespecified practical margin.

Freeze the margin, preserve the pair, narrow the claim

Margin before outcomeName its source, owner, version, date, and rationale. The calculator supplies no universal acceptable margin.
Same analysis unitMatch the exact prompt, planned repetition, engine and surface, market, access condition, and binary endpoint.
Bounded conclusionPassing supports compatibility inside this band for this panel—not equality, invariance, causality, durability, or interchangeability.

Use this calculator only for one fixed binary endpoint such as confirmed brand mention, confirmed brand/domain citation, or confirmed active recommendation. It does not fit rankings, visibility scores, sentiment, counts, unmatched samples, or two independent groups.

Browser-local equivalence record

Ask whether the plausible change stays inside your margin

Name the decision and margin, affirm only supported assumptions, then add one row per exact matched prompt-and-repetition unit.

01 · Analysis identity

Name one endpoint and two comparable periods

Enter the study or decision name.
Enter the organization.
Name the reviewer.
Define one binary endpoint.
Enter the before label.
Enter the before start.
Enter the before end.
Enter the after label.
Enter the after start.
Enter the after end.
Enter the engine and surface.
Enter the market and language.
Enter the access condition.
Name the panel and version.
State the decision.
02 · Prespecified margin and family

Document what “negligible” meant before the result

The symmetric band is −margin to +margin in absolute percentage points. No value is prefilled.

Enter a margin greater than 0 and below 100.
Name the margin source.
Enter the source version and date.
Name the margin owner.
Enter a whole number from 1 to 999.
Document the margin rationale.
State the eligibility and exclusion rule.
03 · Reviewer-supported gates

Affirm only what the record supports

Unchecked gates do not hide descriptive arithmetic. They withhold the equivalence decision.

04 · Matched evidence rows

Preserve every supplied pair and exclusion

Blank rows are ignored. Missing, failed, unavailable, unreviewed, and ineligible states never become absence. The case-sensitive row key is stable row ID + repetition ID.

#Stable row IDExact promptRepetitionBeforeAfterBefore evidenceAfter evidenceConditionsNoteAction
Maximum 250 supplied rows. Duplicate keys exclude every duplicate instance.
05 · Calculate or challenge the rule

Run the supplied record or load a worked branch

Complete the identity, margin record, and at least one matched evidence row.

Formula and eligibility rules

  • Eligible matched pair = both periods have reviewed binary evidence—confirmed presence or confirmed absence—for the same row key under confirmed matching conditions.
  • Stable absence = absent before and after. Gain = absent before, present after. Loss = present before, absent after. Stable presence = present before and after.
  • Paired change = (gains − losses) ÷ eligible pairs, in percentage points.
  • Paired score interval = Newcombe method 10: Wilson score intervals for both marginal proportions, combined with the continuity-corrected within-pair correlation documented for correlated binary data.
  • Equivalence rule = the complete 1 − 2α interval must be strictly above the negative margin and strictly below the positive margin. Touching either bound fails strict inclusion.
Why an independent-proportions calculator is not a substituteThese observations are paired because the same planned unit is measured twice. Treating the periods as independent discards the transition structure and applies the wrong variance model.

What the result can and cannot support

A supported result means the score-based interval for this fixed binary endpoint, matched panel, conditions, and evidence window lies inside the supplied practical band. It does not mean the periods are equal, unchanged in every respect, interchangeable, permanently stable, adequately powered by design, commercially acceptable for every stakeholder, or caused by—or protected from—any intervention.

An inconclusive result means equivalence was not established. The interval may touch or cross a margin because the point change is large, the evidence is sparse, the panel is variable, or the interval remains broad. It is not automatically evidence of a meaningful difference.

Exploratory or multiple-endpoint use receives the descriptive interval but no unadjusted equivalence decision. This calculator does not select a multiplicity correction, choose a margin from observed outcomes, compute sample size, or certify a measurement program.

Method sources

These sources support the statistical construction. They do not choose a defensible margin, validate a supplied prompt panel, confirm matching or independence, or replace statistical review for a consequential decision.

Frequently asked questions

How do you test whether matched AI visibility results are equivalent?

Choose a negligible-change margin before reviewing outcomes, preserve the same binary prompt-and-repetition units in both periods, calculate a paired-proportion confidence interval, and support equivalence only when the complete interval lies strictly inside both margins and every design gate is satisfied.

Does a non-significant before-and-after test prove no meaningful change?

No. Failing to detect a difference is not evidence that the plausible difference is small enough. Equivalence requires a prespecified margin and an interval that fits completely inside it.

What if the interval touches one margin?

The result is inconclusive under the strict rule. Boundary contact is not rounded into equivalence, and it is not converted into proof of a meaningful difference.

Should missing or failed answers count as brand absence?

No. Missing, failed, unavailable, unreviewed, and ineligible observations remain excluded evidence. The memo discloses coverage and every supplied row disposition.

Does this calculator upload observation data?

No. Validation, calculation, memo generation, copying, and Markdown download happen locally in the browser.