Galley
A/B test analysis

Know whether your test actually won.

Your testing tool shows a green arrow and a percentage. Galley tells you whether that number is a real result or just noise, in one plain sentence, with the statistics done properly underneath.

Free to use. No account. Your CSV is read in the browser, never uploaded.

01The problem

Most winners were never checked for luck.

You run a test for a week. The dashboard says the new version is up 12 percent. You ship it, and revenue does not move.

That happens because a percentage on a dashboard is not a result. It is a snapshot of two numbers that would differ from each other even if the two versions were identical. The question that matters is whether the gap is bigger than chance can explain, and almost no ecommerce dashboard answers it clearly.

Galley answers it. It takes the same counts your platform already gives you and runs the tests a statistician would run, then says what they mean in a sentence you can forward to your client or your boss.

The same test, read on different days
One experiment, measured every day for three weeks. On day two it looked like a 23 percent win, which is exactly when most people call it and ship. The gap was noise, and it took another fortnight to settle where it really was: down 7.7 percent. Nothing about the test changed, only the amount of data behind it.
02What you get

Advanced statistics, made readable.

Everything below runs automatically. You do not choose a test or set a parameter.

A straight verdict

Ship it, keep running, or call it flat. One sentence at the top of the report, written in words rather than notation, with the confidence level and the corrected p-value stated plainly beside it.

The money check

Conversion rate can rise while revenue per visitor falls, usually when the winning version sells more of something cheaper. Galley reports revenue per visitor and average order value alongside the rate, so a hollow win shows itself.

Broken tests, caught early

If your traffic split came out uneven, the whole experiment is suspect no matter how good the numbers look. Galley runs a sample ratio mismatch check and warns you before you read anything else.

Import instead of retyping

Export the results from your testing platform, drop the file in, and the form fills itself. Galley recognises the common export shapes and lets you map the columns by hand for anything it does not know.

Under the hood

A real statistics engine. You just never have to run it.

Galley is not a spreadsheet formula with a nice skin on it. It is 906 lines of Python built on scipy, statsmodels and numpy, the same libraries that sit behind published research and professional data science work. All of it executes the moment you press Analyse, and none of it is your problem.

backend/engine.py verbatim
def perform_srm_test(observed, expected_split=None):
    n = len(observed)
    if expected_split is None:
        expected_split = [1 / n] * n
    total = sum(observed)
    expected = [total * p for p in expected_split]
    stat, p = chisquare(observed, f_exp=expected)
    return stat, p

The sample ratio mismatch check, exactly as it runs. It is what tells you the traffic split broke before you trust a single number above it.

What runs on every analysis
  • Two-proportion z-teststatsmodels
  • Chi-square omnibus across all variationsscipy
  • Chi-square goodness of fit for the traffic splitscipy
  • Holm-Bonferroni, Bonferroni or Benjamini-Hochberg correctionstatsmodels
  • Mann-Whitney U on the revenue distributionscipy
  • Bootstrap resampling, ten thousand drawsnumpy
  • Beta-binomial posteriors and expected lossscipy
  • CUPED variance reduction with a t-testscipy
  • Power curves and sample size planningstatsmodels
  • Simpson's paradox detection across segmentsengine
Lines of Python
906
Statistical routines
21
Bootstrap draws
10,000
Settings you configure
0
03How it works

Three steps, about a minute.

Bring your numbers

Drop in a CSV from your testing platform, paste the results table straight out of the dashboard, or type four numbers per variation: visitors, conversions, revenue and units.

Say how long it ran

Duration and start date, so the report can tell you whether you have collected enough data and how much longer you would need to reach a call.

Read the verdict

Significance, confidence intervals, the revenue check and the traffic split, all in one page you can export to PDF and hand to whoever signs off the change.

04Imports

Bring the export you already have.

Galley reads the export formats below without any setup. Anything else falls through to manual column mapping, which takes about fifteen seconds and works with any spreadsheet.

Optimizely Adobe Target

Plus any spreadsheet at all, through manual column mapping.

05AI analysis

Bring your own key for the deep dive.

Every number on the report, the verdict, the significance tests and the charts, is produced without any AI at all. It is arithmetic, and it costs nothing. The AI layer is optional and sits on top of it.

Connect an API key and the same figures get read back to you: a written report structured for whoever has to sign off the change, and a short explanation under each chart saying what it shows and why it matters.

The written report

An executive summary, a trade-off analysis across conversion rate, average order value and revenue per visitor, a risk assessment weighing the Bayesian probabilities against your test duration, and a recommendation. Written for a stakeholder, not for an analyst.

Explanations under each chart

Bootstrap distributions and Beta posteriors are not self-explanatory. Ask for it and each chart gets a couple of sentences telling you what you are looking at, grounded in your actual numbers rather than a generic description of the chart type.

Supported providers, one key each
Claude OpenAI
How your key is handled
  • It is saved in your own browser, one key per provider, and stays there between visits.
  • It travels with the request so the call can be made on your behalf, then it is used and dropped. It is never written to a database and never logged.
  • There is no account for it to be attached to, and Galley holds no key of its own.
  • Your provider bills you directly at their rates. Galley adds nothing and takes nothing.

Prefer not to use AI at all? Galley also writes a rule-based report with no key and no external call, covering the same result in plainer terms.

06Privacy

Your file stays on your machine.

The CSV is parsed in your browser. It is never uploaded, and nothing about the file leaves your computer. Only the summary counts you see in the form, visitors and conversions and totals, are sent to run the analysis. There is no account, so there is nothing stored against your name.

Stop shipping tests that never won.

Bring the export from your last experiment and find out what it actually said.

Analyse a test