How to A/B test your videos after Google Optimize

What happened to Google Optimize?

Google Optimize split website visitors between versions of a page and reported the result in Google Analytics. Google's help page on the Optimize sunset says that Optimize and Optimize 360 are no longer available as of September 30, 2023, and that any experiment still running that day ended. The old Google Optimize product page now opens the general Google Marketing Platform page.

Google Analytics 4 (GA4) now accepts experiment data from outside testing tools, and Google publishes an experiment integration guide for the vendors that build them. The testing tool sends an experience_impression event when it assigns a visitor to a version. Google's GA4 integration help page explains that GA4 then builds an audience for each version and keeps an experience-variant dimension that you can use in Explore reports and BigQuery.

Google's A/B testing options, checked October 2026 on Google's help and developer pages

Option	What it tests	Status
Google Optimize and Optimize 360	Versions of a web page	Closed September 30, 2023
GA4 with a connected testing tool	Versions of a web page. The outside tool splits the traffic, and GA4 reports each version.	Available. The testing tool is a separate product from another vendor.
Firebase A/B Testing	Features, settings, and messages in mobile apps	Available for apps
YouTube Studio A/B test titles and thumbnails	Up to 3 titles, thumbnails, or title and thumbnail pairs on one YouTube video	Available on desktop when the channel has advanced features turned on

A page test can swap one embedded video for another, and the YouTube test stops at the title and thumbnail. Neither one sees what happens inside the player. To compare clicks on a button at the two-minute mark, the video player itself has to record which version each viewer saw and what that viewer did.

What to A/B test in an interactive video

Test the parts of the video that lead to the action you want. Pyxl's article on what to A/B test covers headlines, forms, and calls to action on web pages and in email. An interactive video has its own calls to action and lead forms, plus questions that pause the video. Change one element per test.

Video A/B test ideas and the metric that decides each one

Change	Version A and version B	Metric to compare
Button wording	"Learn more" or "Book a 15-minute demo" on the end screen	Clicks on that button per view
Button timing	Button at the product reveal, or only at the end	Clicks on that button per view
Button color or style	Brand color or a high-contrast color	Clicks on that button per view
Lead form timing	Form before playback, or at the two-minute mark	Leads per view and average watched percentage
Question placement	Question right after the key point, or at the end	Share of correct answers and average watched percentage
Captions	Captions on when the video starts, or off until the viewer turns them on	Average watched percentage
Navigation	A chapter menu, or one linear path	Average watched percentage and clicks on the main call to action

Before you spend a test on color, read CXL's article on which button color converts best. In a video, the scene behind the button changes from shot to shot, so check both colors for contrast in every scene where the button appears.

For a captions test, upload a VTT or SRT file, or create a transcript to get an English track. Select Make default on the track for the version that starts with captions on. Our help article on closed captions has the steps.

For a navigation test, a chapter menu lets viewers jump to the part they need. With dynamic personalized navigation, each viewer sees only the routes that match what you already know about them.

Our guide to video CTAs covers which kind of call to action fits each goal, which helps you choose the two versions.

How to A/B test an interactive video, step by step

1. Write the hypothesis and pick one metric

Write one sentence before you build anything, for example: "Changing the end-screen button from 'Learn more' to 'Book a 15-minute demo' raises demo clicks per view." Name the metric that decides the test and the smallest lift that would change what you do. Pick a metric the video records for every view, such as clicks on one button, answers to one question, leads, or watched percentage. Our guide to video engagement metrics explains what each one measures.

The button's position affects how long the test takes. Across 8.07 million views of 18,315 Mindstamp videos (2018 to October 2026), the average view played 46.5% of the video. A button at the end reaches only the viewers who stay, so a test on an end-screen button collects clicks more slowly than a test on a button in the first minute.

2. Build the two versions

You can build the versions in one video or in two:

• Add both buttons to one video at the same point on the timeline, then use conditional logic to show version B's button only when a variable named variant equals b. Set version A's button to show when variant does not equal b, so a viewer who arrives without a version still sees a button. Conditional logic is on Pro and higher plans.
• Duplicate the video with its interactions and change one element in the copy. This method needs no conditional logic. Each copy counts as a video on your plan, and the results sit in two reports.

The one-video method keeps everything in one report. The help article on creating a video with conditional logic shows where the Show if and Hide if settings are.

3. Split viewers at random

Mindstamp does not assign viewers to versions at random, so the split comes from the tool that sends the traffic:

• A website testing tool shows half of visitors a page whose video embed ends in ?variant=a and the other half a page whose embed ends in ?variant=b.
• An email platform's A/B split sends half of the list a share link that ends in ?variant=a and the other half a link that ends in ?variant=b.

Keep each person in one version. Website testing tools do this with a cookie, and in email each recipient gets one link. Do not split by date (version A this week, version B next week), because the audience changes from week to week.

4. Tag each view with its version

Add the version to the player address as a URL parameter:

https://share.mindstamp.com/w/VIDEO_ID?variant=b
https://embed.mindstamp.com/e/VIDEO_ID?variant=b

The player reads name, email, phone, and custom_id as the viewer's identity, and a few parameters such as t and autoplay control playback. Other parameters, such as variant or utm_source, become variables that Mindstamp stores with the view on Core and higher plans. If the version is in your page address and not in the embed code, use the short script in our help article on passing URL parameters into an embed. It copies the page's parameters onto the Mindstamp player.

5. Set the sample size before launch

Decide how many views each version needs before the test starts. The number depends on your current rate and the smallest lift you want to detect:

Views needed per version, from the standard two-proportion formula with a two-sided 5% significance level and 80% power. Calculators differ slightly by method.

Current click rate	Rate you want to detect	Views per version
5%	7%	About 2,200
10%	12%	About 3,800
10%	13%	About 1,800
20%	25%	About 1,100

Small lifts need many more views. A move from a 10% click rate to 11% needs about 14,750 views per version. If a video gets a few hundred views a month, test a large change, such as a different offer, instead of a new shade of the same button. Evan Miller's sample size calculator runs this math for your own numbers.

6. Wait for the full sample

If you check the results every day and stop when one version looks ahead, you will often pick a winner that is not real. In the 2010 article How Not To Run an A/B Test, Evan Miller works through an example: a change with no real effect, checked after every observation at a 5% significance level, looks like a winner 26.1% of the time. Miller's fix is to commit to a sample size in advance. Run the test in whole weeks so the result includes weekday and weekend viewers, and read it once the planned views are in.

7. Compare the results in Mindstamp

For one video with both versions, open Reporting, then Views. Filter to the video, type variant in the Variable filter, and enter a or b as the value. Compare watched percentage and interactions for each version. For one row per view with a column for each interaction and each variable, export the Views list as a CSV file with Include Interactions, Variables, and Location selected. Then count clicks by the variant column in a spreadsheet. That option appears only when the list is filtered to one video, group, or playlist. The help article on using video data to improve messaging describes the same filter.

For two copies, open each video's report and compare the same metric over the same dates. Leave your own test views out of the count. Previews in the editor do not record normal viewer responses, but a test of the live link does.

8. Send video events to GA4 or your testing tool (optional)

Our Google Analytics integration sends watch-progress events to GA4 at 10, 25, 50, 75, 90, 95, and 100 percent of the video. Answers and clicks stay in Mindstamp reports. If your testing tool or GA4 needs a click inside the video as its goal, listen on your page for the player's viewer_interaction message and pass it on. The embed events guide lists every event, and JavaScript player events are on Core and higher plans. The example below passes each click or answer to GA4 through the Google tag (gtag.js) that is already on your page.

const playerOrigins = ['https://embed.mindstamp.com', 'https://interact.video']; window.addEventListener('message', (event) => { if (!playerOrigins.includes(event.origin)) return; const message = event.data; if (!message || message.event !== 'viewer_interaction') return; // Send the button or question label only, never viewer details gtag('event', 'video_interaction', { interaction_label: message.info.data.parent_value });
});

Send only the label. Player events include the viewer's name and email when the viewer is identified, and Google's Analytics PII guidance says no data Google could recognize as personally identifiable information, such as email addresses, may be passed to Google.

How to read the result

• Decide with the metric you named in step 1. Other numbers in the report can suggest the next test.
• Check that both versions got about the same number of views. A large gap in a 50/50 split can point to a setup problem, such as a link without the parameter.
• If the difference is not significant at the planned sample size, treat the versions as equal and keep the one that is easier to maintain.
• Write down the result, including a loss, and the test you will run next.

Common questions

Does Google still have an A/B testing tool?

Google Optimize, Google's website testing tool, closed on September 30, 2023. Google's help page now sends website testers to third-party tools that report into GA4, and it names AB Tasty, Optimizely, and VWO. Firebase A/B Testing covers mobile apps, and YouTube Studio tests titles and thumbnails.

Can Google Analytics 4 run an A/B test by itself?

No. GA4 does not split traffic. It receives experiment data from a connected testing tool and reports each version as an audience.

Can I A/B test a YouTube video?

In YouTube Studio you can test up to 3 titles, thumbnails, or both on one video. When the test ends, YouTube shows the version with the highest watch time to all viewers, and it says a test should finish within two weeks. The test does not cover buttons or questions inside the video. For those, add interactions to the video in a player that records them. In Mindstamp, a YouTube link works for testing only, so upload the original file before you publish the test.

How long should a video A/B test run?

Run it until each version reaches the sample size you set before launch, in whole weeks. A video with low traffic can take months to reach a few thousand views per version, so test larger changes on those videos.

What is the difference between A/B testing and multivariate testing?

An A/B test compares two versions that differ in one element. A multivariate test changes several elements at once and compares every combination, so it needs much more traffic. Pyxl's walkthrough of Google Optimize and Mindstamp for A/B testing interactive videos (published 2021, updated 2024) explains both methods. Its Optimize setup steps no longer work because the product closed.

Plan your first video test

Start with your most-viewed video and test the wording of its main button. The most-viewed video reaches its sample size first, and a wording change is the smallest edit to build. Write the hypothesis, add both buttons with a variant condition, send the two links through your testing tool or email platform, and read the result when each version reaches its planned views. Our page on interactive video A/B testing shows more test ideas. For a low-traffic video, read our post on testing video on a budget.

Sources and further reading

• 99firms, A/B testing statistics: a list of third-party figures about A/B testing. An earlier version of this article quoted its figure that 58% of companies use A/B testing for conversion rate optimization. The list credits that figure to Invesp. We could not open Invesp's page to check the original source and year, so we removed the figure.
• Outgrow, 9 must-have conversion rate optimization tools in 2020: a 2020 list of conversion tools. An earlier version of this article quoted a claim from it that testing can raise conversion rates by up to 300%. We could not trace that figure to a study, so we removed it.