This guide covers how to run an A/B test for a website on the server instead of in the client (browser). For example, you might want to redirect your users to entirely different versions of your website, or test different fonts or different features that need to run a different code path on the server depending on which variation a visitor gets to see.
First you need to create an A/B test experiment in Matomo (formerly Piwik): read the A/B testing user guide to learn more.
When you are asked on which target pages the experiment should be activated, we recommend selecting "Visitors enter this experiment on any page".
In most cases, nothing needs to be done as long as the regular Matomo JavaScript Tracking Code is embedded into your website. Learn more about this step in the Embedding the A/B Testing framework.
The generated experiment code (_paq.push(['AbTesting::create', {...) does not need to be embedded into your website.
To implement the actual experiment, you can use any A/B testing framework of your choice. For example PlanOut by Facebook (Java, PHP, JavaScript, Go, Ruby), or Vanity (Ruby). For PHP we provide our own PHP Experiments framework.
When you choose an A/B testing framework, it is important that the framework lets you know which variation was chosen for a user. This will be important for the next step when you have to track in Matomo which variation's name was used when a user entered into an experiment.
Using an A/B testing framework could look as follows (the following example is in PHP using our InnoCraft PHP Experiments framework):
use InnoCraft\Experiments\Experiment;
$variations = [['name' => 'variation1'], ['name' => 'variation2']];
$experiment = new Experiment('theExperimentName', $variations);
$activated = $experiment->getActivatedVariation();
if ($activated->getName() == 'variation1') {
/* do something variation1 */
} elseif ($activated->getName() == 'variation2') {
/* do something variation2 */
}
// Important: let Matomo know that you have entered the current visitor into an experiment.
// We recommend escaping the experiment name and variation name if needed to prevent possible XSS.
$script = $experiment->getTrackingScript($experiment->getExperimentName(), $activated->getName());
echo $script; // prints eg "<script ...>_paq.push(['AbTesting::enter', {experiment: 'theExperimentName', variation: 'variation1'}]);"
So far you have created and implemented the experiment server-side, so your users get to see different variations of your website. Now you need to let Matomo know which variation was activated for your current user by adding a JavaScript tracking code to your HTML:
var _paq = window._paq = window._paq || [];
_paq.push(['AbTesting::enter', {experiment: 'theExperimentNameOrId', variation: 'myVariation'}]);
This tracking code lets Matomo know that you have entered the current visitor into an experiment.
experiment - The name of the experiment
variation - The name of the variation you have entered the current visitor into. If you have shown the
original version of your website, use original.
The Matomo JavaScript Tracker will pick up this information when the website is loaded and will send a tracking request to your Matomo Tracking API.
Sometimes you might want to compare how entirely different pages or layouts perform against each other.
For example you might have just implemented a new design for your website and want to make sure that your conversion rates will be at least the same with the new design. To do this you can run an experiment and send some of your users to the new layout before you use the new design permanently for all of your users. Matomo A/B tests lets you compare how your most important metrics are impacted for the old and the new design. When you run such an experiment we usually recommend to only send a few percentage of your users to the newly created layout or website just in case your new layout performs significantly worse.
When you redirect users to a different URL make sure to use an HTTP 302 (temporary) redirect and not an HTTP 301 redirect. An example looks as follows:
/**
* Example using InnoCraft PHP Experiments framework
*/
use InnoCraft\Experiments\Experiment;
$variations = [['name' => 'newDesign', 'url' => '/newDesign.php']];
$experiment = new Experiment('theExperimentName', $variations);
$activated = $experiment->getActivatedVariation();
// automatically redirects to newDesign.php if the "newDesign" version gets chosen for a user and makes
// sure to track the chosen variation on the next request.
$activated->run();
// if user was not redirected because the original version was chosen, we need to let Matomo know that
// the original version was activated.
$script = $experiment->getTrackingScript($experiment->getExperimentName(), $activated->getName());
echo $script; // prints eg "<script ...>_paq.push(['AbTesting::enter', {experiment: 'theExperimentName', variation: 'original'}]);"
Alternative version performing the redirect manually:
/**
* in index.php:
*/
echo "window._paq = window._paq || [];window._paq.push(['AbTesting::enter', {experiment: 'theExperimentName', variation: 'original'}]);"
$variations = [['name' => 'newDesign']];
$experiment = new Experiment('theExperimentName', $variations);
$activated = $experiment->getActivatedVariation();
if ($activated->getName() === 'newDesign') {
// make sure to persist the selected variation so when newDesign.php is loaded you can
// let Matomo know which variation was activated
header("Location: /newDesign.php", true, 302);
exit;
} else {
// do nothing and show original variation
echo "window._paq = window._paq || [];window._paq.push(['AbTesting::enter', {experiment: 'theExperimentName', variation: 'original'}]);"
}
/**
* in newDesign.php:
*/
echo "window._paq = window._paq || [];window._paq.push(['AbTesting::enter', {experiment: 'theExperimentName', variation: 'newDesign'}]);"
You can also implement a simple A/B testing framework yourself. An A/B test framework basically works as follows:
original version randomly.window._paq = window._paq || [];window._paq.push(['AbTesting::enter', {experiment: 'theExperimentName', variation: 'variationNameOrIdActivatedForCurrentVisitor'}]);When an experiment is finished:
Happy experimenting!