A/B testing with modules
Recently one of our clients asked us if it was possible to try some A / B testing on some of their modules and the contents inside them. A cursory glance at the Joomla extensions directory revealed that no such module exists for Joomla. As a result we took it upon ourselves to create this module ourselves and adding a little twist. Instead of testing just one module, we decided it might be interesting if the client could test pools of modules against each other. So for example, custom HTML content vs. image galleries modules. In addition we wanted to put in a mechanism to control how often one pool is displayed so you could create a ratio analysis. For example a poll should be displayed 2/3 of the time vs. 1/3 of the time for the other pool. Below you will find a few screen shots of the administrative back end and version to download of the module.
Above you see the parameter layout of the module. You can specify a percentage by which Pool A should show in relation to Pool B. Below that are two lists from which you can select modules to be assigned to Pool A and B. The modules are described by postion | name so that you know which module you are referring to if you have multiple modules of the same kind across various pages.
There are however some caveats:
- Module of the menu type ie. mod_mainmenu have been purposefully omitted as they should not appear in a random testing scenario
- Contextual modules will not work properly. Modules that rely on information presented as part of the surrounding page do not have access to that page information.
| File | Description | File size |
|---|---|---|
 Verge Module to test A/B Module Display | A meta module that loads a random module from an A/B pool based on ratio | 3 Kb |
Comments
For example, if I'm testing a banner in my left sidebar, how do I define that clicking the banner is the goal and then how do I see which version wins? Reply
unfortunately we have not extended the code yet to include this, because in the specific scenario we had external mechanisms (google analytics, joomlastats) to show how many clicks each test received.
Tracking statistics like that internally probably would also be better for a full fledged component rather than just a module. If we find the time, it would be neat to do this though.
Thank you very much for your feedback. It is greatly appreciated. Reply