...
If preezie is embedded on a single page then this is straight forward:
Split traffic | Page trigger | Segment | Goal A: User conversion rate (sales per user) | Goal B: Sessions per user | Goal C: Bounce rate |
|---|---|---|---|---|---|
50% Test | myhomepage.com | 40% non-preezie users | 3.2% | 2.4 | 23% |
10% preezie users | 4.5% | 3.2 | 0% (preezie counts as a significant event) | ||
50% |
Control | myhomepage.com | 50% non-preezie users | 3.1% | 2.6 | 25% |
Once you can see these the buckets are gaining a good level of traffic (e.g. use a an A/A test to understand how long your website traffic needs to achieve even conversion rates), you can start to compare the preezie 10% against the 50% who never saw it and the 40% who did see it but didn’t engage.
...
If you take 10% of your control bucket raw numbers (e.g. users, conversions, sessions, bounces) and your preezie bucket numbers put them into a signifance calculation tool:
OR
https://abtestguide.com/bayesian/
You can use this method these methods to understand both:
How preezie performs vs those who didn’t see it (i.e. the 10% / 50% buckets)
How preezie performs vs those who did but didn’t interact (i.e. the 10% / 40% buckets)
...
Here we’ll track those who saw it, those who clicked and compare against the inverse of both. Our main goal of exit intents are to keep users on the website, so our goals are now:
Split traffic | Page trigger | Segment | Goal A: Exit rate | Goal B: New user pages/session | Goal C: User conversion rate (sales per user) |
|---|---|---|---|---|---|
50% Test | any | 25% did not see preezie | 30% | 2.1 | 4.2% |
10% saw preezie and did not click it | 20% | 2.3 | 4.5% | ||
15% clicked on preezie | 6% | 4.3 | 6.8% | ||
50% Control | any | 50% no preezie loaded | 30% | 2.2 | 4.1% |
Here you can compare your control bucket with those who did not see it to ensure the behaviour is the same across buckets (see article on A/A testing).
...