Focus on the Family is a global ministry that offers video series on topics such as faith, marriage, and parenting.
This is the second test that we have done in this series of lowering the donation amount needed to receive the premium offer. The first test we did was with Cherish Your Spouse, you can find the write-up for it here. We saw a significant impact on donor conversion and revenue by lowering the donation ask to 30 from 50, we wanted to see if that held true across all verticals of Focus's acquisition offers.
We believe that decreasing the the donation amount to get the premium for people who have just signed up for a video series course will achieve a higher donor conversion rate.
Scroll left and right to see table data
Treatment Name | Samples | Conversions | Conversion Rate | Relative Difference | Confidence | |
---|---|---|---|---|---|---|
C | Control
![]() | 1,720 | 118 | 6.9% | ||
T1 | $30 Premium
![]() | 796 | 80 | 10.1% | 46.5% | 99.0% |
Total | 2,516 | 198 | 7.9% |
This experiment has a required sample size of 547 in order to be valid. Since the experiment had a total sample size of 2,516, and the level of confidence is above 95%, the experiment results are valid.
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:
The key learning from this experiment is that decreasing the donation amount to get the premium for people who have just signed up for a video series results in a higher donor conversion rate. The treatment group had a 46.5% increase in donations for all traffic with a 99.1% level of statistical confidence compared to the control group.
The experiment also revealed that mobile, new, and tablet visitors had significant increases in donations, indicating that the treatment was effective across different segments. Mobile visitors had a 408% increase in donations, new visitors had a 534% increase, and tablet visitors had a 2527% increase.
Based on this result and the results of the other offers where a lower appeal amount was tested, we will use a lower donation amount for the time being.
Experiment recorded by Bailey Wilcox from Focus on the Family
Experiment recorded on 5/5/2023