How asking for less money on a faith premium offer donation page impacts donor conversion

Organization: Focus on the Family

Experiment ID: 145707

Metric Measured: Donations

Timeframe: 4/19/2023 - 5/8/2023

Experiment Summary

Focus on the Family is a global ministry that offers video series on topics such as faith, marriage, and parenting.

This is the second test that we have done in this series of lowering the donation amount needed to receive the premium offer. The first test we did was with Cherish Your Spouse, you can find the write-up for it here. We saw a significant impact on donor conversion and revenue by lowering the donation ask to 30 from 50, we wanted to see if that held true across all verticals of Focus's acquisition offers.

Hypothesis

We believe that decreasing the the donation amount to get the premium for people who have just signed up for a video series course will achieve a higher donor conversion rate.

Design

Control

$30 Premium

Results

Scroll left and right to see table data

   Treatment Name Samples Conversions Conversion Rate Relative Difference Confidence
Control
1,720 118 6.9%
T1
$30 Premium
796 80 10.1% 46.5% 99.0%
   Total 2,516 198 7.9%

This experiment has a required sample size of 547 in order to be valid. Since the experiment had a total sample size of 2,516, and the level of confidence is above 95%, the experiment results are valid.

Flux Metrics Affected

The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results:

T
C
A
0% in traffic
46.5% increase in conversion rate
0% in average gift
46.5% increase in revenue

Key Learnings

The key learning from this experiment is that decreasing the donation amount to get the premium for people who have just signed up for a video series results in a higher donor conversion rate. The treatment group had a 46.5% increase in donations for all traffic with a 99.1% level of statistical confidence compared to the control group.

The experiment also revealed that mobile, new, and tablet visitors had significant increases in donations, indicating that the treatment was effective across different segments. Mobile visitors had a 408% increase in donations, new visitors had a 534% increase, and tablet visitors had a 2527% increase.

Based on this result and the results of the other offers where a lower appeal amount was tested, we will use a lower donation amount for the time being.

Experiment Tags

Email

Experiment recorded by Bailey Wilcox from Focus on the Family

Experiment recorded on 5/5/2023