Recent experiments had resulted in a significant improvement in Buckner's active acquisition campaign known as "7 Scriptures." The greatest lift was the result of technical optimization of the Facebook ads. With this in mind, we wanted to take this one step further and continue our testing of the various advertising techniques available to us through Facebook.
One of the ways ads are shown is based upon the method of bidding. Historically, we had set daily spending limits for each audience which equally spread the spend across each one. We had the hypothesis that if we allowed Facebook's algorithm to determine the allocation of spend, we may see better results. To test this, we set the budget at the campaign level, using Facebook's Budget Optimization tool instead of the ad set level (setting a daily limit per ad set). All ads and targeting remained the same.
At the time of the test, each campaign was spending the same amount per day ($83 and change).
Does Facebook's Budget Optimization tool create more opportunities to get instant donors than setting the budget at the ad set level?
|Treatment Name||Samples||Conversions||Conversion Rate||Relative Difference||Confidence|
|C||Ad Set Level Optimization||33,439||30||0.1%||0.0%|
|T1||Daily Budget Optimization||30,439||23||0.1%||-15.8%||46.7%|
This experiment has a required sample size of 317,901 in order to be valid. Since the experiment had a total sample size of 63,878, and the level of confidence is not above 95%, the experiment results are not valid.
The Flux Metrics analyze the three primary metrics that affect revenue (traffic, conversion rate, and average gift). This experiment produced the following results with a 46.7% level of confidence:
This test was not statistically valid, but the campaign that had daily limits set for each ad set did slightly increase instant donor conversion in the time span of the test. After further discussion, we may believe that setting daily limits for the Budget Optimization was not the best route since it still limited Facebook's algorithm to daily amounts. Instead, we are going to next test setting a campaign spending limit that allows for the media to be spent whenever the algorithm determines it is best. This would potentially allow for increased spending on days that show increased conversion opportunity.