AgilOne | Cloud-Based Predictive Business Analytics » Structured Testing

Things to consider for Fractional Revenue Attribution

AgilOne Marketing — Sun, 18 Mar 2012 04:06:30 +0000

Revenue attribution is the hottest topic these days. Proliferation of online media, requires reshuffling marketing spend across many more spend categories. Traditional funnel-engineering type work is good, but static, and doesn’t address a few key issues.

1) The transient nature of marketing spend effectiveness that comes and goes with changing keywords, banners offers

2) It does not address the problem in a customer-centric manner (in fact orders are placed by customers who clicked on a keyword, or received a catalog)

The new marketing spend effectiveness paradigm involves understanding causality of relationship between marketing and sales at a transactional level using statistical methods to fractionally attribute. There are five elements at play;

Order of events: what sequencing (0rder) of actions lead to sales transactions
Combined effects: what is the joint effects of marketing touches
Frequency: how many touches are required to convert a prospect to a buyer
Time decay: How the effects of marketing->sales decay with time passed
Effectiveness: what is the relative efficacy of each vehicle is different (e.g., banner view is not the same effectiveness as a 52 pg. catalog)

How this problem can be expressed in mathematical terms and the solution is quite sophisticated and i can not get into it since this is our core IP at Agilone.

Once an attribution could be made, the next issue is how to measure the effects of overspending, which i will get into in the next post. The inherent problem in fractional attribution is how to make sure that by increasing marketing spend on one vehicle will most likely (and not by causality) reduce the effectiveness of other existing spend elements.

Simpsons Paradox: Fundamental but common mistakes when analyzing multiple A/B tests

AgilOne Marketing — Wed, 21 Apr 2010 00:15:17 +0000

Most marketers test various concepts through simple AB testing or other more advanced structured testing approaches (fractional factorials, Taguchi methods etc.). A common mistake that most analysts make involves calculating the lift on the overall gains, rather then individual tests. In cases where the test/control sizes are similar proportions, they lead to the same answer, however the reverse is not true. The answers could even lead to contradicting conclusions, as they will be outlined below.

Suppose we have two groups within our file (this could be more, but for simplicity, we’ll stick with two). These two groups could be anything, for example, it could be

Male versus female,
People who bought X and who didn’t buy X
People who are highly responsive to marketing or non-responsive
High value customers , Low value customers

Suppose we’re trying to examine the effectiveness and lift of a specific campaign on these groups.

Examine the calculation below:

		A	B	C	D	E	F
		Test			Control			=C-F	=(C-F)*A
		Total Marketed	Responders	Response Rate	Total Marketed	Responders	Response Rate	Lift	Incremental Responders
X	Group 1	1000	60	6.0%	200	4	2.0%	4.0%	40
Y	Group 2	500	10	2.0%	400	4	1.0%	1.0%	5
								Group 1+2	45.00

X+Y	Total	1500	70	4.7%	600	8	1.3%	3.3%	50.00

If you follow the above example, the incremental customers coming in from each group is 40+5 = 45. However, when we sum up the test and control groups into one group, where we have 1500 test and 600 control subjects, then the incremental customers is calculated at 50.

How come? Why are the two results different? Which one is correct? Let me know at omer.artun@agilone.com what you think. Answer will come in a few days…