Real marketing data is messy, skewed, and full of rare events that models struggle to learn from. When accuracy becomes the goal, models can look impressive while quietly getting the most important decisions wrong. This article explains why that happens, how imbalanced data creates false confidence, and what marketers should do instead.
If you ever get your teeth into Game Theory, you discover why A/B Testing patterns can run you down a rabbit hole.
That’s before you even consider the Ur-challenge of any business enterprise - The Principle Agent Problem. When you build an incentive structure to reward certain results, you can incentivize your agents to lie to you about the effectiveness of your policies.
It doesn’t have to work on everyone
It presumably has to work on an increasing slice of retail market cap with each iteration.
Is there any evidence that Temu’s splash screen is generating more revenue per visitor than Amazon’s?
My ideal (personal) version has a protocol axis for the matrix, and the players can choose to wait. The payoff comes from ongoing constructive interference.
you discover why A/B Testing patterns can run you down a rabbit hole.
Multivariate statistics is a subdivision of statisticsencompassing the simultaneous observation and analysis of more than one outcome variable, i.e., multivariate random variables. Multivariate statistics concerns understanding the different aims and background of each of the different forms of multivariate analysis, and how they relate to each other. The practical application of multivariate statistics to a particular problem may involve several types of univariate and multivariate analyses in order to understand the relationships between variables and their relevance to the problem being studied.
That’s before you even consider the Ur-challenge of any business enterprise - The Principle Agent Problem. When you build an incentive structure to reward certain results, you can incentivize your agents to lie to you about the effectiveness of your policies.
What agents?
Is there any evidence that Temu’s splash screen is generating more revenue per visitor than Amazon’s?
No idea, but if they’re doing it all the time then I doubt it’s detrimental for profitability.
the conflict in interests and priorities that arises when one person or entity (the “agent”) takes actions on behalf of another person or entity (the “principal”)
The owners/investors in the company aren’t the same people doing the yeomen’s work of UX design. Often, these are two entirely separate companies.
if they’re doing it all the time then I doubt it’s detrimental for profitability.
That’s not reliance on Multivariate Testing, that’s dogmatic belief in the infallibility of business management.
The owners/investors in the company aren’t the same people doing the yeomen’s work of UX design. Often, these are two entirely separate companies.
Then we can’t speculate on how well they’re compensated or how. They clearly did a stellar job despite the cursed content though.
That’s not reliance on Multivariate Testing, that’s dogmatic belief in the infallibility of business management.
No. It’s an empirical observation. If a shop near you is doing 80% off all the time and not looking under the weather after a while you start to think they’re doing something clever behind the scenes. No dogmatic belief needed.
They clearly did a stellar job despite the cursed content though.
Idk if I’d call 120 seconds of unskippable flash animation “stellar”.
If a shop near you is doing 80% off all the time and not looking under the weather after a while you start to think they’re doing something clever behind the scenes.
If they’re doing 80% off all the time they aren’t doing something clever. They made a decision once a long time ago and haven’t updated their marketing model since.
If you’re just stacking longer and longer click-bait marketing gimmicks on your splash screen, that’s the opposite of “clever”. That’s a business model of slapping the feed bar ten times because you saw it drop a nugget of food the first time.
If they’re doing 80% off all the time they aren’t doing something clever. They made a decision once a long time ago and haven’t updated their marketing model since.
You’re reducing things to one dimension again. Just because the sign says 80% off doesn’t mean that’s the whole schtick. The profit cold be supply side for all we know. The trick is that continuous conversion is happening, the clever bits could happen anywhere.
If you’re just stacking longer and longer click-bait marketing gimmicks on your splash screen, that’s the opposite of “clever”. That’s a business model of slapping the feed bar ten times because you saw it drop a nugget of food the first time.
Not if it’s algorithmic which I’ve been trying to say. Adaptive systems can do that. Heck MAB is already a thing in production. It’s not too hard to hook it to MDPI and close the loop.
If you ever get your teeth into Game Theory, you discover why A/B Testing patterns can run you down a rabbit hole.
That’s before you even consider the Ur-challenge of any business enterprise - The Principle Agent Problem. When you build an incentive structure to reward certain results, you can incentivize your agents to lie to you about the effectiveness of your policies.
It presumably has to work on an increasing slice of retail market cap with each iteration.
Is there any evidence that Temu’s splash screen is generating more revenue per visitor than Amazon’s?
My ideal (personal) version has a protocol axis for the matrix, and the players can choose to wait. The payoff comes from ongoing constructive interference.
Multivariate testing isn’t A/B testing, it’s applied multivariate statistics that employs A/B testing as an operation.
Tap for spoiler
Multivariate statistics is a subdivision of statisticsencompassing the simultaneous observation and analysis of more than one outcome variable, i.e., multivariate random variables. Multivariate statistics concerns understanding the different aims and background of each of the different forms of multivariate analysis, and how they relate to each other. The practical application of multivariate statistics to a particular problem may involve several types of univariate and multivariate analyses in order to understand the relationships between variables and their relevance to the problem being studied.
What agents?
No idea, but if they’re doing it all the time then I doubt it’s detrimental for profitability.
Principal Agent Theory
The owners/investors in the company aren’t the same people doing the yeomen’s work of UX design. Often, these are two entirely separate companies.
That’s not reliance on Multivariate Testing, that’s dogmatic belief in the infallibility of business management.
Thanks for introducing me to that!
Then we can’t speculate on how well they’re compensated or how. They clearly did a stellar job despite the cursed content though.
No. It’s an empirical observation. If a shop near you is doing 80% off all the time and not looking under the weather after a while you start to think they’re doing something clever behind the scenes. No dogmatic belief needed.
Idk if I’d call 120 seconds of unskippable flash animation “stellar”.
If they’re doing 80% off all the time they aren’t doing something clever. They made a decision once a long time ago and haven’t updated their marketing model since.
If you’re just stacking longer and longer click-bait marketing gimmicks on your splash screen, that’s the opposite of “clever”. That’s a business model of slapping the feed bar ten times because you saw it drop a nugget of food the first time.
You’re reducing things to one dimension again. Just because the sign says 80% off doesn’t mean that’s the whole schtick. The profit cold be supply side for all we know. The trick is that continuous conversion is happening, the clever bits could happen anywhere.
Not if it’s algorithmic which I’ve been trying to say. Adaptive systems can do that. Heck MAB is already a thing in production. It’s not too hard to hook it to MDPI and close the loop.