I’m a bit confused why we are assuming the low performance is the fault ML. I would think instead that this low performance is deliberate because it locally optimizes Facebooks KPIs.
My understanding is that Facebook has a lot of data on the elasticity of ad buyers. If Facebook were to have even more precise algorithms, then they would likely charge even more money for impressions and clicks. Presumably buyers may be reticent to pay even higher prices.
Alternatively more precise targeting would highly bias the money people spend toward larger companies that can afford better keywords.
If you look at the communication, it seems that the Ad people didn't have a clear idea how well-targeted anything was.
It's easy to see how the statistical problems compound.
FB does not have exact earning levels, so you have to infer that from, say, likes. Let's say you can build a model salary(likeBMW, likeTravel, ...) = %likeBMW + %likeTravel +...
This gives you an estimate which is (70-80)% accurate, so you predict >£250k/yr 70pc of the time. In c. 25% you mispredict.
Now it seems to me that this 25% is going to compound across several categories: when you say "College AND HighEarner AND ..." you get more than 50% of your target group not matching this exact criteria (all you need to fail is one condition to fail to match this conjunction).
And according to FB comms, it looks like >50% didn't match client's chosen criteria.
I think this is the right analysis of the issue. ML systems of this kind are very bad at making targeted judgements. It's really little more accurate than guessing the mean of something (eg., salary) for your group.
All ML has to do, for FB/Google/etc. is improve targeting a few percent to have a significant value proposition.
However, the propaganda has it that ML systems can "target" you, etc. Only in the way a nuclear bomb "targets" a house.
Could it be a misunderstanding of the outcome, or perhaps bias in Facebooks algorithms? The best predictor of your target is: your target. In this case it’s predicting who will buy your product. Using a proxy such as income and education is a good start but surely cannot be the best approach because it artificially limits the search space. The best target is some ML-determined combination of features. My guess is that for niche items such as expensive watches and cars, Facebook’s targeting may be too liberal with lower income earners. But I think this the result of being very good at targeting the general audience for medium-low priced goods and services.
My understanding is that Facebook has a lot of data on the elasticity of ad buyers. If Facebook were to have even more precise algorithms, then they would likely charge even more money for impressions and clicks. Presumably buyers may be reticent to pay even higher prices.
Alternatively more precise targeting would highly bias the money people spend toward larger companies that can afford better keywords.