This kind of ranking is also not correct. You have to compare the outcomes , not their p values. Ranking by p values is just silly, just like ranking by avg metric is silly.
Startups in general have to
make decisions with high signal, thinking that 100 improvements of 1% p=0.05 will actually compound in an environment with so much noise is delusion.
I’d say doing this kind of silliness in a startup is just ceremonial, helpful long term if people feel they are doing a good job optimizing a compounding metric, even though it never materializes.
Startups in general have to make decisions with high signal, thinking that 100 improvements of 1% p=0.05 will actually compound in an environment with so much noise is delusion.
I’d say doing this kind of silliness in a startup is just ceremonial, helpful long term if people feel they are doing a good job optimizing a compounding metric, even though it never materializes.