The Mathematics of Paul Graham's Bias Test

A major problem in detecting biased decisionmaking is the problem of unknown inputs. For example, suppose a venture capitalist has a public portfolio which has funded 75% male founders and 25% female. Is this venture capitalist biased against women? It's impossible to know - perhaps only 10% of his applicants were …

more ...



Testing for Truth vs Maximizing Revenue

I've written a blog post describing the testing philosophy of SmartStats, VWO's new Bayesian powered A/B testing engine. In it, I discuss the difference between hypothesis testing (with the goal of finding truth) and making good decisions (with the goal of maximizing revenue). Go check it out on the …

more ...

Wingify releases Bayesian A/B tester

I've written a number of posts here about a/b testing, and readers have probably observed that I favor the Bayesian approach. I'm very happy to announce that Wingify (my employer) has release SmartStats - a fully Bayesian A/B testing engine. I've always maintained that you should A/B test …

more ...