If an artificial intelligence aims to maximise risk-adjusted return, then under mild conditions it is disproportionately likely to pick an unethical strategy unless the objective function allows sufficiently for this risk. Even if the proportion ${ \eta}$ of available unethical strategies is small, the probability ${ p_U}$ of picking an unethical strategy can become large; indeed unless returns are fat-tailed ${ p_U}$ tends to unity as the strategy space becomes large. We define an Unethical Odds Ratio Upsilon (${ \Upsilon}$) that allows us to calculate ${ p_U}$ from ${ \eta}$, and we derive a simple formula for the limit of ${ \Upsilon}$ as the strategy space becomes large. We give an algorithm for estimating ${ \Upsilon}$ and ${ p_U}$ in finite cases and discuss how to deal with infinite strategy spaces. We show how this principle can be used to help detect unethical strategies and to estimate ${ \eta}$. Finally we sketch some policy implications of this work.
Categories: q-fin.RM cs.LG stat.ML