Making experiments pay

strategy+business, June 2, 2020

by Theodore Kinni




Illustration by AndSim

The current pandemic is a dramatic case study in why experiments matter. How else are we to know which tests accurately identify COVID-19 infections and their telltale antibodies, whether and how well drugs already at our disposal mitigate the suffering inflicted by the coronavirus, or whether the vaccines currently in development will actually protect us from it? Randomized controlled trials are the only way to answer these questions short of playing Russian roulette with the lives of large numbers of people.

Experiments have been around for a long time. In The Power of Experiments, an introductory paean to the benefits of experimentation for corporate and government decision making, Harvard Business School professors Michael Luca and Max Bazerman peg the first recorded experiment to the reign of King Nebuchadnezzar in Babylonia circa 600 BC.

As the Old Testament story goes, Nebuchadnezzar attacked Jerusalem and took a group of young Israelites, including one named Daniel, as servants. He ordered that they be acculturated for three years, in part by being fed the same food as the royal court, which the Israelites considered “ritually unclean.” But Daniel proposed that he and three other prisoners be fed only vegetables and water for 10 days, at which point a guard would compare their health to that of the rest of the prisoners, who would have been fed the local fare. At the end of the experiment, the guard judged the vegetarians healthier, and Daniel and his three comrades were able to continue eating the less objectionable diet.

About 2,600 years later, Luca and Bazerman write, “We are in the early days of the age of experiments.” By this, they mean that experiments have spread far beyond the boundaries of science labs and medical trials. In 2018, Google ran more than 10,000 experiments. Amazon, Facebook, Uber, Yelp, and TripAdvisor run thousands of experiments per year. As these names indicate, technology companies, especially those with digital platforms, are at the forefront of corporate experimentation. That’s because it is relatively easy to run experiments on their large, captive audiences. Read the rest here.