What do you think?
Rate this book
The Voltage Effect: How to Make Good Ideas Great and Great Ideas Scale
A leading economist answers one of today’s trickiest questions: Why do some great ideas make it big while others fail to take off?
“Brilliant, practical, and grounded in the very latest research, this is by far the best book I’ve ever read on the how and why of scaling.”—Angela Duckworth, CEO of Character Lab and New York Times bestselling author of Grit
“Scale” has become a favored buzzword in the startup world. But scale isn't just about accumulating more users or capturing more market share. It's about whether an idea that takes hold in a small group can do the same in a much larger one—whether you’re growing a small business, rolling out a diversity and inclusion program, or delivering billions of doses of a vaccine.
Translating an idea into widespread impact, says University of Chicago economist John A. List, depends on one thing only: whether it can achieve “high voltage”—the ability to be replicated at scale.
In The Voltage Effect, List explains that scalable ideas share a common set of attributes, while any number of attributes can doom an unscalable idea. Drawing on his original research, as well as fascinating examples from the realms of business, policymaking, education, and public health, he identifies five measurable vital signs that a scalable idea must possess, and offers proven strategies for avoiding voltage drops and engineering voltage gains. You’ll learn:
• How celebrity chef Jamie Oliver expanded his restaurant empire by focusing on scalable “ingredients” (until it collapsed because talent doesn’t scale)
• Why the failure to detect false positives early on caused the Reagan-era drug-prevention program to backfire at scale
• How governments could deliver more services to more citizens if they focused on the last dollar spent
• How one education center leveraged positive spillovers to narrow the achievement gap across the entire community
• Why the right set of incentives, applied at scale, can boost voter turnout, increase clean energy use, encourage patients to consistently take their prescribed medication, and more.
By understanding the science of scaling, we can drive change in our schools, workplaces, communities, and society at large. Because a better world can only be built at scale.
“Brilliant, practical, and grounded in the very latest research, this is by far the best book I’ve ever read on the how and why of scaling.”—Angela Duckworth, CEO of Character Lab and New York Times bestselling author of Grit
“Scale” has become a favored buzzword in the startup world. But scale isn't just about accumulating more users or capturing more market share. It's about whether an idea that takes hold in a small group can do the same in a much larger one—whether you’re growing a small business, rolling out a diversity and inclusion program, or delivering billions of doses of a vaccine.
Translating an idea into widespread impact, says University of Chicago economist John A. List, depends on one thing only: whether it can achieve “high voltage”—the ability to be replicated at scale.
In The Voltage Effect, List explains that scalable ideas share a common set of attributes, while any number of attributes can doom an unscalable idea. Drawing on his original research, as well as fascinating examples from the realms of business, policymaking, education, and public health, he identifies five measurable vital signs that a scalable idea must possess, and offers proven strategies for avoiding voltage drops and engineering voltage gains. You’ll learn:
• How celebrity chef Jamie Oliver expanded his restaurant empire by focusing on scalable “ingredients” (until it collapsed because talent doesn’t scale)
• Why the failure to detect false positives early on caused the Reagan-era drug-prevention program to backfire at scale
• How governments could deliver more services to more citizens if they focused on the last dollar spent
• How one education center leveraged positive spillovers to narrow the achievement gap across the entire community
• Why the right set of incentives, applied at scale, can boost voter turnout, increase clean energy use, encourage patients to consistently take their prescribed medication, and more.
By understanding the science of scaling, we can drive change in our schools, workplaces, communities, and society at large. Because a better world can only be built at scale.
288 pages, Hardcover
First published February 1, 2022
About the author
John A. List
18 books48 followersProfessor John A. List is the Kenneth C. Griffin Distinguished Service Professor in Economics at the University of Chicago. His research focuses on combining field experiments with economic theory to deepen our understanding of the economic science. In the early 1990s, List pioneered field experiments as a methodology for testing behavioral theories and learning about behavioral principles that are shared across different domains. He co-authored the international best seller, The Why Axis, in 2013. List was elected a Member of the American Academy of Arts and Sciences in 2011, and a Fellow of the Econometric Society in 2015. List received the 2010 Kenneth Galbraith Award, the 2008 Arrow Prize for Senior Economists for his research in behavioral economics in the field, and was the 2012 Yrjo Jahnsson Lecture Prize recipient. He is a current Editor of the Journal of Political Economy.
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 30 of 135 reviews
April 30, 2022
Thanks to Currency Publishing for a hardcover of this great book in exchange for a fair review.
Chapter 8: Quitting is for Winners. Yes!
Great book with lots of insight and great research. Not the boring business book you might at first think. 😊
Chapter 8: Quitting is for Winners. Yes!
Great book with lots of insight and great research. Not the boring business book you might at first think. 😊
February 27, 2022
It's rare for someone to tell you how much they hate workers. That's what you'll find in this book – but I get ahead of myself.
For context, I don't read a lot of business books, so this book may be no better or no worse than your average rah-rah airport book for harried execs looking for a new edge. The reason I got it is because Tim Harford (FT, BBC) recommended it on Twitter. I will have to be more skeptical of Harford's recs in future because this book wasn't just disappointing, it was quite odious.
Disappointing, because the book generally covers stuff I already knew from reading around the subject of, you know, how to run a successful business, for many years. If you haven't done that, you might learn something about sampling bias, the need for replication, marginal economics, behavioural incentives, etc. There's nothing wrong with being an introductory text, of course – people have to learn this stuff from somewhere - but it would be a mistake to expect anything novel here.
Odious, for two reasons. Firstly, the author trumpets the "clawback approach" to make workers more productive. This is where you give workers, like underpaid teachers, a bonus at the start of the year, and then claw it back if they don't perform well. This is simply cruel. There are a lot of things one can do to "improve" workers' productivity, things that might even be replicable and effective. That doesn't mean you should do them.
Secondly, the author implies that at Uber, where he was chief economist for a few years, the shit hit the fan only after he started working in the summer of 2016. An April 2016 Guardian roundup of Uber lawsuits shows this is not the case. It would be one thing if the author didn't suggest he was concerned about the bad stuff going on at Uber, but he does, at least somewhat lamely (e.g. luckily for him, he wasn't in the office much, and he wasn't exposed to the peons, let alone the drivers). My point is that anyone can take the Uber dollar if they want, but don't pretend you didn't know what they were up to.
Other assorted observations:
* The author takes credit for telling GoPuff, an instant grocery delivery company, to diversify its product offerings in order to attract more customers beyond college student. No shit?!?! Are we really to believe no-one else at the company thought of this?
* Of course there is no comment on the fact that GoPuff, and indeed Uber and Lyft, are not actually profitable.
* He spends a bunch of time talking about the demise of the "Jamie's Italian" restaurant chain in a way that is at the very least, unsupported by actual evidence, and likely ignores more complex stuff going on like the market's increasing appetite for authenticity, etc.
* Apparently people only pay taxes because they're afraid of being punished.
* The author describes himself as "socially liberal, fiscally conservative." No further comment required.
* He worked at the White House under George W. Bush but had no idea the Iraq War was coming. Dude – just own it!
* Supposedly Uber's business model scaled but its culture didn't. This presumes that Uber's culture was once good and then turned bad at scale: press X to Doubt.
For context, I don't read a lot of business books, so this book may be no better or no worse than your average rah-rah airport book for harried execs looking for a new edge. The reason I got it is because Tim Harford (FT, BBC) recommended it on Twitter. I will have to be more skeptical of Harford's recs in future because this book wasn't just disappointing, it was quite odious.
Disappointing, because the book generally covers stuff I already knew from reading around the subject of, you know, how to run a successful business, for many years. If you haven't done that, you might learn something about sampling bias, the need for replication, marginal economics, behavioural incentives, etc. There's nothing wrong with being an introductory text, of course – people have to learn this stuff from somewhere - but it would be a mistake to expect anything novel here.
Odious, for two reasons. Firstly, the author trumpets the "clawback approach" to make workers more productive. This is where you give workers, like underpaid teachers, a bonus at the start of the year, and then claw it back if they don't perform well. This is simply cruel. There are a lot of things one can do to "improve" workers' productivity, things that might even be replicable and effective. That doesn't mean you should do them.
Secondly, the author implies that at Uber, where he was chief economist for a few years, the shit hit the fan only after he started working in the summer of 2016. An April 2016 Guardian roundup of Uber lawsuits shows this is not the case. It would be one thing if the author didn't suggest he was concerned about the bad stuff going on at Uber, but he does, at least somewhat lamely (e.g. luckily for him, he wasn't in the office much, and he wasn't exposed to the peons, let alone the drivers). My point is that anyone can take the Uber dollar if they want, but don't pretend you didn't know what they were up to.
Other assorted observations:
* The author takes credit for telling GoPuff, an instant grocery delivery company, to diversify its product offerings in order to attract more customers beyond college student. No shit?!?! Are we really to believe no-one else at the company thought of this?
* Of course there is no comment on the fact that GoPuff, and indeed Uber and Lyft, are not actually profitable.
* He spends a bunch of time talking about the demise of the "Jamie's Italian" restaurant chain in a way that is at the very least, unsupported by actual evidence, and likely ignores more complex stuff going on like the market's increasing appetite for authenticity, etc.
* Apparently people only pay taxes because they're afraid of being punished.
* The author describes himself as "socially liberal, fiscally conservative." No further comment required.
* He worked at the White House under George W. Bush but had no idea the Iraq War was coming. Dude – just own it!
* Supposedly Uber's business model scaled but its culture didn't. This presumes that Uber's culture was once good and then turned bad at scale: press X to Doubt.
February 19, 2022
People who think about building the next Amazon or Google often wonder, what is the recipe? What are the fatal mistakes that kill an organization before it scales? This book will resonate with anyone who has worked in Silicon Valley, because those are the questions it explores. It analyzes a number of common ways in which things go wrong, and what might be done to avoid them.
The first part of the book is a brief summary of recent behavioral economics findings. It talks about some common biases that lead us to believe that a small experiment will replicate at scale. Example: McDonalds wants to know if people would like a more expensive burger, so it runs a focus group. The participants love it. They launch the product, but it flops. It turns out that the focus group was not at all representative of the customer base, it was a self-selected bunch of McDonalds fans. This is selection bias.
The author draws many examples from his tenure as the resident economist for Uber and later Lyft. These companies were rich sources of live data, and experimental playgrounds for all sorts of tricks. One of the most important lessons of this book (which is well known in Silicon Valley) is that before investing heavily in a new product or service, you want to do a cheap and small test first. The crux of the matter is ensuring that the test gives you valuable information. This means that designing these experiments is completely nontrivial, and List shares numerous examples of his failures and successes.
Another important point that will be familiar to Silicon Valley readers is the trap of the Sunk Cost Fallacy. This is, failing to recognizing the signals of failure and cutting your losses because it feels bad emotionally. You do not want to continue wasting money and time on something that has a high chance of failure, when you could instead be trying something new. A less well known example is the marginal utility of a dollar spent. You may be spending $100 on Facebook ads and $100 on Google. Is that the right split? Do not look at the average value, but at the value added by the last dollar. It could be that ten more dollars spent on Google gives you 50x the value of spending them on Facebook, so shifting to 90/110 would give you much more for your money. Diminishing returns often go unnoticed when you're not paying attention.
The book is a quick read, and very enjoyable. I thought List spends perhaps a bit too much time panning Uber with the benefit of hindsight. Also, even though he makes a good attempt at being rigorous and scientific there is a fair amount of anecdotal evidence and opinion in the mix. Those are minor flaws in my opinion. In short, this book is worth reading if any of these are true:
- you are a fan of behavioral economics.
- you want to start a highly scalable, venture funded startup.
- you are interested in how to implement effective public policy at a national scale.
If you are already running a startup and most of this book surprises you, then you probably were not well prepared. Read on and reduce your odds of failure!
The first part of the book is a brief summary of recent behavioral economics findings. It talks about some common biases that lead us to believe that a small experiment will replicate at scale. Example: McDonalds wants to know if people would like a more expensive burger, so it runs a focus group. The participants love it. They launch the product, but it flops. It turns out that the focus group was not at all representative of the customer base, it was a self-selected bunch of McDonalds fans. This is selection bias.
The author draws many examples from his tenure as the resident economist for Uber and later Lyft. These companies were rich sources of live data, and experimental playgrounds for all sorts of tricks. One of the most important lessons of this book (which is well known in Silicon Valley) is that before investing heavily in a new product or service, you want to do a cheap and small test first. The crux of the matter is ensuring that the test gives you valuable information. This means that designing these experiments is completely nontrivial, and List shares numerous examples of his failures and successes.
Another important point that will be familiar to Silicon Valley readers is the trap of the Sunk Cost Fallacy. This is, failing to recognizing the signals of failure and cutting your losses because it feels bad emotionally. You do not want to continue wasting money and time on something that has a high chance of failure, when you could instead be trying something new. A less well known example is the marginal utility of a dollar spent. You may be spending $100 on Facebook ads and $100 on Google. Is that the right split? Do not look at the average value, but at the value added by the last dollar. It could be that ten more dollars spent on Google gives you 50x the value of spending them on Facebook, so shifting to 90/110 would give you much more for your money. Diminishing returns often go unnoticed when you're not paying attention.
The book is a quick read, and very enjoyable. I thought List spends perhaps a bit too much time panning Uber with the benefit of hindsight. Also, even though he makes a good attempt at being rigorous and scientific there is a fair amount of anecdotal evidence and opinion in the mix. Those are minor flaws in my opinion. In short, this book is worth reading if any of these are true:
- you are a fan of behavioral economics.
- you want to start a highly scalable, venture funded startup.
- you are interested in how to implement effective public policy at a national scale.
If you are already running a startup and most of this book surprises you, then you probably were not well prepared. Read on and reduce your odds of failure!
September 2, 2022
Scaling, especially for non-profits and social programs, is an important dilemma in need of effective solutions. This book features some good wordsmithing around that, like the analogy about ingredients instead of chefs. And it promotes the general idea that we should do more of what works and less of what is useless or harmful. Unfortunately, the specific content consists of quite a lot of sloppy hogwash.
Alternatives for understanding about successful scaling:
Factfulness: Ten Reasons We're Wrong About the World – and Why Things Are Better Than You Think
Prescription for a Healthy Nation: A New Approach to Improving Our Lives by Fixing Our Everyday World
The Health Of Nations
Nerd addendum:
-His experience/credibility with regard to social programs is mainly the Chicago Heights Early Childhood Center. But with a quick search I couldn't find evidence that this program is a scalable way to improve the percent of poor kids who are ready for kindergarten.
-One of the main examples he uses in the first part of the book is the drug-absuse-prevention program DARE! and he makes a big deal of describing this as a "false positive." I looked up the reference he cites for this about the original Honolulu study. It's not even a false positive. It's just a weak evaluation of a weak program. They did pre-post surveys of kids and teachers and reported on irrelevant process measures like people "found the program enjoyable." This is not a solid experimental design and it does not include the relevant outcome of drug use. So this should never have been convincing as a basis for scaling it up across the country for decades. That would have been very important to explain for this topic of scaling, as opposed to improperly using this as an example of a "false positive."
-He incessantly refers to Uber and Lyft as "ride-sharing" services. This seems like propaganda. The business model does not involve people doing some sort of carpooling. Serious news outlets use "ride-hailing" to describe these companies.
-The stuff about "risk compensation" (p.90) is dangerously wrong:(https://slate.com/technology/2021/11/...). People trot this out all the time to oppose preventive measures. But the original paper on how seatbelts make people crash into brick walls was weak. The overall impact of seatbelt laws, airbags and other such things has been to save many lives. He also mentions bicycle helmets here, completely missing the evidence that indicates the exact opposite of what he's talking about, i.e. American adult bicyclists who wear helmets tend to be more risk-averse than people who don't wear helmets. He also neglects how in places where bicycling has been scaled up (Denmark, Netherlands, etc.) bicycling is much safer and people mostly don't wear helmets (because bicycling is so safe). How does that work? That would have been good to explore in a book about scaling. But it shows how the book goes out of its way to avoid basic public health principles, even though that's an area where humanity has very successfully scaled up lots of things.
-There's an anecdote about the company called Arivale that for an exorbitant annual fee was supposedly going to help people live longer. He depicts this as a "promising experiment" that failed to scale only because it was too expensive. However, once again he fails to present solid evidence (like, for example, an experiment) that it actually worked in the first place to do what it was supposed to do, i.e. prevent illness and extend life.
Alternatives for understanding about successful scaling:
Factfulness: Ten Reasons We're Wrong About the World – and Why Things Are Better Than You Think
Prescription for a Healthy Nation: A New Approach to Improving Our Lives by Fixing Our Everyday World
The Health Of Nations
Nerd addendum:
-His experience/credibility with regard to social programs is mainly the Chicago Heights Early Childhood Center. But with a quick search I couldn't find evidence that this program is a scalable way to improve the percent of poor kids who are ready for kindergarten.
-One of the main examples he uses in the first part of the book is the drug-absuse-prevention program DARE! and he makes a big deal of describing this as a "false positive." I looked up the reference he cites for this about the original Honolulu study. It's not even a false positive. It's just a weak evaluation of a weak program. They did pre-post surveys of kids and teachers and reported on irrelevant process measures like people "found the program enjoyable." This is not a solid experimental design and it does not include the relevant outcome of drug use. So this should never have been convincing as a basis for scaling it up across the country for decades. That would have been very important to explain for this topic of scaling, as opposed to improperly using this as an example of a "false positive."
-He incessantly refers to Uber and Lyft as "ride-sharing" services. This seems like propaganda. The business model does not involve people doing some sort of carpooling. Serious news outlets use "ride-hailing" to describe these companies.
-The stuff about "risk compensation" (p.90) is dangerously wrong:(https://slate.com/technology/2021/11/...). People trot this out all the time to oppose preventive measures. But the original paper on how seatbelts make people crash into brick walls was weak. The overall impact of seatbelt laws, airbags and other such things has been to save many lives. He also mentions bicycle helmets here, completely missing the evidence that indicates the exact opposite of what he's talking about, i.e. American adult bicyclists who wear helmets tend to be more risk-averse than people who don't wear helmets. He also neglects how in places where bicycling has been scaled up (Denmark, Netherlands, etc.) bicycling is much safer and people mostly don't wear helmets (because bicycling is so safe). How does that work? That would have been good to explore in a book about scaling. But it shows how the book goes out of its way to avoid basic public health principles, even though that's an area where humanity has very successfully scaled up lots of things.
-There's an anecdote about the company called Arivale that for an exorbitant annual fee was supposedly going to help people live longer. He depicts this as a "promising experiment" that failed to scale only because it was too expensive. However, once again he fails to present solid evidence (like, for example, an experiment) that it actually worked in the first place to do what it was supposed to do, i.e. prevent illness and extend life.
March 12, 2022
An enjoyable, reliable and wide-ranging guide to how to scale ideas. The book includes a lot of economic lessons applied to questions like how to scale a business, a nonprofit, or a policy response. Most of these are familiar to students of economics and psychology but it is their application that is most interesting, including ideas like the dangers of false positives, generalizing from non-representative samples, lack of spillovers and economies of scale, and the importance of thinking on the margin.
What sets the book apart, like John Lists's work, is that it is relentlessly empirical throughout. It takes the economic and psychological concepts but brings them to the data in ways that often end up surprising List including questions like do equal opportunity job advertisements help recruit underrepresented minorities, does paying a higher wage necessarily elicit more effort, and will Lyft membership plan's increase its profits.
In addition, there are a wide range of practical examples, many of which List was directly involved in, including setting up a preschool, working for Uber and Lyft, consulting with governments on tax evasion, with Chrysler on wellness, etc. In almost all of them theory helps to guide the thinking but it is really experiments and feedback that are most critical.
I also enjoyed getting to know List a little through the book. It begins with a dedication to his eight! children (talk about scale). It discusses how he got into economics (he came from a family of truckers, got into college on a golf scholarship, did his PhD and Wyoming, and got one job offer out of the 150 he applied for--but then eventually ended up at the University of Chicago), and a lot of the ways he integrates empiricism into the issues he also cares about (like charity and public policy).
Highly recommended.
What sets the book apart, like John Lists's work, is that it is relentlessly empirical throughout. It takes the economic and psychological concepts but brings them to the data in ways that often end up surprising List including questions like do equal opportunity job advertisements help recruit underrepresented minorities, does paying a higher wage necessarily elicit more effort, and will Lyft membership plan's increase its profits.
In addition, there are a wide range of practical examples, many of which List was directly involved in, including setting up a preschool, working for Uber and Lyft, consulting with governments on tax evasion, with Chrysler on wellness, etc. In almost all of them theory helps to guide the thinking but it is really experiments and feedback that are most critical.
I also enjoyed getting to know List a little through the book. It begins with a dedication to his eight! children (talk about scale). It discusses how he got into economics (he came from a family of truckers, got into college on a golf scholarship, did his PhD and Wyoming, and got one job offer out of the 150 he applied for--but then eventually ended up at the University of Chicago), and a lot of the ways he integrates empiricism into the issues he also cares about (like charity and public policy).
Highly recommended.
May 7, 2022
This book was great! At first glance, it may seem like a generic business book about scaling organizations, but List’s background in running field experiments makes for a much more compelling read. He has experience conducting experiments in academia, at ridesharing companies, for the federal government, and in program evaluation for an education nonprofit in Chicago, among others. Drawing from this experience, List explains some key reasons why ideas that succeed in pilot studies never take off (e.g., not generalizing to a broader audience, unintended spillovers, and unsustainable costs), as well as some tips to get around these and scale ideas better. His diverse background makes this relevant to people across many fields, and while it’s light on technical details, it was both informative and easy to read! I loved hearing about all of the clever field experiments and their results. Definitely recommended.
February 21, 2023
John does a great job of making Econ easy for everyone to understand, and maybe because I enjoy the subject, I’d say he even does a good job at making it interesting and entertaining.
July 21, 2023
An interesting read! A sensible structure and academic approach, The Voltage Effect is smart, although dry at times. I like how one overall message is about how economics isn't some invisible hand, but rather an interconnecting system of human behaviors and experiences.
March 29, 2022
Awesome book on false positives and scaling. Real world examples throughout. Reminded me of Good to Great. Couldn't put it down!
February 11, 2022
What’s your secret sauce—the chef, or the ingredients? This is but one of several important questions you need to ask yourself in order to successfully scale, according to The Voltage Effect—a Michelin star cookbook in the world of scaling that gives a fresh perspective on what to do (and not to do) in your scaling journey. Much like a well-crafted meal, a scalable idea requires several essential ingredients, ingredients which Head Chef John List calls the five vital signs. Through bite-sized chapters topped off with captivating real-life stories on scaling failures, successes, and everything in between, Chef List explores the ins and outs of how to best utilize your ingredients to scale your next delicious idea. Whether you’re an entrepreneur, policymaker, or just interested in how people take ideas to the next level, The Voltage Effect is sure to offer some serious (and scalable) food for thought.
May 27, 2022
Muy buen libro con aplicaciones concretas de la teoría económica para la gestión exitosa de las empresas, particularmente para lograr crecer de manera rentable. Contiene joyas que hacen que valga mucho la pena, como la importancia de entender conceptos económicos fundamentales para la toma de decisiones como el costo de oportunidad, la ventaja comparativa, el análisis costo-beneficio y los costos hundidos (¡que deben quedar hundidos en el pasado y no afectar nuestras decisiones a futuro!). También vale mucho la pena su explicación del valor de la agilidad y la experimentación que nos permite probar ideas que luego puedan ser escalables y de alto impacto, así como la manera correcta de implementar estos experimentos. Tal vez el punto más débil del libro es que repetidamente se empecina en criticar a Uber, empresa para la que trabajó varios años y contra la que parece tener algún agravio (por más que varias de las críticas estén justificadas) y la constante repetición de las mantras woke que abundan en la academia hoy en día.
April 11, 2022
It’s a decent book, but sometimes dumbed down to a degree that I found annoying. I don’t think I’ll ever use the term “voltage” in the way List hopes people will.
List is a very interesting and creative scholar, and I did enjoy hearing about his journey.
Based on his academic and applied work in industry, he derives some factors that may help you in assessing whether you really are ready to take a product or intervention to scale:
Are your inferences from the research that you have done reliable? Check for biases in the way people tend to process information. Check to be sure that the populations that you have studied are representative of those that you want to scale to.
Is the product or intervention something that requires special ingredients that are difficult to scale? For example, success at a smaller scale may have depended on the expert or specialized input of individuals who are rare. You should test your intervention with the types of inputs that you would use at scale.
Does the benefit from your product or intervention depend on people complying, and to what extent can you really count on this compliance or take up happening at scale?
These were reasonable enough. It was a nice discussion, although only about ⅓ of the book.
The rest was much less interesting.
For example the discussion of “spillovers” didn’t seem like it was on a strong evidentiary foundation. Eg, I know the cash transfer study and the discussion cherry picked the evidence it presented. The idea that positive spillovers masked a tenfold benefit in the preschool treatment seems crazy. List seems to be falling for the same tendency to tell a good story that he warns against earlier in the book.
He also derives factors that can “boost voltage”—ie, help to improve performance at scale:
Leverage peer effects, such as people’s desire to maintain a good image in front of their peers, and loss aversion.
Focus on marginal returns not average returns (and especially not returns relative to sink costs) and opportunity costs ( meaning sometimes it’s good to quit).
This all seems nice in theory but it’s treated at such a surface level that we never get into the dilemmas that arise when putting these ideas into practice.
Then there was the facile bashing of “equal opportunity employer” policies, after one study, without showing much commitment to getting EOE strategies right. Again, too superficial.
It’s a light read, and you’ll get some tidbits here and there.
List is a very interesting and creative scholar, and I did enjoy hearing about his journey.
Based on his academic and applied work in industry, he derives some factors that may help you in assessing whether you really are ready to take a product or intervention to scale:
Are your inferences from the research that you have done reliable? Check for biases in the way people tend to process information. Check to be sure that the populations that you have studied are representative of those that you want to scale to.
Is the product or intervention something that requires special ingredients that are difficult to scale? For example, success at a smaller scale may have depended on the expert or specialized input of individuals who are rare. You should test your intervention with the types of inputs that you would use at scale.
Does the benefit from your product or intervention depend on people complying, and to what extent can you really count on this compliance or take up happening at scale?
These were reasonable enough. It was a nice discussion, although only about ⅓ of the book.
The rest was much less interesting.
For example the discussion of “spillovers” didn’t seem like it was on a strong evidentiary foundation. Eg, I know the cash transfer study and the discussion cherry picked the evidence it presented. The idea that positive spillovers masked a tenfold benefit in the preschool treatment seems crazy. List seems to be falling for the same tendency to tell a good story that he warns against earlier in the book.
He also derives factors that can “boost voltage”—ie, help to improve performance at scale:
Leverage peer effects, such as people’s desire to maintain a good image in front of their peers, and loss aversion.
Focus on marginal returns not average returns (and especially not returns relative to sink costs) and opportunity costs ( meaning sometimes it’s good to quit).
This all seems nice in theory but it’s treated at such a surface level that we never get into the dilemmas that arise when putting these ideas into practice.
Then there was the facile bashing of “equal opportunity employer” policies, after one study, without showing much commitment to getting EOE strategies right. Again, too superficial.
It’s a light read, and you’ll get some tidbits here and there.
February 4, 2023
The Voltage Effect: How to Make Good Ideas Great and Great Ideas ScaleAlso see at: https://positivehealthleadership.org/...
I greatly respect John List, Ph.D., and many of his recommendations. I have read a lot of his work and heard him in presentations. I have also listened to him on Freakonomics. After hearing a review of his new book on Freakonomics, "Why Do Most Ideas Fail to Scale?" (the book is "The Voltage Effect: How to make good ideas great and great ideas scale"), I got a copy and read it.
Good Ideas
Many great ideas are contained, specifically about why ventures may not scale. For instance, he emphasizes it is important not to be misled by false positives. This is when good results happen, but it is not with a representative sample, thus falsely showing the idea may be successfully scaled. He documents how this happened with Nancy Reagan's "Just Say No" anti-drug campaign, explaining why it was not as successful or even validated. He also explained the importance of knowing the intended audience. Here he cited an example where McDonald's had sampled devoted McDonald's customers to test the new "Arch Deluxe" rather than testing it with the typical customer. He explained this was why the "Arch Deluxe" was not a successful new product and why it was necessary to use actual customers, not devotees.
He also explains that when scaling, it is essential to know if the chef, the leader, or the ingredients make the product successful. As he explains, it is easier to scale ingredients than people. Here he also explained the importance of spillover effects. As he made apparent, minor issues become more prominent when ideas are scaled. This can be documented by "General Equilibrium Effects" based on the theory. This theory explains how expectations can be disrupted because when one area changes, all areas change to adapt to that change. As an example, he explained how when Uber raised driver salaries in hopes of helping them earn more take-home pay, more drivers then drove for Uber. The salary increase increased the number of drivers, leading to fewer rides being given per driver. This meant the drivers did not earn the desired raise. This was a great example of General Equilibrium Effects.
He also explained that intervention spillovers could be positive or negative. As a positive example, he explained the spillover of Herd Immunity happens when many people in the community are vaccinated. He also warned that if costs were too high, it could not scale. While many ideas were good, as I kept reading, something was nagging at me, and something seemed off. It was not until I got to Chapter 7, and when he began to focus on how to scale, that I realized what was nagging me.
Fatal Flaw?
What was nagging me was that the book was about improving the parts without accounting for the whole system and the dynamic interactions or "Systems Appreciation" in Deming's Profound Knowledge. To compound his inattention to the entire system, he also failed to account for the environmental impacts of any venture. It was as if he equated the impact on the environment at the cost of 0. This must be accounted for because the environment is an asset on which every venture and all of us depend. Treating nature with no value encourages its misuse. Ventures should operate such that it supports regeneration because this can be the only way to ensure actual value and improvement, as it supports ongoing viability and profitability for everyone and everything.
System Improvement
As Dr. Ackoff explains (see this powerful presentation titled, "If Russ Ackoff Had Given a TED Talk"), a system is not a sum of its parts but a product of its interactions. Further, he explains that if a system improves by improving the parts taken separately, you can be sure the performance of the whole will not be improved. This is the fatal flaw in List's book. He discusses methods to enhance the parts without improving the system. In Dr. Deming's terms, he has not had an "Appreciation for a System."
https://www.youtube.com/watch?v=OqEeI...
One example he discussed in the book was investing on marginal returns, or the area with the most significant return on the last dollar spent. This may sometimes work. However, it mistakenly encourages management by results or managing by watching the scoreboard rather than continually improving the process. Managing results will result in higher and higher variance, higher costs, and lower profits. (see Red Bead Experiment) In the book Dr. List even relied on a faulty example, explaining hiring more people did not produce the same returns because the new group was not as productive. This mistakenly placed responsibility on the people rather than the system from which results are generated.
Understanding Variation - Contribution from Allen Scott also cited information obtained in personal communication with quality expert Don Wheeler, PhD
In The Science of Using Science: Towards an Understanding of the Threats to Scaling Experiments, John List et al. states, "Policymakers are increasingly turning to insights gained from the experimental method as a means of informing public policies." They argue that knowing when evidence becomes actionable requires information about the population and the situation. Further, they suggest this information is vital to understanding if scaling will work.
Their writing seems to suggest more than experimental methods are necessary. Their concern, relying only on the scientific method can lead to a vast waste of resources, a missed opportunity to improve people's lives, and a diminution in the public's trust in the scientific method's ability to contribute to policymaking.
Actionable Evidence
Dr. Walter A. Shewhart, in 1924 at Bell Labs, developed process behavior charts to determine when evidence becomes actionable. These charts could identify appropriate statistical evidence by separating the noise from the signal. These charts provided an observational improvement method that plotted data over time.
In Understanding Variation: The Key to Managing Chaos, Dr. Donald J. Wheeler documents that process behavior charts work and have been thoroughly proven. Further, it seems hard sciences can use experimental methods and hold many variables constant. However, social sciences must deal with unknown cause-and-effect relationships. These unknowns make the decision to scale problematic without more information. In such an environment, observational studies are needed rather than experiments. If a test program is broad enough and predictable, reliable evidence will be gained about scaling. If, however, the evidence is localized and unpredictable, the proof will be problematic.
List suggests this in his book, "Voltage Effect," when he explains misleading evidence and false positives lead to misinformed choices to scale. As he explains, observational studies can be better than experiments when deciding to scale if they are representative enough to be predictable.
As I see it, the problem is the assumption that we will know all the essential factors. Experimentation cannot identify the unknown factors; only observation does this. Don Wheeler, PhD
Real-Life Example
For example, despite experimental evidence about the value of Hormone Replacement Therapy (HRT), observation studies of over 8000 women over ten years showed that post-menopausal hormone replacement therapy changed the likelihood of heart attack from 2% to 3%. This study made it clear that HRT benefits did not outweigh the risks.
For more on experimental studies versus observational studies, see the recent Quality Digest article by Dr. Wheeler: Different Approaches to Process Improvement. Does your approach do what you need? We recommend Dr. Greger's review of this topic in his linked short Nutritional Facts.org July 4, 2022 vlog post; Observational Studies Show Similar Results to Randomized Trials.
Quitting Shouldn't Exist...
Another concern I had with List's book related to quitting. Dr. List emphasizes the need to get better at quitting and the need to quit. I am not sure why he chose to describe it as quitting. He equated quitting with Failure, but Failure doesn't exist (see Failure Doesn't Exist…). Most entrepreneurs' drive to succeed and do well would stay the same. Thus the aim would not be consistent with quitting. This is why the idea of quitting is a confusing reference. For instance, using his personal example, he explained that he chose not to make a difference by being a professional golfer but as an academic professor. Thus by his admission, he did not quit wanting to make a difference; he just pivoted.
To me, pivoting, a term used often by the NSF iCorps program and others, is a better way to encourage entrepreneurs and is a method to help them succeed and scale. A pivot should occur when an entrepreneur discovers, after researching the idea, the market, and customers, that the picture is a no-go or not a good idea to scale. After discovering the concept may be problematic, they must pivot to a variant or alternative. From my perspective, this is better terminology than quitting and allows the entrepreneur to carry forward the many assets and skills gained toward the pivoted aim of the venture.
While I encourage you to read Lists' book, "The Voltage Effect: How to make good ideas great and great ideas scale," please keep in mind that any venture must contribute to systemic improvement. I am concerned List did not adequately account for the system's impact, especially when parts are maximized, as he suggests. This was a bit confusing since he did discuss "General Equilibrium Effects" and then ignored it throughout the text.
As Russ Ackoff makes clear, improving the parts cannot improve the system. As we all seek to contribute toward comprehensive improvements, we must focus on creating net-positive, pervasive, reciprocal, selfish, selfless, synergistic interactions so everyone and everything benefits. Please share how you practice paneugenesis.
I greatly respect John List, Ph.D., and many of his recommendations. I have read a lot of his work and heard him in presentations. I have also listened to him on Freakonomics. After hearing a review of his new book on Freakonomics, "Why Do Most Ideas Fail to Scale?" (the book is "The Voltage Effect: How to make good ideas great and great ideas scale"), I got a copy and read it.
Good Ideas
Many great ideas are contained, specifically about why ventures may not scale. For instance, he emphasizes it is important not to be misled by false positives. This is when good results happen, but it is not with a representative sample, thus falsely showing the idea may be successfully scaled. He documents how this happened with Nancy Reagan's "Just Say No" anti-drug campaign, explaining why it was not as successful or even validated. He also explained the importance of knowing the intended audience. Here he cited an example where McDonald's had sampled devoted McDonald's customers to test the new "Arch Deluxe" rather than testing it with the typical customer. He explained this was why the "Arch Deluxe" was not a successful new product and why it was necessary to use actual customers, not devotees.
He also explains that when scaling, it is essential to know if the chef, the leader, or the ingredients make the product successful. As he explains, it is easier to scale ingredients than people. Here he also explained the importance of spillover effects. As he made apparent, minor issues become more prominent when ideas are scaled. This can be documented by "General Equilibrium Effects" based on the theory. This theory explains how expectations can be disrupted because when one area changes, all areas change to adapt to that change. As an example, he explained how when Uber raised driver salaries in hopes of helping them earn more take-home pay, more drivers then drove for Uber. The salary increase increased the number of drivers, leading to fewer rides being given per driver. This meant the drivers did not earn the desired raise. This was a great example of General Equilibrium Effects.
He also explained that intervention spillovers could be positive or negative. As a positive example, he explained the spillover of Herd Immunity happens when many people in the community are vaccinated. He also warned that if costs were too high, it could not scale. While many ideas were good, as I kept reading, something was nagging at me, and something seemed off. It was not until I got to Chapter 7, and when he began to focus on how to scale, that I realized what was nagging me.
Fatal Flaw?
What was nagging me was that the book was about improving the parts without accounting for the whole system and the dynamic interactions or "Systems Appreciation" in Deming's Profound Knowledge. To compound his inattention to the entire system, he also failed to account for the environmental impacts of any venture. It was as if he equated the impact on the environment at the cost of 0. This must be accounted for because the environment is an asset on which every venture and all of us depend. Treating nature with no value encourages its misuse. Ventures should operate such that it supports regeneration because this can be the only way to ensure actual value and improvement, as it supports ongoing viability and profitability for everyone and everything.
System Improvement
As Dr. Ackoff explains (see this powerful presentation titled, "If Russ Ackoff Had Given a TED Talk"), a system is not a sum of its parts but a product of its interactions. Further, he explains that if a system improves by improving the parts taken separately, you can be sure the performance of the whole will not be improved. This is the fatal flaw in List's book. He discusses methods to enhance the parts without improving the system. In Dr. Deming's terms, he has not had an "Appreciation for a System."
https://www.youtube.com/watch?v=OqEeI...
One example he discussed in the book was investing on marginal returns, or the area with the most significant return on the last dollar spent. This may sometimes work. However, it mistakenly encourages management by results or managing by watching the scoreboard rather than continually improving the process. Managing results will result in higher and higher variance, higher costs, and lower profits. (see Red Bead Experiment) In the book Dr. List even relied on a faulty example, explaining hiring more people did not produce the same returns because the new group was not as productive. This mistakenly placed responsibility on the people rather than the system from which results are generated.
Understanding Variation - Contribution from Allen Scott also cited information obtained in personal communication with quality expert Don Wheeler, PhD
In The Science of Using Science: Towards an Understanding of the Threats to Scaling Experiments, John List et al. states, "Policymakers are increasingly turning to insights gained from the experimental method as a means of informing public policies." They argue that knowing when evidence becomes actionable requires information about the population and the situation. Further, they suggest this information is vital to understanding if scaling will work.
Their writing seems to suggest more than experimental methods are necessary. Their concern, relying only on the scientific method can lead to a vast waste of resources, a missed opportunity to improve people's lives, and a diminution in the public's trust in the scientific method's ability to contribute to policymaking.
Actionable Evidence
Dr. Walter A. Shewhart, in 1924 at Bell Labs, developed process behavior charts to determine when evidence becomes actionable. These charts could identify appropriate statistical evidence by separating the noise from the signal. These charts provided an observational improvement method that plotted data over time.
In Understanding Variation: The Key to Managing Chaos, Dr. Donald J. Wheeler documents that process behavior charts work and have been thoroughly proven. Further, it seems hard sciences can use experimental methods and hold many variables constant. However, social sciences must deal with unknown cause-and-effect relationships. These unknowns make the decision to scale problematic without more information. In such an environment, observational studies are needed rather than experiments. If a test program is broad enough and predictable, reliable evidence will be gained about scaling. If, however, the evidence is localized and unpredictable, the proof will be problematic.
List suggests this in his book, "Voltage Effect," when he explains misleading evidence and false positives lead to misinformed choices to scale. As he explains, observational studies can be better than experiments when deciding to scale if they are representative enough to be predictable.
As I see it, the problem is the assumption that we will know all the essential factors. Experimentation cannot identify the unknown factors; only observation does this. Don Wheeler, PhD
Real-Life Example
For example, despite experimental evidence about the value of Hormone Replacement Therapy (HRT), observation studies of over 8000 women over ten years showed that post-menopausal hormone replacement therapy changed the likelihood of heart attack from 2% to 3%. This study made it clear that HRT benefits did not outweigh the risks.
For more on experimental studies versus observational studies, see the recent Quality Digest article by Dr. Wheeler: Different Approaches to Process Improvement. Does your approach do what you need? We recommend Dr. Greger's review of this topic in his linked short Nutritional Facts.org July 4, 2022 vlog post; Observational Studies Show Similar Results to Randomized Trials.
Quitting Shouldn't Exist...
Another concern I had with List's book related to quitting. Dr. List emphasizes the need to get better at quitting and the need to quit. I am not sure why he chose to describe it as quitting. He equated quitting with Failure, but Failure doesn't exist (see Failure Doesn't Exist…). Most entrepreneurs' drive to succeed and do well would stay the same. Thus the aim would not be consistent with quitting. This is why the idea of quitting is a confusing reference. For instance, using his personal example, he explained that he chose not to make a difference by being a professional golfer but as an academic professor. Thus by his admission, he did not quit wanting to make a difference; he just pivoted.
To me, pivoting, a term used often by the NSF iCorps program and others, is a better way to encourage entrepreneurs and is a method to help them succeed and scale. A pivot should occur when an entrepreneur discovers, after researching the idea, the market, and customers, that the picture is a no-go or not a good idea to scale. After discovering the concept may be problematic, they must pivot to a variant or alternative. From my perspective, this is better terminology than quitting and allows the entrepreneur to carry forward the many assets and skills gained toward the pivoted aim of the venture.
While I encourage you to read Lists' book, "The Voltage Effect: How to make good ideas great and great ideas scale," please keep in mind that any venture must contribute to systemic improvement. I am concerned List did not adequately account for the system's impact, especially when parts are maximized, as he suggests. This was a bit confusing since he did discuss "General Equilibrium Effects" and then ignored it throughout the text.
As Russ Ackoff makes clear, improving the parts cannot improve the system. As we all seek to contribute toward comprehensive improvements, we must focus on creating net-positive, pervasive, reciprocal, selfish, selfless, synergistic interactions so everyone and everything benefits. Please share how you practice paneugenesis.
September 6, 2022
Inside the Strategy Room podcast
https://podcasts.apple.com/us/podcast...
https://www.mckinsey.com/business-fun...
Avoid mistakes. The first thing people need to think about in the idea search phase or when considering scaling is, what are the warts? If there is a wart, such as that the addressable market is too small, how can you expand it before you scale or as you scale?
Every organization should have a scale unit that includes a naysayer who says, ‘I don’t think that idea will work, and I will generate data to prove it.
It’s one thing to state your ambitions and preferences in a survey and a completely different thing to back them with real money.
Many restaurants have great success, and their owners say, “If I could scale to 50 or 100 restaurants, I’d literally be cooking,” but a lot of them fail. If the initial success was based on the chef—a unique human—it will never scale because unique humans don’t scale. If you can convert that unique human into a process, sometimes that works.
When you scale across countries, the rules and customs change, minimum-wage laws may change, and if those changes prevent you from scaling your inputs, you’re dead.
“The ‘more is less’ phenomenon,” based on an experiment where I auctioned off ten baseball cards as a group. Then, in another auction, I added three defective cards to the original ten. You would think that people would bid more because it’s the ten cards plus the three defective ones, which together still have value. But they bid less because they looked at the three defective ones and interpreted the other ten as being of lower quality. Streaming wars may also exemplify the dilution principle: once you add mediocre content around baseline strong content, people’s perception of all the content drops.
Nonfinancial incentives are important, too, such as social image, social pressure, self-image. Social incentives are all around us, and we need to take advantage of them to implement incentives that can scale.
https://podcasts.apple.com/us/podcast...
https://www.mckinsey.com/business-fun...
Avoid mistakes. The first thing people need to think about in the idea search phase or when considering scaling is, what are the warts? If there is a wart, such as that the addressable market is too small, how can you expand it before you scale or as you scale?
Every organization should have a scale unit that includes a naysayer who says, ‘I don’t think that idea will work, and I will generate data to prove it.
It’s one thing to state your ambitions and preferences in a survey and a completely different thing to back them with real money.
Many restaurants have great success, and their owners say, “If I could scale to 50 or 100 restaurants, I’d literally be cooking,” but a lot of them fail. If the initial success was based on the chef—a unique human—it will never scale because unique humans don’t scale. If you can convert that unique human into a process, sometimes that works.
When you scale across countries, the rules and customs change, minimum-wage laws may change, and if those changes prevent you from scaling your inputs, you’re dead.
“The ‘more is less’ phenomenon,” based on an experiment where I auctioned off ten baseball cards as a group. Then, in another auction, I added three defective cards to the original ten. You would think that people would bid more because it’s the ten cards plus the three defective ones, which together still have value. But they bid less because they looked at the three defective ones and interpreted the other ten as being of lower quality. Streaming wars may also exemplify the dilution principle: once you add mediocre content around baseline strong content, people’s perception of all the content drops.
Nonfinancial incentives are important, too, such as social image, social pressure, self-image. Social incentives are all around us, and we need to take advantage of them to implement incentives that can scale.
Read
April 8, 2023According to John List, the “Voltage Effect” is a compilation of 5 “vital signs” of the scalability of a business idea.
The 5 vital signs are:
- No false positives in small scale tests,
- Have a representative audience in small scale tests,
- No reliance on hard to find human talent or resources (they get harder to find as you scale up),
- No spillover effects (no unintended consequences that appear when scaling an idea up),
- Improving unit economics (the cost per unit completed must stay flat or ideally go down as you scale up).
3 best takeaways:
- Always get a second (or even third or fourth) opinion before making a substantial medical decision such as surgery or intense courses of medication.
- Quitting is for winners. Always generate additional good options to ensure good quitting decisions as you better understand your opportunity cost.
- When Uber tried to increase their driver wages they initially succeeded. However, as the wages were better more drivers joined which created an oversupply of drivers. Ultimately driver earnings returned back to the prior amount but with the spillover effect that drivers were working longer.
Checkout my substack if you want to read my full notes https://lachysnotes.substack.com/p/th...
The 5 vital signs are:
- No false positives in small scale tests,
- Have a representative audience in small scale tests,
- No reliance on hard to find human talent or resources (they get harder to find as you scale up),
- No spillover effects (no unintended consequences that appear when scaling an idea up),
- Improving unit economics (the cost per unit completed must stay flat or ideally go down as you scale up).
3 best takeaways:
- Always get a second (or even third or fourth) opinion before making a substantial medical decision such as surgery or intense courses of medication.
- Quitting is for winners. Always generate additional good options to ensure good quitting decisions as you better understand your opportunity cost.
- When Uber tried to increase their driver wages they initially succeeded. However, as the wages were better more drivers joined which created an oversupply of drivers. Ultimately driver earnings returned back to the prior amount but with the spillover effect that drivers were working longer.
Checkout my substack if you want to read my full notes https://lachysnotes.substack.com/p/th...
December 29, 2022
I had the pleasure of meeting Mr. List at my university where he was promoting the book and lecturing. The man is no stranger to dad jokes and corny humor as is evident in his book. I’m sure Mr. List has extensive experience writing professional and academic papers and that is not what I was expecting in “The Voltage Effect,” but this book’s prose was a little too relaxed in my opinion.
As far as the content is concerned, I did not really glean anything substantial although I am a finance graduate, so I can see where someone without an economic/finance background could find the book helpful with introductory economic ideas. The way that List mentions his social and economic experiments to support his arguments feels like having a conversation with someone who says, “oh yeah and this one time…”
Buy and large, it was a decent read, and List is certainly an interesting man to read about, but as far as content, I do not think I learned anything substantial beyond introductory economics.
Reader beware: there are countless times List says, “up the voltage” or something along those lines.
As far as the content is concerned, I did not really glean anything substantial although I am a finance graduate, so I can see where someone without an economic/finance background could find the book helpful with introductory economic ideas. The way that List mentions his social and economic experiments to support his arguments feels like having a conversation with someone who says, “oh yeah and this one time…”
Buy and large, it was a decent read, and List is certainly an interesting man to read about, but as far as content, I do not think I learned anything substantial beyond introductory economics.
Reader beware: there are countless times List says, “up the voltage” or something along those lines.
August 6, 2022
I give it 3.5 stars but rounded it to 4 stars. The title "The Voltage Effect" isn't intuitively associated with "scaling" so the author had to explain it. It felt like much of the book was like that... the author had to explain the connection between the organization's actions and failure to scale (as opposed to a typical business failure regardless of size). Some of the information is useful but not new. The book should have started with the premise that scaling failure occurs at the weakness link. For Uber, it was its toxic work environment. For Jamie Oliver's restaurant chain, it was its key Executives. Perhaps just getting and keeping the right people is half the battle of scaling.
January 2, 2023
Lo interesante de este libro es la genialidad de John List de tomar conceptos básicos de economía y ordenarlos en una tipo-receta para escalar ideas/proyectos/empresas/equipos. John basa sus recomendaciones a partir de su propia experiencia como profesor, asesor económico de empresas, y padre. Esto aleja al libro de un volumen más de economía, y lo lleva a un libro accesible a una audiencia mucho mayor. Entre tanta pretensión que envuelve la profesión de economista, da cierto alivio encontrar libros como este.
March 23, 2022
John List writes fascinating stories and data in a humble and provocative way. I learned quite a bit through the read and I was grateful for the many economic definitions he shared. My only unmet desire was a slightly deeper dive into the economics, maybe some formulas or graphs, but other than that it was a gem full of practical applications for several areas of my life.
April 14, 2022
Great book with lots of stories on why scaling works and don't work and how to run field experiments on big datasets.
Main reason for me to not give it five stars is that lot of the basics has already been covered in other books
Main reason for me to not give it five stars is that lot of the basics has already been covered in other books
April 16, 2022
I thoroughly enjoyed this book.
I got a lot out of it.
How to scale ideas, behavioural economics, and what makes programs and policies work.
A lot of this is pertinent to my career, and I really appreciate List's excellent writing in his own exciting career.
I would highly recommend this if you are interested in behavioural economics - something I love. Think Kahneman or Ariely.
Admittedly, I found the last quarter of this book a little underwhelming, but the rest of the book was outstanding.
4.4/5
I got a lot out of it.
How to scale ideas, behavioural economics, and what makes programs and policies work.
A lot of this is pertinent to my career, and I really appreciate List's excellent writing in his own exciting career.
I would highly recommend this if you are interested in behavioural economics - something I love. Think Kahneman or Ariely.
Admittedly, I found the last quarter of this book a little underwhelming, but the rest of the book was outstanding.
4.4/5
June 22, 2022
I’ve always been interested in knowing more about the University of Chicago economist John List and so when I heard he had a book on a topic that could be interesting, I was eager to read it. The Voltage Effect is mainly focused on scaling ideas, but he uses it as a vehicle to talk more about his research that uses field experiments to test different hypothesis and try out novel approaches to things.
To start off, he described what exactly he meant by voltage in this context. “These cases are all examples of a voltage drop: when an enterprise falls apart at scale and positive results fizzle. (The term ‘voltage drop’ comes from the literature of implementation science and can be traced to the work of Amy Kilbourne and co-authors.) Voltage drops are what happens when the great electric charge of potential that drives people and organizations dissipates, leaving behind dashed hopes, not to mention squandered money, hard work, and time…According to Straight Talk on Evidence…between 50 and 90 percent of programs will lose voltage at scale.” (13)
The structure of the book is to layout 5 things that can cause problems with scaling, and then go into 4 things that you should seek out when trying to scale. “It [the book] unpacks the Five Vital Signs, or the five key signature elements that will cause voltage drops and prevent an idea from taking off. The first is false positives – these are cases where there was never any voltage in the first place, though it appeared otherwise. The second is overestimating how big a slice of the pie your idea can capture. Often this is the result of failing to know your audience – or assuming that the small subset of people who have bought into your idea are more representative of the general population than they actually are, so that when you expand your idea it falls short for a broader set of people . The third is failing to evaluate whether your initial success depends on unscalable ingredients – unique circumstances that can’t be replicated at scale. The fourth is when the implementation of your idea has unintended consequences or spillovers, that backfire against that same idea. And the fifth is the ‘supply-side economics’ of scaling – for instance, will your idea be too costly to sustain at scale?” (16)
The section on false positives was pretty intuitive – you can’t scale something if it doesn’t actually work. He had good examples of testing things at a small scale, and in multiple domains. Don’t only test it in one type of city, but make sure to test it various places. In his own experiments, he found success in one small experiment, but thankfully did another small experiment before rolling out the policy to a wider company. He found it didn’t work, and saved money and credibility by not rolling it out. I liked the idea of having an idea subject to independent replication. “Of course, showing data in a way that highlights the strengths of an idea or downplays the weaknesses is much different from knowingly falsifying data, but the solution to both is the same: independent replication, which is just as crucial for businesses as it is for scientists. If someone has an idea, it should be someone else with no stake in the idea and who doesn’t stand to benefit financially who should test it out, or at least replicate it before it is shipped. Otherwise, incentives potentially conflict with full honesty.” (42) I’m always a fan of having it set up to have pushback on an idea, and encourage a person or people to play the role of devil’s advocate. “As we have seen, organizations don’t always incentivize employees to speak the truth, and in many cases the person who comes up with an idea is also the sole tester of the idea. This speaks more broadly to the need for every business and organization to have a devil’s advocate deputy, team, and/or function built into its structure – in other words, a force that is always pushing for more data, more proof…By now I hope it’s clear that the most hazardous obstacle to successful scaling is not ignorance. It is the illusion of knowledge, arising from either misleading data, hidden biases, or outright deception.” (43)
The know your audience section was pretty good as well. A big emphasis was not having a one size fits all approach to things, but instead letting people in different places tailor aspects to meet the needs of the people in that area. He also talked about the importance of finding the right set of people to test an idea on. “This example highlights that there is always the risk – or temptation – that researchers may deliberately seek out a specific population that stands to benefit most from the program, product, or medication in order to show large effects, since this can increase the chance of public recognition and further funding or investment. Likewise, it may be less expensive to convince people to participate in a study if those people expect to benefit from it.” (59)
One of my favorite parts of the book was the section on “Is it the Chef or the Ingredients,” focusing on what the non-negotiables are in your organization. What can you compromise on and what can you do without? The example of many restaurants having an amazing chef that can’t scale versus Jamie Oliver having restaurants that focus on great, simple ingredients that many chefs could produce was vivid and memorable. “For an idea or enterprise – not just restaurant chains [like those of Jamie Oliver] – to hold strong at scale, you need to know what the drivers of high performance are and do everything in your power to keep them in place. To achieve this, before anything else you must determine if your secret sauce is the ‘chef’ or the ‘ingredients.’ In other words, dose your success at small scale rest largely on the people indispensable to your idea or product, say the engineer who built the platform your business runs on, or the celebrity spokesperson who fundraises for your nonprofit – or is it the idea or product itself? If it involves people, a key piece to understand is whether those responsible for implementing the idea will be faithful to its ingredients…Knowing this isn’t half the battle. It’s the whole battle. If your answer is the ‘chef’ (people), there will likely be a limit to how big you can get, since, as we’ve seen, people with unique skills are inherently unscalable…When it comes to ingredients, you must know your negotiables and non-negotiables, then figure out whether your non-negotiable ingredients – the ones your enterprise can’t survive without – are in fact scalable.” (75)
I liked the section on spillovers as well. It talks a lot about unintended consequences of actions, like how there can be unintended consequences to additional safety features, or negative aspects to having wage transparency. “While Peltzman’s paper [on how increased auto safety regulation didn’t lead to fewer injuries] was controversial at the time – unsurprisingly, it was politicized by pro- and anti-regulation advocates – much research in the intervening years has borne out similar conclusions in other domains. It turns out people have a tendency to engage in riskier behaviors when measures are imposed to keep them safer. Give a biker a safely helmet and he rides more recklessly – and, even worse, cars around him drive more haphazardly…In short, safety measures have the potential to undermine their own purpose. This phenomenon – which came to be known as the Peltzman effect – is often used as a lens for studying risk compensation, the theory that we make different choices depending on how secure we feel in any given situation (i.e., we take more risk when we feel more protected and less when we perceive that we are vulnerable).” (90) I had heard about instances of head trauma going up after bike helmet laws were passed, so this reinforced that prior of mine. I thought the part on incentivizing workers was interesting, too. Money can incentivize the people you pay more, but it can also disincentivize the people you pay less. “As it turned out, it wasn’t the extra $5 that incentivized the higher-paid solicitors to work harder. It was the $5 less that disincentivized the lower-paid workers when they knew others were making more than them. The fact was, at odds with my original belief in the very first summer, the solicitors did talk about pay and this served to disincentivize the lower wage earners. This group shirked their duties by visiting fewer houses, and even engaged in more theft, pocketing donations at a much higher rate than the higher-paid solicitors. This is an example of the psychological phenomenon called resentful demoralization. It’s the flip side of the famous John Henry effect, which is the bias introduced into experiments when members of the control group are aware that they are being compared to the experimental group and react by trying harder than they typically would. In my field experiment, the solicitors did the opposite. The feelings of resentment resulting from the knowledge of the wage disparity (i.e., that they were being underpaid compared to their peers) created an unintended effect that undercut our fundraising…The idea is that salary data should be made publicly available – at least inside the company – so that everyone knows how much everyone else is making, from the bottom all the way up to the top. Based on my experience with the baseball fundraising, one would expect that making salaries transparent could drive resentful demoralization if people saw peers making more.” (99) There is more nuance to it, and they found that when managers make more than their workers thought they did, they worked even harder, reinforcing the idea that workers work hard when they feel like they can have a good return to their effort.
The Cost Trap section made sense – good ideas have to make a profit, not just revenue. One thing that was interesting was the idea of being able to scale with good, but not great, workers. “It may sound counterintuitive, or even idiotic, not to search out the best talent in the early stages of your endeavor. When it comes to, say, designing new, innovative hardware that will be scaled, I’m not saying to choose mediocre computer engineers. After all, the hardware or digital interface must be of high quality; that’s a non-negotiable. But if maintaining that hardware at scale will require forty thousand technicians, the reality is that not all of them will be five-star workers, so those in the development phase shouldn’t be, either. Hiring less-excellent technicians is admittedly not ideal, but it is a negotiable…Ideal conditions are not realistic in most cases, so you must ask yourself a question infused with a rude dose of reality: will you actually be able to hire the best people at scale, or will either budget restriction or the finite pool of talented candidates make this impracticable?” (126) I agree that you will have to have workers like this eventually, but I’m not sure I totally buy into his assertion that you should use them the entire time to see how mediocre people do running the show.
I did like the part on Incentives that Scale. This had a lot of stories from his time at Uber, but also studies related to reducing jet fuel usage and improving tax collection in the Dominican Republic. I thought the part that was most interesting was dealing with incentives for teachers and students. While many find the idea of paying teachers more for higher test scores or students more for better performance unappealing, it seems like it works for certain cohorts. They tested giving teachers a bonus if their students did well, and then giving teachers a bonus, and then clawing it back if their students didn’t do well, and found that the clawback resulted in better performance from the teachers. They then tried a similar experiment with students. “This time we took it in the opposite direction of the clawback, and told some students that if they performed will on the test, they would receive their reward one month after the test. Suddenly, the incentives didn’t matter. That is, rewards – even large ones – delivered with a delay didn’t impact students’ performance at all. This finding suggests that one explanation for students’ low investment in their own education, and their high dropout rate, is that the current returns (get into college, get a higher-paying job, et cetera) are delivered with too long a delay to sufficiently motivate some students. After all, if delaying an incentive by just one month can tank motivation, the abstract prospect of better opportunities in the far-off future clearly won’t be very persuasive…When it comes to incentives, timing is everything.” (155)
I liked the chapter on Revolution on the Margins. I read the blog Marginal Revolution with some regularity, but was not as familiar with the context of the name as I could have been. It’s often easier to calculate the average benefit or cost than the marginal benefit or cost, but when making decisions, it’s better to focus on the margin rather than the average. He talked about this in the context of working for the Council of Economic Advisors. “This chore [benefit-cost analysis of implementing policies on a large scale] is an important one because the more than one hundred federal agencies issue approximately 4,500 new rulemaking notices each year. Of those, about fifty to a hundred per year meet the necessary condition of being ‘economically significant’ (more than $100 million yearly in either benefits or costs). Every economically significant proposal, then, receives a formal analysis of the benefits and costs.” (161) I then liked the extra context on the benefits and importance of looking at the impact of the next item and not just the averages. “And – staying true to my economics training – once I looked at all the charts and graphs and figures across the various agencies, I knew that if we wanted to identify and prioritize the policies that got the most out of taxpayer money, just as Justice Breyer had suggested [in his book Breaking the Vicious Cycle which advocated that the obligation of government was to use money to scale initiatives that improve as many lives as possible], we needed to look not at the positive impact per dollar spent on average, but the positive impact of the last dollar spent. That’s because the benefit-cost averages that lumped all the dollars together were obscuring more specific figures revealing that certain policies became much less impactful the more they were scaled…In the late nineteenth century, the field of economics took an intellectual leap forward that came to be known as the Marginal Revolution…Going beyond the limited concept of supply versus demand, Jevons, Menger, and Walras introduced the utility function, or the theory of utility, into the discussion of value…Everything we spend money on provides a certain amount of satisfaction, or utility, whether we are paying to own an object, use a service, or have an experience. And this level of satisfaction determines the value we receive from goods and services…Jevons, Menger, and Walras posited that utility isn’t static: that goods and services – broken into ‘units’ – have different value to consumers depending on if they are the first or last unit consumed, or somewhere in between. The value of that final, most recent unit is referred to as the marginal utility, and it is rarely the same as the value averaged across all units.” (165)
The chapter called Quitting Is for Winners also had some good content. A bit of it was focused on the sunk cost fallacy. He had some nice examples from his own life in moving on from a focus on golf to focus instead on academics. “Pursuing such objectives requires tremendous sacrifices, the most significant of which is the opportunity costs of paths not taken. This is why it is so devastating when an idea you pour your heart and soul and time into fails to scale. It’s not just the voltage you lose. It’s all the other promising opportunities you turned down in the process…But if you quit at the right time (and ignore that sunk cost), then you can move on to scale something else – something with a better shot at success. This is what I call optimal quitting. Sometimes you have to leave behind that professional golf career you’ve been dreaming of…in order to shift gears and find a better one. And the sooner you do this, the lower the opportunity cost you’ll pay…It requires an effort that runs counter to our deep-rooted heuristics and fast way of thinking…You can see the dangers of this tunnel-vision type of thinking when aiming to scale. Rather than imagine what other ideas they could spend their time pursuing, people often zero in on various aspects of the idea they have already invested time and resources into…When you have lots of alternatives, quitting will be much less painful, both emotionally and practically.” (191) It seemed like some of his comments on comparative advantage were oversimplifying the concept so much it might not have been accurate any more, but overall an interesting chapter on the benefits of not ignoring opportunity cost and not wasting time on less useful pursuits.
The last chapter was on culture. Given that he had worked at Uber during their early days, he saw some interesting things. The chapter talks a bit about how people act differently when having communal goals versus individual goals. “Research suggests that deep trust is a powerful factor in enabling organizations to scale, in part because it promotes cooperation and functional teamwork is essential for growth, but for other reasons as well. The lack of trust at Uber was in large part the natural endgame of an organization that was a meritocracy in name but not in spirit: people didn’t trust that the objective value of their contributions (time, ideas, and effort) would be appreciated. In other words, employees didn’t feel respected…Sure, Uber was great at attacking complacent and lazy thinking, but here is the irony: the leaders at Uber were complacent in how they thought about scaling up the company culture. No one in a leadership position – myself included – forcefully questioned Uber’s culture to the same extent employees were pushed to question its business ideas and practices…He [Travis Kalanick] knew he had made big mistakes, and he showed remorse, not just because he lost his job but because he felt that he had let his Uber team down. I don’t believe Travis Kalanick is a bad person. He is a good person who made several bad calls…at scale.” (211)
(Continued in comments)
To start off, he described what exactly he meant by voltage in this context. “These cases are all examples of a voltage drop: when an enterprise falls apart at scale and positive results fizzle. (The term ‘voltage drop’ comes from the literature of implementation science and can be traced to the work of Amy Kilbourne and co-authors.) Voltage drops are what happens when the great electric charge of potential that drives people and organizations dissipates, leaving behind dashed hopes, not to mention squandered money, hard work, and time…According to Straight Talk on Evidence…between 50 and 90 percent of programs will lose voltage at scale.” (13)
The structure of the book is to layout 5 things that can cause problems with scaling, and then go into 4 things that you should seek out when trying to scale. “It [the book] unpacks the Five Vital Signs, or the five key signature elements that will cause voltage drops and prevent an idea from taking off. The first is false positives – these are cases where there was never any voltage in the first place, though it appeared otherwise. The second is overestimating how big a slice of the pie your idea can capture. Often this is the result of failing to know your audience – or assuming that the small subset of people who have bought into your idea are more representative of the general population than they actually are, so that when you expand your idea it falls short for a broader set of people . The third is failing to evaluate whether your initial success depends on unscalable ingredients – unique circumstances that can’t be replicated at scale. The fourth is when the implementation of your idea has unintended consequences or spillovers, that backfire against that same idea. And the fifth is the ‘supply-side economics’ of scaling – for instance, will your idea be too costly to sustain at scale?” (16)
The section on false positives was pretty intuitive – you can’t scale something if it doesn’t actually work. He had good examples of testing things at a small scale, and in multiple domains. Don’t only test it in one type of city, but make sure to test it various places. In his own experiments, he found success in one small experiment, but thankfully did another small experiment before rolling out the policy to a wider company. He found it didn’t work, and saved money and credibility by not rolling it out. I liked the idea of having an idea subject to independent replication. “Of course, showing data in a way that highlights the strengths of an idea or downplays the weaknesses is much different from knowingly falsifying data, but the solution to both is the same: independent replication, which is just as crucial for businesses as it is for scientists. If someone has an idea, it should be someone else with no stake in the idea and who doesn’t stand to benefit financially who should test it out, or at least replicate it before it is shipped. Otherwise, incentives potentially conflict with full honesty.” (42) I’m always a fan of having it set up to have pushback on an idea, and encourage a person or people to play the role of devil’s advocate. “As we have seen, organizations don’t always incentivize employees to speak the truth, and in many cases the person who comes up with an idea is also the sole tester of the idea. This speaks more broadly to the need for every business and organization to have a devil’s advocate deputy, team, and/or function built into its structure – in other words, a force that is always pushing for more data, more proof…By now I hope it’s clear that the most hazardous obstacle to successful scaling is not ignorance. It is the illusion of knowledge, arising from either misleading data, hidden biases, or outright deception.” (43)
The know your audience section was pretty good as well. A big emphasis was not having a one size fits all approach to things, but instead letting people in different places tailor aspects to meet the needs of the people in that area. He also talked about the importance of finding the right set of people to test an idea on. “This example highlights that there is always the risk – or temptation – that researchers may deliberately seek out a specific population that stands to benefit most from the program, product, or medication in order to show large effects, since this can increase the chance of public recognition and further funding or investment. Likewise, it may be less expensive to convince people to participate in a study if those people expect to benefit from it.” (59)
One of my favorite parts of the book was the section on “Is it the Chef or the Ingredients,” focusing on what the non-negotiables are in your organization. What can you compromise on and what can you do without? The example of many restaurants having an amazing chef that can’t scale versus Jamie Oliver having restaurants that focus on great, simple ingredients that many chefs could produce was vivid and memorable. “For an idea or enterprise – not just restaurant chains [like those of Jamie Oliver] – to hold strong at scale, you need to know what the drivers of high performance are and do everything in your power to keep them in place. To achieve this, before anything else you must determine if your secret sauce is the ‘chef’ or the ‘ingredients.’ In other words, dose your success at small scale rest largely on the people indispensable to your idea or product, say the engineer who built the platform your business runs on, or the celebrity spokesperson who fundraises for your nonprofit – or is it the idea or product itself? If it involves people, a key piece to understand is whether those responsible for implementing the idea will be faithful to its ingredients…Knowing this isn’t half the battle. It’s the whole battle. If your answer is the ‘chef’ (people), there will likely be a limit to how big you can get, since, as we’ve seen, people with unique skills are inherently unscalable…When it comes to ingredients, you must know your negotiables and non-negotiables, then figure out whether your non-negotiable ingredients – the ones your enterprise can’t survive without – are in fact scalable.” (75)
I liked the section on spillovers as well. It talks a lot about unintended consequences of actions, like how there can be unintended consequences to additional safety features, or negative aspects to having wage transparency. “While Peltzman’s paper [on how increased auto safety regulation didn’t lead to fewer injuries] was controversial at the time – unsurprisingly, it was politicized by pro- and anti-regulation advocates – much research in the intervening years has borne out similar conclusions in other domains. It turns out people have a tendency to engage in riskier behaviors when measures are imposed to keep them safer. Give a biker a safely helmet and he rides more recklessly – and, even worse, cars around him drive more haphazardly…In short, safety measures have the potential to undermine their own purpose. This phenomenon – which came to be known as the Peltzman effect – is often used as a lens for studying risk compensation, the theory that we make different choices depending on how secure we feel in any given situation (i.e., we take more risk when we feel more protected and less when we perceive that we are vulnerable).” (90) I had heard about instances of head trauma going up after bike helmet laws were passed, so this reinforced that prior of mine. I thought the part on incentivizing workers was interesting, too. Money can incentivize the people you pay more, but it can also disincentivize the people you pay less. “As it turned out, it wasn’t the extra $5 that incentivized the higher-paid solicitors to work harder. It was the $5 less that disincentivized the lower-paid workers when they knew others were making more than them. The fact was, at odds with my original belief in the very first summer, the solicitors did talk about pay and this served to disincentivize the lower wage earners. This group shirked their duties by visiting fewer houses, and even engaged in more theft, pocketing donations at a much higher rate than the higher-paid solicitors. This is an example of the psychological phenomenon called resentful demoralization. It’s the flip side of the famous John Henry effect, which is the bias introduced into experiments when members of the control group are aware that they are being compared to the experimental group and react by trying harder than they typically would. In my field experiment, the solicitors did the opposite. The feelings of resentment resulting from the knowledge of the wage disparity (i.e., that they were being underpaid compared to their peers) created an unintended effect that undercut our fundraising…The idea is that salary data should be made publicly available – at least inside the company – so that everyone knows how much everyone else is making, from the bottom all the way up to the top. Based on my experience with the baseball fundraising, one would expect that making salaries transparent could drive resentful demoralization if people saw peers making more.” (99) There is more nuance to it, and they found that when managers make more than their workers thought they did, they worked even harder, reinforcing the idea that workers work hard when they feel like they can have a good return to their effort.
The Cost Trap section made sense – good ideas have to make a profit, not just revenue. One thing that was interesting was the idea of being able to scale with good, but not great, workers. “It may sound counterintuitive, or even idiotic, not to search out the best talent in the early stages of your endeavor. When it comes to, say, designing new, innovative hardware that will be scaled, I’m not saying to choose mediocre computer engineers. After all, the hardware or digital interface must be of high quality; that’s a non-negotiable. But if maintaining that hardware at scale will require forty thousand technicians, the reality is that not all of them will be five-star workers, so those in the development phase shouldn’t be, either. Hiring less-excellent technicians is admittedly not ideal, but it is a negotiable…Ideal conditions are not realistic in most cases, so you must ask yourself a question infused with a rude dose of reality: will you actually be able to hire the best people at scale, or will either budget restriction or the finite pool of talented candidates make this impracticable?” (126) I agree that you will have to have workers like this eventually, but I’m not sure I totally buy into his assertion that you should use them the entire time to see how mediocre people do running the show.
I did like the part on Incentives that Scale. This had a lot of stories from his time at Uber, but also studies related to reducing jet fuel usage and improving tax collection in the Dominican Republic. I thought the part that was most interesting was dealing with incentives for teachers and students. While many find the idea of paying teachers more for higher test scores or students more for better performance unappealing, it seems like it works for certain cohorts. They tested giving teachers a bonus if their students did well, and then giving teachers a bonus, and then clawing it back if their students didn’t do well, and found that the clawback resulted in better performance from the teachers. They then tried a similar experiment with students. “This time we took it in the opposite direction of the clawback, and told some students that if they performed will on the test, they would receive their reward one month after the test. Suddenly, the incentives didn’t matter. That is, rewards – even large ones – delivered with a delay didn’t impact students’ performance at all. This finding suggests that one explanation for students’ low investment in their own education, and their high dropout rate, is that the current returns (get into college, get a higher-paying job, et cetera) are delivered with too long a delay to sufficiently motivate some students. After all, if delaying an incentive by just one month can tank motivation, the abstract prospect of better opportunities in the far-off future clearly won’t be very persuasive…When it comes to incentives, timing is everything.” (155)
I liked the chapter on Revolution on the Margins. I read the blog Marginal Revolution with some regularity, but was not as familiar with the context of the name as I could have been. It’s often easier to calculate the average benefit or cost than the marginal benefit or cost, but when making decisions, it’s better to focus on the margin rather than the average. He talked about this in the context of working for the Council of Economic Advisors. “This chore [benefit-cost analysis of implementing policies on a large scale] is an important one because the more than one hundred federal agencies issue approximately 4,500 new rulemaking notices each year. Of those, about fifty to a hundred per year meet the necessary condition of being ‘economically significant’ (more than $100 million yearly in either benefits or costs). Every economically significant proposal, then, receives a formal analysis of the benefits and costs.” (161) I then liked the extra context on the benefits and importance of looking at the impact of the next item and not just the averages. “And – staying true to my economics training – once I looked at all the charts and graphs and figures across the various agencies, I knew that if we wanted to identify and prioritize the policies that got the most out of taxpayer money, just as Justice Breyer had suggested [in his book Breaking the Vicious Cycle which advocated that the obligation of government was to use money to scale initiatives that improve as many lives as possible], we needed to look not at the positive impact per dollar spent on average, but the positive impact of the last dollar spent. That’s because the benefit-cost averages that lumped all the dollars together were obscuring more specific figures revealing that certain policies became much less impactful the more they were scaled…In the late nineteenth century, the field of economics took an intellectual leap forward that came to be known as the Marginal Revolution…Going beyond the limited concept of supply versus demand, Jevons, Menger, and Walras introduced the utility function, or the theory of utility, into the discussion of value…Everything we spend money on provides a certain amount of satisfaction, or utility, whether we are paying to own an object, use a service, or have an experience. And this level of satisfaction determines the value we receive from goods and services…Jevons, Menger, and Walras posited that utility isn’t static: that goods and services – broken into ‘units’ – have different value to consumers depending on if they are the first or last unit consumed, or somewhere in between. The value of that final, most recent unit is referred to as the marginal utility, and it is rarely the same as the value averaged across all units.” (165)
The chapter called Quitting Is for Winners also had some good content. A bit of it was focused on the sunk cost fallacy. He had some nice examples from his own life in moving on from a focus on golf to focus instead on academics. “Pursuing such objectives requires tremendous sacrifices, the most significant of which is the opportunity costs of paths not taken. This is why it is so devastating when an idea you pour your heart and soul and time into fails to scale. It’s not just the voltage you lose. It’s all the other promising opportunities you turned down in the process…But if you quit at the right time (and ignore that sunk cost), then you can move on to scale something else – something with a better shot at success. This is what I call optimal quitting. Sometimes you have to leave behind that professional golf career you’ve been dreaming of…in order to shift gears and find a better one. And the sooner you do this, the lower the opportunity cost you’ll pay…It requires an effort that runs counter to our deep-rooted heuristics and fast way of thinking…You can see the dangers of this tunnel-vision type of thinking when aiming to scale. Rather than imagine what other ideas they could spend their time pursuing, people often zero in on various aspects of the idea they have already invested time and resources into…When you have lots of alternatives, quitting will be much less painful, both emotionally and practically.” (191) It seemed like some of his comments on comparative advantage were oversimplifying the concept so much it might not have been accurate any more, but overall an interesting chapter on the benefits of not ignoring opportunity cost and not wasting time on less useful pursuits.
The last chapter was on culture. Given that he had worked at Uber during their early days, he saw some interesting things. The chapter talks a bit about how people act differently when having communal goals versus individual goals. “Research suggests that deep trust is a powerful factor in enabling organizations to scale, in part because it promotes cooperation and functional teamwork is essential for growth, but for other reasons as well. The lack of trust at Uber was in large part the natural endgame of an organization that was a meritocracy in name but not in spirit: people didn’t trust that the objective value of their contributions (time, ideas, and effort) would be appreciated. In other words, employees didn’t feel respected…Sure, Uber was great at attacking complacent and lazy thinking, but here is the irony: the leaders at Uber were complacent in how they thought about scaling up the company culture. No one in a leadership position – myself included – forcefully questioned Uber’s culture to the same extent employees were pushed to question its business ideas and practices…He [Travis Kalanick] knew he had made big mistakes, and he showed remorse, not just because he lost his job but because he felt that he had let his Uber team down. I don’t believe Travis Kalanick is a bad person. He is a good person who made several bad calls…at scale.” (211)
(Continued in comments)
December 9, 2022
Love the idea and discussion about scaling. Definitely some worthy tips and discussion about scaling in many ways. I think the book was just a bit dry for me and I lost focus on the various discussions. It wasn’t super long but perhaps it needed to be even shorter.
August 2, 2023
Upscale to scale up. .
So you’re an entrepreneur with a new business idea to add to your repertoire. It's fresh and innovative, and you're certain your new product or service has the potential to make a big splash in the marketplace. You're excited about the prospect and already envisioning your idea's exponential growth.
But here’s a big, important question you must expect to encounter at some point: Can your idea truly scale and become profitable in the long term?
Scalability is the capacity for an idea or business model to expand without compromising its core integrity or functionality. Essentially, it’s about more than just getting bigger or faster. For your business to scale successfully, it should grow sustainably and in a way that adds value to your business every step of the way.
In short, scaling is equal parts art and science.
Want to know the long story of it? In this book, you’ll discover the common pitfalls that masquerade as signs of growth but can lead you astray, so you’re prepped. You'll also learn why knowing your audience is critical in assessing your idea’s scalability and why managing the costs of scaling is crucial.
But that's not all. As you go along, you'll also uncover the four best-kept secrets that can unlock the scalability of your idea. From exploring scalable incentives to scaling your organization’s culture, each secret offers unique insights and practical applications to transform your thinking about growth.
So, get ready to upscale your approach to success.
-
The three-part foundation of scalability
As you're gearing up to take your idea or business to the next level, it's crucial to know and master the three-step foundation of scalability.
First, beware of the voltage drop, which happens when an idea that works well on a small scale loses its profitability as it expands. An infamous example of this concept is Theranos, the health-tech company led by Elizabeth Holmes, which raised over $700 million from investors. Theranos was valued at $9 billion due to its supposedly revolutionary portable blood-testing technology that didn’t even exist. Holmes resorted to faking results using other companies' machines to save face. Eventually, her inability to scale this nonexistent technology was exposed and her company’s downfall became one of history's most notorious tales of business collapse.
Second, you need to know your audience well. Success in scaling depends on knowing what your target audience needs. Kmart's Blue Light Special sales gimmick exemplifies the perils of misunderstanding the audience. The initiative floundered when Kmart’s corporate team standardized discounted items without considering their regional customers’ preferences.
Third, avoid the cost trap in scaling. Ideally, you should achieve economies of scale using fixed costs better as you scale. The principle means achieving greater efficiency and cost reduction as the size of the operation increases. An idea that becomes more expensive as it grows is a major red flag. So it’s important to see if your budget constraints can hamper replicating your early successes on a larger scale. Don’t be afraid to ask: Will scaling compromise any nonnegotiables?
Here’s another cautionary tale: Arivale. Despite raising $50 million in capital funds, becoming Startup of the Year in 2016, and ticking all the boxes for successful scaling, Arivale fell into the cost trap. Its highly personalized services, backed by complex scientific processes, became more expensive as it expanded, defying the principle of economies of scale.
To recap, successful scaling is a delicate balancing act. Take time to identify the right parts of your business to amplify. By taking underperforming products out of the equation, understanding your audience better, and keeping costs under control along the way, it’s easier to play the scaling game.
With these foundations laid, you can now delve into the four secrets to unlock your idea's scalability potential.
-
Scale through well-planned incentives
Since we’ve established the groundwork for scaling, it's time to reveal the first secret weapon of scaling: incentives. They're like the little nudges that get everyone moving in the right direction – from your employees to your customers.
Remember the early days of Uber? The no-tipping policy was a big win for passengers. Then, things changed when drivers started asking for tips, likely nudged by Lyft's in-app tipping feature. So Uber jumped on the tipping bandwagon. More drivers signed up, but it didn't magically improve the service or make drivers richer. The surprising bit? Only 1 percent of folks consistently tipped. But this whole tipping saga showed how powerful it can be to tweak and adapt incentives to fit your situation.
Let's dig deeper into the concept of incentives. Most of the time, you do the work and get rewarded. What if you got the reward first and had to work to keep it? That's the clawback approach. It plays on your instinct to keep what you have.
Wanlida Group, a Chinese electronics manufacturer, used this clawback strategy brilliantly. The company gave its employees an upfront bonus that was theirs to keep as long as they met their weekly production goal. And guess what? This strategy worked like a charm, boosting productivity by over 1 percent. The great thing about the clawback approach? It works across cultures and situations and is super scalable. Even small bonuses can change behavior and productivity when seen as something to lose.
But there are some pitfalls to watch out for. First, people with a lot of risk experiences are more relaxed about potential losses. And second, ethical considerations are huge. You can't set unrealistic targets and stress your team out; you've got to pay out the bonuses when your people hit their targets.
Now that you know the perks of using incentives as a business strategy, let’s head into the next secret: marginal thinking.
-
Always think on the margins
You know those moments when you stand in the grocery store aisle, debating whether to get that delicious candy bar or go for the healthier apple instead? You're doing what economists call a cost-benefit analysis. It's the same thought process when deciding on a gym membership or a new apartment.
As it happens, this thinking is also useful when trying to scale your business.
Thanks to the marginal revolution in the late nineteenth century, we’re now more aware than ever of the value we get from things without going too deep into the weeds. It's why gold costs more than food and diamonds are more expensive than water. It's about satisfaction or what economists call “utility.” When you're in the driver's seat of a business, you need to keep practicing marginal thinking. Collecting data isn't enough – you need to dig into it, to see what works, what doesn't, and where there might be room for improvement. Just like when you decided on that apple because you knew you'd feel better for it later.
Now, here’s a learning curve to marginal thinking that no one likes: mistakes. We all make them. When you think on the margins for the sake of your business, you’ll retrace some sunk costs. These are resources you've already spent that you can't get back. It's tempting to throw in more time and money to fix them, but that's rarely a good idea. You need to be able to cut your losses.
That's only easy sometimes, of course. It can be tough to admit you've made a mistake – especially if it affects your reputation or job – but it’s always worth it. Some businesses even switch things up, letting different employees take a fresh look at things every few months. It helps them avoid getting stuck in the past.
Take Lyft’s early days, for example. During an executive meeting, CEO Logan Green and his team examined a spreadsheet of their returns on ad expenditure. A startling pattern emerged: marginal returns from Facebook ads were far lower than Google ones. In hindsight, this wasn't a data error but a failure to think on the margins. Thankfully, Lyft decided to optimize its marginal gains by reallocating the budget from Facebook to Google. This way, it could strategically save costs, especially when COVID-19 hit the ride-sharing industry.
Just like Green and his team, you need to think on the margins. More often than not, it’s the best way to shake things up and revolutionize your business.
With marginal thinking now under your belt, let's navigate to the next secret of scaling that will keep you on your toes: quitting when you have to.
-
Sometimes you need to quit to win
Knowing when to quit is just as important as scaling your best ideas. It may seem contradictory to the common advice of staying resilient. Still, it actually aligns with the principle of opportunity cost – the idea that choosing the best path for you means giving up the possibilities of another.
Successful quitting allows you to invest your time, effort, and resources wisely. In other words, it's about making a good trade-off.
Consider an alternate universe where the author and economist, John List, is a pro golfer instead. While such a life might be fulfilling on a personal level, List's skills as an economist have enabled him to make more significant contributions to his field and society. This isn’t to diminish the value of golfers but to emphasize the importance of recognizing and leveraging our unique abilities. In List's case, his impact as an economist provides the greatest return. Sometimes, it's crucial to understand when to quit a particular skill and pivot toward another that allows us to make a greater impact in life.
Here's another example from List's own experience. One day, his son faced a decision between two baseball bats. He’d saved enough money for the top-of-the-line bat, but his father introduced him to opportunity cost. By choosing the cheaper bat, he’d still have enough money left to buy a new baseball glove. This simple lesson in weighing the trade-offs between options demonstrates the broader concept of opportunity cost in decision-making.
Of course, we’re the first to admit that recognizing when to quit can be challenging. After all, quitting involves acknowledging sunk costs and facing the uncertainty of what comes next. But it's crucial to remember that these sunk costs – in terms of time, money, or both – are in the past. They shouldn’t factor into your decisions about the future. Instead of obsessing over these points for reflection, focus more on what you can do now and in the future to maximize your resources.
Not convinced about the power of quitting? Check out this experiment List once conducted with a colleague in 2013. Participants were asked to flip a virtual coin to help them make unsure decisions like quitting a job or ending a relationship. The results showed that those who embraced change and decided to quit were happier six months later than those who’d stuck to the status quo.
In short, recognize when it's time to quit, and fearlessly embrace the opportunity to pivot toward something that could spell a better, more lasting outcome for you. While letting go of an exciting yet unprofitable idea is hard, its benefits far outweigh the emotional and practical costs. So evaluate your path regularly, consider your opportunity costs, and don't hesitate to change your direction if necessary.
Now that you know what it takes to scale your business, it’s time to learn how to scale your company culture. After all, the people you invest in can make or break your business!
-
Scaling “people power” is just as important
Scaling your business operations is a surefire path to success, but don't overlook the significance of scaling your company culture alongside it. Your culture, with its values and behaviors, can make or break your journey to greatness.
So how do you build a scalable company culture? Well, there are three key steps: avoid the perils of meritocracy, prioritize trust and teamwork, and be generous with your apologies when things go awry.
Let's talk about meritocracy gone wrong. Uber serves as a glaring example of how things can all go downhill. The company boasted about rewarding merit but, in reality, privilege and politics hijacked the process. This created a toxic environment where only the loudest voices at the top were heard, leaving the rest of the employees feeling ignored and undervalued. Pair that with its hyper-aggressive culture and you've got a recipe for talent drain and a repellent effect on potential hires.
On the flip side, trust and teamwork can set the stage for a healthier and more productive workplace. Just look at Netflix for inspiration. The streaming TV service provider has cultivated a culture of freedom and responsibility where employees are trusted to do their jobs without being suffocated by micromanagement. This penchant for trust ultimately breeds better performance and creates a positive atmosphere. Additionally, Netflix understands the power of collective achievement. It ties compensation to its overall success, encouraging cooperation and healthy competition. That's how it could scale without compromising its overall business performance.
But even with the best intentions, there’ll be moments when you stumble. Mistakes happen, but it's how you handle them that matters – and it all begins with embedding a culture of apologizing from the get-go. When trust is shaken, well-crafted apologies from your company can work wonders. Take Uber’s case, for instance. All it took for the company to handle reports of serious driver mishaps over the years was a genuine apology to its customers. Unfortunately, that didn't happen, and the company's reputation eventually took a hit.
So in conclusion, scaling your company culture is paramount. It's not just about the numbers; it's about the people working in the environment you create. A healthy and inclusive culture doesn't happen by accident; it requires intentional and consistent efforts on your part.
-
You've discovered the secret ingredients to scaling successfully: trusting your instincts, planning and delivering valuable incentives, thinking marginally, letting go of bad ideas and practices, and building a scalable company culture.
So go ahead – take these steps to foolproof your scaling strategy and unleash the full potential of your business.
So you’re an entrepreneur with a new business idea to add to your repertoire. It's fresh and innovative, and you're certain your new product or service has the potential to make a big splash in the marketplace. You're excited about the prospect and already envisioning your idea's exponential growth.
But here’s a big, important question you must expect to encounter at some point: Can your idea truly scale and become profitable in the long term?
Scalability is the capacity for an idea or business model to expand without compromising its core integrity or functionality. Essentially, it’s about more than just getting bigger or faster. For your business to scale successfully, it should grow sustainably and in a way that adds value to your business every step of the way.
In short, scaling is equal parts art and science.
Want to know the long story of it? In this book, you’ll discover the common pitfalls that masquerade as signs of growth but can lead you astray, so you’re prepped. You'll also learn why knowing your audience is critical in assessing your idea’s scalability and why managing the costs of scaling is crucial.
But that's not all. As you go along, you'll also uncover the four best-kept secrets that can unlock the scalability of your idea. From exploring scalable incentives to scaling your organization’s culture, each secret offers unique insights and practical applications to transform your thinking about growth.
So, get ready to upscale your approach to success.
-
The three-part foundation of scalability
As you're gearing up to take your idea or business to the next level, it's crucial to know and master the three-step foundation of scalability.
First, beware of the voltage drop, which happens when an idea that works well on a small scale loses its profitability as it expands. An infamous example of this concept is Theranos, the health-tech company led by Elizabeth Holmes, which raised over $700 million from investors. Theranos was valued at $9 billion due to its supposedly revolutionary portable blood-testing technology that didn’t even exist. Holmes resorted to faking results using other companies' machines to save face. Eventually, her inability to scale this nonexistent technology was exposed and her company’s downfall became one of history's most notorious tales of business collapse.
Second, you need to know your audience well. Success in scaling depends on knowing what your target audience needs. Kmart's Blue Light Special sales gimmick exemplifies the perils of misunderstanding the audience. The initiative floundered when Kmart’s corporate team standardized discounted items without considering their regional customers’ preferences.
Third, avoid the cost trap in scaling. Ideally, you should achieve economies of scale using fixed costs better as you scale. The principle means achieving greater efficiency and cost reduction as the size of the operation increases. An idea that becomes more expensive as it grows is a major red flag. So it’s important to see if your budget constraints can hamper replicating your early successes on a larger scale. Don’t be afraid to ask: Will scaling compromise any nonnegotiables?
Here’s another cautionary tale: Arivale. Despite raising $50 million in capital funds, becoming Startup of the Year in 2016, and ticking all the boxes for successful scaling, Arivale fell into the cost trap. Its highly personalized services, backed by complex scientific processes, became more expensive as it expanded, defying the principle of economies of scale.
To recap, successful scaling is a delicate balancing act. Take time to identify the right parts of your business to amplify. By taking underperforming products out of the equation, understanding your audience better, and keeping costs under control along the way, it’s easier to play the scaling game.
With these foundations laid, you can now delve into the four secrets to unlock your idea's scalability potential.
-
Scale through well-planned incentives
Since we’ve established the groundwork for scaling, it's time to reveal the first secret weapon of scaling: incentives. They're like the little nudges that get everyone moving in the right direction – from your employees to your customers.
Remember the early days of Uber? The no-tipping policy was a big win for passengers. Then, things changed when drivers started asking for tips, likely nudged by Lyft's in-app tipping feature. So Uber jumped on the tipping bandwagon. More drivers signed up, but it didn't magically improve the service or make drivers richer. The surprising bit? Only 1 percent of folks consistently tipped. But this whole tipping saga showed how powerful it can be to tweak and adapt incentives to fit your situation.
Let's dig deeper into the concept of incentives. Most of the time, you do the work and get rewarded. What if you got the reward first and had to work to keep it? That's the clawback approach. It plays on your instinct to keep what you have.
Wanlida Group, a Chinese electronics manufacturer, used this clawback strategy brilliantly. The company gave its employees an upfront bonus that was theirs to keep as long as they met their weekly production goal. And guess what? This strategy worked like a charm, boosting productivity by over 1 percent. The great thing about the clawback approach? It works across cultures and situations and is super scalable. Even small bonuses can change behavior and productivity when seen as something to lose.
But there are some pitfalls to watch out for. First, people with a lot of risk experiences are more relaxed about potential losses. And second, ethical considerations are huge. You can't set unrealistic targets and stress your team out; you've got to pay out the bonuses when your people hit their targets.
Now that you know the perks of using incentives as a business strategy, let’s head into the next secret: marginal thinking.
-
Always think on the margins
You know those moments when you stand in the grocery store aisle, debating whether to get that delicious candy bar or go for the healthier apple instead? You're doing what economists call a cost-benefit analysis. It's the same thought process when deciding on a gym membership or a new apartment.
As it happens, this thinking is also useful when trying to scale your business.
Thanks to the marginal revolution in the late nineteenth century, we’re now more aware than ever of the value we get from things without going too deep into the weeds. It's why gold costs more than food and diamonds are more expensive than water. It's about satisfaction or what economists call “utility.” When you're in the driver's seat of a business, you need to keep practicing marginal thinking. Collecting data isn't enough – you need to dig into it, to see what works, what doesn't, and where there might be room for improvement. Just like when you decided on that apple because you knew you'd feel better for it later.
Now, here’s a learning curve to marginal thinking that no one likes: mistakes. We all make them. When you think on the margins for the sake of your business, you’ll retrace some sunk costs. These are resources you've already spent that you can't get back. It's tempting to throw in more time and money to fix them, but that's rarely a good idea. You need to be able to cut your losses.
That's only easy sometimes, of course. It can be tough to admit you've made a mistake – especially if it affects your reputation or job – but it’s always worth it. Some businesses even switch things up, letting different employees take a fresh look at things every few months. It helps them avoid getting stuck in the past.
Take Lyft’s early days, for example. During an executive meeting, CEO Logan Green and his team examined a spreadsheet of their returns on ad expenditure. A startling pattern emerged: marginal returns from Facebook ads were far lower than Google ones. In hindsight, this wasn't a data error but a failure to think on the margins. Thankfully, Lyft decided to optimize its marginal gains by reallocating the budget from Facebook to Google. This way, it could strategically save costs, especially when COVID-19 hit the ride-sharing industry.
Just like Green and his team, you need to think on the margins. More often than not, it’s the best way to shake things up and revolutionize your business.
With marginal thinking now under your belt, let's navigate to the next secret of scaling that will keep you on your toes: quitting when you have to.
-
Sometimes you need to quit to win
Knowing when to quit is just as important as scaling your best ideas. It may seem contradictory to the common advice of staying resilient. Still, it actually aligns with the principle of opportunity cost – the idea that choosing the best path for you means giving up the possibilities of another.
Successful quitting allows you to invest your time, effort, and resources wisely. In other words, it's about making a good trade-off.
Consider an alternate universe where the author and economist, John List, is a pro golfer instead. While such a life might be fulfilling on a personal level, List's skills as an economist have enabled him to make more significant contributions to his field and society. This isn’t to diminish the value of golfers but to emphasize the importance of recognizing and leveraging our unique abilities. In List's case, his impact as an economist provides the greatest return. Sometimes, it's crucial to understand when to quit a particular skill and pivot toward another that allows us to make a greater impact in life.
Here's another example from List's own experience. One day, his son faced a decision between two baseball bats. He’d saved enough money for the top-of-the-line bat, but his father introduced him to opportunity cost. By choosing the cheaper bat, he’d still have enough money left to buy a new baseball glove. This simple lesson in weighing the trade-offs between options demonstrates the broader concept of opportunity cost in decision-making.
Of course, we’re the first to admit that recognizing when to quit can be challenging. After all, quitting involves acknowledging sunk costs and facing the uncertainty of what comes next. But it's crucial to remember that these sunk costs – in terms of time, money, or both – are in the past. They shouldn’t factor into your decisions about the future. Instead of obsessing over these points for reflection, focus more on what you can do now and in the future to maximize your resources.
Not convinced about the power of quitting? Check out this experiment List once conducted with a colleague in 2013. Participants were asked to flip a virtual coin to help them make unsure decisions like quitting a job or ending a relationship. The results showed that those who embraced change and decided to quit were happier six months later than those who’d stuck to the status quo.
In short, recognize when it's time to quit, and fearlessly embrace the opportunity to pivot toward something that could spell a better, more lasting outcome for you. While letting go of an exciting yet unprofitable idea is hard, its benefits far outweigh the emotional and practical costs. So evaluate your path regularly, consider your opportunity costs, and don't hesitate to change your direction if necessary.
Now that you know what it takes to scale your business, it’s time to learn how to scale your company culture. After all, the people you invest in can make or break your business!
-
Scaling “people power” is just as important
Scaling your business operations is a surefire path to success, but don't overlook the significance of scaling your company culture alongside it. Your culture, with its values and behaviors, can make or break your journey to greatness.
So how do you build a scalable company culture? Well, there are three key steps: avoid the perils of meritocracy, prioritize trust and teamwork, and be generous with your apologies when things go awry.
Let's talk about meritocracy gone wrong. Uber serves as a glaring example of how things can all go downhill. The company boasted about rewarding merit but, in reality, privilege and politics hijacked the process. This created a toxic environment where only the loudest voices at the top were heard, leaving the rest of the employees feeling ignored and undervalued. Pair that with its hyper-aggressive culture and you've got a recipe for talent drain and a repellent effect on potential hires.
On the flip side, trust and teamwork can set the stage for a healthier and more productive workplace. Just look at Netflix for inspiration. The streaming TV service provider has cultivated a culture of freedom and responsibility where employees are trusted to do their jobs without being suffocated by micromanagement. This penchant for trust ultimately breeds better performance and creates a positive atmosphere. Additionally, Netflix understands the power of collective achievement. It ties compensation to its overall success, encouraging cooperation and healthy competition. That's how it could scale without compromising its overall business performance.
But even with the best intentions, there’ll be moments when you stumble. Mistakes happen, but it's how you handle them that matters – and it all begins with embedding a culture of apologizing from the get-go. When trust is shaken, well-crafted apologies from your company can work wonders. Take Uber’s case, for instance. All it took for the company to handle reports of serious driver mishaps over the years was a genuine apology to its customers. Unfortunately, that didn't happen, and the company's reputation eventually took a hit.
So in conclusion, scaling your company culture is paramount. It's not just about the numbers; it's about the people working in the environment you create. A healthy and inclusive culture doesn't happen by accident; it requires intentional and consistent efforts on your part.
-
You've discovered the secret ingredients to scaling successfully: trusting your instincts, planning and delivering valuable incentives, thinking marginally, letting go of bad ideas and practices, and building a scalable company culture.
So go ahead – take these steps to foolproof your scaling strategy and unleash the full potential of your business.
February 14, 2022
fascinating. quite informative.
February 21, 2022
Interesting books with aspects of business, economics and psychology.
Nominally, it's about common errors when "scaling up". Taking a restaurant to a business with over a dozen locations; rolling out an experimental program from one school to hundreds of schools; that kind of thing. Not all of the studies and advice references seem to be directly related to that, but it is all interesting.
I feel that some of the main lessons of the book are:
1) make sure your test group is fully representative or random, and preferably, do some intermediate tests with more diverse groups before rolling out en masse. It may have only worked as expected for the small population in your initial test.
2) identify what "special sauce" your business or program depends on, such as staff, ingredients, or other resources you use. Could this work anywhere, with people you can hire in any city (or ingredients you can get from multiple sources?) or does it depend on a finite amount of resources that can't accommodate a larger effort? Maybe you have an amazingly talented employee that holds it all together - unti they leave the company. Maybe it works because you hire truly elite employees for your first location... but then as you expand, you run out of the cream of the crop and have to hire more average people.
Nominally, it's about common errors when "scaling up". Taking a restaurant to a business with over a dozen locations; rolling out an experimental program from one school to hundreds of schools; that kind of thing. Not all of the studies and advice references seem to be directly related to that, but it is all interesting.
I feel that some of the main lessons of the book are:
1) make sure your test group is fully representative or random, and preferably, do some intermediate tests with more diverse groups before rolling out en masse. It may have only worked as expected for the small population in your initial test.
2) identify what "special sauce" your business or program depends on, such as staff, ingredients, or other resources you use. Could this work anywhere, with people you can hire in any city (or ingredients you can get from multiple sources?) or does it depend on a finite amount of resources that can't accommodate a larger effort? Maybe you have an amazingly talented employee that holds it all together - unti they leave the company. Maybe it works because you hire truly elite employees for your first location... but then as you expand, you run out of the cream of the crop and have to hire more average people.
November 14, 2023
This book was really good and full of interesting pieces of evidence backed by intriguing context and real stories ranging from US government to Uber and Lyft. I was intrigued the entire time and took away tangible learnings.
As always, shout out to my fans ~ my wife, my mom, and my sister … in no particular order as I don’t dare rank you three.
As always, shout out to my fans ~ my wife, my mom, and my sister … in no particular order as I don’t dare rank you three.
July 30, 2023
This is a remarkable book by an experienced academic consultant who has worked intensively with the management at multiple tech companies. His topic seems to be fairly straightforward, and almost quaint: there is much wisdom in attempting big changes via small, initial steps, assessing success, and then placing significant resources behind an idea that works and is manageable.
Professor John A. List is the Kenneth C. Griffin Distinguished Service Professor in Economics at the University of Chicago. His research focuses on combining field experiments with economic theory to deepen our understanding of the economic science The author's real expertise comes through when he recounts some of his consulting work with large companies such as Uber and Lyft. He was trained as a behavioral economist and has applied that largely to the business landscape. However there’s some data in recent news that casts some doubt on the overall long-term benefits of making business decisions purely on behavioral economics. Of note, the idea of “ nudging“ important decisions in increments seems to be under attack. As an interested observer, who tends to take a wider view of decision-making algorithms as they can be used by a human mind and not a computer, this book is a welcome foray into specific market tested ideas.
The chapters on company culture are insightful and sharp. I think that there’s some great work being done on modern business culture, and how we can improve it from a start-up level. However, other misfires such as DEI, and significant drop offs and quality of hiring when large companies disregard quality and experience in favor of box checking, has also been addressed late in this book. Professor List looks at it from a data, driven economic standpoint, which I think is crucial to making sure we are making legitimate scientific decisions on management. Hiring quality people (not "brilliant jerks" or "tokens" as a CEO mentions here in a late chapter) is an extremely challenging endeavor for any company. Even more reason to spend more time on why we are hiring and for what reason. Each individual employee is an important cog in that. This includes all the leaders as well. Understanding this and comprehending it is crucial to success.
Overall, I think book may have barely missed the mark when it comes to delegating or discussing specific details of behavioral modifications, who makes those modificaitons and how it can help with larger companies. Also, as a start-up with limited resources, how can a company achieve some of the data driven benefits of economic thinking without excessive resource burn? That I am not sure.
One humorous observation from this book is how consulting economists in general tend to favor incentives as the only way to drive performance. I do feel that simply paying people more for doing their job may not always result in higher performance, only that the employee now feels that they are more valued. However, there are some great observations from data here in this book that indicates that giving someone a bonus ahead of time with strings attached, i.e. they will lose their bonus, if performance is not achieved, tends to result in higher enterprise performance, than simply promising someone a bonus if they succeed in hitting benchmarks. Essentially, people are loath to lose what they already have in their pocket, rather than a promise of future gain. How would you structure this as a company? Likely send out bonuses ahead of time, even half a bonus deposited at the start of the measurement, with the other half promised when benchmarks are hit.
However, all would be in agreement that such a bonus will be clawed back if performance is not achieved. This has worked in several public school teacher studies that has resulted in significantly higher performance by their students. This is a unique insight that should all help us understand how performance should work with front loader bonuses that can be clawed back, which certainly improves workers performance from the get-go, since they don’t want to lose something they already have. The clawback provision is effective in many executive ranks, and primarily in the insurance industry. I do think that more work should be done in the real world on the front lines with start ups, and even large companies with clawback provisions as a mechanism of performance.
At this time I think three stars is appropriate, with a caveat surrounding the real conclusions we can achieve from such an white-tower oriented approach. I think much of this remains academic, but with long-term applications to companies that have the resources to implement them. Also, let’s not forget that we are in the middle of a historic boom cycle in equity, stock, market, and financing. Companies simply just have more money to invest in non-revenue generating activities, such as studies and culture, and improving workers performance. And now we face a tightening economic future that is quickly becoming apparent. Currently, I feel that a lot of these academic visions and discussions may be at risk, since they are not direct revenue-generating activities that a traditional executive would dedicate resources to develop. This will still remain one of the top 10 books I’ll be here economics, up there with Thaler and Sunstein's "Nudge".
Professor John A. List is the Kenneth C. Griffin Distinguished Service Professor in Economics at the University of Chicago. His research focuses on combining field experiments with economic theory to deepen our understanding of the economic science The author's real expertise comes through when he recounts some of his consulting work with large companies such as Uber and Lyft. He was trained as a behavioral economist and has applied that largely to the business landscape. However there’s some data in recent news that casts some doubt on the overall long-term benefits of making business decisions purely on behavioral economics. Of note, the idea of “ nudging“ important decisions in increments seems to be under attack. As an interested observer, who tends to take a wider view of decision-making algorithms as they can be used by a human mind and not a computer, this book is a welcome foray into specific market tested ideas.
The chapters on company culture are insightful and sharp. I think that there’s some great work being done on modern business culture, and how we can improve it from a start-up level. However, other misfires such as DEI, and significant drop offs and quality of hiring when large companies disregard quality and experience in favor of box checking, has also been addressed late in this book. Professor List looks at it from a data, driven economic standpoint, which I think is crucial to making sure we are making legitimate scientific decisions on management. Hiring quality people (not "brilliant jerks" or "tokens" as a CEO mentions here in a late chapter) is an extremely challenging endeavor for any company. Even more reason to spend more time on why we are hiring and for what reason. Each individual employee is an important cog in that. This includes all the leaders as well. Understanding this and comprehending it is crucial to success.
Overall, I think book may have barely missed the mark when it comes to delegating or discussing specific details of behavioral modifications, who makes those modificaitons and how it can help with larger companies. Also, as a start-up with limited resources, how can a company achieve some of the data driven benefits of economic thinking without excessive resource burn? That I am not sure.
One humorous observation from this book is how consulting economists in general tend to favor incentives as the only way to drive performance. I do feel that simply paying people more for doing their job may not always result in higher performance, only that the employee now feels that they are more valued. However, there are some great observations from data here in this book that indicates that giving someone a bonus ahead of time with strings attached, i.e. they will lose their bonus, if performance is not achieved, tends to result in higher enterprise performance, than simply promising someone a bonus if they succeed in hitting benchmarks. Essentially, people are loath to lose what they already have in their pocket, rather than a promise of future gain. How would you structure this as a company? Likely send out bonuses ahead of time, even half a bonus deposited at the start of the measurement, with the other half promised when benchmarks are hit.
However, all would be in agreement that such a bonus will be clawed back if performance is not achieved. This has worked in several public school teacher studies that has resulted in significantly higher performance by their students. This is a unique insight that should all help us understand how performance should work with front loader bonuses that can be clawed back, which certainly improves workers performance from the get-go, since they don’t want to lose something they already have. The clawback provision is effective in many executive ranks, and primarily in the insurance industry. I do think that more work should be done in the real world on the front lines with start ups, and even large companies with clawback provisions as a mechanism of performance.
At this time I think three stars is appropriate, with a caveat surrounding the real conclusions we can achieve from such an white-tower oriented approach. I think much of this remains academic, but with long-term applications to companies that have the resources to implement them. Also, let’s not forget that we are in the middle of a historic boom cycle in equity, stock, market, and financing. Companies simply just have more money to invest in non-revenue generating activities, such as studies and culture, and improving workers performance. And now we face a tightening economic future that is quickly becoming apparent. Currently, I feel that a lot of these academic visions and discussions may be at risk, since they are not direct revenue-generating activities that a traditional executive would dedicate resources to develop. This will still remain one of the top 10 books I’ll be here economics, up there with Thaler and Sunstein's "Nudge".
February 18, 2022
Interesting insights from the research done at private sector, but the constant hubris of taking credit for well stablished ideas (eg claiming that he came up with the idea of evaluating the cost-benefit of public policies in the margin) made it hard to enjoy
March 16, 2022
The Voltage Effect is a much-needed addition to the econ/business/management literature: finally, a book that not only shares *ideas* but actually focuses on their implementation – in particular, how to make them work at scale.
The book offers a framework and practical toolkit for scaling that can directly be applied to real-world business ideas or policies. It is basically structured as a checklist, reviewing chapter by chapter various issues to be aware of (hello, false positives and spillovers!) and techniques to deploy (think smart incentive design) to ensure the scalability of an idea. And if that sounds dry and academic to you, you'll be pleasantly surprised: I don't think there is a single concept in the book that is not illustrated through examples (from business, policy and the non-profit sphere), and the author dutifully translates his "economese" to practical terms throughout.
Many aspects elevate this book above the "business-self-help" genre, most importantly its rigorous research basis (as you would expect from a professor whose academic work has attracted over 65k citations!) and the relatability of its author (if you get annoyed with people who need to mention their Harvard MBA every other paragraph, then you'll probably find it refreshing to learn from a down-to-earth Midwesterner who worked his way to the very top of the economics profession after going to a less-know college on a golf scholarship, and still finds time to coach his son's baseball team).
And even if you don't think of yourself as someone interested in econ/business, you'll probably find this an eye-opening read with many applications to everyday life. If this is the case, I'd suggest you start straight at Chapter 8 (Quitting is for Winners) – my guess is that you'll be tempted to read through the whole rest of the book afterwards :)
The book offers a framework and practical toolkit for scaling that can directly be applied to real-world business ideas or policies. It is basically structured as a checklist, reviewing chapter by chapter various issues to be aware of (hello, false positives and spillovers!) and techniques to deploy (think smart incentive design) to ensure the scalability of an idea. And if that sounds dry and academic to you, you'll be pleasantly surprised: I don't think there is a single concept in the book that is not illustrated through examples (from business, policy and the non-profit sphere), and the author dutifully translates his "economese" to practical terms throughout.
Many aspects elevate this book above the "business-self-help" genre, most importantly its rigorous research basis (as you would expect from a professor whose academic work has attracted over 65k citations!) and the relatability of its author (if you get annoyed with people who need to mention their Harvard MBA every other paragraph, then you'll probably find it refreshing to learn from a down-to-earth Midwesterner who worked his way to the very top of the economics profession after going to a less-know college on a golf scholarship, and still finds time to coach his son's baseball team).
And even if you don't think of yourself as someone interested in econ/business, you'll probably find this an eye-opening read with many applications to everyday life. If this is the case, I'd suggest you start straight at Chapter 8 (Quitting is for Winners) – my guess is that you'll be tempted to read through the whole rest of the book afterwards :)
Displaying 1 - 30 of 135 reviews