What do you think?
Rate this book
The Primacy of Doubt: From Quantum Physics to Climate Change, How the Science of Uncertainty Can Help Us Understand Our Chaotic World
“Quite possibly the best popular science book I’ve ever read” ( Popular Science ) shows how the tools that enabled us to overcome the uncertainty of the weather will enable us to find new answers to modern science's most pressing questions Why does your weather app say “There’s a 10% chance of rain” instead of “It will be sunny tomorrow”? In large part this is due to the insight of Tim Palmer, who made uncertainty essential to the study of weather and climate. Now he wants to apply it to how we study everything else. In The Primacy of Doubt , Palmer argues that embracing the mathematics of uncertainty is vital to understanding ourselves and the universe around us. Whether we want to predict climate change or market crashes, understand how the brain is able to outpace supercomputers, or find a theory that links quantum and cosmological physics, Palmer shows how his vision of mathematical uncertainty provides new insights into some of the deepest problems in science. The result is a revolution—one that shows that power begins by embracing what we don’t know.
320 pages, Hardcover
Published October 18, 2022
110 people are currently reading
930 people want to read
About the author
Tim Palmer
4 books1 followerTimothy Noel Palmer, FRS, CBE, is a Royal Society Research Professor in the Department of Physics at the University of Oxford who pioneered the development of operational ensemble weather and climate forecasting. He is a Fellow of the Royal Society, an International Member of the US National Academy of Sciences, a recipient of the Institute of Physics Dirac Gold Medal and a lead author of the Intergovernmental Panel on Climate Change. His PhD was in general relativity theory.
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 30 of 44 reviews
October 18, 2022
This is quite possibly the best popular science book I've ever read (and I've read many hundreds). To describe what Tim Palmer, a physicist turned meteorologist, does in simple terms does not do it justice. But essentially he explores the nature of (mathematically) chaotic systems and shows how we can deal better with uncertainty, even using his expertise to propose a different way to look at the lack of local reality in quantum physics.
This is interesting stuff anyway, but what is astounding is the way that Palmer rattles through a series of topics that are quite difficult to get your head around and, in several diverse cases, gives the most approachable explanation of the topic I've ever seen.
I'm not saying this book is an easy read, by the way. You do have to think about what you are reading, and I had to go back over a couple of sections to make sure it sunk in. But it is so rewarding of the effort.
In terms of this broad enlightening nature, the first of the three sections in the book stands out head and shoulders above the rest. Palmer starts by exploring chaos and gives the best explanation of the behaviour of chaotic systems, state space and attractors I've come across. Then he throws in Cantor sets, then shows the relationship of weather forecasts to all this, and introduces p-adic numbers (arguably the only bit that could have been better explained). He then shows graphically (literally, not metaphorically) how the introduction of noise can make models of chaotic systems work better. Finally in this section, he takes on quantum uncertainty, with one of the only explanations of the use of Bell's inequality I've ever seen that is at least vaguely comprehensible.
I don't usually go into that much detail in a review, but just wanted to show how much is crammed into the first 80 or so pages.
In the second section, Palmer addresses the use of Monte Carlo methods and ensembles in making at least partly successful predictions of chaotic systems, such as the weather, the climate and pandemics. Usually, the applications of the theory are the most interesting bits of a book, but somehow this isn't quite as engaging as the theory in the first section, though things really liven up when we get onto economics, and how economists are stuck in the fairly useless state meteorologists were before the great storm of 1987, when they used single-run forecasts, rather than ensembles. He also shows fairly bluntly that economists have failed in the development of the kind of models that can handle a chaotic system like the economy.
Finally, in the third section, Palmer addresses the big picture. He starts with an alternative interpretation of quantum theory that effectively enables hidden variables, using an approach that he describes as involving 'counterfactual indefiniteness', a concept he calls the 'cosmological invariant set' and invariant set theory. How much this will appeal probably depends how you feel about quantum interpretations, or get worried about the idea that until a quantum system interacts with the outside world it doesn't have real values for things like the location of particles. This part felt a bit hand-wavy, partly, I think because it needed too much of the mathematics behind it (which we sensibly don't see) to get a handle on it.
To end this more speculative section, Palmer takes on things like consciousness, free will and God - not bad going for a relatively short book. Finishing The Primacy of Doubt is like getting off one of those exciting roller coaster rides, when your immediate inclination is to think 'I want to do that again, but I'll have a bit of a break first.' I will be reading this book again, without doubt. Remarkable.
This is interesting stuff anyway, but what is astounding is the way that Palmer rattles through a series of topics that are quite difficult to get your head around and, in several diverse cases, gives the most approachable explanation of the topic I've ever seen.
I'm not saying this book is an easy read, by the way. You do have to think about what you are reading, and I had to go back over a couple of sections to make sure it sunk in. But it is so rewarding of the effort.
In terms of this broad enlightening nature, the first of the three sections in the book stands out head and shoulders above the rest. Palmer starts by exploring chaos and gives the best explanation of the behaviour of chaotic systems, state space and attractors I've come across. Then he throws in Cantor sets, then shows the relationship of weather forecasts to all this, and introduces p-adic numbers (arguably the only bit that could have been better explained). He then shows graphically (literally, not metaphorically) how the introduction of noise can make models of chaotic systems work better. Finally in this section, he takes on quantum uncertainty, with one of the only explanations of the use of Bell's inequality I've ever seen that is at least vaguely comprehensible.
I don't usually go into that much detail in a review, but just wanted to show how much is crammed into the first 80 or so pages.
In the second section, Palmer addresses the use of Monte Carlo methods and ensembles in making at least partly successful predictions of chaotic systems, such as the weather, the climate and pandemics. Usually, the applications of the theory are the most interesting bits of a book, but somehow this isn't quite as engaging as the theory in the first section, though things really liven up when we get onto economics, and how economists are stuck in the fairly useless state meteorologists were before the great storm of 1987, when they used single-run forecasts, rather than ensembles. He also shows fairly bluntly that economists have failed in the development of the kind of models that can handle a chaotic system like the economy.
Finally, in the third section, Palmer addresses the big picture. He starts with an alternative interpretation of quantum theory that effectively enables hidden variables, using an approach that he describes as involving 'counterfactual indefiniteness', a concept he calls the 'cosmological invariant set' and invariant set theory. How much this will appeal probably depends how you feel about quantum interpretations, or get worried about the idea that until a quantum system interacts with the outside world it doesn't have real values for things like the location of particles. This part felt a bit hand-wavy, partly, I think because it needed too much of the mathematics behind it (which we sensibly don't see) to get a handle on it.
To end this more speculative section, Palmer takes on things like consciousness, free will and God - not bad going for a relatively short book. Finishing The Primacy of Doubt is like getting off one of those exciting roller coaster rides, when your immediate inclination is to think 'I want to do that again, but I'll have a bit of a break first.' I will be reading this book again, without doubt. Remarkable.
November 2, 2023
‘What’s the point of this story? Well, like Apollo 13, we can think of planet Earth as a damaged spacecraft hurtling through the emptiness of space. Unlike Apollo 13—planet Earth holds many billions of human astronauts—Planet Earth has not been damaged because of some design flaw, as was the case for Apollo 13; it has been damaged by its astronauts.’
Not full of fluff and waffles like some general/pop science books can sometimes be. It offers enough to keep readers more ‘informed’ about the current ‘climate’ status/issues. Palmer’s writing is engaging enough without ‘trying to be’ entertaining. Except for the ‘golf’ analogies? Jokes aside, an easy 5* shelving.
‘This strikingly paralleled strategies on tackling climate change: try to cut emissions and get to net-zero as quickly as possible, or do little that might threaten economic growth. As far as I could tell, those who advocated minimal restriction to the economy during the pandemic tended to also advocate minimal restrictions on the use of fossil fuels.’
‘As with global warming, this means that the expected number of deaths cannot be reliably estimated using a model simulation run with the most likely values of the uncertain parameters—The expected number of deaths is higher than the number predicted using the most likely values of the parameters. In the case of the second, more laissez-faire policy, the estimated cumulative number of deaths at day three hundred using the most likely value of the parameters is around 25,000. However, the expected number of deaths based on the full ensemble with perturbed parameters is significantly larger than this, around 40,000.’
Palmer’s delivery of his ideas and arguments (or rather its efficacy) is impressive—especially in terms of universal relatability/understanding. Like Beckman’s book, Math Without Numbers on pure maths, Palmer seems to have spent great effort in creating the most accessible and simplest form of ‘writing’ to ensure the highest chances of ‘understanding’ as possible (for any readers, without the need of any extra intellectual ‘primer’ to his text); yet without discounting/shaving off any of its textual significance/details.
‘—exascale computing would be needed to even contemplate building a digital ensemble of planet Earth, and perhaps it won’t really be a serious option until we have zettascale computers, capable of performing one billion trillion computations a second—quantum computers, or photonic processors as well as silicon processors, and high-performance low-power imprecise chips that produce noise in hardware—And since handling uncertainty underpins all sciences, the project to build a digital ensemble of global society would bring together the various approaches to representing uncertainty that have developed in the different subdisciplines. This project really would recognise the primacy of doubt.
A first step in this direction is the European Union’s Destination Earth (DestinE) project4. Evolving from an earlier EU flagship project proposal, ExtremeEarth, that some colleagues and I developed a few years ago—DestinE is the first step towards building a digital ensemble of global society. Sadly, the UK having left the EU5, UK scientists will not be contributing to DestinE.’
Certainly not the kind of book that will make you feel like you’re essentially taking in information you already know but from a different source and/but in a slightly altered form of in-take. A refreshing read; not ‘dense’ at all. I like my ‘general’ science books to be a bit more ‘challenging’, but because Palmer covers quite a decent amount of ‘topics’, I didn’t care if it was not (even) more ‘in-depth’. To be fair, if he had done otherwise, it might result in an overly bulky book that would ultimately ‘look’ too intimidating for general readers.
‘The mathematician and physicist John von Neumann, based at Princeton University, put together a team of meteorologists led by Jule Charney. Since these models are based on laws of physics such as the Navier-Stokes equation, I will henceforth call them “physics based”, to distinguish them from statistical-empirical models (sometimes called data-driven models) of the type that Fitzroy, Blanford and Walker developed. To run their physics-based models, Charney’s team used the first fully programmable electronic digital computer, known as ENIAC, which stood for “electronic numerical integrator and computer”.’
“Ulam and von Neumann decided—“Monte Carlo”—a simple example of the Monte Carlo method at work. Suppose you want to estimate the value of p / 4 without using a calculator. The area of a circle is (π / 4) × D2 where D is its diameter. The Monte Carlo way to estimate π would be to take a sheet of paper and draw on it a square (of area D2) which just contains the circle. You then randomly scatter a number of small seeds on the sheet of paper. It is the random scattering of small seeds that is the repeated operation mentioned earlier. Now count the total number, N1, of seeds that lie inside the square. Of these count the number, N2, that also lie within the circle. If indeed the seeds are randomly distributed inside the square, then the ratio N2 / N1 will get closer and closer to p / 4 the greater the number of seeds that are scattered on the sheet of paper. The circumstances of Ulam’s eureka moment are similar to those of many other eminent scientists. The moments occurred when they were relaxing—Why do eureka moments seem to occur during moments of relaxation?’
I like reading ‘von Neumann’ mentioned/referenced in a book (especially one that is not a ‘textbook’), and written well/with strong relevance to the text. Palmer didn’t disappoint. While some readers might not enjoy this part, I like and appreciate it very much that Palmer includes some minor mathematical bits plus relevant graphs/diagrams to support/explain his arguments in a more elaborate/detailed way. If you don't 'get' the maths/etc., it wouldn't really make any significant dent to your understanding of the text; but if you do, it's a tasty cherry on top.
‘There’s an old North English saying: “There’s nowt so queer as folk!” For sure, there’s nowt as uncertain as folk—I discussed the representation of unresolved scales in models of fluid turbulence using random noise. Such noise can in principle also provide a representation of some of the uncertainty associated with structural model error—even with the uncertainty of folk. Now whilst models like COVIDSim do have some inherent stochasticity in their equations, the associated noise processes do not contribute much to the ensemble spread.’
‘We concluded in the previous chapter that noise in the brain is a necessary part of the answer. But is it sufficient? Is consciousness a necessary part of understanding, as Penrose argues it must be? I don’t know for sure, but if it is, then developing creative AI will require a close synergy between quantum and classical computing, in addition to the synergy discussed in the previous chapter between power-intensive and low-power classical computing.
If this is so, then we have a long way to go before we create truly intelligent machines. “The singularity”, the day when machines completely take over, is likely not a few years away, as some commentators believe; it is many decades, potentially centuries away. We will need our creative brains for quite some time yet.’
The idea of ‘noise’, or rather its relevance and importance to future 'research' (one of them being human consciousness x machines) is something I find particularly interesting (albeit rather hypothetical/speculative? But not far-fetched? Especially with relation to the recent research/studies of inter-brain neural synchronisation (the 'extension' of the human consciousness, etc.; which seems all very 'Matrix-like' to my silly brain)? But it is unclear (to me at least) which particular area of studies/research he is thinking of?); also reminding me of The Silentiary by Antonio di Benedetto (and with all that in mind/primed with all that 'knowledge', re-reading Antonio di Benedetto's will feel slightly like reading 'sci-fi'). Palmer’s 'optimistic' views on the scientific/mathematical relevance (and research) to ‘noise’, I found truly fascinating, albeit seeming to be a little looser than ideal.
‘For many people it seems implausible that the works of Mozart, Shakespeare and Einstein could emerge from such a collection of particles. Following ideas formulated by Plato and Descartes, maybe there really are two different worlds: the physical world and the spiritual world. As a physicist, I would find it hard to accept such a notion since it would imply there is a core aspect of the world that cannot be studied by the scientific method. Of course, this might be the case, but I am not currently inclined to give up the idea that science can explain such matters. Nevertheless, I completely understand why the idea of emergence of conscious creative creatures from elementary particles seems completely implausible. As one scientific colleague put it, do the electrons and protons in the brain somehow know that Mozart was a superior composer to Mendelssohn?’
If you have only time and attention for one, I wouldn't skip Chapter 10 (excerpts below). My least favourite is definitely the final chapter because it is in that chapter that Palmer indulged in waxing bit too 'romantically' for my taste; and I thought the writing/arguments weren't very strong (for instance, I'd prefer that he also go into the issues of reproduction, bioethics, etc. if he wanted to make some points about 'God'/spirituality in a purely conventional sense (which he then argues is necessary in order to sustain and amplify the 'meaningfulness' of common human activities like playing golf and popping babies, etc.); but that doesn’t stray from the fact that the entire book is well worth a proper read.
‘We are edging towards a topic which some people might feel is taboo. How much is a human life worth?—there were just short of 25,000 people killed on the UK roads in 2020. If we valued a human life as literally priceless, then it would make sense to impose a ten-mile-per-hour speed limit on all roads. However, most of us probably feel that the extra time it would take to travel is not justified—That is to say, we implicitly put a value, in terms of our time, on a human life.’
‘Sabine Hossenfelder put it in a rather graphic way (apparently referring to something that drunken Germans do from railway bridges): science does not tell you not to pee onto high-voltage electricity lines; it tells you that urine is a good conductor of electricity.
Just as with weather prediction, a cost-loss analysis can help you make a decision about whether to take anticipatory action regarding climate change—this requires putting a value on things which do not have an unambiguous value, like living in a future hell on earth.’
February 13, 2023
Off to a rousing start. Feynman is his scientific hero! (me too). Special order from the Calif State Library's great ZIP program!
Brian Clegg loves it! His review is nearby. Top of the chart, as of 1/9/23. He says,
"This is quite possibly the best popular science book I've ever read (and I've read many hundreds)."
Nature's short review:
"Anyone intrigued by the uncertainty of weather forecasts will appreciate this important, if complicated, book. Physicist Tim Palmer has spent much of his career researching ensemble prediction — using many models with varying initial conditions, rather than a single model. Here he ranges over climate change, conflicts, consciousness, financial crashes and pandemics. Uncertainty is, he argues, more complicated than is often assumed; in his view, inspired by physicist Richard Feynman, doubt is the primary essence of knowing."
A wide-ranging book, about just what it says. Start with the publisher's summary above. Then move on to Brian Clegg's 5-star review, nearby. I didn't like it quite as much as he did -- in fact, I set it aside for a couple of weeks -- and I did quite a lot of skimming. But there's a good deal of meat here, especially on the subject of Climate Change, and some interesting speculation on other topics.
Climate Change! The scientific topic of the new century, where the science often takes second place to politics, sadly. The Climate Catastrophe stuff. Fortunately, Tim Palmer takes a more even-handed approach, and if he leans more towards the "serious problem" side than I do -- more on this in a moment -- he at least acknowledges that our scientific understanding of the Earth's climate is still rudimentary. In fact, the main premise of the book is that a chaotic system, which our climate certainly is, can't have a deterministic solution. The newspaper accounts attributing a flood or severe storm to "Climate Change" are BS, and usually political BS.
OK, personal experience: I'm a retired geologist. I earned a B.S. in geology and chemistry at Rice University, and a M.S. in geochemistry at the University of North Carolina (Chapel Hill). I've been interested in paleoclimates since student days, am well-read on the topic, and have considerable experience applying statistical analyses to geological data. The amount of warming in the past century, around 1 deg C (1.8 deg F), suggests to me that future warming will be slow, gentle and won't cause serious problems.
One of my retirement projects has been to contribute to Wikipedia, and there I got involved with their Climate Change pages, and fell unknowingly into the Wiki Climate Wars, which were at their worst about 5 years ago. I got involved in a silly dispute, and ended up being permanently banned from editing on that topic -- forever! Now, I don't particularly care about that, as it's very stressful dealing with the Wiki Climate Warriors -- who are still there, and (unsurprisingly) still unbanned -- but don't go there for an unbiased view of the topic! Bah. What a pity.
Anyway, back to the book. As Clegg says, not an easy read. But, if you bog down, skim ahead until he starts to make sense again: a reading approach the author himself encourages. 4-star book, recommended reading.
Brian Clegg loves it! His review is nearby. Top of the chart, as of 1/9/23. He says,
"This is quite possibly the best popular science book I've ever read (and I've read many hundreds)."
Nature's short review:
"Anyone intrigued by the uncertainty of weather forecasts will appreciate this important, if complicated, book. Physicist Tim Palmer has spent much of his career researching ensemble prediction — using many models with varying initial conditions, rather than a single model. Here he ranges over climate change, conflicts, consciousness, financial crashes and pandemics. Uncertainty is, he argues, more complicated than is often assumed; in his view, inspired by physicist Richard Feynman, doubt is the primary essence of knowing."
A wide-ranging book, about just what it says. Start with the publisher's summary above. Then move on to Brian Clegg's 5-star review, nearby. I didn't like it quite as much as he did -- in fact, I set it aside for a couple of weeks -- and I did quite a lot of skimming. But there's a good deal of meat here, especially on the subject of Climate Change, and some interesting speculation on other topics.
Climate Change! The scientific topic of the new century, where the science often takes second place to politics, sadly. The Climate Catastrophe stuff. Fortunately, Tim Palmer takes a more even-handed approach, and if he leans more towards the "serious problem" side than I do -- more on this in a moment -- he at least acknowledges that our scientific understanding of the Earth's climate is still rudimentary. In fact, the main premise of the book is that a chaotic system, which our climate certainly is, can't have a deterministic solution. The newspaper accounts attributing a flood or severe storm to "Climate Change" are BS, and usually political BS.
OK, personal experience: I'm a retired geologist. I earned a B.S. in geology and chemistry at Rice University, and a M.S. in geochemistry at the University of North Carolina (Chapel Hill). I've been interested in paleoclimates since student days, am well-read on the topic, and have considerable experience applying statistical analyses to geological data. The amount of warming in the past century, around 1 deg C (1.8 deg F), suggests to me that future warming will be slow, gentle and won't cause serious problems.
One of my retirement projects has been to contribute to Wikipedia, and there I got involved with their Climate Change pages, and fell unknowingly into the Wiki Climate Wars, which were at their worst about 5 years ago. I got involved in a silly dispute, and ended up being permanently banned from editing on that topic -- forever! Now, I don't particularly care about that, as it's very stressful dealing with the Wiki Climate Warriors -- who are still there, and (unsurprisingly) still unbanned -- but don't go there for an unbiased view of the topic! Bah. What a pity.
Anyway, back to the book. As Clegg says, not an easy read. But, if you bog down, skim ahead until he starts to make sense again: a reading approach the author himself encourages. 4-star book, recommended reading.
December 22, 2022
Interesting, thought provoking, but not quite in the ‘mind-blowing’ category of my popular science reads. But then few books are in that latter category.
The author outlines several scientific fields of human endeavour (if one counts economics!), where a lack of certainty in the mathematical modelling of a phenomenon can still be handled to give useful results, and even sometimes be turned to our advantage. Perhaps the baseline equations are non-linear in a way that can give wild fluctuations, even chaos, over time (population growth models, or even models of the many interacting objects in our solar system!). In a related manner some equations are extremely sensitive to the initial conditions (E.g. weather forecasting and its famous ‘butterfly effect’).
Given the author’s background he concentrates initially on the challenges behind weather forecasting and then broadens the discussion. He shows how modern powerful computers have allowed the sensitivity to initial conditions in weather models to be managed by running many parallel models of the atmosphere, differing just slightly in their initial set up, and then noting how different the results are. This is ‘ensemble modelling’. Often one gets many similar outcomes from the parallel models, giving confidence in a fairly reliable prediction. Occasionally too many different results appear, indicating an unstable, unpredictable weather trend. Either way, it’s demonstrated why no weather prediction is ever expected to have a good reliability beyond 2-3 weeks.
In addition, he has a chapter which sums up, as well as I’ve seen anywhere, the status behind modelling climate change (commonly termed global warming). The author demonstrates and supports the success of models predicting warming based on human activities and CO2, but also gives caveats as to the accuracy of current modelling (mainly poor modelling of cloud extent, and its importance in trapping or reflecting solar radiation).
In later chapters the author extends the potential for ensemble modelling, looking at multiple runs of different relevant models to look for useful trends, in the fields of economics, political decision making, pandemic modelling (the book is very up to date on CoVid!). Even eventually chapters on the problem of understanding quantum mechanics and thought processes!
The author does say these latter chapters are speculative and I wasn’t persuaded but I found them thought provoking nonetheless as whenever someone speculates on consciousness and free will, for example. Perhaps a bit too much influence of Roger Penrose here with his thoughts on the brain as having some ‘quantum computing’ aspects.
I’m also not so surprised by the content as others may be. In my engineering career ensemble modelling (we often called it ‘sensitivity studies’) was commonly used for studying mechanical failure scenarios when you weren’t clear on how accurately the starting scenario was known. This advanced a great deal in my lifetime as computers have allowed the analysis to develop from taking many days to complete a range of scenarios to, nowadays, minutes or an hour, tops! Similarly the author touches on Monte Carlo analyses, where a controlled randomness can be used to start analysis scenarios in order to see how sensitive results are to initial conditions. Again, not uncommon in some types of structural analysis. But the author does summarise these features well.
I like to review the knowledge requirements readers may need for popular science books. This is difficult here. The book contains no equations at all, but quite of lot of informative, technically sophisticated, diagrams. The underlying mathematical concepts are not simple (non-linear equations, fractals, phase space, etc.). The basic aim is to explore how useful mathematical modelling of various subjects can be achieved even in unpromising situations. So I guess the reader really does have to care about the role of mathematical modelling, how it’s achieved and how close to reality it is. Later in the book the reader will need to be concerned about the philosophical status of quantum mechanics, and consciousness!
So, make your choice - if you’re a serious geek, with mathematical interests, then this book covers its chosen territory very well; if you’re more humanities based, and not so well disposed to the role maths and physics can play, then this book will be a struggle. However, the chapter on climate change, which does rely on the previous chapters on modelling the weather, deserves a further mention for its exceptional clarity.
For me, 4*. The early chapters on mathematical modelling of physical phenomena are excellent. As it became more speculative, especially for topics which are hard to quantify, I wasn’t persuaded that the detail given was worth the return I got from them.
The author outlines several scientific fields of human endeavour (if one counts economics!), where a lack of certainty in the mathematical modelling of a phenomenon can still be handled to give useful results, and even sometimes be turned to our advantage. Perhaps the baseline equations are non-linear in a way that can give wild fluctuations, even chaos, over time (population growth models, or even models of the many interacting objects in our solar system!). In a related manner some equations are extremely sensitive to the initial conditions (E.g. weather forecasting and its famous ‘butterfly effect’).
Given the author’s background he concentrates initially on the challenges behind weather forecasting and then broadens the discussion. He shows how modern powerful computers have allowed the sensitivity to initial conditions in weather models to be managed by running many parallel models of the atmosphere, differing just slightly in their initial set up, and then noting how different the results are. This is ‘ensemble modelling’. Often one gets many similar outcomes from the parallel models, giving confidence in a fairly reliable prediction. Occasionally too many different results appear, indicating an unstable, unpredictable weather trend. Either way, it’s demonstrated why no weather prediction is ever expected to have a good reliability beyond 2-3 weeks.
In addition, he has a chapter which sums up, as well as I’ve seen anywhere, the status behind modelling climate change (commonly termed global warming). The author demonstrates and supports the success of models predicting warming based on human activities and CO2, but also gives caveats as to the accuracy of current modelling (mainly poor modelling of cloud extent, and its importance in trapping or reflecting solar radiation).
In later chapters the author extends the potential for ensemble modelling, looking at multiple runs of different relevant models to look for useful trends, in the fields of economics, political decision making, pandemic modelling (the book is very up to date on CoVid!). Even eventually chapters on the problem of understanding quantum mechanics and thought processes!
The author does say these latter chapters are speculative and I wasn’t persuaded but I found them thought provoking nonetheless as whenever someone speculates on consciousness and free will, for example. Perhaps a bit too much influence of Roger Penrose here with his thoughts on the brain as having some ‘quantum computing’ aspects.
I’m also not so surprised by the content as others may be. In my engineering career ensemble modelling (we often called it ‘sensitivity studies’) was commonly used for studying mechanical failure scenarios when you weren’t clear on how accurately the starting scenario was known. This advanced a great deal in my lifetime as computers have allowed the analysis to develop from taking many days to complete a range of scenarios to, nowadays, minutes or an hour, tops! Similarly the author touches on Monte Carlo analyses, where a controlled randomness can be used to start analysis scenarios in order to see how sensitive results are to initial conditions. Again, not uncommon in some types of structural analysis. But the author does summarise these features well.
I like to review the knowledge requirements readers may need for popular science books. This is difficult here. The book contains no equations at all, but quite of lot of informative, technically sophisticated, diagrams. The underlying mathematical concepts are not simple (non-linear equations, fractals, phase space, etc.). The basic aim is to explore how useful mathematical modelling of various subjects can be achieved even in unpromising situations. So I guess the reader really does have to care about the role of mathematical modelling, how it’s achieved and how close to reality it is. Later in the book the reader will need to be concerned about the philosophical status of quantum mechanics, and consciousness!
So, make your choice - if you’re a serious geek, with mathematical interests, then this book covers its chosen territory very well; if you’re more humanities based, and not so well disposed to the role maths and physics can play, then this book will be a struggle. However, the chapter on climate change, which does rely on the previous chapters on modelling the weather, deserves a further mention for its exceptional clarity.
For me, 4*. The early chapters on mathematical modelling of physical phenomena are excellent. As it became more speculative, especially for topics which are hard to quantify, I wasn’t persuaded that the detail given was worth the return I got from them.
October 16, 2022
This is one of the best NetGalley books I've read in a long time. I tend to read new science titles that focus on math, physics, or astronomy, and I prefer they aren't popularized too much in their approach. PRIMACY OF DOUBT meets the bill, stopping short of actual math but definitely giving a qualitative treatment to a difficult topic. The author talks through how various chaotic systems (not just "chaos theory") can benefit from being analyzed using ensemble systems that contain some level of random noise to allow for uncertanities in the initial conditions. This applies to weather, climate, pandemic forecasting, econmics, even our brains. If you don't mind thinking a little while you read and can put in the effort to slow down and follow along, this one is well worth it.
October 30, 2022
The Primacy of Doubt : From Quantum Physics to Climate Change, How the Science of Uncertainty Can Help Us Understand Our Chaotic World (2022) by Tim Palmer explores profound uncertainty in physics in Chaos and Quantum theory. Palmer is a highly acclaimed physicist who now works at Oxford after many years of working on ensemble weather forecasts.
The early chapters on chaos theory and quantum theory and in particular why ensemble forecasts are used and how they enable a varying window of uncertainty in forecasts are really great. Palmer knows the material inside out and manages to give very good explanations of chaos theory.
As with most books about mathematical constructs there is, by necessity to make the book generally readable, not much math. Which is a pity, but clearly these kinds of books sell many more copies than introductory textbooks.
The Primacy of Doubt extends the views on chaos theory into climate forecasting and Palmer believes that the use of ensembles of models means that the models are accurate and useful.
Palmer also discusses economic modelling and finds it wanting and suggests that the inability of economic forecasts to be of much use could be overcome by using ensemble models with many agents.
Finally the book discusses the brain and Palmer suggests that the efficiency of the brain is in part due to the harnessing of quantum effects to provide some randomness in conjunction with remarkable low energy computation.
The Primacy of Doubt is a fine popular science book that imparts knowledge and insights into chaos and quantum theory from a highly accomplished physicist.
The early chapters on chaos theory and quantum theory and in particular why ensemble forecasts are used and how they enable a varying window of uncertainty in forecasts are really great. Palmer knows the material inside out and manages to give very good explanations of chaos theory.
As with most books about mathematical constructs there is, by necessity to make the book generally readable, not much math. Which is a pity, but clearly these kinds of books sell many more copies than introductory textbooks.
The Primacy of Doubt extends the views on chaos theory into climate forecasting and Palmer believes that the use of ensembles of models means that the models are accurate and useful.
Palmer also discusses economic modelling and finds it wanting and suggests that the inability of economic forecasts to be of much use could be overcome by using ensemble models with many agents.
Finally the book discusses the brain and Palmer suggests that the efficiency of the brain is in part due to the harnessing of quantum effects to provide some randomness in conjunction with remarkable low energy computation.
The Primacy of Doubt is a fine popular science book that imparts knowledge and insights into chaos and quantum theory from a highly accomplished physicist.
February 13, 2023
Positive and negatives.
One side of the book is an insightful exploration of how the world around us is permeated by a very specific kind of uncertainty, different from that of quantum mechanics. The author then explains how physicists and climate sci3ntists model complex systems using ensembles of predictions that yield probabilistic forecasts, that than by relying on single deterministic models.
Unfortunately, the other side of the book devolves into "physicist writes a pop science book bingo", with allusions to the uncertainty principle, the incompleteness theorem, the unification of qm and gravity, the nature of causality, consciousness and even God. The book is heavily influenced by James Gleick and Roger Penrose, but I feel the material flounders when stretched beyond what the science supports.
To fill another bingo square, "What can be said at all can be said clearly; and whereof one cannot speak thereof one must be silent."
Still, worth a read, especially regarding modeling concepts, pitfalls and approaches. Wish the author had covered other applicatitions of ensemble forecasting such as elections, or covered agent-based models more in depth. In the end though, any book that gives food for thought is a good book, and this book certainly provides plenty of that.
One side of the book is an insightful exploration of how the world around us is permeated by a very specific kind of uncertainty, different from that of quantum mechanics. The author then explains how physicists and climate sci3ntists model complex systems using ensembles of predictions that yield probabilistic forecasts, that than by relying on single deterministic models.
Unfortunately, the other side of the book devolves into "physicist writes a pop science book bingo", with allusions to the uncertainty principle, the incompleteness theorem, the unification of qm and gravity, the nature of causality, consciousness and even God. The book is heavily influenced by James Gleick and Roger Penrose, but I feel the material flounders when stretched beyond what the science supports.
To fill another bingo square, "What can be said at all can be said clearly; and whereof one cannot speak thereof one must be silent."
Still, worth a read, especially regarding modeling concepts, pitfalls and approaches. Wish the author had covered other applicatitions of ensemble forecasting such as elections, or covered agent-based models more in depth. In the end though, any book that gives food for thought is a good book, and this book certainly provides plenty of that.
October 12, 2023
I was super excited to read this book. I felt that the book seemed to be heading in the direction of establishing just how fundamental and existential chaos theory is. As someone who had worked with chaotic systems in my undergrad thesis, I was hoping to dive deeper into the epistemic and perhaps ontological aspects of the uncertainty inherent in these systems. I thus have many thoughts.
I think Palmer did a good job of push for the geometry of chaos being a good way to discuss and think about these systems more than just individual solutions. Kind of reminds me of people’s personal preferences in how to handle General Relativity.
The book is divided into three parts. The first explains the theories, and I quite enjoyed this. As someone who likes to think in graphs, I liked the explanation that a very simple view of climate change is one wherein we shift the graph up like a forcing mechanism. So even if the underlying weather is chaotic, we still get more or higher peaks.
The second part of the book was good in that we got so many examples of how chaos affects modelling. It also tackles how noise/randomness can be used in ensembles etc. to better our models. There’s a cool clarification of the difference between prediction and projection. Projections are results generated by assuming certain parameters (eg. Assuming a lockdown is implemented, and then modelling the results.) It also delivered a very succinct and compelling argument that the saying, “Listen to the science,” in dealing with global issues doesn’t really make sense. The science gives us projections, yes. However, it is not a question of science what we choose to value. Therefore, decisions on potentially slowing global development to curb carbon emissions is a value-judgment question, not a scientific one.
These discussions ranging from climate, pandemics, economics etc. were all really interesting. However, it sort of felt a bit repetitive. You just change the topic in question, but the author’s discussion of modelling etc. for each was quite similar.
The third part of the book is the most compelling but also the most iffy. The author does warn us that this part is speculative, but I don’t think being speculative gives you the excuse to be so hand wavy at times.
The author asks whether alternate hypothetical decisions/events are consistent with the laws of physics. For example, if Archduke Ferdinand’s bodyguard told the driver in time to take another route. Would that have been another world that our laws of physics permit? Or could that other world just not happen because it is a violation of the laws of physics (sort of block universe). He then posits that the universe is chaotic, so perhaps the universe’s state space is made up of a fractal attractor he calls the cosmological invariant set. Other hypothetical universes exist on the fractal structures on the set, but aren’t crossed by our particular trajectory. Then worlds that could just not happen, lie outside the set. This is a pretty neat idea, but he doesn’t really present a convincing argument for why this is a likely interpretation.
Earlier in the book, he showed how noise and signal annealing can help with lower resolution systems simulating higher dimensional ones. He then argues that noise is inherent in circuits. The brain’s neurons and such also contain noise. He talks about the low power thinking mode, where the 20J energy of the brain is spread. And high power thinking mode during concentration. He says that perhaps the eureka moments that occur when not concentrating on something are due to the noise (like from the quantum interactions of ions) is what leads us to be able to consider other ideas that break away from the train of logic in our concentration. This is quite compelling, and I would have appreciated a less handwavy approach.
The most egregious and sort of cringey part is when he begins to suggest that the noise may be what allows humans to access other nearby trajectories in the cosmological state space. That may apparently by why, humans have such a penchant for what if questions. There really was no evidence for this, and it felt like idle thoughts at this point. He also says that if some semblance of the big crunch occurs, then we can sort of see each other again in the next universe, with conditions that are different but on the invariant set. This is cool speculation, but I think it is quite dangerous to meander to what can sound like quantum hocus pocus. Like please try not to court the Deepak Chopra fans. Maybe this stuff has more basis than I am giving it credit for. But it is the author’s responsibility to expound on that. I would have given the book 4 stars if it wasn’t for these parts.
I think Palmer did a good job of push for the geometry of chaos being a good way to discuss and think about these systems more than just individual solutions. Kind of reminds me of people’s personal preferences in how to handle General Relativity.
The book is divided into three parts. The first explains the theories, and I quite enjoyed this. As someone who likes to think in graphs, I liked the explanation that a very simple view of climate change is one wherein we shift the graph up like a forcing mechanism. So even if the underlying weather is chaotic, we still get more or higher peaks.
The second part of the book was good in that we got so many examples of how chaos affects modelling. It also tackles how noise/randomness can be used in ensembles etc. to better our models. There’s a cool clarification of the difference between prediction and projection. Projections are results generated by assuming certain parameters (eg. Assuming a lockdown is implemented, and then modelling the results.) It also delivered a very succinct and compelling argument that the saying, “Listen to the science,” in dealing with global issues doesn’t really make sense. The science gives us projections, yes. However, it is not a question of science what we choose to value. Therefore, decisions on potentially slowing global development to curb carbon emissions is a value-judgment question, not a scientific one.
These discussions ranging from climate, pandemics, economics etc. were all really interesting. However, it sort of felt a bit repetitive. You just change the topic in question, but the author’s discussion of modelling etc. for each was quite similar.
The third part of the book is the most compelling but also the most iffy. The author does warn us that this part is speculative, but I don’t think being speculative gives you the excuse to be so hand wavy at times.
The author asks whether alternate hypothetical decisions/events are consistent with the laws of physics. For example, if Archduke Ferdinand’s bodyguard told the driver in time to take another route. Would that have been another world that our laws of physics permit? Or could that other world just not happen because it is a violation of the laws of physics (sort of block universe). He then posits that the universe is chaotic, so perhaps the universe’s state space is made up of a fractal attractor he calls the cosmological invariant set. Other hypothetical universes exist on the fractal structures on the set, but aren’t crossed by our particular trajectory. Then worlds that could just not happen, lie outside the set. This is a pretty neat idea, but he doesn’t really present a convincing argument for why this is a likely interpretation.
Earlier in the book, he showed how noise and signal annealing can help with lower resolution systems simulating higher dimensional ones. He then argues that noise is inherent in circuits. The brain’s neurons and such also contain noise. He talks about the low power thinking mode, where the 20J energy of the brain is spread. And high power thinking mode during concentration. He says that perhaps the eureka moments that occur when not concentrating on something are due to the noise (like from the quantum interactions of ions) is what leads us to be able to consider other ideas that break away from the train of logic in our concentration. This is quite compelling, and I would have appreciated a less handwavy approach.
The most egregious and sort of cringey part is when he begins to suggest that the noise may be what allows humans to access other nearby trajectories in the cosmological state space. That may apparently by why, humans have such a penchant for what if questions. There really was no evidence for this, and it felt like idle thoughts at this point. He also says that if some semblance of the big crunch occurs, then we can sort of see each other again in the next universe, with conditions that are different but on the invariant set. This is cool speculation, but I think it is quite dangerous to meander to what can sound like quantum hocus pocus. Like please try not to court the Deepak Chopra fans. Maybe this stuff has more basis than I am giving it credit for. But it is the author’s responsibility to expound on that. I would have given the book 4 stars if it wasn’t for these parts.
January 6, 2023
He lost me as he jumped between topics barely explained.
Why's Roger Penrose doing some Bell experiment?
Diagrams were too few, and kindle display makes them less clear.
Why does the second law of thermodynamics show "heat can’t be converted into work in such a way"?
There are too many typos that an editor should catch:
"ellipsoid—an egg-shaped two-dimensional figure." It's 3D not 2D.
"the orbits of the three bodies never repeat" with legend n = 4 gravitating bodies, yet picture shows 3 bodies
"quite unique" is bad usage.
"endnote 4—is well worth watching." How? can't find the twitter note from S.Tardivel
Maybe instead or before I should read James Gleick’s book Chaos.
March 15, 2024
To me the best science books trigger new thought processes which alter and / or confirm one's world view. This book does all that in an engaging and illuminating 5-star way. If you are a black hole, in terms of gobbling up knowledge and perspective, this is 5-stars you should gobble up. That being said you will not get indigestion since the book is devoid of the arcane, consisting mostly of storied narrative.
Author Tim Palmer is both a physicist and meteorologist who develops mathematical computerized weather models, all of which gives him a broad wholistic perspective. This book casts a wide net, with chapter topics ranging from physics to climate change to pandemics to brain function. At the same time the book dives deep into the main premise that the universe often exhibits chaotic uncertainty in proceeding towards a "fractal attractor", thus scientific predictions should bound this uncertainty by running ensembles of predictions with rationally generated slightly varied inputs, or what Palmer would term information enhancing "noise".
Readers of this book might be "triggered" into relating the storied narrative to their own world of the arcane, which for me is Networked Higgs Mechanisms (NHM's) related to dark matter, dark energy, and quantum gravity. NHM's point to a "fractal attractor" scale dimension of the universe as being 2.167D. Various galactic surveys put galactic dimensionality at somewhere between 2.0D and 2.5D, such that galaxy formation science would benefit from ensemble modeling, especially since recent James Webb Space Telescope observations have largely negated current galaxy formation science models.
Author Tim Palmer is both a physicist and meteorologist who develops mathematical computerized weather models, all of which gives him a broad wholistic perspective. This book casts a wide net, with chapter topics ranging from physics to climate change to pandemics to brain function. At the same time the book dives deep into the main premise that the universe often exhibits chaotic uncertainty in proceeding towards a "fractal attractor", thus scientific predictions should bound this uncertainty by running ensembles of predictions with rationally generated slightly varied inputs, or what Palmer would term information enhancing "noise".
Readers of this book might be "triggered" into relating the storied narrative to their own world of the arcane, which for me is Networked Higgs Mechanisms (NHM's) related to dark matter, dark energy, and quantum gravity. NHM's point to a "fractal attractor" scale dimension of the universe as being 2.167D. Various galactic surveys put galactic dimensionality at somewhere between 2.0D and 2.5D, such that galaxy formation science would benefit from ensemble modeling, especially since recent James Webb Space Telescope observations have largely negated current galaxy formation science models.
December 21, 2023
Wow, you really pulled it out at the end there, Tim.
This was going to be 3 stars because, well, it was near impossible to understand.
I was brushing up on differential equations while reading, watching youtube videos, googling, and I was still dying. Relearning differential equations did help a little. I don't know what's going on with the science books this year, but I'm guessing the editors are either too smart to where everything makes perfect sense even with the gaps in explanation, or they just don't understand anything at all and only point out syntactical errors. Authors/publishers? need to consult people who have some mathematical ability but who are not knowledgeable about the specific topic of their book.
That said, the things covered in this book are fascinating. It is basically about chaos theory, but the author explores how it applies (or could possibly apply) to many subjects, including (among other things) climate science, quantum physics, and consciousness. I did appreciate how fairly the author treated the topic of climate change. He was very matter of fact, not using emotional appeals or sensationalism. However, his chapters on quantum physics and consciousness were my favorite. His ideas there were very new to me. I don't think they're floating around in the zeitgeist, at least at this point, and I do think they're worth a read, just for the pure purpose of rearranging your mental furniture.
This was going to be 3 stars because, well, it was near impossible to understand.
I was brushing up on differential equations while reading, watching youtube videos, googling, and I was still dying. Relearning differential equations did help a little. I don't know what's going on with the science books this year, but I'm guessing the editors are either too smart to where everything makes perfect sense even with the gaps in explanation, or they just don't understand anything at all and only point out syntactical errors. Authors/publishers? need to consult people who have some mathematical ability but who are not knowledgeable about the specific topic of their book.
That said, the things covered in this book are fascinating. It is basically about chaos theory, but the author explores how it applies (or could possibly apply) to many subjects, including (among other things) climate science, quantum physics, and consciousness. I did appreciate how fairly the author treated the topic of climate change. He was very matter of fact, not using emotional appeals or sensationalism. However, his chapters on quantum physics and consciousness were my favorite. His ideas there were very new to me. I don't think they're floating around in the zeitgeist, at least at this point, and I do think they're worth a read, just for the pure purpose of rearranging your mental furniture.
March 22, 2024
This book was no light work…it took me a solid year to get through it. Especially the last 30% where it started to get super “woo-woo”.
Some parts were definitely over my head (I am no where near a mathematician or quantum physicist) and I had to go back and re-read several sections to just to try to comprehend what the author was saying. But overall, I really enjoyed the insights this book provided and I appreciate the author’s attempt to make some heavy concepts more accessible to the public. Science communication is hard these days so I give this book major props.
Some parts were definitely over my head (I am no where near a mathematician or quantum physicist) and I had to go back and re-read several sections to just to try to comprehend what the author was saying. But overall, I really enjoyed the insights this book provided and I appreciate the author’s attempt to make some heavy concepts more accessible to the public. Science communication is hard these days so I give this book major props.
May 27, 2024
A collection of very intriguing ideas, not all equally explored or explained . Severe topical whiplash between sections renders the books quite unfocused. It could have been two quite compelling books: one about the interesting applications of ensemble forecasting and stochasticity to different problem spaces and one about the authors particular speculations about metaphysics.
October 12, 2023
The butterfly effect as we know it is a film, a lofty theory, and a hackneyed topic to demonstrate your sophistication. As with many other scientific theories, it is popularly misunderstood. Yet it is a phenomenon discovered by Edward Lorenz so important that it has pried open a whole new discipline.
Tim Palmer gave a wonderful account of theoretical development surrounding meteorology. I got to appreciate the beauty of combating chaos with chaos: introducing stochasticity in a model. Palmer’s credit here, lies in taking multi-model projections and convoking them with a probability or frequentist map. While technical details were unavoidable, he made sure they were included for explanation’s sake.
Therefore, I was taken aback that towards the end of the book he decided to breach my façade of understanding and machine-gunned me with the problems of string theory and the like. I didn’t ask for this. Since this is a popular science book, I did not quite see how secondary education in physics could enable a thorough comprehension in this part.
Touching on the current issues is the most enlightening part though. As an undertaking conceived in the pandemic, the book lends itself to the in-depth discussion of various applications, including but not limited to covid, climate change, conflicts and commerce. These chapters are very informative and brimming with a humble undertone that remind me of Vaclav Smil. Perhaps no distinguished scientists are arrogant.
I hope his hope will bear fruit—a CERN for making forecasts. Only concerted efforts like the Marshall Plan can benefit the countries and regions that have limited resources to pool, be it human or monetary. As for his ambition to devise a provable alternative to string theory, as an outsider I can only wish him good luck.
Tim Palmer gave a wonderful account of theoretical development surrounding meteorology. I got to appreciate the beauty of combating chaos with chaos: introducing stochasticity in a model. Palmer’s credit here, lies in taking multi-model projections and convoking them with a probability or frequentist map. While technical details were unavoidable, he made sure they were included for explanation’s sake.
Therefore, I was taken aback that towards the end of the book he decided to breach my façade of understanding and machine-gunned me with the problems of string theory and the like. I didn’t ask for this. Since this is a popular science book, I did not quite see how secondary education in physics could enable a thorough comprehension in this part.
Touching on the current issues is the most enlightening part though. As an undertaking conceived in the pandemic, the book lends itself to the in-depth discussion of various applications, including but not limited to covid, climate change, conflicts and commerce. These chapters are very informative and brimming with a humble undertone that remind me of Vaclav Smil. Perhaps no distinguished scientists are arrogant.
I hope his hope will bear fruit—a CERN for making forecasts. Only concerted efforts like the Marshall Plan can benefit the countries and regions that have limited resources to pool, be it human or monetary. As for his ambition to devise a provable alternative to string theory, as an outsider I can only wish him good luck.
December 30, 2022
Note: I received an advanced reading copy of this book from NetGalley.
I really enjoyed this book. Palmer writes about a lot of different topics, but his explanations were clear and they are ones I'm sympathetic with (the parts about doubt and uncertainty in models). I also really liked that he separated his personal new ideas about quantum theory in its own section at the very end instead of sprinkling it through the book. It makes it easier for the reader to understand what's standard knowledge and what's the author's own beliefs. Overall, I enjoyed this book.
Personally, I also enjoyed Palmer explaining how he got into the work he does. As a scientist myself, seeing how people land in the careers they are currently in is fascinating.
I really enjoyed this book. Palmer writes about a lot of different topics, but his explanations were clear and they are ones I'm sympathetic with (the parts about doubt and uncertainty in models). I also really liked that he separated his personal new ideas about quantum theory in its own section at the very end instead of sprinkling it through the book. It makes it easier for the reader to understand what's standard knowledge and what's the author's own beliefs. Overall, I enjoyed this book.
Personally, I also enjoyed Palmer explaining how he got into the work he does. As a scientist myself, seeing how people land in the careers they are currently in is fascinating.
January 21, 2023
[short update Jan 2023]
A good explanation of chaos theory and how it can be used to understand complex systems
For those of us who find much of theoretical physics impenetrable, Palmer offers a possible solution that he believes Einstein would like.
He uses his expertise in weather systems and modeling challenges to give us insights that he himself discovered.
His discussion of error and uncertainty using climate change was helpful. However his suggestion that uncertainty in the modeling predictions is sufficient to be a climate change skeptic, baffled me - tipping points (changes that we can't undo) that are possible but low probability can't be ignored if the result is the death of the planet as we know it.
Palmer fixed this in later chapter where he did address the need to include risk when accessing a course of action. And he does mention that this certainly applies to the climate crisis. Back in the Climate Chpt, he should have at least included a forward reference.
The chapters about free will, consciousness, and the role of random noise in the workings of our neural brain are fascinating. Palmer suggests that our creativity is aided by random neural firings. And there's no reason why AI computers can't simulate that same process and come up with outputs that weren't possible in the original program fed into the computer.
A good explanation of chaos theory and how it can be used to understand complex systems
For those of us who find much of theoretical physics impenetrable, Palmer offers a possible solution that he believes Einstein would like.
He uses his expertise in weather systems and modeling challenges to give us insights that he himself discovered.
His discussion of error and uncertainty using climate change was helpful. However his suggestion that uncertainty in the modeling predictions is sufficient to be a climate change skeptic, baffled me - tipping points (changes that we can't undo) that are possible but low probability can't be ignored if the result is the death of the planet as we know it.
Palmer fixed this in later chapter where he did address the need to include risk when accessing a course of action. And he does mention that this certainly applies to the climate crisis. Back in the Climate Chpt, he should have at least included a forward reference.
The chapters about free will, consciousness, and the role of random noise in the workings of our neural brain are fascinating. Palmer suggests that our creativity is aided by random neural firings. And there's no reason why AI computers can't simulate that same process and come up with outputs that weren't possible in the original program fed into the computer.
July 5, 2024
The book layout is awful and too many concepts are covered. The author takes nonlinear dynamics and applies it to the weather, economics, war, quantum mechanics, the mind, and spirituality. Suffice it to say it’s a mess for that reason. I picked up this book because I was interested in Palmer’s explanation of quantum mechanics (which boils down to hidden variables via fractal dimensions or moving the uncertainty from ontological to epistemological) but instead, I got a lot more… and less than I bargained for. If each of these topics was broken out into its own book with detailed conceptual and mathematical breakdowns that would be better. I guess I’ll have to read his papers for that.
Note: the author makes an interesting observation in the book’s twelfth chapter on the mind where he argues we are always rational even when it appears we aren’t. Perfect rationality all the time requires a large amount of energy and for most actions that energy expense cannot be justified for maximizing Darwinian fitness. I completely agree with this perspective and most behavioral economists don’t understand this.
Note: the author makes an interesting observation in the book’s twelfth chapter on the mind where he argues we are always rational even when it appears we aren’t. Perfect rationality all the time requires a large amount of energy and for most actions that energy expense cannot be justified for maximizing Darwinian fitness. I completely agree with this perspective and most behavioral economists don’t understand this.
July 31, 2023
Interesting book that is hard to summarize. The author spends a lot of time discussing climate modeling, and this was quite interesting to learn more about. He sometimes goes into great detail that goes over my head, there is certainly a bit of this book that is skippable. He often talked about adding noise to the climate system, among other potential systems. This was an interesting concept to me, that because of our uncertainty and natural chaos that may exist, stochastic noise can help better model some processes. He talked about using Monte Carlo calculations to help make predictions in chaotic systems. I'm sure I'm not summarizing this well, but these were my main takeaways. Not sure I'm taking much from this book, but I did find it somewhat interesting.
October 26, 2024
Highly overrated. His conjectures about quantum mechanics and that p-adic numbers will become important in physics are extremely unlikely to be correct. And his repeated claim that the crash of 2008 was unpredictable is idiotic. Anybody who bothered to look into the data at least as early as 2005 was able to predict it. He should stick to topics he understands well.
Also, his explanations are simplistic and unnecessary for anybody who knows anything about the topics but completely unintelligible for anybody who doesn't know the topics.
Also, his explanations are simplistic and unnecessary for anybody who knows anything about the topics but completely unintelligible for anybody who doesn't know the topics.
December 11, 2022
I haven’t read a pop sci book this good in a long time. I know chaos theory well but learned a bunch of new things, like its connections with algorithmic undecidability. The hot takes on economics were fun; Palmer argues economics is overly fixated on parsimony and elegance and does not care enough about forecasting accuracy. The ideas about the low energy usage of the brain and its connections with creativity were fascinating.
October 11, 2023
A mixed bag. Not the most crystal clear writing, but it has some interesting, if unconvincing, ideas.
May 25, 2024
The Primacy of Doubt by Tim Palmer is about uncertainty and doubt and how it affects our understanding of our world and our lives. I listened to this as an audiobook, but a warning, there are diagrams in this book which the author describes, and I don’t know if it’s just me but I was unable to find them online.
Thoughts while reading:
-It was interesting to learn about ensemble forecasting, which is using models to predict a situation by slightly changing multiple variables. I know next to nothing about these kinds of things so this was new to me
-learning about how predicting more than three bodies in a system (like the solar system) is impossible, and how planets can randomly be ejected was very unsettling. It reminds me of being a kid and reading science picture books about how the sun would eventually swallow the earth; the idea of not knowing what the future holds reminds me of that anxiety I felt as a child
-I honestly had to listen to chapter 2 twice, and I’m not sure I fully understand it. I vaguely grasp the idea of states. I know what sets are due to something else I read. The Lorenz equations were interesting in that initial conditions can drastically affect the outcome, but my understanding of even the simplified version is tenuous at best. Interesting how there’s two sets of numbers, with real numbers and the other kind, and how, with somethings, when you magnify them, you get Euclidean geometry, but with other things when you magnify them, you just get small pieces of the thing (fractals)
-it's interesting how after a period of time, even if you have detailed information about the initial conditions, certain situations become entirely unpredictable
-I liked learning about the history of climate science and interesting use of the Monte Carlo method to make predictions using randomness
-what's that chapter on climate change was quite informative, though it was also anxiety inducing. It gave a clear idea of what our current models are capable of, and what they're not capable of, such as predicting those random extreme events that we have already experienced
-it's quite fascinating to think that models could be used to predict both markets or conflicts (wars). I had never considered that in economist might be resistant to the idea of using Ensemble forecast since they like neat and parsimonious models. And it's really interesting to think that using models to predict conflict could help prevent them
-I found part 3 really difficult to understand, though it’s still interesting. I guess the problem is that I know too little about physics to make sense of a lot of the ideas that may or may not be wrong. Though since I have read Thinking, Fast and Slow, it was interesting to hear about how those processes could potentially work due to the effect of the width of neurons and random noise, which can help spur creativity
Overall, I'd give this a 3 out of 5. The ideas are interesting, but it was a dense read, and difficult for someone who knows little about math and physics to understand
Thoughts while reading:
-It was interesting to learn about ensemble forecasting, which is using models to predict a situation by slightly changing multiple variables. I know next to nothing about these kinds of things so this was new to me
-learning about how predicting more than three bodies in a system (like the solar system) is impossible, and how planets can randomly be ejected was very unsettling. It reminds me of being a kid and reading science picture books about how the sun would eventually swallow the earth; the idea of not knowing what the future holds reminds me of that anxiety I felt as a child
-I honestly had to listen to chapter 2 twice, and I’m not sure I fully understand it. I vaguely grasp the idea of states. I know what sets are due to something else I read. The Lorenz equations were interesting in that initial conditions can drastically affect the outcome, but my understanding of even the simplified version is tenuous at best. Interesting how there’s two sets of numbers, with real numbers and the other kind, and how, with somethings, when you magnify them, you get Euclidean geometry, but with other things when you magnify them, you just get small pieces of the thing (fractals)
-it's interesting how after a period of time, even if you have detailed information about the initial conditions, certain situations become entirely unpredictable
-I liked learning about the history of climate science and interesting use of the Monte Carlo method to make predictions using randomness
-what's that chapter on climate change was quite informative, though it was also anxiety inducing. It gave a clear idea of what our current models are capable of, and what they're not capable of, such as predicting those random extreme events that we have already experienced
-it's quite fascinating to think that models could be used to predict both markets or conflicts (wars). I had never considered that in economist might be resistant to the idea of using Ensemble forecast since they like neat and parsimonious models. And it's really interesting to think that using models to predict conflict could help prevent them
-I found part 3 really difficult to understand, though it’s still interesting. I guess the problem is that I know too little about physics to make sense of a lot of the ideas that may or may not be wrong. Though since I have read Thinking, Fast and Slow, it was interesting to hear about how those processes could potentially work due to the effect of the width of neurons and random noise, which can help spur creativity
Overall, I'd give this a 3 out of 5. The ideas are interesting, but it was a dense read, and difficult for someone who knows little about math and physics to understand
This entire review has been hidden because of spoilers.
December 11, 2023
Great book about DOUBT
This book is about uncetainty and doubt, from physics, to weather forecast, then to our humanity. It's an amzing book that would fundamentally change the readers mind about thinking, about noise's role universally around us. The correct way of addressing the noise is to work within it's uncertainty and that uncertainty often leads to creativity and solutions. Unfortunately, most humans are stuck within the wrong way of thinking in rejecting the noise, filtering the data around us before any processes even start. People tend to set a clear goal of some excellence in their mind in order to achieve that. Often than not, they fail. In this sense, the book of "Why Greatness Cannot Be Planned" is addressing this related issues as well.
Here is GPT4's summary:
Central to Palmer's thesis is the idea that doubt and uncertainty are not weaknesses in scientific endeavor but are, in fact, fundamental to gaining a deeper and more accurate understanding of complex systems like the Earth's climate. He argues that acknowledging and embracing uncertainty can lead to more robust scientific models and predictions, especially in fields that deal with inherently unpredictable systems.
Palmer also delves into the history and philosophy of science, examining how scientific theories have evolved and how our understanding of uncertainty has changed over time. He emphasizes the importance of probabilistic thinking in science, advocating for a shift from deterministic to probabilistic models in areas like climate forecasting.
The book is not just a scientific discourse but also touches on the implications of this approach for public policy and how society understands and reacts to scientific predictions. Palmer's insights are particularly relevant in the context of climate change, where understanding and communicating uncertainties are crucial for effective policy-making and public engagement.
This book is about uncetainty and doubt, from physics, to weather forecast, then to our humanity. It's an amzing book that would fundamentally change the readers mind about thinking, about noise's role universally around us. The correct way of addressing the noise is to work within it's uncertainty and that uncertainty often leads to creativity and solutions. Unfortunately, most humans are stuck within the wrong way of thinking in rejecting the noise, filtering the data around us before any processes even start. People tend to set a clear goal of some excellence in their mind in order to achieve that. Often than not, they fail. In this sense, the book of "Why Greatness Cannot Be Planned" is addressing this related issues as well.
Here is GPT4's summary:
Central to Palmer's thesis is the idea that doubt and uncertainty are not weaknesses in scientific endeavor but are, in fact, fundamental to gaining a deeper and more accurate understanding of complex systems like the Earth's climate. He argues that acknowledging and embracing uncertainty can lead to more robust scientific models and predictions, especially in fields that deal with inherently unpredictable systems.
Palmer also delves into the history and philosophy of science, examining how scientific theories have evolved and how our understanding of uncertainty has changed over time. He emphasizes the importance of probabilistic thinking in science, advocating for a shift from deterministic to probabilistic models in areas like climate forecasting.
The book is not just a scientific discourse but also touches on the implications of this approach for public policy and how society understands and reacts to scientific predictions. Palmer's insights are particularly relevant in the context of climate change, where understanding and communicating uncertainties are crucial for effective policy-making and public engagement.
May 27, 2025
The workings of speculative minds can be very fascinating, especially when presented in cogent form as sign of thought independence from mainstream. This book is a great example of intellectual speculations, smoothly infused with a modicum of biographical points of reference and organized in a progression of abstraction and generality. Tim Palmer is an expert in chaos theory, and particularly in ensemble forecast methodology which he contributed to establish to great benefit for the world. His insight into the theory are the root of his intellectual edifice, which hosts rooms dedicated to weather forecast and climate change, but also to infections, financial trends, foundations of quantum mechanics, consciousness and god - in fact, the Kantian triad of World, Mind and God culminate the book. The main message is the proposition of extending the use of ensemble forecasts to all relevant domains of human activity - which would quite clearly sounds like having an hammer for all possible nails around if this did not make pretty much sense, as the author elegantly explains - and more generally the need to give precise description to uncertainties, both in the lower and higher bounds, so that this can inform (though not dictate) decisions. He is also quite right in recriminating that the (re-)discovery of chaos theory in the XX century should be given a similar status as relativity theory and quantum mechanics. For Palmer the geometry of chaos, namely the cosmological invariant set, is the key to solve many if not most of standing issues in science, including quantum mechanics (not all counterfactuals are legitimate, hence Bell's theorem does not necessarily imply non-locality or irreality) and free will (echoes of counterfactuals). Though the notes do contain some technical discussions, and certainly precise references, the main text could benefit from more technical content to better satisfy also readers in the know. Praise for the sharp intellectual stand nevertheless.
June 10, 2024
The author does a nice job explaining complex topics, and starts off with a great premise about skepticism in science. When he's explaining how to interpret the global warming models, he's shedding light on an important topic. I feel though that he eventually loses the forest for the trees, and then gets mired in the weeds of GIGO (garbage in--garbage out).
If you put the wrong "parameters" into a computer model, you get wrong predictions. And unfortunately a lot of what he's talking about with the mathematical modeling of uncertainty is missing major inputs. Things like the 2008 financial crash or the COVID pandemic are not just about fluctuations in big chaotic systems. With the pandemic for example, he narrows the question down to "flattening the curve" to avoid overwhelming the medical system. That is what authorities were talking about, so it's reasonable for him to look at that. Nevertheless, I think it's misleading to portray that issue as an equation to solve about infectiousness and R0, when it has everything to do with an insane social/health/financial policy of running hospitals like Toyota factories. The lesson to learn from that experience is not that we need more mathematicians with bigger supercomputers churning out fancier models. The real moral of the story is that we need hospitals and clinics to have slack so that we're ready when a school bus crashes or there's a pandemic, and also so that the doctors and nurses taking care of you routinely aren't rushed and burned out.
If equations are the solution to everything, then why not model that? Or better yet, make it happen and use your math to measure the real-world benefit.
If you put the wrong "parameters" into a computer model, you get wrong predictions. And unfortunately a lot of what he's talking about with the mathematical modeling of uncertainty is missing major inputs. Things like the 2008 financial crash or the COVID pandemic are not just about fluctuations in big chaotic systems. With the pandemic for example, he narrows the question down to "flattening the curve" to avoid overwhelming the medical system. That is what authorities were talking about, so it's reasonable for him to look at that. Nevertheless, I think it's misleading to portray that issue as an equation to solve about infectiousness and R0, when it has everything to do with an insane social/health/financial policy of running hospitals like Toyota factories. The lesson to learn from that experience is not that we need more mathematicians with bigger supercomputers churning out fancier models. The real moral of the story is that we need hospitals and clinics to have slack so that we're ready when a school bus crashes or there's a pandemic, and also so that the doctors and nurses taking care of you routinely aren't rushed and burned out.
If equations are the solution to everything, then why not model that? Or better yet, make it happen and use your math to measure the real-world benefit.
December 8, 2023
Great book that provides insight on weather forecasting (among other things!) and it’s evolution from the 19th century to today.
there are some ideas and/or concepts here that I rarely see discussed scientifically (or honestly attempted objectively) in books for the public and enjoyed some of the more involved explanations. I enjoy learning/reading from experts (lookup Palmer’s background!) in any specific scientific field so reading Tim Palmer’s thoughts/ideas was a win for me!
I personally found some of the math a little bit too dense (that’s on me) and the extrapolation of what’s at the core of this book to free will and consciousness being at odds with recent books I’ve read on said topic (one that comes to mind is “The Hidden Spring” by Mark Solms). Palmer explicitly points out that he is off the path of his professional knowledge in tackling such questions and that his aim is to simply add something to the discussion. this I applaud and wish more science authors did this even though my understanding and the logic Palmer uses here in this book differ. this is strictly speaking for the very tail end of the book and the portion regarding free will and consciousness.
the book is divided into 3 parts; parts of I and all of part II were excellent reading for me. part III veers off of a bit from I & II but is prefaced as such while also being the smallest part of the book.
there are some ideas and/or concepts here that I rarely see discussed scientifically (or honestly attempted objectively) in books for the public and enjoyed some of the more involved explanations. I enjoy learning/reading from experts (lookup Palmer’s background!) in any specific scientific field so reading Tim Palmer’s thoughts/ideas was a win for me!
I personally found some of the math a little bit too dense (that’s on me) and the extrapolation of what’s at the core of this book to free will and consciousness being at odds with recent books I’ve read on said topic (one that comes to mind is “The Hidden Spring” by Mark Solms). Palmer explicitly points out that he is off the path of his professional knowledge in tackling such questions and that his aim is to simply add something to the discussion. this I applaud and wish more science authors did this even though my understanding and the logic Palmer uses here in this book differ. this is strictly speaking for the very tail end of the book and the portion regarding free will and consciousness.
the book is divided into 3 parts; parts of I and all of part II were excellent reading for me. part III veers off of a bit from I & II but is prefaced as such while also being the smallest part of the book.
December 26, 2024
Not a book for listening at 2x, which is what I did. Would have to go back with a more focused approach to really understand much of it. A great examples is this (paraphrased from next big idea club) "Suppose the universe evolved on a fractal. One consequence would be that the laws of physics would be unpredictable, but not indeterminate. However, the timeless nature of the fractal means that the Big Bang initial conditions would be no more fundamental than the state of the universe when the murderer committed his crime. In a self-referential way, this means that the murder’s actions determine the structure of the fractal geometry as much as the fractal geometry determines the actions of the murderer. Going one step further, if the laws of physics describe this fractal geometry, then the murderer’s actions contribute as much to the laws of physics as the laws of physics contribute to the murderer’s actions. From this perspective, the concepts of determinism and moral responsibility can mutually coexist. " Frankly, I don't get that. Seems like a distinction without a difference. Nonetheless, there's enough in here to spur one's imagination and the explanation of the 3 body problem is alone worth "the price of admission" so to speak.
February 5, 2025
Brian Clegg, the doyen of pop-sci reviewers has said this is the best popular science book he has read and it's hard to disagree. Consistently engaging, astonishing and mind-expanding, it opens roads less travelled than most popularisations. I learned more about climate science from it's one chapter than I had ever known before, and it introduced ideas I hadn't even imagined, such as a science of conflicts.
The last part is wildly speculative but nonetheless just as fascinating as the rest of the book.
The last part is wildly speculative but nonetheless just as fascinating as the rest of the book.
July 24, 2024
Fascinating dive into how modeling helps makes sense out of complex systems, from weather and climate, to conduct along borders. A very interesting take on how noise can in some ways improve modeling.
I also really enjoyed the final, speculative chapters on consciousness and free will. A bit of a departure from the rest of the book, but in line enough with the general themes that they didn’t seem out of place. I would love a deeper dive into this section.
I also really enjoyed the final, speculative chapters on consciousness and free will. A bit of a departure from the rest of the book, but in line enough with the general themes that they didn’t seem out of place. I would love a deeper dive into this section.
July 1, 2024
An interesting and thought-provoking book. Fortunately, I had some understanding of quantum issues, uncertainty, entropy and some maths as in simulated annealing. But, it lost me in a few places, and once you loose contact, it is hard to follow the subsequent reasoning. That aside, if you want some ‘heavy’ physics, maths and philosophy, you may find this interesting.
Displaying 1 - 30 of 44 reviews