I was reading Maggie’s Farm Wednesday (as I do every day, and you should), and saw a link to an excellent article on simulations in American Thinker. Well, actually, the article is about the problem with global warming — climatological — simulations. It doesn’t offer a lot of information on simulations themselves, what they are, and how they work, and that’s why I’m writing this.
By the way, simulations are a real pain in the *ss to teach to undergrads. Trust me. Despite all the fuzzy math nonsense (or perhaps because of it), undergrads are very uncomfortable with uncertainty. But that’s another topic for another time.
Think of a (mathematical) function. The function takes values as input, processes the values, and spits out a value. A simulation is a lot like a function, with three crucial exceptions. First, a simulation has no real input values, but uses instead simulated input. Second, because the input is simulated, instead of running the simulation only once (as you would with a function), you have to run a simulation many times (iterations), then statistically analyze the output — the simulated output, calculated from the simulated input. Third, because the output of the simulation is simulated output and there are multiple outputs (because the simulation must be run many times), it must be statistically analyzed for reliability and the multiple results must be analyzed statistically.
A function is certain. A simulation is uncertain.
So how does a simulation simulate input values? Usually, by taking real data, analyzing it statistically to determine its frequency and distribution, then using statistics to generate input values using the same frequency and distribution. Note that in order for this to work, one must assume that the data are stable, that is, that the frequency and distribution of the data will not change over time.
The part of the simulation that corresponds to a function is known as the model. Obviously, only an accurate model can produce reliable results (output), and the more accurate the model, the more reliable the results.
Simulations can be powerful tools for making predictions, and are used in many fields, including business. However, because simulations use simulated (that is, not real) input and result in simulated (that is, not real) output, they have no evidentiary power — that is, you cannot use a simulation as evidence for anything, nor can you call the output of a simulation (real) data.
Let’s look at two different simulations.
The first is a business model. The owner of a bakery wants to see if increasing the number of ovens will increase his sales (and therefore, profits). The input of the simulation will be his sales data over the last year. The model will take into account the production per oven, fixed and variable costs, etc., and will generate the revenue and profit for several different numbers of ovens. The consultant can then statistically analyze the results for each number of ovens, and give the owner an answer.
Note that the model is based on hard data. Production per oven, fixed and variable costs, for example, are all absolute values. The model does not need to estimate anything, because all of the data are known.
Now let’s look at a climatological simulation, which generates the frequency and distribution of climatological data, and uses it to simulate input data (and will eventually output climatological data). Given the instability of the climate, this is problematic (if the data used to create the input are not stable, the simulation cannot be reliable). But the real problem — and difference from the business simulation — is in the model.
In the business model as I pointed out above, the data are all absolute, that is, known. In the climatological model, the data in the model are estimated, because climatology is a young science, and nobody is certain how, say, CO2 or water vapor affects the climate. A climatologist, given enough data, can make an educated guess. But it’s still estimated.
You can use estimated data in your model, but doing so inserts another layer of uncertainty into the simulation results. Let me explain.
Since the data in our model are estimated, we must use statistics to determine their validity. Since Mr. Lewis uses dice, I will as well, sticking for the sake of simplicity and clarity to rolling one die.
If you roll a die, the probability that you will roll, say, a 1 is 1/6. If you roll the die a second time, the probability that you will roll, say, a second 1 is 1/6 * 1/6, or 1/36. If you roll the die a third time, the probability that you will roll, say, a third 1 is 1/6 * 1/6 * 1/6, or 1/216, and so forth.
Estimated variables in a simulation model must be treated in the same way as rolling a die, because each is uncertain, and each involves probability. Assuming that our climatologist is ethical, each of the estimated variables in his model should fall within the statistical norm of reliability, or be 95% reliable. Given the complexity of climatological models, hundreds of such estimated variables would be necessary, but for clarity’s sake, we will say the model includes only 50 such variables.
That means that the reliablility of the simulation model is 0.95^50 (0.95 raised to the fiftieth power), or 0.0769, or 7.7%. So even if we didn’t have the uncertainty of the simulated input (not to mention the additional uncertainty of assuming that the data are stable), even if there were no uncertainty in our ouputs themselves, our simulation results would only 7.7% reliable.
Climatological simulations cannot be taken very seriously. They can certainly never be taken as evidence or proof, as no simulation can be, because they are simulations. They aren’t real.
What disturbs me about all this global warming warfare is that the climatologists know this. They know that their models have no evidentiary power. Yet, they disingenuously claim the reverse. This isn’t science. It’s politics. It’s dishonest. And it’s a breach of professional ethics and integrity.
This whole global warming thing may be perfectly valid, of course. But simulations won’t tell us one way or the other.