Why Error Bars Matter

[Paprika]

I will be using a simple scenario to show why error bars matter: you hang a thermometer by the window and for three days in a row you measure the temperature at noon. The reading for all 3 days is 25 degrees. Are you justified to say that the noon temperature for all 3 days was the same, or that there was a constant trend? No.

All measurement with instruments are imprecise, and therefore have an asssociated uncertainty. Let's say the uncertainty of your thermometer is 1 degree. Your results will be as such, with the cap indicating the uncertainty of the data points:



This data could represent many possible situations.


For example, the blue line where there was no change. Or the red line, where there was a decrease at a constant rate. Or the green line: an increase at a constant rate. Or the yellow line: an increase followed by a decrease. You get the idea. Assuming no other information about the temprature, the data allows for many such contradicting interpretations of trends, all which are unjustified.

So what are you justified in concluding? Assuming that there was no additional error (for example, you read the thermometer properly to prevent parallax error) you can say that the noon temperature remained between 26 and 24 degrees. You can say that the temperature difference from the first day to the third day must have been at most 2 degrees, and other things like that.

If using the same thermometer to make another reading on the fourth day which registers 28 degrees (with an uncertainty of 1 degree), it is justified to say that the noon temperature of the fourth day was higher than the previous three days:



It should be obvious why.

[shunyadragon]

OK, what now?

[Jonathandavid]

Actually, there is a subtle but critical misunderstanding about theory choice behind Paprika's post. If I can find the time I will post on it.

[Paprika]

OK, what now?

Have I made any error in the original post?

[Roy]

Actually, there is a subtle but critical misunderstanding about theory choice behind Paprika's post.

And a maths error too. Maybe other problems depending on whether the error bars are absolute or the 95% kind.

Roy

P.S. Paprika, since you've asked: "You can say that the temperature difference from the first day to the third day must have been at most 1 degree" should be 2 degrees.

[Paprika]

P.S. Paprika, since you've asked: "You can say that the temperature difference from the first day to the third day must have been at most 1 degree" should be 2 degrees.

Oops.

[shunyadragon]

Have I made any error in the original post?

Not an error, but an incomplete explanation of what your want. Do you understand the statistics behind what you are requested, and failed to give an adequate explanation of what you want. There is also the accuracy of the equipment which may be expressed in a range of measurement, which your question is not clear on this issue.

[Jonathandavid]

Okay, here goes.

If error bars allow for different curves, then any of the curves that fit within the error bars are acceptable hypotheses. However, this causes a logical problem, because we can fit an infinite amount of curves in that area. Do we have to eliminate an infinite amount of hypotheses?

Philosopher Nelson Goodman came up with a famous thought experiment. Imagine that you're doing research on emeralds. You find an emerald, and note its color. It is green. The next you find is also green, as is the third one you find, etc. The obvious hypothesis one would conclude here is: "All emeralds are green". But let's consider an alternative hypothesis: "All emeralds are grue". This means that all emeralds you discover after some point t in the future will appear blue rather than green; this is the property known as "grue". It implies that all emeralds found up to the moment of t will appear green. This hypothesis is actually corroborated by the evidence, and fits the data just as well. And since you can place point t at any moment in the future, there are an infinite amount of possible hypotheses.

This holds true for any conclusion that is reached by inductive reasoning. Induction is, simply put, drawing a conclusion by taking many observations and extrapolating a theory from that. After seeing a hundred swans that happened to be white, you would inductively conclude "all swans are white". The conclusion that the temperature has remained constant for the period measured after taking three measurements is certainly an example of induction.

So how does this translate to curves? Well, for every few points, you can fit an infinite amount of hypothetical curves through them. The thing is, you don't actually need the error bars to fit an infinite amount of hypotheses to a set of data points. It is possible to fit a line showing an average climbing trend exactly through the three points at 25 degrees C. Error bars do not logically increase the amount of possible curves that could fit the data, so they do not actually make the straight line a less likely solution. The chance that the flat line is correct is mathematically 0 anyway.

But what do the error bars mean then? Well, the biggest mistake Paprika makes is that he treats the error bars as a priori constraints for acceptable hypotheses. They are not, error bars are not used that way. While there are statistical tests that compare a set of data to a randomly generated alternative, the actual theories involved decide what hypotheses will be considered alternatives for theory choice (though I use the word "hypothesis" as a special instance of "theory", it is common in philosophy to leave out reference to a hypothesis altogether, because a theory is never completely logically proven).

When theories need to be compared, that's when the error bars come into play. They are tools in the practice of science. If someone actually wants to argue that temperature has been climbing in the example given by Paprika, the error bars allow to argue that, because they do not refute that hypothesis. But simply pointing out that you can also draw a climbing line within the boundaries of the error bars elicits a "so what?" response from scientists (as illustrated in this thread) because simply drawing a line is meaningless.

I am leaving open here how theories are constructed and refuted, my basic point is that error bars are not the space within which hypotheses are generated, but that they are tools for theory choice.

[Paprika]

I am leaving open here how theories are constructed and refuted, my basic point is that error bars are not the space within which hypotheses are generated, but that they are tools for theory choice.

Makes sense. I stand corrected.