Announcement

Collapse

Natural Science 301 Guidelines

This is an open forum area for all members for discussions on all issues of science and origins. This area will and does get volatile at times, but we ask that it be kept to a dull roar, and moderators will intervene to keep the peace if necessary. This means obvious trolling and flaming that becomes a problem will be dealt with, and you might find yourself in the doghouse.

As usual, Tweb rules apply. If you haven't read them now would be a good time.

Forum Rules: Here
See more
See less

Scientists Are Wrong All the Time, and That’s Fantastic

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Scientists Are Wrong All the Time, and That’s Fantastic

    Source: Wired.com


    On February 28, 1998, the eminent medical journal The Lancet published an observational study of 12 children: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive development disorder in children. It might not sound sexy, but once the media read beyond the title, into the study’s descriptions of how those nasty-sounding symptoms appeared just after the kids got vaccinated, the impact was clear: The measles-mumps-rubella vaccine can cause autism.

    This was the famous study by Andrew Wakefield, the one that many credit with launching the current hyper-virulent form of anti-vaccination sentiment. Wakefield is maybe the most prominent modern scientist who got it wrong—majorly wrong, dangerously wrong, barred-from-medical-practice wrong.

    But scientists are wrong all the time, in far more innocuous ways. And that’s OK. In fact, it’s great.

    When a researcher gets proved wrong, that means the scientific method is working. Scientists make progress by re-doing each other’s experiments—replicating them to see if they can get the same result. More often than not, they can’t. “Failure to reproduce is a good thing,” says Ivan Oransky, co-founder of Retraction Watch. “It happens a lot more than we know about.” That could be because the research was outright fraudulent, like Wakefield’s. But there are plenty of other ways to get a bum result—as the Public Libary of Science’s new collection of negative results, launched this week, will highlight in excruciating detail.

    You might have a particularly loosey-goosey postdoc doing your pipetting. You might have picked a weird patient population that shows a one-time spike in drug efficacy. Or you might have just gotten a weird statistical fluke. No matter how an experiment got screwed up, “negative results can be extremely exciting and useful—sometimes even more useful than positive results,” says John Ioannidis, a biologist at Stanford who published a now-famous paper suggesting that most scientific studies are wrong.

    The problem with science isn’t that scientists can be wrong: It’s that when they’re proven wrong, it’s way too hard for people to find out.

    Negative results, like the one that definitively refuted Wakefield’s paper, don’t make the news. Fun game: Bet you can’t name the lead author of that paper. (It’s okay, neither could we. But keep reading to find out!) It’s way easier for journalists to write a splashy headline about a provocative new discovery (guilty) than a glum dismissal of yet another hypothesis, and scientific journals play into that bias all the time as they pick studies to publish.

    “All of the incentives in science are aligned against publishing negative results or failures to replicate,” says Oransky. Scientists feel pressure to produce exciting results because that’s what big-name journals want—it doesn’t look great for the covers of Science and Nature to scream “Whoops, we were wrong!”—and scientists desperately need those high-profile publications to secure funding and tenure. “People are forced to claim significance, or something new, extravagant, unusual, and positive,” says Ioannidis.

    Plus, scientists don’t like to step on each other’s toes. “They feel a lot of pressure not to contradict each other,” says Elizabeth Iorns, the CEO of Science Exchange. “There’s a lot of evidence that if you do that, it’ll be negative for your career.”

    When the politics of scientific publishing prevent negative results from getting out there, science can’t advance, and potentially dangerous errors—whether due to fraud or an honest mistake—go unchecked. Which is why lots of scientific publications, including PLOS, have recently begun to emphasize reproducibility and negative results.

    Big-name journals have said they want to make data more transparent and accessible, so scientists can easily repeat analyses. Others, like the Journal of Negative Results in BioMedicine, are devoted to publishing only negative results. PLOS One’s collection of negative, null, and inconclusive papers, called The Missing Pieces, is now putting the spotlight on papers that contradict previous findings. PLOS thought—and we agree—it’s time to give them the attention they deserve. Negative results, step up:

    Vaccines and Autism. Wakefield’s 1998 study reported a possible link between the measles-mumps-rubella vaccine and the onset of autism in children with gastrointestinal problems. More than 20 studies have since ruled out any connection, but they didn’t focus on children with gastrointestinal problems. So in 2008, researchers led by Mady Hornig conducted a case study that did. Again, they found no evidence linking the vaccine with autism.

    Psychic Ability. In 2011, Daryl Bem, a psychologist at Cornell, conducted nine experiments that seemed to suggest people could be psychic. Extraordinary claims require extraordinary evidence, so researchers replicated one of the experiments three times in 2012. As the newer paper states, “all three replication attempts failed to produce significant effects and thus do not support the existence of psychic ability.” Bummer.

    Priming and Performance. In a highly cited study from 2001, John Bargh, a psychologist at Yale, found that people who were exposed to words like “strive” or “attain,” did better on a cognitive task. Researchers did two experiments in 2013 to reproduce the original findings. They could not.

    Running Out of Self Control. Some research has suggested that when trying to exercise self-control, you really are exercising—like flexing a muscle. After a while, you get too tired and can no longer control yourself. But a 2014 study wasn’t able to reproduce this effect at all.

    Buddies and Memory. In 2007, Sid Horton, a psychologist at Northwestern University, found that people who associated an object with a specific person were able to name a picture of that object faster when the person was present. But in a valiant display of self-abnegation, Horton tried to reproduce his results in 2014—and failed.


    Source

    © Copyright Original Source



    It's a wonderful thing that we now have places to go to find these negative result papers. Like the author stated, the public never easily finds out when science is wrong. Now hopefully the media will be more willing to correct their erroneous scientific reporting since they have these resources.
    Last edited by Jesse; 02-28-2015, 04:38 AM.
    "Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies. The robber baron's cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience." ― C.S. Lewis, God in the Dock: Essays on Theology (Making of Modern Theology)

  • #2
    While it's great to make it easier to publish negative results, and while it's inevitable that science is sometimes wrong, there's no reason science has to be wrong as often as it is. There are broad fields of science where most new results are probably wrong, and that's just ridiculous. More rigorous statistical standards would help a lot.

    Comment


    • #3
      Originally posted by sfs1 View Post
      While it's great to make it easier to publish negative results, and while it's inevitable that science is sometimes wrong, there's no reason science has to be wrong as often as it is. There are broad fields of science where most new results are probably wrong, and that's just ridiculous. More rigorous statistical standards would help a lot.
      You have to be more specific here, a broad negative brush has little meaning unless you can do better then 'there is no reason science is wrong as it often is.'

      Are you a scientist that can provide some basis for the above?

      The bottom line is out of 10s of thousands of research papers and projects every year, science is not often wrong. There is also an excellent track record of science finding and correcting errors. Some scientific knowledge is not totally correct or accurate, because of the limited knowledge, but when new knowledge and research becomes available science's self corrective methods correct previous research. Scientists are the ones that correct errors and problems in science not philosophers and theologians.

      You, of course, can cite errors and problems, but not significant when compared to many published works where science is right on.
      Glendower: I can call spirits from the vasty deep.
      Hotspur: Why, so can I, or so can any man;
      But will they come when you do call for them? Shakespeare’s Henry IV, Part 1, Act III:

      go with the flow the river knows . . .

      Frank

      I do not know, therefore everything is in pencil.

      Comment


      • #4
        Originally posted by shunyadragon View Post
        You have to be more specific here, a broad negative brush has little meaning unless you can do better then 'there is no reason science is wrong as it often is.'

        Are you a scientist that can provide some basis for the above?

        The bottom line is out of 10s of thousands of research papers and projects every year, science is not often wrong. There is also an excellent track record of science finding and correcting errors. Some scientific knowledge is not totally correct or accurate, because of the limited knowledge, but when new knowledge and research becomes available science's self corrective methods correct previous research. Scientists are the ones that correct errors and problems in science not philosophers and theologians.

        You, of course, can cite errors and problems, but not significant when compared to many published works where science is right on.
        Yes, I'm a scientist (see here, although the physics citation count is highly inflated), and yes I can provide some basis. The most important relevant reference is this. More details on hypothesis testing, and how many researchers use it badly, can be found here. This recent paper, besides providing suggestions for improvement, also gives a quick review of the state of knowledge on the subject.

        Comment


        • #5
          What the general public doesn't understand well is that, in general, natural science gives the best possible explanation with the extant data.

          There are hypotheses/theories that have a probability of correctness close to 100%, others are more tentative, but they're testable/falsifiable and do give explanations that fit the data, and are not WAGs.

          K54

          Comment


          • #6
            Originally posted by Jesse View Post
            It's a wonderful thing that we now have places to go to find these negative result papers. Like the author stated, the public never easily finds out when science is wrong. Now hopefully the media will be more willing to correct their erroneous scientific reporting since they have these resources.
            Interesting article. I wonder how much of the problem has to do more with how the media chooses to sensationalize headlines. I don't know how many times I've read a headline that says "scientists cure cancer!!", and then you read the article and you find out they were able to successfully treat a certain type of cancer in some mice under certain conditions, or you find out that the cancer is destroyed...along with the host. Or how many times in the last few years have we seen headlines strongly hinting at having found life on other planets, and when you actually read the article, its mostly guess work about planets that may or may not be life supporting under the right circumstances?

            Are scientists themselves encouraged to make their results known to the mainstream, or, is that something that certain journals do, or is it something that the university and labs the scientists work for do? Or...who's in charge of that? How much financial incentive through grants, and whatnot, is there for scientist to publish positive results (looking back through the article, I guess they do mention that "scientists desperately need those high-profile publications to secure funding and tenure"). Is there a way to curb or put a stop to that? Should there be?

            I also notice that most of the negative results from later experiments highlighted in the article are from psychological studies. Is there a higher or lower rate of negative findings in certain branches than there are in others? For instance, is physics an area with a higher degree of reliability in initial experiments, compared to, say, biology?

            Comment


            • #7
              Originally posted by klaus54 View Post
              What the general public doesn't understand well is that, in general, natural science gives the best possible explanation with the extant data.

              There are hypotheses/theories that have a probability of correctness close to 100%, others are more tentative, but they're testable/falsifiable and do give explanations that fit the data, and are not WAGs.

              K54
              What's a WAG?

              Comment


              • #8
                Originally posted by Adrift View Post
                What's a WAG?
                If I told you, I'd have to kill you.



                K54

                Comment


                • #9
                  Originally posted by Adrift View Post
                  What's a WAG?
                  Wild Ass Guess. As opposed to SWAG -- a Scientific Wild Ass Guess.

                  I'm always still in trouble again

                  "You're by far the worst poster on TWeb" and "TWeb's biggest liar" --starlight (the guy who says Stalin was a right-winger)
                  "Overall I would rate the withdrawal from Afghanistan as by far the best thing Biden's done" --Starlight
                  "Of course, human life begins at fertilization that’s not the argument." --Tassman

                  Comment


                  • #10
                    Originally posted by rogue06 View Post
                    Wild Ass Guess. As opposed to SWAG -- a Scientific Wild Ass Guess.
                    And then there's swag, which is just pure abomination.

                    Comment


                    • #11
                      Originally posted by Adrift View Post
                      Interesting article. I wonder how much of the problem has to do more with how the media chooses to sensationalize headlines. I don't know how many times I've read a headline that says "scientists cure cancer!!", and then you read the article and you find out they were able to successfully treat a certain type of cancer in some mice under certain conditions, or you find out that the cancer is destroyed...along with the host. Or how many times in the last few years have we seen headlines strongly hinting at having found life on other planets, and when you actually read the article, its mostly guess work about planets that may or may not be life supporting under the right circumstances?

                      Are scientists themselves encouraged to make their results known to the mainstream, or, is that something that certain journals do, or is it something that the university and labs the scientists work for do? Or...who's in charge of that? How much financial incentive through grants, and whatnot, is there for scientist to publish positive results (looking back through the article, I guess they do mention that "scientists desperately need those high-profile publications to secure funding and tenure"). Is there a way to curb or put a stop to that? Should there be?

                      I also notice that most of the negative results from later experiments highlighted in the article are from psychological studies. Is there a higher or lower rate of negative findings in certain branches than there are in others? For instance, is physics an area with a higher degree of reliability in initial experiments, compared to, say, biology?
                      Time for this again

                      science-news-cycle.jpg

                      I'm always still in trouble again

                      "You're by far the worst poster on TWeb" and "TWeb's biggest liar" --starlight (the guy who says Stalin was a right-winger)
                      "Overall I would rate the withdrawal from Afghanistan as by far the best thing Biden's done" --Starlight
                      "Of course, human life begins at fertilization that’s not the argument." --Tassman

                      Comment


                      • #12
                        Originally posted by rogue06 View Post
                        Time for this again

                        [ATTACH=CONFIG]4252[/ATTACH]
                        Looks about right.

                        Comment


                        • #13
                          Originally posted by Chrawnus View Post
                          And then there's swag, which is just pure abomination.
                          Don't you look down on my generation.






























                          Swag.
                          "Kahahaha! Let's get lunatic!"-Add LP
                          "And the Devil did grin, for his darling sin is pride that apes humility"-Samuel Taylor Coleridge
                          Oh ye of little fiber. Do you not know what I've done for you? You will obey. ~Cerealman for Prez.

                          Comment


                          • #14
                            fat fingers and slow brain

                            Originally posted by Adrift View Post
                            Looks about right.
                            Symptomatic of the public's misunderstanding of natural scientific method is the confusion of the word "proof". In science and police work, proof means "evidence", in mathematics/logic it mean 100% certainty within an axiom system.

                            Also, "theory" is misunderstood. "Theory" in police work and common parlance means either a hunch, WAG, or what scientists would call an "hypothesis."

                            OTOH, a scientific theory is a well-tested collection of hypotheses on a common theme, or a "great theory" which is the best available explanation of a broad range of data, e.g., Plate Tectonics and biological evolution by genetic diversity, natural selection, genetic drift, natural selection, changing biomes over time, extinction producing vacated niches, adaptation of other species to these niches, catastrophes, ...

                            Of course anti-evolutionists LOVE the "only a theory" spin, spoken out of either pure ignorance or outright deceit by their "leaders" to the credulous.

                            K54
                            Last edited by klaus54; 02-28-2015, 05:35 PM.

                            Comment


                            • #15
                              Originally posted by klaus54 View Post
                              Symptomatic of the public's misunderstanding of natural scientific method is the confusion of the word "proof". In science and police work, proof means "evidence", in mathematics/logic it mean 100% certainty within an axiom system.

                              Also, "theory" is misunderstood. "Theory" in police work and common parlance means either a hunch, WAG, or what scientists would call an "hypothesis."

                              OTOH, a scientific theory is a well-tested collection of hypotheses on a common theme, or a "great theory" which is the best available explanation of a broad range of data, e.g., Plate Tectonics and biological evolution by genetic diversity, natural selection, genetic drift, natural selection, changing biomes over time, extinction producing vacated niches, adaptation of other species to these niches, catastrophes, ...

                              Of course anti-evolutionists LOVE the "only a theory" spin, spoken out of either pure ignorance or outright deceit by their "leaders" to the credulous.

                              K54
                              Started a thread explaining What "theory" means in science awhile back.

                              I'm always still in trouble again

                              "You're by far the worst poster on TWeb" and "TWeb's biggest liar" --starlight (the guy who says Stalin was a right-winger)
                              "Overall I would rate the withdrawal from Afghanistan as by far the best thing Biden's done" --Starlight
                              "Of course, human life begins at fertilization that’s not the argument." --Tassman

                              Comment

                              Related Threads

                              Collapse

                              Topics Statistics Last Post
                              Started by Hypatia_Alexandria, 03-18-2024, 12:15 PM
                              48 responses
                              135 views
                              0 likes
                              Last Post Sparko
                              by Sparko
                               
                              Started by Sparko, 03-07-2024, 08:52 AM
                              16 responses
                              74 views
                              0 likes
                              Last Post shunyadragon  
                              Started by rogue06, 02-28-2024, 11:06 AM
                              6 responses
                              47 views
                              0 likes
                              Last Post shunyadragon  
                              Working...
                              X