View previous topic :: View next topic |
Author |
Message |
huffdaddy
Joined: 25 Nov 2005
|
Posted: Wed Sep 13, 2006 5:38 am Post subject: the voracity of science |
|
|
http://www.newscientist.com/article.ns?id=dn7915
Quote: |
Most scientific papers are probably wrong
Most published scientific research papers are wrong, according to a new analysis. Assuming that the new paper is itself correct, problems with experimental and statistical methods mean that there is less than a 50% chance that the results of any randomly chosen scientific paper are true.
John Ioannidis, an epidemiologist at the University of Ioannina School of Medicine in Greece, says that small sample sizes, poor study design, researcher bias, and selective reporting and other problems combine to make most research findings false. But even large, well-designed studies are not always right, meaning that scientists and the public have to be wary of reported findings. |
|
|
Back to top |
|
 |
gang ah jee

Joined: 14 Jan 2003 Location: city of paper
|
Posted: Wed Sep 13, 2006 5:49 am Post subject: Re: the voracity of science |
|
|
huffdaddy wrote: |
John Ioannidis, an epidemiologist at the University of Ioannina School of Medicine in Greece |
I'm no scientist, but I'd suggest that he's probably guilty of a lot of mistakes that he claims are present in the papers that he surveyed. In any case, as the article points out, research reports aren't read as gospel.
Also, is p=<0.5 convention outside of the life sciences? |
|
Back to top |
|
 |
daskalos
Joined: 19 May 2006 Location: The Road to Ithaca
|
Posted: Wed Sep 13, 2006 5:52 am Post subject: |
|
|
Voracity? Does science really eat that much? It should be ashamed of itself for such gluttony, but you know science -- it's probably not.  |
|
Back to top |
|
 |
huffdaddy
Joined: 25 Nov 2005
|
Posted: Wed Sep 13, 2006 6:18 am Post subject: |
|
|
daskalos wrote: |
Voracity? |
Joke from the old Usenet days. |
|
Back to top |
|
 |
daskalos
Joined: 19 May 2006 Location: The Road to Ithaca
|
Posted: Wed Sep 13, 2006 6:25 am Post subject: |
|
|
Oh. Damnit I hate ignorance. Especially when it's mine.  |
|
Back to top |
|
 |
mindmetoo
Joined: 02 Feb 2004
|
Posted: Wed Sep 13, 2006 4:30 pm Post subject: |
|
|
Humans make mistakes and are capable of fraud. If it was so bad, rockets would blow up on launch pads, nuclear power plants would explode, computers wouldn't turn on. When research is based on previous research, if a study is really flawed, it becomes obvious. We could not get from scientific paper to space station.
It's telling the study is published in a widely read science magazine. I'm just curious how many other organizations publish such self criticism so widely? Does the Mormon church quickly distribute a study that says, despite a core tenant of its faith, its Indian members, in fact, have no genes that can be traced back to the Middle East? |
|
Back to top |
|
 |
atlhockey

Joined: 20 Aug 2006 Location: Jeonju City
|
Posted: Wed Sep 13, 2006 4:47 pm Post subject: |
|
|
As mindmetoo pointed out, science is often enough wrong or "off" a bit, but it is also self-correcting. That's why no study is taken as valid until it's been not only peer-reviewed but replicated several times by independent researchers. And often not even until the original idea has been expanded upon in research as well.
That "50%" number is sort of telling too. In chemistry and physics research, when you describe a phenomenon, it should occur pretty much every time under the conditions you define, because particles are predictable. When you get into life science and especially social science, even if you find a significant effect, with variability between humans it's hardly going to be "true" all the time. The threshhold for significance in psychology is much lower than in chemistry. I'm not sure if this study looked at differences between fields, but it's going to be grossly inaccurate if it didn't. |
|
Back to top |
|
 |
laogaiguk

Joined: 06 Dec 2005 Location: somewhere in Korea
|
Posted: Wed Sep 13, 2006 4:49 pm Post subject: |
|
|
atlhockey wrote: |
That "50%" number is sort of telling too. In chemistry and physics research, when you describe a phenomenon, it should occur pretty much every time under the conditions you define, because particles are predictable. When you get into life science and especially social science, even if you find a significant effect, with variability between humans it's hardly going to be "true" all the time. The threshhold for significance in psychology is much lower than in chemistry. I'm not sure if this study looked at differences between fields, but it's going to be grossly inaccurate if it didn't. |
I was thinking the same thing. Comparing Chemistry and Sociology just isn't fair. |
|
Back to top |
|
 |
gang ah jee

Joined: 14 Jan 2003 Location: city of paper
|
Posted: Wed Sep 13, 2006 5:06 pm Post subject: |
|
|
laogaiguk wrote: |
I was thinking the same thing. Comparing Chemistry and Sociology just isn't fair. |
Right, which is why I though it was strange that the article claims that statistical significance is achieved "if the odds are only 1 in 20 that the result could be pure chance" or p=<.5. I understood that outside of social sciences and medicine signficance was set much lower at p=<.1 or less. |
|
Back to top |
|
 |
Thunndarr

Joined: 30 Sep 2003
|
Posted: Wed Sep 13, 2006 5:19 pm Post subject: |
|
|
mindmetoo wrote: |
Does the Mormon church quickly distribute a study that says, despite a core tenant of its faith, its Indian members, in fact, have no genes that can be traced back to the Middle East? |
Tsk tsk tsk. I know you know better. |
|
Back to top |
|
 |
RACETRAITOR
Joined: 24 Oct 2005 Location: Seoul, South Korea
|
Posted: Wed Sep 13, 2006 6:48 pm Post subject: |
|
|
Science is not wrong. Scientific theories and hypotheses are sometimes wrong at certain stages, but there is a very simple method of correcting them.
I was just thinking the other day, two of the most cherished names in science over the last century or so, Albert Einstein and Sigmund Freud, have both nearly been totally disproven out of existence. |
|
Back to top |
|
 |
huffdaddy
Joined: 25 Nov 2005
|
Posted: Wed Sep 13, 2006 7:22 pm Post subject: |
|
|
Found the original article. Looks like it's more of a theoretical statistical analysis of biomedical research. So yes, his findings probably have a good chance of being wrong. Although he's definitely on the right track.
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1182327 |
|
Back to top |
|
 |
yodanole
Joined: 02 Mar 2003 Location: La Florida
|
Posted: Mon Sep 18, 2006 7:24 am Post subject: |
|
|
The only way to get new ideas out in the public eye academically, is to take a chance. Darwin waited several years, but his ideas were so radical he didn't really have to worry about being pre-empted by ongoing research. |
|
Back to top |
|
 |
n3ptne
Joined: 14 Sep 2005 Location: Poh*A*ng City
|
Posted: Mon Sep 18, 2006 9:45 am Post subject: |
|
|
It is admittedly true that Science is only relevant, or true, under the paticular variables that are defined...
However, the aim of Science is to subjugate all aspects of everything, be they social or technical, into underlying and fundamental concepts that include all variables and under which all things are always "true".
The fact Science is not there yet, in fact not even close to being there yet, is moot. A scientist simply acknowledges that "there" exists, and that the level of scientific knowledge is steadily progressing towards it.
Theories and hypothesis are all self-admittedly flawed, and constantly adjusting to compensate for new experimentally verified data. This doesn't suggest that Science is wrong or even unreliable. |
|
Back to top |
|
 |
on_me_head_son

Joined: 26 May 2004 Location: Seoul
|
Posted: Mon Sep 18, 2006 6:17 pm Post subject: |
|
|
yodanole wrote: |
The only way to get new ideas out in the public eye academically, is to take a chance. Darwin waited several years, but his ideas were so radical he didn't really have to worry about being pre-empted by ongoing research. |
Not a criticism but an interesting fact!
Darwin was forced to go public with his theory earlier than intended as Alfred Russel Wallace was also developing the same theory. Instead of competing with each other they jointly presented the paper
'On the Tendency of Species to form Varieties; and on the Perpetuation of Varieties and Species by Natural Means of Selection' to the Linnean Society a year before Darwin published 'The Origin of Species'. |
|
Back to top |
|
 |
|