Social scientists do actual peer-review and find 60% of social science results exaggerated

Intended for constructive conversations. Exhibits of polarizing tribalism will be deleted.
Locked
jacob
Site Admin
Posts: 15910
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
Contact:

Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by jacob »


Dragline
Posts: 4436
Joined: Wed Aug 24, 2011 1:50 am

Re: Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by Dragline »

I would have thought the figure would be even higher.

User avatar
GandK
Posts: 2059
Joined: Mon Sep 19, 2011 1:00 pm

Re: Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by GandK »

Lies, damn lies, and sociology. :lol:

vexed87
Posts: 1521
Joined: Fri Feb 20, 2015 8:02 am
Location: Yorkshire, UK

Re: Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by vexed87 »

:lol:

Chad
Posts: 3844
Joined: Fri Jul 23, 2010 3:10 pm

Re: Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by Chad »

It's not surprising given the subject and the difficulty of these types of studies.

luxagraf
Posts: 215
Joined: Tue Nov 26, 2013 4:32 pm
Contact:

Re: Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by luxagraf »

Reminds me of that line, "any field with the word 'science' in its name is guaranteed not to be a science."

Which I think I got from An Introduction to General Systems Thinking.

tonyedgecombe
Posts: 450
Joined: Thu Aug 30, 2012 2:11 pm
Location: Oxford, UK Walkscore: 3

Re: Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by tonyedgecombe »

A bit like any country with democratic in the name isn't democratic.

vexed87
Posts: 1521
Joined: Fri Feb 20, 2015 8:02 am
Location: Yorkshire, UK

Re: Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by vexed87 »

The biggest scourge in the sciences remains the data that doesn't get published because it contradicts the findings expected or preferred. This is a huge problem and always will be. It takes a special person to throw their life and soul into a piece of research, get one or two outliers which destroy the credibility of your findings and be completely honest in your papers. There is also selection bias from the publishers that means failed research doesn't get published nearly as often as it should. In reality a failed paper is just important as success as they are lessons learnt. The reason that data or failed research is omitted is partly due to the way that science is funded. You don't get research grants for failing to draw reliable results, so there's pressure to perform and produce findings with substance.

Tyler9000
Posts: 1758
Joined: Fri Jun 01, 2012 11:45 pm

Re: Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by Tyler9000 »

Lol at the Futurama reference.

I think there are two lessons here.

First, just because something is justified with numbers and a study does not mean it is true. The human condition is particularly complicated, so "proving" things is a shaky exercise.

Second, the root cause here is not the field being studied but the effect of outside human influence and confirmation bias. One should be careful not to believe this problem is isolated to the social sciences.

Dragline
Posts: 4436
Joined: Wed Aug 24, 2011 1:50 am

Re: Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by Dragline »

Here's a timely article and podcast addressing some of these issues:

http://fivethirtyeight.com/datalab/podc ... r-science/

User avatar
Ego
Posts: 6359
Joined: Wed Nov 23, 2011 12:42 am

Re: Social scientists do actual peer-review and find 60% of social science results exaggerated

Post by Ego »

Vox interviewed the guy in charge of the Reproducability Project responsible for this study.

http://www.vox.com/2015/8/27/9212161/ps ... eplication

Accurate replication is the problem. He says it is difficult to follow the original methods. So, it seems, this social science result is exaggerated.

Locked