"The weirdest people in the world?" Henrich, Heine, Norenzayan (2010)

A classic paper in BEHAVIORAL AND BRAIN SCIENCES (2010), cautioning any research based on human-subjects studies in design, engineering, and social sciences.  Good for our AME PhD methods course.


The weirdest people in the world? 

Joseph Henrich 
Steven J. Heine
Ara Norenzayan 
Department of Psychology, University of British Columbia, Vancouver

Abstract
Behavioral scientists routinely publish broad claims about human psychology and behavior in the world’s top journals based on samples drawn entirely from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies. Researchers – often implicitly – assume that either there is little variation across human populations, or that these “standard subjects” are as representative of the species as any other population. Are these assumptions justified? Here, our review of the comparative database from across the behavioral sciences suggests both that there is substantial variability in experimental results across populations and that WEIRD subjects are particularly unusual compared with the rest of the species – frequent outliers. The domains reviewed include visual perception, fairness, cooperation, spatial reasoning, categorization and inferential induction, moral reasoning, reasoning styles, self-concepts and related motivations, and the heritability of IQ. The findings suggest that members of WEIRD societies, including young children, are among the least representative populations one could find for generalizing about humans. Many of these findings involve domains that are associated with fundamental aspects of psychology, motivation, and behavior – hence, there are no obvious a priori grounds for claiming that a particular behavioral phenomenon is universal based on sampling from a single subpopulation. Overall, these empirical patterns suggests that we need to be less cavalier in addressing questions of human nature on the basis of data drawn from this particularly thin, and rather unusual, slice of humanity. We close by proposing ways to structurally re-organize the behavioral sciences to best tackle these challenges.

plus

Anthony Brandt (Music, Rice), gave a great talk at the Neuroscience of art, innovation and creativity conference dismantling some sweeping claims about universals in musical aesthetic experience and music.

methodology iterate Boulder NCAR study

Hi Brandon, 

As you develop trust thanks to Melissa (and Linda) 

It’d be very very useful to  observe live working sessions — rather than indirect report thru abstract questionnaires.

situations: either of single users working alone , or ensemble.  (In latter case maybe a work session in which you are participant is best)


• live event, write down your impressions of what was going on, afterward
choice: participate in the event as a user, vs as explicit fly on the wall (after you establish trust and permission from others to do this)

• live event with you as participant, with camera + audio recording the entire room to capture all inter-body activity NOT just the screen, head or hands! We will send video snips to  Satinder Gill @ Cambridge for advice o iterating the method.

Iteratively refining the methodology is key to abductive science.


• User views recording with you and does talk aloud: what was she thinking, what was she / peers doing?  
Easiest way to do this is to recurse: play video on Quicktime Player and simply create a second screen grab movie in Quicktime Player 

These are known and effective  techniques in HCI user  experience / design research.  Let's apply them :)


As for trust, I’d suggest that we position your work NOT to replace the "pro's" work with supercomputer simulations, but as entirely different genus with different intent and distinct features:

INTENT:
(1) exploring general features of models
(2) qualitative speculation, backed by quantitative models
(3) more as an instrument for scaffolding ensemble conversation than for intense individual work

FEATURES:

(1) Realtime 
(2) Sketch , minimum power needed to scaffold live scientific creation rather than compete on resolution.
(3) NOT merely educational toy, but as prop for cognitive research in how knowledge is collectively created in improvisatory but highly structured and skilled ways.

Have fun!
Xin Wei

methodology iterate Boulder NCAR study

Hi Brandon, 

As you develop trust thanks to Melissa (and Linda) 

It’d be very very useful to  observe live working sessions — rather than indirect report thru abstract questionnaires.

situations: either of single users working alone , or ensemble.  (In latter case maybe a work session in which you are participant is best)


• live event, write down your impressions of what was going on, afterward
choice: participate in the event as a user, vs as explicit fly on the wall (after you establish trust and permission from others to do this)

• live event with you as participant, with camera + audio recording the entire room to capture all inter-body activity NOT just the screen, head or hands! We will send video snips to  Satinder Gill @ Cambridge for advice o iterating the method.

Iteratively refining the methodology is key to abductive science.


• User views recording with you and does talk aloud: what was she thinking, what was she / peers doing?  
Easiest way to do this is to recurse: play video on Quicktime Player and simply create a second screen grab movie in Quicktime Player 

These are known and effective  techniques in HCI user  experience / design research.  Let's apply them :)


As for trust, I’d suggest that we post your work NOT to replace but entirely different genus un two ways:


(1) Realtime
(2) SKETCH , minimum power needed to scaffold live scientific creation rather than compete on resolution.
(3) NOT merely educational toy, but as prop for cognitive research, meta-science

Have fun!
Xin Wei

methodology iterate Boulder NCAR study

Hi Brandon, 

As you develop trust thanks to Melissa (and Linda) 

It’d be very very useful to  observe live working sessions — rather than indirect report thru abstract questionnaires.

situations: either of single users working alone , or ensemble.  (In latter case maybe a work session in which you are participant is best)


• live event, write down your impressions of what was going on, afterward
choice: participate in the event as a user, vs as explicit fly on the wall (after you establish trust and permission from others to do this)

• live event with you as participant, with camera + audio recording the entire room to capture all inter-body activity NOT just the screen, head or hands! We will send video snips to  Satinder Gill @ Cambridge for advice o iterating the method.

Iteratively refining the methodology is key to abductive science.


• User views recording with you and does talk aloud: what was she thinking, what was she / peers doing?  
Easiest way to do this is to recurse: play video on Quicktime Player and simply create a second screen grab movie in Quicktime Player 

These are known and effective  techniques in HCI user  experience / design research.  Let's apply them :)


As for trust, I’d suggest that we post your work NOT to replace but entirely different genus un two ways:


(1) Realtime
(2) SKETCH , minimum power needed to scaffold live scientific creation rather than compete on resolution.
(3) NOT merely educational toy, but as prop for cognitive research, meta-science

Have fun!
Xin Wei

methodology iterate Boulder NCAR study

Hi Brandon, 

As you develop trust thanks to Melissa (and Linda) 

It’d be very very useful to  observe live working sessions — rather than indirect report thru abstract questionnaires.

situations: either of single users working alone , or ensemble.  (In latter case maybe a work session in which you are participant is best)


• live event, write down your impressions of what was going on, afterward
choice: participate in the event as a user, vs as explicit fly on the wall (after you establish trust and permission from others to do this)

• live event with you as participant, with camera + audio recording the entire room to capture all inter-body activity NOT just the screen, head or hands! We will send video snips to  Satinder Gill @ Cambridge for advice o iterating the method.

Iteratively refining the methodology is key to abductive science.


• User views recording with you and does talk aloud: what was she thinking, what was she / peers doing?  
Easiest way to do this is to recurse: play video on Quicktime Player and simply create a second screen grab movie in Quicktime Player 

These are known and effective  techniques in HCI user  experience / design research.  Let's apply them :)


As for trust, I’d suggest that we post your work NOT to replace but entirely different genus un two ways:


(1) Realtime
(2) SKETCH , minimum power needed to scaffold live scientific creation rather than compete on resolution.
(3) NOT merely educational toy, but as prop for cognitive research, meta-science

Have fun!
Xin Wei

methodology iterate Boulder NCAR study

Hi Brandon, 

As you develop trust thanks to Melissa (and Linda) 

It’d be very very useful to  observe live working sessions — rather than indirect report thru abstract questionnaires.

situations: either of single users working alone , or ensemble.  (In latter case maybe a work session in which you are participant is best)


• live event, write down your impressions of what was going on, afterward
choice: participate in the event as a user, vs as explicit fly on the wall (after you establish trust and permission from others to do this)

• live event with you as participant, with camera + audio recording the entire room to capture all inter-body activity NOT just the screen, head or hands! We will send video snips to  Satinder Gill @ Cambridge for advice o iterating the method.

Iteratively refining the methodology is key to abductive science.


• User views recording with you and does talk aloud: what was she thinking, what was she / peers doing?  
Easiest way to do this is to recurse: play video on Quicktime Player and simply create a second screen grab movie in Quicktime Player 

These are known and effective  techniques in HCI user  experience / design research.  Let's apply them :)


As for trust, I’d suggest that we post your work NOT to replace but entirely different genus un two ways:


(1) Realtime
(2) SKETCH , minimum power needed to scaffold live scientific creation rather than compete on resolution.
(3) NOT merely educational toy, but as prop for cognitive research, meta-science

Have fun!
Xin Wei