what is responsive media?

Let me share a response to Brandon, because it may be more broadly useful. :)

(See “Resistance Is Fertile: Gesture and Agency in the Field of Responsive Media,” Configurations, 2002, 10:439–472)


I introduced the notion “responsive media”

after making the Gardens as prototype examples of what I called playspaces,

refusing to call them

      interactive spaces (refusing Shannon telementationalist turn-taking),

      or ambiente intelligente, (there is no intelligence “in the world” any more than there are numbers “in the world")

      or immersive environments (we are always and everywhere immersed)

 

I formed the Topological Media Lab in 2001 to study gesture and movement from 

experiential as well as computational perspectives, using what I called "responsive media."


The touchstone models for responsive media were / are:

a pool of water,

a resonant violin (not a piano),

clay or dough.

 

This blurb about Responsive Media Research at the TML described it pretty well:

 

Zhuangzi Cook Ting (Was: Movement Session Planning)



Yes!  Zhuangzi's Cook Ting story one my lifetime favourite touchstones  It informs a profoundly richer way to approach the design of sociotechnical systems.   Thanks to Jonathan for referencing this valuable dual-language archive!   Since to translate is to write, here’s Burton Watson’s rendering, which also gets the essence, somewhat more fluidly:

Cook Ting was cutting up an ox for Lord Wen–hui. At every touch of his hand, every heave of his shoulder, every move of his feet, every thrust of his knee—zip! zoop! He slithered the knife along with a zing, and all was in perfect rhythm, as though he were performing the dance of the Mulberry Grove or keeping time to the Ching-shou music. ‘Ah, this is marvelous!’, said Lord Wen-hui. ‘Imagine a skill reaching such heights.’ 

Cook Ting laid down his knife and replied, ‘What I care about is the Way, which goes beyond skill. When I first began cutting an oxen all I could see was the ox itself. After three years I no longer saw the whole ox. And now — now I go at it by spirit and don’t look with my eyes. Perception and understanding have come to a stop and spirit moves where it wants. I go along with the natural makeup, strike in the big hollows, guide the knife through the big openings, and follow things as they are. So I never touch the smallest ligament or tendon, much less a main joint. ‘A good cook changes the knife once a year – because he cuts. A mediocre cook changes the knife once a month – because he hacks. I’ve had this knife of mine for nineteen years and I’ve cut up thousands of oxen with it, and yet the blade is as good as though it had just come from the grindstone. There are spaces between the joints, and the blade of the knife has really no thickness. If you insert what has no thickness into such spaces, then there’s plenty of room – more than enough for the blade to play with. That’s why after nineteen years the blade of my knife is still as good as when it first came from the grindstone.

_______________________________________________________

On Mar 30, 2017, at 10:15 AM, Jonathan Bratt <jdbratt@asu.edu> wrote:

Had to follow up Brenda's post with some Zhuangzi.


His cook was cutting up an ox for the ruler Wen Hui. Whenever he applied his hand, leaned forward with his shoulder, planted his foot, and employed the pressure of his knee, in the audible ripping off of the skin, and slicing operation of the knife, the sounds were all in regular cadence. Movements and sounds proceeded as in the dance of 'the Mulberry Forest' and the blended notes of the King Shou. The ruler said, 'Ah! Admirable! That your art should have become so perfect!' (Having finished his operation), the cook laid down his knife, and replied to the remark, 'What your servant loves is the method of the Dao, something in advance of any art. When I first began to cut up an ox, I saw nothing but the (entire) carcase. After three years I ceased to see it as a whole. Now I deal with it in a spirit-like manner, and do not look at it with my eyes. The use of my senses is discarded, and my spirit acts as it wills. Observing the natural lines, (my knife) slips through the great crevices and slides through the great cavities, taking advantage of the facilities thus presented. My art avoids the membranous ligatures, and much more the great bones. A good cook changes his knife every year; (it may have been injured) in cutting - an ordinary cook changes his every month - (it may have been) broken. Now my knife has been in use for nineteen years; it has cut up several thousand oxen, and yet its edge is as sharp as if it had newly come from the whetstone. There are the interstices of the joints, and the edge of the knife has no (appreciable) thickness; when that which is so thin enters where the interstice is, how easily it moves along! The blade has more than room enough. Nevertheless, whenever I come to a complicated joint, and see that there will be some difficulty, I proceed anxiously and with caution, not allowing my eyes to wander from the place, and moving my hand slowly. Then by a very slight movement of the knife, the part is quickly separated, and drops like (a clod of) earth to the ground. Then standing up with the knife in my hand, I look all round, and in a leisurely manner, with an air of satisfaction, wipe it clean, and put it in its sheath.' The ruler Wen Hui said, 'Excellent! I have heard the words of my cook, and learned from them the nourishment of (our) life.'



The instigating thread that inspired this quote from Zhuangzi:


On Mar 28, 2017, at 11:42 AM, Jessica Rajko <jessica.rajko@asu.edu> wrote:

Hi Friends,

Can we work together to plan sometime in late April? Here is what I’ve heard so far.

Don’t use the word time
Think about time 

Oh the oxymoron :)

JR
 



On Mar 29, 2017, at 10:10 AM, Garrett Laroy Johnson <gljohns6@asu.edu> wrote:

Hi Jessica and Sharon, 

cc: PHuN co-directors and Xin Wei 

We’re very much looking forward to talking about a praxis workshop for the Fall! Could we find some time April 21, 1:30 or 2pm? 

A couple of signposts/take-aways from the Synthesis Campfire yesterday: 

-enacted event and experience as a set of critical practices  

-vis a vis the critical discourse: what are non-reductive distillations of this knowledge who are not co-present (papers, presentation, videos, films, performative presentations?) 

-how to draw on embodied / practical knowledge without reifying notions of self?       

best, 
Garrett 


From: Xin Wei Sha
Sent: Wednesday, March 29, 2017 10:30 AM
To: Garrett Johnson; Jessica Rajko; Sharon McCaman (Student); Celina Osuna; Angela Sakrison (Student); Jonathan Bratt (Student); Zachary Thomas
Cc: Adam Nocek; Lauren Hayes; Brenda McCaffrey (Student)
Subject: Re: Movement Session Planning
 
Attachments available until Apr 28, 2017
In any transdisciplinary conversation, there’s always the risk of peace (as Stengers observed so beautifully in her essay ), of superficial agreement and understanding when that is not the case.  No chance of such superficial comity here ! ;)   In more conventional, terms, we’re going through some vocabulary impedance matching. 

No not “time” — I say: instead of “time” can we think about how organisms (in both biological and Whitehead’s senses) and their milieux co-construct, evolve, transform, change, exfoliate, … one another.

Longo and I would like to suggest that any quantity, be it duration (not Newtonian or time as a scalar parameter ) be recognized as the result of an act of measurement, which implicates apparatus, and expectant theory.

Seriously heeding Adam, we should not spend too too much time on the “humanist" accounts of experience, including phenomenology.  But we could for at least learning purposes, as a transition, start from late Merleau-Ponty (rather than his “disembodied" alternatives). To that end, someone should dig for Don Lande’s translation of Merleau-Ponty Phenomenology of Perception, and see if we can get a pdf of the Body chapters.

Patiently,
Xin Wei

Click to Download
Simondon_MEOT_part_2_Rebolledo2011-x.pdf
2.4 MB
Click to Download
PP Table of Contents.pdf
52 KB
Click to Download
Harrison_Dominion_of_the_Dead_ch1.pdf
21.3 MB
___________________________________________________________ 
Sha Xin Wei • +1-650-815-9962 • skype: shaxinwei



  



From: Adam Nocek
Sent: Wednesday, March 29, 2017 1:38:02 PM
To: Xin Wei Sha; Garrett Johnson; Jessica Rajko; Sharon McCaman (Student); Celina Osuna; Angela Sakrison (Student); Jonathan Bratt (Student); Zachary Thomas
Cc: Lauren Hayes; Brenda McCaffrey (Student)
Subject: Re: Movement Session Planning
 

Hi, Xin Wei and all,

I couldn't agree with you more, Xin Wei, though I don't want to hate on phenomenology too much. I think Varela, Thompson, et al. have done a good deal to extend the reaches of phenomenology, but I think that's exactly why we need to be ever so careful. The neuro/cog-sci brand of phenomenology is incredibly seductive and is indebted to a very particular history of the field -- Husserl and a _selective_ reading of M-P. There are other engagements with phenomenology that might be better-suited for "niche construction." In his Nature Course, M-P speaks at length about the ethologist, Jakob von Uexküll (Umwelt), as does Deleuze and Guattari in ATP (and so does Heidegger in The Fundamental Concepts of Metaphysics, BTW). There is a very productive engagement to be had between M-P and D&G via von Uexküll. This would surely open up a different kind of engagement with phenomenology; it is also one that would have to take account of Raymond Ruyer's work ...this of course depends upon very careful and patient thinking:)

 

Thanks!

Adam Nocek






From: Brenda McCaffrey (Student)
Sent: Thursday, March 30, 2017 9:50:35 AM
To: Adam Nocek; Xin Wei Sha; Garrett Johnson; Jessica Rajko; Sharon McCaman (Student); Celina Osuna; Angela Sakrison (Student); Jonathan Bratt (Student); Zachary Thomas
Cc: Lauren Hayes

Subject: Re: Movement Session Planning
 

I started this semester by dipping my toe into somatic practices, and over the past month I've slammed hard onto the ground and have been transformed.  I have always been 'sensitive' so I had no idea how far I could go beyond this.  I am becoming proprioceptive and can never go back.  I no longer sense.  I perceive subtle shifts in positions of internal organs.  I perceive subtle rhythms and flows in my body.  All movement comes from this and flows back into it.  Pre-reflection has emerged as a kind of super power.


I've played the piano since I was very young, and yesterday I sat down to play and something new occurred; I was able to play through my body in conscious and intentional ways to create music that was magical.


Yes, critical thinking about phenomenology is a fun intellectual exercise.  Linear.  As Giuseppe might say, try spiraling instead.


-Brenda




From: Garrett Johnson
Sent: April 3, 2017

Thanks all for this rich conversation, I’m looking forward to meeting to plan something for the Fall and see what we can get into. Looping in my collaborator Britta Peterson (Director of Dance, American University) who is also interested in this discussion. 
All best,
Garrett 




From: Brenda McCaffrey
Sent: April 3, 2017

 Beautiful!

B

[Synthesis] Notes on media processing, shoot, and post-production (Was: Shoot timespace 3' video)

Much stronger second version of Timespace video document.  Compliments to the team.

Let me share basic points about video documentation that I think are important for all Synthesis work.

(0) Action is better this time.  But in general please do NOT merely face the scrim -- think a bit about physical action and inter subjective relational action, before going on stage.  Pay attention and relate to each other, not to screen.   In future shoots with time, plan to bring people of different ages and sexes to visit and try out the environment.  ( build and use a sc local friends list )

(1) Music
Never add music in post.  
No post-production music is better in long run bc music or narrative added as overlay in post, blocks other commentary for contexts over the coming decades that you can never anticipate.  (Or it would need to be scripted and rehearsed for specific audience like a SIGGRAPH jury, which is a whole other order of work than simply documenting one instrument.). And most 'popular' music will go stale and sound silly in a very short time.  (Fashion obsoletes by design.)  Avoid stock music. (Re-read Peter Brook about the Deadly Theatre.)

More 'diagetic' sound 
One should  always be able to run video thru actual sc. sound synth from video output.  Compromise would be run your edited video thru some of Julian's instruments.

(2) _After_ front camera segments, it would be good to edit in the top camera segments, which hint at the potential of sc system's multi-perspective system.   Also this way there's only one Timespace video in our Vimeo.  

(3) And as appropriate, basic copyright / URL in black final segment "Synthesis 2017 | synthesiscenter.net"

(4) All videos should  have in Vimeo's text caption the names of people involved in media coding, shoot & post; date, location, as well as "Synthesis 2017 | synthesiscenter.net"

(5) It was smart to use overhead theatrical lights ( and with warm tone! ).  In general,  experiment with the lighting array to optimize results for POV.  In this case, it may have been better to dim the row of lights closest to the scrim. Not only would that lessen spill onto the scrim, possibly strengthening its image, this would sculpt space so that bodies would move in and out of visibility and create a zone of action within which the instrument takes effect.

In all responsive video work in real space, always consider design the physical lighting. Physical lighting and proper camera lensing, exposure, focus, framing eliminates many weaknesses that can't be easily fixed in post, and make possible many powerful effects.

Analog conditioning is part of your media instrument and at least as important as the coding.

Much stronger results -- compliments to the team.

some theses to be nailed onto the Church of Big Data

(1) Cristian S. Calude, Giuseppe Longo. "The Deluge of Spurious Correlations in Big Data.” Foundations of Science (March 2016): 1-18.

Abstract:
Very large databases are a major opportunity for science and data analytics is a re- markable new eld of investigation in computer science. The effectiveness of these tools is used to support a philosophy against the scientific method as developed throughout history. According to this view, computer-discovered correlations should replace under- standing and guide prediction and action. Consequently, there will be no need to give scientific meaning to phenomena, by proposing, say, causal relations, since regularities in very large databases are enough: with enough data, the numbers speak for themselves . The end of science is proclaimed. Using classical results from ergodic theory, Ramsey theory and algorithmic information theory, we show that this philosophy is wrong. For example, we prove that very large databases have to contain arbitrary correlations. These correlations appear only due to the size, not the nature, of data. They can be found in randomly generated, large enough databases, which as we will prove implies that most correlations are spurious. Too much information tends to behave like very little information. The scientific method can be enriched by computer mining in immense databases, but not replaced by it.


(2)  Sha Xin Wei, Gabriele Carotti-Sha.  "Big Data." AI & Society, Springer Online(April 2016): 1-4.
http://link.springer.com/article/10.1007/s00146-016-0662-7/fulltext.html

Abstract: 
Big Data is a term popular among administrators and business circles motivating quite a lot of investment today. Part of it is rebranding. But rather than old wine in new bottle, it may be more likely watered wine in new bottles branded to take old wine’s market. David Donoho, one of the foremost statisticians in the world and a visionary of data science, observes that much of what passes for Big Data is a bit of software engineering plus a bit of statistics. Knowledge is power. But a smidgen of knowledge plus poor judgment can do a lot of damage, especially when billions of dollars implicate billions of lives.

(3) Complessità, scienza e democrazia / interview with Giuseppe Longo

_______________________________________________________
_______________________________________________________

Re: Hadoop, pig, dynamical systems tools

Hi Brenda,

David Donoho has an extensive critical review of big data techniques in
http://courses.csail.mit.edu/18.337/2015/docs/50YearsDataScience.pdf

But I'd like to learn about HADOOP from your direct encounter, and understand how you think we can use such big data mining techniques to multi-dimensional activity / ensemble-body / gesture tracking.  It may be good to talk with Qiao to get his opinion on things.  I trust his scientific judgment most ...

And here's a curnudgeonly rant:

Glad to see you working thru all this,
Xin Wei

On Aug 8, 2016, at 8:40 PM, Brenda McCaffrey <brendamc@asu.edu> wrote:

Hi Xin Wei and friends,

Quick update:

*  Thanks to Connor for helping me get much of the Java script working within Eclipse.
*  I'm meeting with Mike K. tomorrow morning at 10am to look at the code.
*  I'm learning more about Hadoop and Apache Pig and it may be very exciting for us, although I have yet to find anyone in our realm who has knowledge of it.  Here are a few things I've learned:
  • Hadoop is an open source data management system that evolved from Google's behavioral search system about 10 years ago.
  • Hadoop is used by Amazon, Yahoo, Facebook and others to track and manage behavioral data.  For example, that's how Facebook knows where you move your mouse.
  • Hadoop allows massively parallel processing of large data sets.  It can be set up on unlimited low-cost computers to run infinitely parallel.
  • Apache Pig is a scripting language that manages Hadoop queries.
Apparently, Hadoop and Pig are primarily used in business systems.  The gentleman who wrote these dynamical systems codes (Jacob Perkins when he was at UT Austin), did so to demonstrate that one could solve complex scientific problems without massive data sets by using parallel processing.  Eureka!

I continue to be intrigued by this approach since it implies to me that we may be able to solve very complex, real-time dynamical systems problems by using relatively simple parallel computing frameworks in conjunction with Java.

I'm going to get the fundamental Java to work (I hope) to demonstrate it's functionality, but the real power of these programs is the ability to slice the simulations into partitions that can be evaluated in parallel to converge on the regions of interest.  This could be especially useful for time-domain data coming in from complex systems for which we don't know the closed form analytical models.  (Video?)

These are my thoughts for now.  I'm not a CS person so I may be completely in left field about this, but it looks promising.

Thanks for listening.
-Brenda


On Sun, Aug 7, 2016 at 4:10 AM, <shaxinwei@gmail.com> wrote:

Dr Brenda McCaffrey, PhD researcher at Synthesis, is working on a project to develop some dynamical system based tools for modeling human movement.

Can anyone help with the following query:

I’m converging on an approach for Java simulation of Lyapunov exponents based on an amazing set of programs created by a gentleman named Jacob Perkins (https://github.com/thedatachef/sounder/tree/master/udf/src/main/java/sounder/pig/chaos) that he developed using the Sprott algorithm.  This is really deep stuff and uses Hadoop and pig as well as Java.  Do we have resources in these areas?  A couple of hours with an expert would change my life!

Xin Wei

"The weirdest people in the world?" Henrich, Heine, Norenzayan (2010)

A classic paper in BEHAVIORAL AND BRAIN SCIENCES (2010), cautioning any research based on human-subjects studies in design, engineering, and social sciences.  Good for our AME PhD methods course.


The weirdest people in the world? 

Joseph Henrich 
Steven J. Heine
Ara Norenzayan 
Department of Psychology, University of British Columbia, Vancouver

Abstract
Behavioral scientists routinely publish broad claims about human psychology and behavior in the world’s top journals based on samples drawn entirely from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies. Researchers – often implicitly – assume that either there is little variation across human populations, or that these “standard subjects” are as representative of the species as any other population. Are these assumptions justified? Here, our review of the comparative database from across the behavioral sciences suggests both that there is substantial variability in experimental results across populations and that WEIRD subjects are particularly unusual compared with the rest of the species – frequent outliers. The domains reviewed include visual perception, fairness, cooperation, spatial reasoning, categorization and inferential induction, moral reasoning, reasoning styles, self-concepts and related motivations, and the heritability of IQ. The findings suggest that members of WEIRD societies, including young children, are among the least representative populations one could find for generalizing about humans. Many of these findings involve domains that are associated with fundamental aspects of psychology, motivation, and behavior – hence, there are no obvious a priori grounds for claiming that a particular behavioral phenomenon is universal based on sampling from a single subpopulation. Overall, these empirical patterns suggests that we need to be less cavalier in addressing questions of human nature on the basis of data drawn from this particularly thin, and rather unusual, slice of humanity. We close by proposing ways to structurally re-organize the behavioral sciences to best tackle these challenges.

plus

Anthony Brandt (Music, Rice), gave a great talk at the Neuroscience of art, innovation and creativity conference dismantling some sweeping claims about universals in musical aesthetic experience and music.

Re: [Synthesis] "The weirdest people in the world?" Henrich, Heine, Norenzayan (2010)

This looks fantastic. Thanks for sharing!

From: <synthesis-research@googlegroups.com> on behalf of "xinwei@mindspring.com" <shaxinwei@gmail.com>
Date: Thursday, August 4, 2016 at 8:58 AM
To: Ron Broglio <rbroglio@earthlink.net>, Grisha Coleman <Grisha.Coleman@asu.edu>, Todd Ingalls <Todd.Ingalls@asu.edu>, Loren Olson <Loren.Olson@asu.edu>, Pavan Turaga <pturaga@asu.edu>, Christian Ziegler <Christian.Ziegler@asu.edu>, Kimberlee Swisher <kimberlee.swisher@gmail.com>, Dehlia Hannah <dhannah1@asu.edu>, Stacey Kuznetsov <kstace@asu.edu>, Christopher Roberts <cmrober2@asu.edu>, Byron Lahey <Byron.Lahey@asu.edu>, Lauren Hayes <laurensarahhayes@gmail.com>, Jessica Rajko <jessica.rajko@asu.edu>, Steven Tepper <Steven.Tepper@asu.edu>, David Tinapple <david.tinapple@asu.edu>, Edward Finn <edfinn@asu.edu>, Garth Paine <Garth.Paine@asu.edu>, Adam Nocek <Adam.Nocek@asu.edu>, Xin Wei Sha <Xinwei.Sha@asu.edu>, Tamara Underiner <Tamara.Underiner@asu.edu>
Cc: Posthaven Post By Email <post@synthesis.posthaven.com>, "synthesis-research@googlegroups.com" <synthesis-research@googlegroups.com>
Subject: [Synthesis] "The weirdest people in the world?" Henrich, Heine, Norenzayan (2010)

A classic paper in BEHAVIORAL AND BRAIN SCIENCES (2010), cautioning any research based on human-subjects studies in design, engineering, and social sciences.  Good for our AME PhD methods course.


The weirdest people in the world? 

Joseph Henrich 
Steven J. Heine
Ara Norenzayan 
Department of Psychology, University of British Columbia, Vancouver

Abstract
Behavioral scientists routinely publish broad claims about human psychology and behavior in the world’s top journals based on samples drawn entirely from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies. Researchers – often implicitly – assume that either there is little variation across human populations, or that these “standard subjects” are as representative of the species as any other population. Are these assumptions justified? Here, our review of the comparative database from across the behavioral sciences suggests both that there is substantial variability in experimental results across populations and that WEIRD subjects are particularly unusual compared with the rest of the species – frequent outliers. The domains reviewed include visual perception, fairness, cooperation, spatial reasoning, categorization and inferential induction, moral reasoning, reasoning styles, self-concepts and related motivations, and the heritability of IQ. The findings suggest that members of WEIRD societies, including young children, are among the least representative populations one could find for generalizing about humans. Many of these findings involve domains that are associated with fundamental aspects of psychology, motivation, and behavior – hence, there are no obvious a priori grounds for claiming that a particular behavioral phenomenon is universal based on sampling from a single subpopulation. Overall, these empirical patterns suggests that we need to be less cavalier in addressing questions of human nature on the basis of data drawn from this particularly thin, and rather unusual, slice of humanity. We close by proposing ways to structurally re-organize the behavioral sciences to best tackle these challenges.

plus

Anthony Brandt (Music, Rice), gave a great talk at the Neuroscience of art, innovation and creativity conference dismantling some sweeping claims about universals in musical aesthetic experience and music.

"The weirdest people in the world?" Henrich, Heine, Norenzayan (2010)

A classic paper in BEHAVIORAL AND BRAIN SCIENCES (2010), cautioning any research based on human-subjects studies in design, engineering, and social sciences.  Good for our AME PhD methods course.


The weirdest people in the world? 

Joseph Henrich 
Steven J. Heine
Ara Norenzayan 
Department of Psychology, University of British Columbia, Vancouver

Abstract
Behavioral scientists routinely publish broad claims about human psychology and behavior in the world’s top journals based on samples drawn entirely from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies. Researchers – often implicitly – assume that either there is little variation across human populations, or that these “standard subjects” are as representative of the species as any other population. Are these assumptions justified? Here, our review of the comparative database from across the behavioral sciences suggests both that there is substantial variability in experimental results across populations and that WEIRD subjects are particularly unusual compared with the rest of the species – frequent outliers. The domains reviewed include visual perception, fairness, cooperation, spatial reasoning, categorization and inferential induction, moral reasoning, reasoning styles, self-concepts and related motivations, and the heritability of IQ. The findings suggest that members of WEIRD societies, including young children, are among the least representative populations one could find for generalizing about humans. Many of these findings involve domains that are associated with fundamental aspects of psychology, motivation, and behavior – hence, there are no obvious a priori grounds for claiming that a particular behavioral phenomenon is universal based on sampling from a single subpopulation. Overall, these empirical patterns suggests that we need to be less cavalier in addressing questions of human nature on the basis of data drawn from this particularly thin, and rather unusual, slice of humanity. We close by proposing ways to structurally re-organize the behavioral sciences to best tackle these challenges.

plus

Anthony Brandt (Music, Rice), gave a great talk at the Neuroscience of art, innovation and creativity conference dismantling some sweeping claims about universals in musical aesthetic experience and music.