puppetry & film animation >> interaction

for 20 years since sponge, my fellow artists and i measured our work not to “new media” or anything to do with “computation” or "interaction” but to the most highly evolved and richest arts: theater, architecture, plastic arts, visual arts…

we aspire to make experiences as powerful as that which were achieved by the likes of Bertoldt Brecht, Sankai Juku, Dumb Type, or Anish Kapoor, Mona Hartoum, Renzo Piano, …

by such standards very very few works of computational media art “measure up” in experiential and conceptual power. but that should impel us to do more. certainly that should prevent us from lapsing into the complacency typical of programmers who discover the pleasure of mapping an input into a color or a frequency, and stop there.

ever since i established the Topological Media Lab in at GaTech, i said we do not do “interactive” stuff. and when i accepted the Canada Research Chair in “new media” i said there is no such thing as new media :)

elegant mappings of motion into graphics by Tobias Grumbler

Jitter artists:

Tobias Gremmler’s elegant, straightforward mappings of motion into graphic renderings could be a nice inspiration for 
mocap —> jitter if we can figure how to work with a sparse set of trackers.


The quality suggests it is not realtime, but this is useful for ideas for jitter instruments that should be doable in Max 7 / gen.
The primitives are lines and points, so not very challenging to render on gpu.   We don’t have the density.   

Could be time to implement camera-based tracker-free mocap as jitter externals.





________________________________________________________________________________________
skype: shaxinwei • mobile: +1-650-815-9962
_________________________________________________________________________________________________


________________________________________________________________________________________
Sha Xin Wei, PhD • Professor and Director • School of Arts, Media and Engineering + Synthesis
Herberger Institute for Design and the Arts + Fulton Schools of Engineering • ASU
Fellow: ASU-Santa Fe Center for Biosocial Complex Systems
Affiliate Professor: Future of Innovation in Society; Computer Science; English
Founding Director, Topological Media Lab
skype: shaxinwei • mobile: +1-650-815-9962
_________________________________________________________________________________________________

Laetitia Sonami Ladys Glove & Michel Waisvisz, Hands. gestural instruments, STEIM

Two powerful performers in the STEIM family: Laetitia Sonami and Michel Waisvisz
STEIM: STUDIO FOR ELECTRO-ACOUSTIC MUSIC, Amsterdam

Laetitia Sonami
Lady’s Glove

Éliane Radigue et Laetitia Sonami, The last of the LADY’s GLOVE | Les Soirées Nomades - juin 2015


An Historical Moment on a Line Between A and B de Laetitia Sonami au Mois Multi 10

Uploaded on Mar 10, 2009
À partir dun instrument insolite et original, son Ladys Glove, Sonami tisse en solo une envoûtante performance audio qui prend forme au rythme de sa gestuelle corporelle. Conçu et développé par lartiste, le Ladys Glove est un gant complexe et élégant relié à un système informatique lui permettant de concevoir des sons en temps réel en fonction des trajectoires imposées par son corps. Avec des collages, des échantillonnages de sons et des voix captées dans différents lieux quelle associe à des images vidéo de lartiste Sue Costabile, Sonami crée en direct des paysages sonores et visuels à même une danse énigmatique et ensorcelante.

À propos de Laetitia Sonami
(Oakland, États-Unis)

Né en France, Laetitia Sonami est installée aux États-Unis depuis 1975 où elle poursuit son travail en musique électronique. Elle réunit la technologie de pointe, la musique et la narration pour en extraire des récits intimes, une forme d'art spontané qui lui est propre. Elle est surtout connue pour son instrument original, le Ladys Glove. Ses performances des dernières années lui ont valu une renommée internationale. Depuis 1996, elle a été invitée dans plusieurs festivals internationaux et a fait des prestations dans les grandes capitales culturelles du monde. Son travail a été récompensé par de nombreux prix. En janvier 1997, le New York Times décrit Sonami comme « a human antenna searching the air for sounds, like a dancer focused on her hands, or like a deity summoning earth-shaking rumbles with a brusque gesture. »


and 

Michel Waisvisz, Hands instrument, STEIM

Michel Waisvisz - 1993

Michel Waisvisz - Hyper Instruments Part 1

Bert Bongers was a key engineer creating these instruments 

art, Chaosmosis 131

The artist - and more generally aesthetic perception - detaches and deterritorializes a segment of the real in such a way as to make it play the role of a partial enunciator. Art confers a function of sense and alterity to a subset of the perceived world. The consequence of this quasi-animistic speech effect of a work of art is that the subjectivity of the artist and the “consumer” is reshaped. …. The work of art, for those who use it, is an activity of unframing, of rupturing sense, of baroque proliferation or extreme impoverishment, which leads to a recreation and a reinvention of the subject itself.

Felix Guattari, Chaosmosis 131

Re: frameless, physically generated animation of shadow (puppets)

Hi all,

Thank you for sharing!  Here are links to some of the projects that I have been also referencing.

A. Remnance of Form

B. Augmented Shadow

C. Shadow (2002) by Scott Snibbe

D. Kumi Yamashita

Much appreciated,
Cooper

On Sat, Apr 2, 2016 at 3:25 PM, Shaun Ylatupa-Mcwhorter <sylatupa@asu.edu> wrote:

My answers to Xin Weis questions,

 

Q1.  Can you think of some interesting / viable ways to do continuous, Mechanical, Analogs

 

A1: Draw examples for the countless biological forms and their interplay

Schema

-- Object and Activity, spatial distance to, Object, temporal distance, Action

Sound

-- Tree Creeking against tree during wind-blow

-- A bush Chiming against chain-link- fence during a walk-by

-- A log sounding against bird beak during dig

-- Grass rustles against feet during walk-by

LIGHT

-- A leaf shades against forest floor during sun shine

-- A water reflects against bridge during street lights

-- A forest glows amidst camp during fire burn

-- A branch bending under weight during a bird-rests

 

Q2.  If you need to resort to electricity, how can you make the electro-mechanical system consume minimum energy?

A2: The least amount of energy can be achieved by only measuring or rendering the minimum spatial or temporal change, by the Object and subject, required so that the desired level of perception, by the observer, is achieved. Don’t collect or measure data with too fine of a grain. Don’t render light or sound to too fine of a resolution. If there is too great of difference between the resolution of the measured data and the desired rendering then either preprocess the data before rendering or do an efficient compression.

 

Brenda,

I love the idea of using peoples motion, fire light.

And I also like the mirrors directing light—especially with the provided video, that would be a challenge to add value to that installation; and it would also be fun to play with.

 

 

From: Adam Nocek [mailto:Adam.Nocek@asu.edu]
Sent: Saturday, April 02, 2016 9:55 AM
To: sxw asu <sxwasu@gmail.com>; Varsha Iyengar <varshaiyengar@hotmail.com>; Sudarshan Seshasayee (Student) <prashanth.sesh@asu.edu>; Sanghyun Yoo (Student) <cooperyoo@asu.edu>; Julian Stein <Julian.Stein@asu.edu>; Connor Rawls (Student) <Connor.Rawls@asu.edu>; Joshua Stark (Student) <Joshua.M.Stark@asu.edu>; Matthew Briggs <matthewjbriggsis@gmail.com>; Loren Olson <Loren.Olson@asu.edu>; post@synthesis.posthaven.com
Cc: synthesis-research@googlegroups.com; Shang Wang <swang158@asu.edu>; Jonathan Bratt (Student) <jdbratt@asu.edu>; Minsoo Kang <mkang29@asu.edu>; Jonatan Lemos <jlemoszu@asu.edu>; Brenda McCaffrey (Student) <brendamc@asu.edu>; Dallas Nichols <drnicho1@asu.edu>; Micky Small <msmall4@asu.edu>; Shaun Ylatupa-Mcwhorter <sylatupa@asu.edu>; Yongshi Zhao <yzhao91@asu.edu>; Peter Weisman <peter.weisman@asu.edu>; Assegid Kidane <Assegid.Kidane@asu.edu>
Subject: RE: frameless, physically generated animation of shadow (puppets)

 

Ahhh... the virtual life of sculpture is animation--animation haunts. I love it!

 

A.J. Nocek, PhD
Assistant Professor, Philosophy of Technology and Science and Technology Studies
Director, Laboratory for Critical Technics (LCT)

School of Arts, Media + Engineering
Herberger Institute for Design and the Arts
Arizona State University, Tempe, AZ 85287-9309
phone:(206) 434-7637

 

From: sxw asu [sxwasu@gmail.com]
Sent: Saturday, April 02, 2016 8:30 AM
To: Varsha Iyengar; Sudarshan Seshasayee (Student); Sanghyun Yoo (Student); Julian Stein; Connor Rawls (Student); Joshua Stark (Student); Matthew Briggs; Loren Olson; post@synthesis.posthaven.com
Cc: synthesis-research@googlegroups.com; Adam Nocek; Shang Wang; Jonathan Bratt (Student); Minsoo Kang; Jonatan Lemos; Brenda McCaffrey (Student); Dallas Nichols; Micky Small; Shaun Ylatupa-Mcwhorter; Yongshi Zhao; Peter Weisman; Assegid Kidane
Subject: frameless, physically generated animation of shadow (puppets)

Over the past couple of years, some of us have been talking about animation, shadow puppets, and analog “computation.”

 

 

No frames.

No digital logic.

No computer.

Analog, continuous computation. 

 

There are many ways to make this analog system responsive to contingent human activity ((i.e. “interactive")

 

Q1.  Can you think of some interesting and mechanically viable ways to do that?

 

Q2.  If you need to resort to electricity, how can you make the electro-mechanical system consume minimum energy?

              

 

 

________________________________________________________________________________________

Herberger Institute for Design and the Arts + Fulton Schools of Engineering • ASU

skype: shaxinwei • mobile: +1-650-815-9962

_________________________________________________________________________________________________

 

processualist vs. schematic analysis

Longo work with colleague at Tufts on cancers is tackles the same methodological problem: taking a processual and field approach instead of a schema approach can yield a far deeper and effective understanding of the emergence of cancers as well as the emergence of terrorism

http://www.nytimes.com/2016/03/28/world/europe/mystery-about-who-will-become-a-terrorist-defies-clear-answers.html?emc=edit_th_20160328&nl=todaysheadlines&nlid=70287705&_r=0

In both cases, this may imply changing the organism’s habitual practices and its ambient.

Eureka! Sainkho Namtchylak, Night Birds (Was: catchiness, traditional music, (metronomically regular) beat, Sainkho Naimtchuk, Gregorian Chant)

Here’s what I wanted to play for you :

Sainkho Namtchylak 
Night Birds
Lost Rivers album (1992)

No computer synthesis.
Riding the fold between human and nonhuman 

_________________________________________________________________________________________________

On Feb 29, 2016, at 6:30 PM, Brenda McCaffrey <brendamc@asu.edu> wrote:

Hi Xin Wei,

This is ravishing.  Thank you.

Hmmm...how do you think our experience of gravity affects our musicality?

-Brenda

On Mon, Feb 29, 2016 at 5:36 PM, Xin Wei Sha <Xinwei.Sha@asu.edu> wrote:
Dear  AME 531 Seminarians,

Thanks again to Dr. Garth Paine for a lovely set of exercises and ear- and mind-opening musical examples.   (I’ve asked for the names of composers and works.)

Let me offer a followup to the discussion we had…

(1) Re. “traditional” music.
The appeal to “traditional” music for different ancient cultures does not distinguish any  universals about  what is tuneful or rhythmic or “catchy” or musical: 

Mongolian Tuva throat singing

Sainkho Naimtchuk, one of the most extraordinary singers in the world

Sainkho Namtchylak - Order To Survive (Germany 2002)
then


And some example of “traditional” music:
Khoomii singing


(2) Metrically regular beat-based music versus other kinds of music — e.g. melismatic song

Talking about tapping toes to a metrically regular beat.  This is an artifact of very very recent invention — of electrical mechanical devices.   The notion of metronomic performance is very recent in human history:

Bonus, Alexander Evan. The Metronomic Performance Practice: A History of Rhythm, Metronomes, and the Mechanization of Musicality. PhD Dissertation, Case Western Reserve Univ., 2010.

If humans have vocalized and sung for say 5000+ years, then the notion of (mechanically periodic) beat occupies a pretty trivial portion of that time.     Beat-oriented music is of course quite common and widespread — even Sainkho Namtchylak sings songs with periodicity.  However as you can hear, she also sings music that is clearly musical, even passionately musical, but whose power and appeal is not based on beat.  This is not new, but ancient.

Gregorian Chant is a glorious example of music that is melismatic, rather than beat-based.  Gregorian Chant ebbs and flows, rises and falls with the intertwining of both the patterns of closed and open vocalisation of Latin speech and the inhalation and exhalation of breath which varies in length and intensity with the sweeps of meaning, syllabification, closing / opening of the nasal, vocal, lung cavities, the stresses of the poetry, all of which lead to a gentle irregularity.  

Singing, like bowed stringed musical instruments, is characterized  continuous flow, breath, but not beat.

One of the most ravishing songs of the past 1500 years is the opening of the Gregorian Pentecost Mass:
Gregorian Chant: Dominica Pentecostes: I. Introitus (Spiritus Domini)

(Domenica di Pentecoste, Introito VIII modo, Schola Gregoriana Mediolanensis, Giovanni Vianini, Milano, Italia)

(
)

(3)  Also, even when the music clearly has temporal accent and pulse, it may not be “catchy” to your ear.  Appealing to “traditional” music won’t guarantee “catchiness-to-your-ear.”  I would hazard that to ears habituated to American popular music from the past 10-30 years, especially post iTunes, traditional Kabuki music may not be so catchy:
Azuma Kabuki Musicians Nagauta Music

And then it may – à chacun son goût (“to each his own taste”).  And therein lies the rub. :  musicality is as much a question of the putative subject (the listening “you”) as it is of object (sound waves, pieces of flesh pressing on pieces of wood or metal at specific times ), and the social-cultural context. ( Social-cultural-geological-cosmological context, if we read farther into Deleuze & Guattari, Simondon, Whitehead. )   Garth’s exercise introduces this subjectification by having one human be the “sensor” for the other.

(4)
In a sense, this whole year-long seminar is devoted to opening up to the universe of pattern available to us when we sail farther from the shores of habits inherited from combinatoric or mechanical, or graph-theoretic  tropes that have become frozen in by the reductive and simplistic forms of  late 20c computationalism.

A grand challenge for those of us who are doing research — adding to the sum of human knowledge using the technologies of our age, which are shot through and through with computational logic — is whether and how we can articulate media = matter + energy + affect with more poetry, nuance, breath.

Rather than reducing music (organized sound) and musical performance to formal computationally popularized patterns, what can we learn from organized sound and (live) performance to inform a more expanded notions of pattern, computation, articulation, language, representation … ?

Xin Wei