is an artist working with code. He crafts interactive installations, sneaky interventions, playful websites, workshops, and toolkits
for other artists working with code. Exploring possibilities of new technologies: to understand how they affect society, to misuse them, and build alternative futures; aiming to share a laugh, spark curiosity, create confusion, and share spaces with magical vibes. Working with machine learning, computer vision, social and surveillance tech spanning commercial and arts spaces. Previously adjunct professor at NYU's ITP
, member of F.A.T. Lab
, community manager for openFrameworks
, and artist in residence at STUDIO for Creative Inquiry
at CMU, and YCAM
in Japan. Work commissioned and shown around the world, including: the V&A
, NTT ICC
, Ars Electronica
, Todays Art
, and Eyebeam
Game that imagines a world where face analysis is key to the latest gig economy app. As a Faceworker, the player is given an opportunity to interrogate in realtime how computer vision and machine learning tools work — to playfully grow an intuition for what it means to see like a machine, and to understand how machines can fail. Created with support from Mozilla Foundation.
Vibe Check 2020 with Lauren McCarthy
Appropriates common surveillance tools including face recognition and expression analysis to catalog the emotional effect exhibition visitors have on one another. Some are identified as evoking expressions of happiness, disgust, sadness, surprise, or boredom. Upon entering the exhibition, visitors are playfully alerted to who these people are, and as they leave, they may find they've earned this distinction themselves.
Command-line tool for showing real time gCO2eq emissions estimates in various formats (equivalent sea ice lost per hour, grams of beef consumed, number of light bulbs, etc) combining CPU and GPU usage. A tool for researchers and hobbyists, playfully intervening at the site of one of the most commonly used debugging/monitoring tools used for machine learning.
Pattern Radio 2019 with Google Creative Lab
Working with a decade of underwater recordings collected by NOAA to visualize humpback whale songs. Techniques from computer vision, signal processing, machine learning, all converge to push the creative potential of machine-aided listening of contemporary performers in a 11 million year old cultural practice.
discrete figures 2018 with Rhizomatiks
Internationally touring dance performance. Explorint the relationships between the performing arts and mathematics, giving rise to mathematical entities that engage with the bodies of human dancers onstage. Collecting a custom dance dataset and training a new network for generation of abstract dance-like movement.
Concerned you may know an ICE employee? Use this helpful app to check. Just point your camera and press the photo button. We'll check against hundreds of ICE employee photos from LinkedIn and show you the closest match. A direct subversion of face recognition technology to watch those who wield power without responsibility.
NSynth Super 2018 with Magenta
Synthesizer for exploring neural interpolations of sonic textures. Four knobs for controlling "reference" sounds and a touchscreen for blending between them using the NSynth algorithm. Contributed to conceptual and interaction design, and wrote the synthesis engine and initial user interface.
Man and Woman In The Middle 2017 with Lauren McCarthy
We each wrote scripts to intercept and modify or reply to text messages from each other, in the hope of syncing our different communication styles and developing a better relationship and conversation.
How We Act Together 2016 with Lauren McCarthy
Prompts you to perform small gestures of social interaction repeatedly until exhausted. Your action triggers a video stream of all the previous participants performing the same action back at you. If you perform longer than anyone before you, your own gesture gets added at the end of the stream.
Bird Sounds 2016 with Google Creative Lab
Thousands of bird sounds, organized using machine learning. Similar bird sounds are placed close together, producing a sonic map that produces unexpected connections between different species.
The Infinite Drum Machine 2016 with Google Creative Lab
Thousands of everyday sounds, organized using machine learning. Experimenting with alternative interfaces for exploring sound libraries and serendipitous opportunities for musical improvisation and composition.
Terrapattern 2016 with Golan Levin, David Newbury, et al
Similar-image search for satellite photos. An open-source tool for discovering "patterns of interest" in unlabeled satellite imagery—a prototype for exploring the unmapped, and the unmappable. Click an interesting spot on Terrapattern's map, and it will find other locations that look similar.
A virtual bungee jump through a real space. Visitors are strapped into a harness and a VR headset showing a live stream from a camera on a wire. The camera meanders through a playful diorama before dropping through the bottom of the box and into the crowd.
Exhausting a Crowd 2015
A crowdsourced description of 12 hours in Piccadilly Circus, London, inspired by the classic 60-page piece of experimental literature from Georges Perec, “An Attempt at Exhausting a Place in Paris”, written from a bench over three days in 1974.
pplkpr 2014-2015 with Lauren McCarthy
An app that tracks, analyzes, and auto-manages your relationships. It uses GPS and subtle changes in heart rhythm to keep track of when you're coming and going and determine your emotional state. The data is connected to people you interact with to determine who should be auto-scheduled into or out of your life.
Augmented Hand Series 2014-2015 with Golan Levin and Chris Sugrue
Real-time interactive software system that presents playful, dreamlike, and uncanny transformations of its visitors' hands. It consists of a box into which the visitor inserts their hand, and a screen which displays their "reimagined" hand—for example, with an extra finger, or with fingers that move autonomously.
Sharing Faces 2013-2014
For eight months we shared photos between Anyang, Korea and Yamaguchi, Japan. Visiting the installation at either location would match your expression and pose in realtime with these photos of someone else who once stood in front of the installation. Thousands of people visited the work, and saw themselves reflected in the face of another person.
A map showing ephemeral connections created by simultaneous listens: every second a few people hit "play" on the same song, at the same time. Developed during inaugural media artist residency at Spotify.
Who Pays Artists? 2014
A sounding board for artists to anonymously share stories about what they've been paid for, and by whom. An intervention and conversation starter in the ongoing dialog about money in the context of the arts.
Social Soul 2014 with Lauren McCarthy
An immersive digital experience inspired by the question: how does it feel to be inside someone else's social media stream? Starts with the stream of the person visiting, then uses a custom algorithm to match participants, displaying their social stream instead. After exiting, users and their connected "soulmate" receive a tweet encouraging offline conversation.
Noodle 2014 with Lauren McCarthy
A crowdsourced robot with the inputs and outputs of a computer but the thoughts and feelings of a person. Noodle uses a natural language interface to program mechanical turk workers, providing them with a body for influencing the real world.
Conversnitch 2013-2014 with Brian House
An eavesdropping lamp/lightbulb that livetweets conversations, using a small microphone with a Raspberry Pi that records audio snippets and uploads them to Mechanical Turk for automated transcription.
us+ 2013 with Lauren McCarthy
A Google Hangout app that analyzes speech and facial expressions to improve conversation. Speech to text is handled by the Chrome Speech API, linguistic analysis is based on LIWC
and inspired by linguistic style matching research, facial analysis is based on the Hangouts Face API.
Going Public 2013
A one month performance coinciding with the Twitter IPO, where my Twitter profile is made publicly accessible by allowing anyone to tweet as "me" simply by direct messaging me a tweet prepended with a "~" (tilde).
FriendFlop 2013 with Lauren McCarthy
A browser extension for Chrome that scrambles the identities on your Twitter or Facebook timelines, dissolving your biases and reminding you that everyone is saying the same shit anyway.
Open Fit 2013 with Lisa Kori
Open source software that investigates several approaches to generating custom tailored pants patterns. Open Fit Lab is an experimental event where we use this software for on-the-spot generation and creation of custom clothes.
Endless glitch facets of a "fractured, frozen" mosaic, a metaphor for our data in the cloud. Commissioned by xo
Social Roulette 2013 with Jonas Lund and Jonas Jongejan
A game with "a 1 in 6 chance of deleting your Facebook account", in fact a hoax that was shut down by Facebook for suggesting the possibility that your account is yours to play games with.
Light Leaks 2013 with Jonas Jongejan
A scattered array of fifty mirror balls reflect light from three projectors, filling a room completely with small reflections, casting patterns that fill the visitor's peripheral vision. Creating a curious space that alternates between a meditative state, and an uneasy imbalance.
Inbox Zero Residency 2013
A one-week residency aimed at clearing my inbox, treating email answering as art, daily digital life as readymade performance. Daily reports on Twitter, supported by GaOk
RAM Dance Toolkit 2013 with YCAM
A toolkit for dancers to communicate with each other in virtual environments. Includes functions to access, recognize, and process motion data to support creation of various scenes, and to provide realtime feedbacks to dancers with code, easily.
ExR3 2013 with Elliot Woods
An anamorphic analog interactive installation that exists coherently in a fractured, mirrored version of a reflected room visible from four points within the real space.
Missing 2012 with The xx, Matt Mets and Aramique
50 robotic speakers turn to face visitors, while a spatialized recording of The xx's song "Missing" follows them through the space.
Mother 2012 with Inmi Lee
Digitally extruding hand gestures in response to linguistic cues as a method of experimental sound visualization.
Blind Self Portrait 2012 with Matt Mets
While the visitor keeps their eyes shut, a moving platform guides a pen in their hand to draw a self-portrait, using computer vision to track their face and generate a line drawing. The result is a machine-aided drawing, a self-portrait you could never draw.
A subway advertisement upgrade kit, built from laser-cut acrylic, rubber bands, and nine razor blades. It transforms a small patch of subway advertisement into an 8×8 grid of pixel stickers, ready for interaction with the public.
ScratchML 2012- with Jamie Wilkinson and Michael Auger
A format for describing turntablism, as well as tools for recording, analyzing, sharing, and even recreating scratch performances with giant robot arms. Initially built as a prototype in at the 319 Scholes Art Hack Day
Experiments with shadow manipulation as an alternative interaction paradigm, exploring the unexpected results of structure in shadow, or form in emptiness. Shadowplay uses ProCamToolkit
which was co-developed by the YCAM Interlab
during a residency.
An experimental projection mapping tool that minimizes the amount of time spent doing calibration. mapamok uses ProCamToolkit
which was co-developed by the YCAM Interlab
during a residency.
Face Substitution Research 2011-, with Arturo Castro
Real time face tracking and substitution experiments including a realization of the Scramble Suit
and an interactive installation Faces
that picks a new face for you every time you blink.
Pocket Topographies 2011-, with Chris Woebken
Ongoing experiments with pocket-sized topographical maps for visualizing geographic data in a physical form. The first piece shows a year of travel around NYC captured surreptitiously by an iPhone. Started during a residency at MakerBot
People Staring at Computers 2011
A photographic intervention. Custom app installed around NYC computer stores, taking a picture each minute and uploading it. Photos exhibited on site, full screen, on every computer.
Sharing Interviews 2011
Interview series about creators sharing work, with an emphasis on open source, media art, and digital communities. The interviews are conducted with EtherPad and stored on GitHub.
Eyeshine 2011 with Golan Levin
Interactive installation which captures, records and replays the retroreflections (red-eye effects) from the eyes of its observers. In doing so it presents an image wholly constructed through the process of being observed.
The Janus Machine 2010, with Zach Lieberman, Theo Watson and Daito Manabe
A 3d photo booth built from a structured light system. Janus is often depicted with two faces, each facing in opposite directions. Here: the real face, and the scan stored by the computer.
Only Everything Lasts Forever 2008-2010
Very long sound composition for MP3: an arrangement of every sound we can uniqely distinguish. An exploration of noise and emptiness, and the auditory implications of sound representation. An open source project towards an MFA.
Night Lights 2009
Building projection with crowd interaction in collaboration with YesYesNo
, featuring three different interaction modes over six different scenes, cycled every hour over five nights in Auckland, NZ.
One-year performance for custom keylogger, tweeting every 140 characters I type, ending June 30, 2010. An exploration of the boundary between "information" and "control", ubiquitous status updates, and personal identity.
Portrait Machine 2009, with Theo Watson
An interactive photography installation that visualizes the connections between visitors, reminding us of our connectedness and uniqueness, creating strong visual patterns and playful juxtapositions.
DIY 3D Scanning 2009-
Ongoing work democratizing realtime 3D scanning with structured light (one projector, one camera).
1¢ Grants 2009
A project to provide 100 grants at 1¢ cent each. Suspended early due to the high overhead cost of mailing pennies.
The Vibrobyte is a wireless haptic interface specialized for co-located musical performance. It was featured in "Telemorphosis" at ICMC 2008
. Technical details were presented at the 156th ASA Meeting
and NIME 2009
I Eat Beats 2008-2009
Skittle-based beat sequencer, a tangible and edible music interface. v2 implements a generic control surface with the aide of a linear polarizer to cancel out the screen.
An appropriation of computability theory. During each brief scene, a random sequence of p'' code is generated and run, while the memory it uses is visualized and sonified. An investigation of emergent behavior and computational dreams.
Please Remember Me When I'm Gone 2008, with Rebecca Weisman
Exploration of personality, death, and emptiness through a human-scale settlement of handmade dwellings combining video, electronics, and recorded voice. Colors from the sky as it moves from dusk to dark, and from dark to dawn, are sampled from video and routed into the huts.
Ongoing noise synth explorations, with inspiration from the Kraakdoos, Theremin, Thingamakit, and BEAM electronics.
DIY 3D Interface 2008-
Ongoing work with DIY capacitive sensing for 3D interaction. v1 used an Arduino, aluminum foil, and six resistors.
Future Fragments 2008
An anti-time-capsule: quotes from seven fellow art students, transcribed phonetically and encoded as colors. Prints of these colors were carried by the artists for a summer. Decoded back into phonemes and re-formed into words, each text offers an indirect account of their respective journeys.
Musique Du Vent 2008
Reclaiming the entropy of unused dryer heat from a vent outside a freshman dorm, redirecting it into a discarded accordion. During the early Winter, the vent creates a small grassy space surrounded by snow. In March, it's mostly thick mud, but still unexpectedly warm and inviting.
An exercise in simplicity: the most basic capacitive sensor, using the smallest possible number of external components. Requires only an unshielded cable connected to a microphone input.
Early Processing work. Don't move too fast, you might scare it. Click to forgive and forget.