Echos of the Mind: An interactive Music Video created from your thoughts.

It's been a while since my last post. Austin has kept me busy over the last few months with so many great interactive art projects. I will say my last one most definitely was my smallest and most intimate and my most gratifying. 

It was not too long ago that when I had returned to Berlin I had made the goal to go back to school which was my driving force for going back and getting my visa. A lot of people didn't know I wanted to go back to school and that is because I've been acting like i've been in school ever since I left. My goal in my work was to practice things I couldn't in my undergrad but were not available inn any of the graduate programs I was interested in: Art Therapy using Video. The field of art therapy I had been interested in was gestalt theory and to me what that meant for video was allowing a way for people to see how they saw their life and the world by magnifying one moment. Fast forward 3 years later and I am creating art that is purely experiential(or I hope is). There has of course been a lot of  "what the heck am I doing moments" but when you are creating an experience it is easier when you realize that you have given a moment to someone that they will never forget.  It was on my last project that I felt that most of all. 

It was an installation for a Neuroscience graduate lab at UT lead by assistant professor Laura Colgin. They studied brain waves and had approached us after we had created something small and fun for a recent party of theirs using EEG headsets and an eye tracker. Laura asked if we could create an installation that would appeal to the old and young, was a 5 min experience, used alpha brain waves that were triggered when you closed your eyes. As someone that typically works with video, I considered what this installation would consist of if the person needed to close their eyes. These people would most likely be older(60-70) since the event was about memory and they happened to be the most concerned on it since they were loosing it the quickest. after a couple of days of creating a couple ideas(recording the brain waves and letting people see it after) I quickly realized this was a great opportunity to bring in a new element I never use....stereophonic sound. We would create an installation that would allow the user to hear a musical composition in stereophonic sound using their alpha brain waves. 

The goal was to create an ephemeral sound that accompanied the fleeting restless mind and aided it in a relaxation. once relaxed the alphas brain waves would increase and the participant could enjoy the complex melody. 

Once the event came around, experience would prove to be much more amazing then expected. The quick set up and 2-5 min experience left kids and adults smiling a minute into their eyes closed. In fact, it became very difficult to tell people it was over because they were enjoying it so much. It was like trying to wake a child from a dream. It was in fact the most pleasing moment I've had watching people interact with any one of my installations and reminded me of why I do what I do. 

 

Tour of a 3-D World using Kinect

I was commissioned for an installation before I left to Berlin and it ended up being one of the quickest projects and most streamlined to-date(mostly because I was leaving to Germany and the deadline was two weeks). So among moving out of my place, my birthday, travelling to Europe, watching Star Wars and spending Christmas with my dad, I also spent a whole day building a computer from scratch for interactive installations and 60 hours building a program for said machine.  

This program is an interactive menu system projected on a wall that allows you to pick between 3-D models of installations using your body that the company will be building over the next year. When you pick a model from the menu, it then allows the user to control the camera perspective in a 3-D space (through tracking in and around and tilting) to study the model as though you were actually there. The controls are all done via infrared from the Microsoft kinect so that the user merely needs to walk up and they have full control using they're right and left hands.

There's also some nice details added to it as well: lighting and texturing on the structure, a transition wipe away effect from the menu, a designated window of space the user must be in to use it and the ability for the program to be controlled and reset from the clients phone. This was also my first project full plug and play for the client. 

It essentially is a way for us to interactively pitch our installations. 

Top 5 Extrasensory Interactive Technologies

I'm a huge advocate of making lists for productivity. Because of that, and my enthusiastic nature in technology, I figured I'd drop some knowledge of what I see as hugely effective and accessible hardware to get to know for doing interactive technology today.  This is my top list of wireless interactive technology in order of personal interest.

5) Holograms:

Holographic imaging is slowly coming into play. Not on the commercial level but in a way where many of the media houses and people in the field are prepping for it. Right now, it is only doable in some old school ways (pepper’s ghost pseudo-holograms) or in the official way in a very small scale but, once it is commercially possible, the abilities it will grant may be endless as media takes a transition from a 2D surface to the ability to be and participate in the 3D realm. You may then element labor for building structures that would take months or 30 people and build it purely digitally using light to exist as if it was a piece of architecture.

4)Infrared for Motion Tracking

Motion tracking infrared sensors are what I use for body tracking for performance and installation. It is most notably used in the Microsoft Kinect and in autofocus camera systems and is very easy to get to know and play with if you have the proper program. The growing interest in performance using the motion tracking has led to a whole field of programmers and creators that specialize in programming things that analyze and react to how the body moves. Being in Berlin, I recently participated in a meetup that brought many people together working in these fields.  Dancers, programmers, and neuroscientists all who were wanting to use this for understanding how the human body moves and how it can be expressed through other means. It's amazing what you can do with something you can purchase at the video game store for $25. The biggest flaw though is the camera to its placement and field of view. The imaging space can be blocked or confused very easily. I've heard the same complaint from researchers using $50,000 gear. The only real alternative without this flaw seems to be #1 in this list.

3)EyeTracking

Eye tracking is a new field experimented with in marketing but seldom used commercially for user design. It seems most attention goes towards the research done for marketing and advertising since the tracking of the eye is also the tracking on the user's attention. The usability of this is staggering for research but is also limited to where there is a camera to catch the eye and proper calibration for the parallax of the head in space. But once calibrated, it is highly effective to use with your computer and also extremely easy to learn to use with the only learning curve being knowing how not to look where you don't consciously want to(since our attention is constantly jetting). This also aided in the retention of concentration, a skill as coveted today as memory. 

 

 

2)Brain Monitoring

Like the previous, brain monitoring is also something emerging relatively quickly that also has an appeal for those training their concentration. That is most of the commercial selling points as of now you can purchase an EEG headset(brain monitoring device) for the price of a gaming system, but it's practicality has not fully formed for commercial use in interactive design because the technology is advancing before the research. To monitor the brain is easy now but to link activity to certain thoughts and actions is very difficult because, unlike a muscle, a thought exists in the mind. There are also many different paradigms of human thought evolving with our technology that almost make it so you must know a bit about neuroscience to know how different brain waves affect us. It becomes increasingly difficult to create complex programs using the hardware for the general consumer because everyone's brains work differently, but the crowd sourcing ability of the community research is growing and with that people are becoming more familiar with them and how they work. Of course, being able to interact with the environment using purely your thoughts would be the end goal but the difficulty in use is often knowing how your own thoughts work which, since thoughts being intangible and fluid things, becomes a daunting task of making them into something quantifiable.

1) Accelerometers/Gyroscopes

I listed this last one with a bias for motion tracking but also for the sake of its original use was to get us into space. Like a compass, the accelerometer and gyroscope combination are used to determine acceleration and direction of acceleration which can be used to program gesture controls and motion tracking in any environment. It is something often put in our devices(phone,game controller,fitbit,etc,) to monitor whether the device is turned in a certain direction. The original of the first one on the space shuttle in 1981 was as big as a small car but now they exist as chips smaller than a penny and cheaper than a quarter.

 This device's application for me is something that can allow anything to be interactive and coordinated in 3-D space(wirelessly with the right gear). With that information you can trigger anything to do anything when it moves a certain way, goes to a certain spot, stops moving and more if you simply attach chip that's smaller than a penny to it. This to me has the most practical application today without needing any research complex research. It simply needs to be applied and experimented with and can instantly turn life into a dynamic wonderland of technology.

I've also considered (since the only way to time travel currently is to go fast) how this may be something for use in knowing how much time has gone by for the average future space explorer/time traveler. But really, that's just something fun to consider a possibly application.

 

Berlin, EEG and Eye Tracking

I'm currently in Berlin learning the language and spending immense amount of time on projects. I have just finished a project for a client I hope to post soon that allows the user to click through a virtual menu system projected on the wall using your body hands for controls. Your choices are currently between two 3D models of installations that will be installed this year. Once you have chosen one, you can then control the camera around the model to explore its facets and scale. It is currently in the prototype and testing stage but was released at a party last week in Houston for user testing.

The next project in line is one using an EEG headset and an eye tracking camera in collaboration with the video artist Jerome Morrison. This project is for a party for the graduate school program for neuroscience at the University of Texas. It will be hosted at The Museum of Human Achievement. We are especially excited because the project is using hardware we have had for a while but have been waiting to use for a project. So when asked if we could do something( knowing already a bit about what we’ve wanted to do in the past) we pitched to create a visual neurobiofeedback program. What this means is that the program is affected by your mind and then your mind is affected by the program and on and on until you step away. Our plan is to use the eye tracker to set changes to where the user looks at the image and then to have the EEG brain waves from the user affect what happen to the image based on concentration (or lack of).  

It is an exciting project and I look forward to its progress.