Using mobile apps like those from Ustream and Qik, a few are even live streaming the show. Taking note of this behavior, a compelling opportunity exists to use these devices to create opportunities for fans to interact with the on stage artist and become participants in the performance.

One of the most ambitious efforts in this area is being led by techno DJ and producer Richie Hawtin. An established music technology innovator, Hawtin gained attention in recent years for tweeting his DJ sets. As part of his current tour performing as his alter ego, Plastikman, he and his collaborators have released the “SYNK” application for the iPhone and the iPod Touch.

During performances audience members can use the app to see real time data from percussion gear, live video from an on stage perspective, and the application will synchronize phone visuals with activity on the stage LED wall.  More details on the app can be found here.

Here’s video of the “Synkotik” mode of the app.

In this LA Weekly Coachella interview, Richie Hawtin describes the motivation for integrating SYNK into his performance.

“I think a great concert experience is when unexpected things happen and when people come together to experience something that goes beyond music. The idea was to throw something into the mix that would heighten that experience. With the app we’re able to send people messages, we’re able to actually synchronize things that are happening on stage with the iPhone apps, we’re even actually allowing people to actually have access to my set-up, you know, triggering sounds. So everyone talks about technology democratizing everything. With our app we will actually blur the lines between the performer and the audience, actually forcing questions of, who is the performer, who is being in control, It is an experiment.”

In addition to the work of Hawtin and his collaborators, it should be noted that there are other efforts in this area, such as Eyebeam’s Open Video Sync project.  It aims to create an open source codebase which will allow multiple iPhones to synchronously playback video using Bluetooth or Wifi transmitted timecode.

Using smartphones apps for interactive concerts is a concept that is still in its infancy.  It is clearly a powerful avenue for tech savvy artists to push the boundaries of live performance.  Furthermore, by creating a way for audiences to participate in the performance, fans are given an additional incentive to buy tickets, an opportunity which cannot be overlooked in today’s music industry where artists are increasingly dependent upon live show revenues.

Researchers at the University of California San Diego Computer Audition Lab have developed a system that could address this problem by allowing people to find music using descriptive words rather than artist names and song titles.  For instance, a person could enter the words “high energy guitars” or “romantic vocals” and then receive a list of tracks that match that description.

The USCD system is capable of ingesting songs and automatically tagging them with annotation data without human intervention. To provide accurate results, the system must first be taught to hear music and describe it using natural language.  The training process uses digital signal processing and machine learning algorithms to expose the system to a broad array of music along with the words people use to describe it.  For example, to be able to accurately identify music that is referred to as “driving rock”, the system must analyze a large number of driving rock songs and then identify signal patterns that make that particular style of song unique.

The researchers have been gathering training data through crowdsourcing using an innovative Facebook game called “Herd-It“.  In this game, users are played a song snippet and asked to associate descriptive words and phrases with it.  Users earn points based on how well their answers match those of previous players.  Here’s a video describing the game.

The research group’s latest work in improving automatic music analysis was recently presented at the 2009 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) in the paper “Dynamic Texture Models of Music,” by Luke Barrington, Antoni Chan, and Gert Lanckriet.

With the continuing decline of the radio DJ as taste maker, web based music search and discovery tools will become increasingly important. With further development, machine learning driven music search systems such as this one could provide an intuitive and compelling method for listeners to find music they will enjoy.

Here are video captures of the performances.

This promotion is an excellent example of using innovative technology and engaging content to capture audience attention. Simultaneously, it provides a unique avenue for artists to promote their music and live shows.

Augmented reality (AR) is a technology that has been around for quite a while, primarily in the academic and research domains. Recently, its mainstream presence has increased due to the development of the Flash version of the Augmented Reality Toolkit (FLARToolkit). The ARToolkit was developed by Dr. Hirokazu Kato with Dr. Mark Billinghurst at the University of Washington’s Human Interface Technology Lab (HITLAB) over ten years ago.

In 2000-2001, I led a team which modified the original C based ARToolkit to work on 3D accelerated desktop Windows PC’s. We used the toolkit to develop interactive augmented reality projections for the band Duran Duran’s live concert tour. Here’s a video of ARToolkit effects that we used in the live shows.

The project was documented in this presentation at the 2002 Augmented Reality Toolkit Workshop:

Jarrell Pair, Jeff Wilson, Jeff Chastine, Maribeth Gandy. “The Duran Duran Project: The Augmented Reality Toolkit in Live Performance”. The First IEEE International Augmented Reality Toolkit Workshop, 2002. Download PDF

Now that it’s possible, what will be the breakthrough applications for mobile live broadcasting?   I think the answer may lie in looking at the trend of celebrities using Twitter.  Twitter is popular with stars because it is simple and easy to maintain.  It can be almost spontaneously updated unlike a traditional blog or personal website.  Celebrities can easily enhance this fan communication channel using Ustream.tv, Qik, Kyte, or Flixwagon.  In particular, mobile broadcasting could be appealing to touring musicians who rarely have an opportunity to sit at a computer and send a well composed personal blog post. Rapper Soulja Boy has been an early adopter in this area by using Kyte’s mobile platform to keep his fans in the loop.  In a similar fashion, Lil Wayne is set to begin using Ustream.tv’s mobile application.  Mobile broadcasting is clearly a concept to keep an eye on over the next year.

Here’s Ustream.tv’s mobile demo video.

Microsoft Tag has recently become publicly available at http://tag.microsoft.com/.  Visit http://gettag.mobi/ to obtain a reader for most camera equipped smart phones.  Potential applications for media distribution, promotion, and advertising are vast.  For example, a band could enhance their show flyers with tags linking to online ticket offers and guest lists.  A couple of years ago I experimented with Semacode, a similar technology, to develop demos showing how tags could be used to distribute music videos and film trailers.

Demo 1: Music Video Distribution With Visual Tags

Demo 2: Film Trailer Distribution With Visual Tags