Wednesday, November 21, 2012

Tunemark Radio with Improved Visualization

Thanks to a new feature of iOS 6, I will be adding significantly improved audio visualization features to Tunemark Radio. Specifically, I am referring to Audio Queue taps, which Apple mentions in their iOS 6 SDK release note, but provide no documentation at all, other than comments in the AudioQueue.h file. If you are looking for the technical details, search for AudioQueueProcessingTapCallback in the AudioQueue.h file. I’ll write up a more detailed description of how it can all be used, along with the hooks I added to RadioKit to support it, in a future post.

Anyhow, once you get over the hurdle of the lack of documentation, you can do some pretty nice things with an Audio Queue Tap. It provides a way to get the raw PCM audio right before it is played, perform adjustments to the audio (apply echo effects, etc.) or just monitor the audio for visualizations.

In my case, I’m interested in using it for adding better visualizations to Tunemark Radio. Since Tunemark Radio is playing Internet radio station streams, it is generally dealing with MP3 or AAC audio, which is in a compressed format and short of implementing your own MP3 or AAC decoder, there’s no easy non-CPU intensive method to get actual waveforms. Instead, in the past I had to resort to just querying the audio queue gain (i.e volume) information and do some rudimentary visualizations with that.

Now that I have real-time access to the raw audio waveforms thanks to the Audio Queue tap, I can do more more dynamic and responsive visualizations. The first one I’ll be adding (which will be available in the next upcoming Tunemark Radio update) is a waveform visualizer. It will look something like this, although it’s a lot more engaging when it’s actually moving:


Basically, the waveform will move in time to the music.

Future updates will include more dynamic visualizers that have various visual effects based on different frequency changes (such as part of an image changing based on bass notes and another part changing based on high frequency notes).

All content copyright © 2009  Brian Stormont, unless otherwise noted.   All rights reserved.