Wednesday, April 29, 2009

DevDiary: Audio stream handling improvements

I'm currently working on some improvements to the audio streaming code used in my radio app engine. Specifically, I'm working on improving the buffering of the audio stream. One of the more common complaints with radio apps in general (not just mine) is the behavior when the wireless connection is unreliable.

A lot of the connection quality issues depend on the data rate of the stream in relation to the current wireless signal quality. If the wireless connection is too poor, then it will be technically impossible for the connection to maintain a fast enough speed to keep up with the audio stream's data rate. If the audio is playing at 64K, but the wireless connection is so bad it only allows 32K of data per second, the stream is going to stop playing . In this specific situation, there's nothing I as a developer can do. I can't violate the laws of physics and get more bandwidth out of a bad wireless connection.

However, there are some things I can do if the wireless signal quality decreases only intermittently. If the signal gets so bad that it is dropped, my radio engine will currently stop the audio playback and attempt to reconnect. As part of this reconnection, the current audio buffers are cleared to start with a clean slate with the new connection. So, when a connection is lost, the end-user will immediately notice this as the audio will stop while the app tries to reconnect. A better idea is to maintain the current buffered audio data and allow that to continue to play while the app attempts to reconnect in the background. This is one of the improvements I've made.

Based on the audio stream bit-rate, this now allows a fairly decent time window for the app to reconnect before the audio buffer runs out of data. With a 128K "live" stream, based on the testing I've done it appears that the furthest ahead in time you can get is about 12.5 seconds. With a 64K stream, that approximately doubles to 22 seconds. So, if the app can reconnect within this time window, there should be no interruption in the audio playing.

I've done a road-test with my new code - I drove a section of the highway where I'd normally lose a connection with my app about a half dozen times. I'm happy to report the audio playback wasn't interrupted once with my new code changes! The app lost its connection to the server several times, but with my improved buffering, it wasn't even noticeable audio-wise.

I'm now working on seeing if it is possible to seamlessly re-stitch the two streams back together by comparing the audio packets. I'm not sure if this will be possible. I haven't yet determined if the streaming server always serves identical MP3/AAC frames for the same point in time in a stream. If it does, then there might be a time overlap between the buffered stream and the newly acquired stream and in that case I can simply compare the packets and find where the two streams overlap and merge the buffers at the point. This way, the streams will be seamlessly merged.

All content copyright © 2009  Brian Stormont, unless otherwise noted.   All rights reserved.