Wednesday, October 21, 2009

Supporting AAC+ via the iPhone SDK

I've been asked a few times about how I've managed to support AAC+ v1 (also known as HE AAC v1) using the iPhone SDK. Even though AAC+ has been supported since iPhone OS 2.2, the process is not currently documented well at all in Apple's iPhone documentation. About 6 months ago, when I had a project requiring AAC+ support, I ended up using one of my iPhone Developer tech support credits to get an answer from Apple. Even then, the answer provided by Apple was fairly vague and it took a few more hours of tinkering for me to get things working.

Since it's apparently still not a well documented procedure, I figured I should share this information. I may be shooting myself in the foot competitively - AAC+ isn't something every radio app currently supports - but I do try to share with the developer community. AAC+ support isn't meant to be some secret feature usable only by some developers - it's officially supported by the SDK. It's just that the documentation is lacking.

This is not meant to be a full tutorial of using the iPhone SDK audio streams and queues. I am assuming you can already play standard audio streams in your app.

When you receive the audiostreamproperty notification for an AAC+ stream, initially the stream property will be AAC. You'll then need enumerate the kAudioFileStreamProperty_FormatList for that property. Retrieve the format list via a call to AudioFileStreamGetProperty(). Loop through the format list and if the stream is in AAC+ format one of the format IDs will be kAudioFormatMPEG4AAC_HE. This is the descriptor you'll need to use for AAC+ support when you call AudioQueueNewOutput().

This last point is key! You must use this new descriptor pulled out of the format list to create your audio queue. If you use the original descriptor, you will only be creating an AAC audio queue rather than an AAC+ one.

Oh, and don't expect this to work on the simulator. For some reason HE AAC is only supported on the real iPhone hardware - the simulator will never return a result of HE AAC - at least it didn't under 2.2.1. I haven't re-tested with any of the 3.x sims.

Also note that the SDK currently only supports AAC+ v1 with SBR (Spectral Band Replication). The PS (parametric stereo) feature of AAC+ v2 does not work and results in a mono audio playback. Also, the AAC+ support only works on OS 2.2 and higher.

Here's an example of how I detect the HE AAC descriptor in an audiostreamproperty notification handler . It's just a snippet of code taken out of context, but hopefully gives you an idea of how it is done:

AudioStreamBasicDescription asbd;
UInt32 asbdSize = sizeof(asbd);
UInt32 formatListSize;
Boolean writable;

err =
AudioFileStreamGetProperty(inAudioFileStream, kAudioFileStreamProperty_DataFormat, &asbdSize, &asbd);
if (err) { /* do something for an error */ }

err =
AudioFileStreamGetPropertyInfo(inAudioFileStream, kAudioFileStreamProperty_FormatList, &formatListSize, &writable);
if (!err) {

        
// get the FormatList data
        void* formatListData = calloc(1, formatListSize);
        err =
AudioFileStreamGetProperty(inAudioFileStream, kAudioFileStreamProperty_FormatList, &formatListSize, formatListData);
        
if (err) { PRINTERROR("get kAudioFilePropertyFormatList"); free(formatListData); ; }
        
        
// Scan through all the supported formats and look for HE AAC
        for (int x = 0; x < formatListSize; x += sizeof(AudioFormatListItem)){
                
AudioStreamBasicDescription *pasbd = formatListData + x;
        
                
if (pasbd->mFormatID == kAudioFormatMPEG4AAC_HE){
                        
NSLog(@“Found HE AAC!");
                        
// HE AAC isn't supported on the simulator for some reason
                        if (!TARGET_IPHONE_SIMULATOR){
                                
memcpy(&asbd, pasbd, sizeof(asbd));
                        }
                        
break;
                }                                
        }
        
free(formatListData);
}

25 comments:

R.Aubin said...

Awesome! Thanks for the write up and the example code. ;)

Andy Buckingham said...

Thank you very much for sharing this code snippet. I'm dropping this in to the existing AudioStreamer class from Cocoa With Love (http://cocoawithlove.com/2009/06/revisiting-old-post-streaming-and.html)

The only problem I'm having is knowing what the value of writable should hold?

Mostly Torn said...

Hi Andy,

writable is just a Boolean. You don't set any value ahead of time - it is a return value from AudioFileStreamGetPropertyInfo()

Anonymous said...

MANY MANY THANKS!

Anonymous said...

Thanks a lot!!!

Neha said...

I verified that there is a parameter for AAC_HE but how would I use that decoder to play my audio file?

Mostly Torn said...

Neha,

You would then use the AudioStreamBasicDescriptor (asbd) for this format when you call AudioQueueNewOutput()

Neha said...

This is what I did
aqData.mDataFormat.mFormatID = kAudioFormatMPEG4AAC_HE
Forced the FormatID before calling AudioQueueNewOutput().
But when I try to play aac+sbr file, don't hear anything.

Mostly Torn said...

Neha,

You can't just force the audio queue to HE AAC in that manner.

You have to do something similar to what I demonstrated in the sample code in this blog post. You have to enumerate through the format lists that were detected by the SDK and then use the one that is HE AAV. Use the ASBD returned to create the audio queue.

This code assumes you are familiar with using the audiostreamproperty notification handler on an audiostream. If this is not what you are doing, then you can not use this approach for playing HE AAC.

Neha said...

How about using the AudioFormatProperty 'kAudioFormatProperty_FormatList' on AudioQueue services?

Charles said...

Neha, if you are using the AudioFile API, then you need to use AudioFileGetPropertyInfo (to get the size of the format list) and then AudioFileGetProperty to get the list itself. Then iterate through the list and look for an AudioStreamBasicDescription that has an AAC-HE format ID. You can then use this to create your queue.

bikeath1337 said...

Thank you very much. It worked for me!

Joe said...

This is great stuff, Brian, thanks so much. Have you figured out how to deliver it to the iPhone via rtsp? We're looking for true streaming of AAC+ to the iPhone. Thank you! - Joe

Mostly Torn said...

Hi Joe,

Unfortunately, there's no native RTSP support on the iPhone so it currently something one would have to develop from scratch.

I've been looking into doing it, but it's probably a several day effort (or more) and I've only had one radio station ask for it so far, so I can't justify the time investment.

pataswing said...

Very helpful post. I managed to get it working without too much trouble.

There's one error in the source provided : you placed a "break;" in the following if statement :

if (err) { PRINTERROR("get kAudioFilePropertyFormatList"); free(formatListData); break; }

This produces an error, as "break" doesn't work in if statements.

I put it in comments and it seems to be working all fine.

Anyways, thanks for sharing !

Mostly Torn said...

Hi Pataswing,

Thanks for the heads-up regarding the break statement. The code snippet was taken from a larger "while" statement (not included in the example), which is why there was a break originally.

Roy said...

Ok... so... you are officially my first man-crush!

Thanks for this write up and the supplied code - I was able to implement a cut down version of it (minus the buttons and labels) in under 30 min.

Thanks again!

Abhinav said...

Hi,

I was trying to play internetradio stations. I was successful yo play mp3 streams but i couldnt play aacp steams.
After searching a lot, i came to know that coreaudio-api doesnt support aacp. But the aacp format streams can be decoded by either of aacp/aach/aac with aacp being the richest in quality and aac being most decodable.

My problem is that i open an AudioFileStreamID by using AudioFileStreamOpen().
When I recieve the data from NSURLConnection callback, i pass the data to AudioFileStreamParseBytes(). In case of mp3 streams, the property callback is called when a property is found. Thus in case of 'ffmt' i.e file format property, i use the AudioFileStreamGetProperty() to get the AudiofileStreamBasicDescription.

But in AACPm the property callback is not even called. Can anyone tell me where i should call the AudiofileStreamGetProperty to get the formatList. i am confused in this.

Abhinav said...

sorry, AACPm is AACP

Jiropole said...

Very cool, your efforts are really appreciated. With your example I was able to acquire the higher-quality aac+ asbd structure. My problem is I can't even decode normal aac streams. Every stream I've tried triggers an error from AudioFileStreamParseBytes: "typ?". If you have a hot minute, can you stream e.g. "http://u10.di.fm:80/di_progressive_aac"? I can stream nearly any MP3 stream so the rest of my streaming code is solid...

Mostly Torn said...

Hello Jiropole,

Yes, that aac stream works fine in my test app. You can verify it yourself if you try out the free Tunemark Radio app. (Choose to add a custom URL).

Cheers,
Brian

Jiropole said...

Thanks, I will grab your app! I don't want to bug you, feel free to pass -- but is it kosher to wait until _ReadyToProducePackets to check all the properties, setup audio queue and set magic cookie? I didn't have any luck servicing individual property changes, so I just wait for the big one. I'm confused because it works for mp3.

Mostly Torn said...

Yes, I also wait until the ReadyToProducePackets event occurs.

Mostly Torn said...

Also, when you are creating the audio queue (via AudioQueueNewOutput()), it is important that you use the AAC property list which you parsed. If you are instead trying to use an MP3 property, that could be the cause of the your problem.

Jiropole said...

Suffice to say I'm a dummy -- it didn't have to do with audio queues -- but I won't go into it here. At least I got it working ;)

By the way, I'm really impressed with your app! I didn't know anyone else was doing more than a super basic visualizer with their radio. The least I can do is send you a gift cert for our app Spark Radio. Please hit me at: jesse [at] handcastmedia [dot] com.


All content copyright © 2009  Brian Stormont, unless otherwise noted.   All rights reserved.