Beta 16: Problems with sound playback

I have had consistent problems with sound playback using the iPhone runtime. With every sound I have tried in all of wav, mp3 and ogg, I experience lag when the sound first plays. Wavs have the largest frame drop. I have also tried different bitrates and sample rates with no difference.

I have attached a simple application which plays a sound when you click a button. You can also see that the frame rate drops whenever this button clicks.

The sound is a 16-bit Mono 128kbps mp3 at 44100hz

Every time I click the button I get 39-42 FPS.

Didn't test your app, but what I do in those cases is first fire the sound, mark somewhere (with a flag) "Sound fired" and then sync the video in the next loop.

The way to do the latter is to place all actions that should happen when the sound is played, before the condition that triggers the sound. And to be sure that those actions happen when the sound has been fired I add a "Is Flag X on?".

Hope this helsp in your case.

Netninja, Yves explained that the OGG is not supported, and I'm using MP3 for the soundtrack and. wav files for the rest, I've also noticed that sometimes a sound associated with a touch of an object does not sound the first and second if so, the bug is random, I think our friends should review very well Clickteam this section, it is crucial.
I guess you are doing.

Yes, this is a problem I have noticed too. I guess that the sound engine of iOS needs some initialisation before working.
I suggest that you play a dummy sound in the title page of your game, like that the delay will not be noticed.

Thanks Francois and this time and we are still beta, so we have time, I think it's better IOS Exporter debug as possible before launching applications to the App Store, at least not that they are made with MMF.

I feel that we are already light years from Gamesalad and I think that Clickteam have a great product under his arm.

Thanks for the feedback guys, but I don't think anyone really noticed the severity of this problem.

Whenever I play a sound effect, my game lags. This isn't the first time I play the sound... this is every time a sound is played. A dummy sound won't help in this situation.

So if you had a game where you wanted to play a sound every time your ship shot a bullet, then it wouldn't be playable. The frame rate drop is very noticeable and I don't think I'm doing anything wrong with my sounds - check the example - the sound is a very simple one.

I don't see how I can release a game where it has such a major performance issue.

As I said. I used to have that problem and fix it with the workaround I posted earlier :) And beleive me when I say that my ship shoots lots of bullets.

If i remember correctly mp3 uses a separate engine than wav files, they are better suited for music and not sound effects.

Use a dummy blank wav to get things started, then the wav lag stops.

The lag in the gampe depends on the sound, its frequency, it length. We have not yet been able to pinpoint which frequency or sound, but safe to say that sounds with lower frequencies will play faster.
To play sounds, I am using the method recommended by Apple. I have two solutions to prepare the sound player for playing a sound, either do it before the sound is played, or after, when the sound has just stopped. I could do the latter, but there will be lags in the game when the sound stop, which would be the same after all.

Francois - so is there a specific sound format that plays better than others? Could you provide some guidelines as to what frequency we should sample the sound at, and other settings? I have not been able to produce a sound that doesn't lag, but perhaps I am using the wrong settings. In my example my sounds are extremely short already.

I've attached another test. This has two .wav files with the same problem, as I mentioned before.

The first button (top one) has the sample rate at 44100Hz, 16 Bit.

The second button (bottom) has the sample rate at 22050Hz, 16 Bit.

Clicking either button will make the frame rate drop to 46 FPS, and quite often to 42 FPS. This is a slight performance improvement over mp3, as we already knew it would be, but it's still an issue. The sample rate in this case doesn't seem to affect the lag. I can't think of any more options to change with the sound format.

42-46 FPS is in the range of 10% slowdown. It's very noticeable, especially in action games. It's quite the deal-breaker!

Please look into this! :(

Have you selected the property "Timer-base movements" in the property of the frame? With this, the speed of the objects should be the same whatever the fps (as it is based on the timer).
I have not yet found a pattern on which frequency causes delay.
I have don some research on the Internet, and I have found one sound engine that plays sounds with minimal delays.
But it is too late to implement in the version of the iOS exporter as it might generate a lot of bugs and delay the product once again.

Thanks for the feedback Francois. I have "Timer-base movements" selected by default, but I am not using built-in movements (I'm using custom) so I don't think that it will make a difference even if it was off.

Does anyone else have this problem with their games or am I doing something wrong?

I'm adding sounds to my game tonight - I'll let you know what I find.

BTW, did you try 8-bit just for comparison and are you using mono or stereo samples? Sorry I cannot download and try your wavtest.mfa at the moment as I'm at work.

I'm using Mono... 8-bit is the only thing I didn't try. But I figured that the sound quality was getting pretty low with the stuff I was doing. I'll try 8-bit as well just to see.

It seems I´m also experiencing this problem:

http://www.clickteam.com/epicenter/ubbthreads.php?ubb=showflat&Number=225995&#Post225 995

Ah doesnt sound (pardon the pun) very good, if its affecting the performance that much :(

It is :( ... read this post: http://www.clickteam.com/epicenter/ubbthreads.php?ubb=showflat&Number=226005#Post2260 05

I've now got sound in my game, using Beta 17. The MP3 I have playing causes no slow down at all - running 60+fps, but I think I read that you said the first MP3 is played by the hardware so that works fine.

I've added in a short sample, maybe a 1/3 second in length. When this plays the FPS on my itouch drops from 62 to 51, returning to full speed when it is not playing. I did another test to play ten instances of the sound every second and it still only dropped to 51. Put a different sound in and it would sometimes drop to 47fps while both are playing.

This is definitely a weak point in the runtime, although it may delay the release I think if you have another method of handling the sound that does not cause this slowdown (or is less of an impact) then it's got my vote.

I have been researching a little and found that audio slowdowns are a common cause of headaches on iPhone when using Apple solutions:

1) http://stackoverflow.com/questions/607942/playing-sounds-in-iphone-games-slowdowns
2) http://www.iphonedevsdk.com/forum/iphone-sdk-development/36211-avaudioplayer-usage.html

To deal with it, as a workaround, many games play each sound at the begining of the game/level/frame at a volume close to zero and then reuse the sound by resetting its position to zero, share players with other sounds, etc.

But there's one developer that, as a result of

http://stackoverflow.com/questions/986983/what-do-you-use-to-play-sound-in-iphone-games

Created its own open-source library for uncompressed sounds like wavs based on OpenAL:

https://github.com/zoul/Finch

According to many devs (see last post in 2 above), by using this library the slowdowns are gone.

You can also find a great explanation on how to implement OpenAL here for the iPhone:

http://www.gehacktes.net/2009/03/iphone-programming-part-6-multiple-sounds-with-openal/

Hope this helps :)

Quote:

---

Originally Posted by AndyH

The MP3 I have playing causes no slow down at all - running 60+fps, but I think I read that you said the first MP3 is played by the hardware so that works fine.

---

Just as you found out, once the sound is actually playing, the game seems to generally run fine. It's the moment that the sound starts to play that causes a frame drop. So playing an MP3 at the start of frame seems to be okay.

Quote:

---

Originally Posted by AndyH

I've added in a short sample, maybe a 1/3 second in length. When this plays the FPS on my itouch drops from 62 to 51, returning to full speed when it is not playing.

---

62 to 51 FPS is a huge slowdown! That's about 18% slower which is definitely noticeable.

Thank you for the link to the audio engine. I'll have a look at it, but as I said earlier, this will not be for release 1, as it is too late, and we have to get it out.

If you have a background music that plays during the frame, you should have it in MP3 format, and play it as the first sound : it will use the MP3 hardware decoder, and wont take any machine time.

> Thank you for the link to the audio engine. I'll have a look at it, but as I said earlier, this will not be for release 1, as it is too late, and we have to get it out.

I know, just wanted to let you know about this.

> If you have a background music that plays during the frame, you should have it in MP3 format, and play it as the first sound : it will use the MP3 hardware decoder, and wont take any machine time.

For sounds (not music), since I cannot wait for version 1.x (being x > 0), this is what I´m going to do:

1) Create an active object that will serve as a sound manager,
2) Give each Alterable Value the name of one wav sound,
3) Let the manager play each sound at the begining of the frame all sounds at volume close to zero,
4) If, for each sound being played, its position is greater than or equal to T time, the manager will pause it,
5) If there is an event that requests a sound (Alterable value of the sound in the manager, say, equal to 1), then the positions of the sounds goes to zero, and the manager resumes it.

All sounds have cannot be uninterruptable ones. I haven´t started yet working on this since I'm working on retina support, so let me know what you think before I start implementing it :)

BH sounds good (excuse the pun), fancy making this into a starter widget for wider use?

No problem. Just let me see if it works first. I will start to implement it early next week. :)

Great idea BHGames! I'm going to try a similar work around. Post any files that you come up with :)

Wait till next week

When you pause the sound, you are going to detect if it is past T time- is this going to have to be a hard coded value for each sample? I guess for a safety buffer we'll have to add a small amount of silence to the end of the samples? And I also guess we're going to need to have an event that checks if the sound is not playing / paused to restart it?

You're right. It is cumbersome, but unfortunately in my case it's the way to go since I'm already in the market. So I cannot wait for a fix in versions 1.x of the exporter (being x > 0) :(

EDIT: I will try no to add a silence at the end of the raw sound. Just a "Lenght minus T millisecs" should suffice, being T a value really small and common to all sounds. If for any reason the sound reaches its end before the evaluation kicks in, yes, I will have to play the sound again. But hopefully, those cases might be exceptions since the evaluations will be done on eahc update loop. Fingers crossed ...

I just did some tests to try and get some workarounds, but I haven't been able to find a solution. I did a similar method to what you suggested BHGames, using the sound position to try and play sounds again. However positioning the sound has the same effect as playing a new sound from the start - the game still lags.

In the attached file, my app plays a sound on a channel and repositions it to the start when you click the button. If you let a sound get to the end then it actually "stops", so you can't reposition it.

Unless a workaround or solution can be found, then I don't think the runtime should be released to the public. Customers who buy it will not be happy.

I'm being stubborn about this because it's a critical feature which is essential to the finished iPhone runtime. Games need sound effects. If the runtime can't play sound without lagging then nobody will be able to make games.

agreed that sound is a critical feature of the exporter, people buying this will want a simple play sound solution, and not having to implement some huge workaround like pausing sounds and preloading them to get it to work.

I think the IOS Exporter can not go out with this lack as strong, and I understand Francois to get the product that was dated for December, but if the product is released and begins to criticize this aspect all work is lost.

It is best to repair this and that version 1.0 be as robust as possible. They are close and as I said before, the mere fact that the IOS Exporter generate Xcode project leaves them light years ahead of competition gamesalad. So there can be failures and less one like this.

Changing the sound routines is not a matter of one week. This is a major change that will necessit several weeks of beta test. We have to get the product out, it is a vital matter for our company.
We will update it as soon as possible after.
And the sound issue is not a problem for everyone : three games arez now on the app store.

And none of them is using sound heavily, Francois. The thing which we are saying is that releasing a product with faulty sound system may only work worst on the long range than delaying runtime for couple weeks time.

Francois I understand your position, what it is to be in charge of a project development cost is very high.

In addition we are beta testers and we will know to handle this situation. Now it is good that the first update to fix IOS Exporter this special topic. That if I think you should solve minor problems for the 1.0 release.

Greetings and thanks for your confidence in us.

Quote:

---

Originally Posted by netninja

I just did some tests to try and get some workarounds, but I haven't been able to find a solution. I did a similar method to what you suggested BHGames, using the sound position to try and play sounds again. However positioning the sound has the same effect as playing a new sound from the start - the game still lags.

In the attached file, my app plays a sound on a channel and repositions it to the start when you click the button. If you let a sound get to the end then it actually "stops", so you can't reposition it.

Unless a workaround or solution can be found, then I don't think the runtime should be released to the public. Customers who buy it will not be happy.

I'm being stubborn about this because it's a critical feature which is essential to the finished iPhone runtime. Games need sound effects. If the runtime can't play sound without lagging then nobody will be able to make games.

---

netninja: I was thinking more in a solution like the one I´m attaching. Unfortunately, it doesn't seem to work on iOS runtime (but it works ok on the PC runtime). So I´m posting it as a bug report.

BHGames - your code works virtually the same as mine, except it isn't looping unnecessarily. The problem will still be present when you reposition the sound as it has the same lag as when you play a sound from the start.

Francois - I understand the need to release the iPhone runtime as quickly as possible. I think that it might upset some of your users, but the choice is obviously up to you. The current games made with the MMF runtime don't have fast movements so if there is lag, it's unnoticeable.

If you still think it's best to release the runtime with this issue, can you please make this sound issue a number one priority to fix after the first release?

> BHGames - your code works virtually the same as mine, except it isn't looping unnecessarily. The problem will still be present when you reposition the sound as it has the same lag as when you play a sound from the start.

So I guess from your words, that my code works on your device (?)