Moving forward - The Fusion Audio Handler

Hey everyone.

I've been pretty caught up in the world lately, so my apologies for staying away from this community for the past year or so.

First, Please login to see this link.!

I've been moving toward actual programming and working on various projects when I have time, but that's basically meant that I haven't used Fusion very much recently. With the impending release of the next version of Fusion, I've realized that keeping some of my more interesting Fusion projects to myself isn't going to matter since they're about to get superseded anyway.

The Please login to see this link. is the best handler/processor of audio at runtime in Fusion because it has full support for object-oriented programming, dynamic control of audio objects, generators, multi-scene support if you need that (not recommended!), internal/external audio directories, and hierarchy of channel importance (default is 0 = auto-dropped, 1 = locked, but you can set the auto-drop cutoff higher and implement importance levels). This is not an extension in and of itself but it will work seamlessly for handling all audio-related tasks, including ones that go through DSP or OpenAL extensions. I can't put myself through working in the Clickteam Windows SDK, so a few things are missing that I would've liked: an OpenAL extension, a custom extension for the minimal audio tokens, and some more efficient processing of audio channels through a central handler extension. Without a custom extension, this is the most efficient way that Fusion can process audio at runtime if you need control over every channel.

This version of the Fusion Audio Handler is a variation of the one that I'm using, which does not inherently support multi-scene audio, so it's possible there are some features or functions that won't work correctly yet (especially with multiple scenes). If you find anything like that, definitely let me know and I'll post an updated version of it.

Making this handler, I discovered a couple of weird little inconsistencies in the way Fusion processes audio-related events, so I implemented workarounds already in the Audio Handler group. The biggest feature that I wish I could've added to this handler was on-call audio dropping--that is, dropping channels as soon as they've ended--but Please login to see this link., and since none of their audio functions are open-source, there's no way to get around that without an extension. To illustrate this:

Create token for channel 1
>Channel 1 plays
>Channel 1 finishes
Token for channel 1 still exists

How do I notify the token for channel 1 that the channel's ended? It was a simple request for Clickteam but that's how stuff goes sometimes. In most cases, the workarounds I have should work sufficiently well by removing all objects that are no longer playing when a new audio object starts. The obvious case where this isn't ideal is when there's already a large workload because this is basically forcing Fusion to perform garbage collection in the middle of that workload. You can also manually call a function that drops all ended channels or you can call another function that drops the oldest unlocked channel that hasn't ended. Again, ordinarily you won't have a reason to call either of these functions (they're called automatically in the handler when needed), but they're definitely available if you'd like to call them yourself.

I believe I added a function to the [Resume on scene start] group that can, if enabled, restore each audio object's position from when the previous frame ended. You most likely don't want to enable this function, but if you have some audio-synchronized events (I highly recommend never switching scenes if you do), you can enable this function to keep everything synchronized. There will be an audible jump (a slight "skip" backwards) when the new scene starts depending how long it takes for Fusion to clear the old scene and start the new scene if you enable this function. Keeping this function disabled, the Fusion Audio Handler will automatically carry over all non-ended audio objects and they'll then continue to affect their original channels when the new scene starts. If you don't have multiple scenes, remove these two groups and then remove the array object from your frame because I personally don't like Fusion's array object and this is the only reason I have it in the Fusion Audio Handler.

The only extension that the Fusion Audio Handler uses is the Internal List Object because of its amazing performance. I was going to finish porting this extension to Android but I haven't gotten around to it. If you need to use the Fusion Audio Handler on Android, switching all of the events from Internal List to an invisible List should give you about the same level of performance that a ported Internal List object would give you.

To add audio to the handler: Create an audio token, assign the necessary values to it (definition type, token type, linked object if applicable, and linked object values if the definition may or may not have a linked object), then call its definition. The definition fills in the repetitive values and then calls the audio handler to add it. You'll want to have as many values in the definition as you can get so that you won't need to change very many values in the game engine.

Second, like I said, I don't see a scenario where my appearances are going to increase; rather, I'm going to be around here less and less. Depending on my availability, I'll have a few other resources to share before I vanish entirely. I haven't had an internet connection at home for the better part of this year so that's been a major roadblock for getting new videos up on YouTube. My goal with that is to start rebranding and changing the direction a little bit coming up. Leave the "snail" stuff behind--I'm thinking just shorten it from "Snail On" to "On". Realistically, I'm not sure what I want to start using for a brand/identity and that's been something else that I've been pondering this year.

I'll post a video up here tomorrow with the link to the Fusion Audio Handler and I'll go into more details in the video. One thing that I do want to say is that I'm not going to charge money for downloading the Fusion Audio Handler or any of my other soon-to-be-released resources. I've been critical of Clickteam in the past for their closed business model, so it'd be fairly hypocritical of me to do the same. Quite bit of work and my time went into creating the Fusion Audio Handler, so I'll be happy if you'd like to donate what you feel it's worth, but I'm not going to ask for anything or put a price on it to download it. I created it for my own projects, first and foremost, and anyone's use of it certainly won't decrease its value to me--in fact, the opposite's true! If you feel it's valuable to you and it offers features or convenience you wouldn't have had otherwise, definitely let me know. I've already made arrangements with people behind some highly professional projects, so keep me in the loop if you'd like to use it or distribute it. Regarding giving me credit: Let me know first since, again, I'm not going to be using the "snail" brand anymore and I haven't decided yet how I want to rebrand.

The other resources that I have are going to be available to the community soon and they're at least as useful as the Fusion Audio Handler is. Not sure yet when I'm releasing those but all of these are pretty related.

As always, I'm happy to hear from people via private message, Steam, YouTube when I check my messages there, and email even if I'm not around here very much. Say hello or ask questions!

Don't have the video up yet but Please login to see this link..

Thanks for the note there. It's the natural evolution of where he was going with it.

Of course it handles positional audio efficiently and easily. That was one of my requirements. A list of what I needed:

Positional audio
-Direct control over pan
-Direct control over volume
Frequency-controlled audio
-Based on speed, position, and several other factors
Locking channels
-Music is never interrupted
-Audio carries over between scenes (whether Fusion's "frames" or custom-made scenes)
Playing and pausing audio
-All playing audio can be paused
-All paused audio can be resumed
-Specific audio can be saved
-Specific audio can be returned
Exact time positioning
-Audio starts only when called
-I don't recall why I took this out of the handler I posted, but I have it in one of my projects
Audio syncing with game time
-Not Fusion's strongest suit at all
-Audio is synced with game time, not real time
Ability to affect audio by object
-Each object receives a definition
-These definitions determine its behavior and how the handler interacts with it

One of the things I'm really unhappy about in Fusion is that it doesn't support lambdas. This means that even with the object definitions, I have to hard-code several functions in the Fusion Audio Handler, namely the easing algorithms. If Fusion supported lambdas, I could've assigned the easing functions in the definitions and then the Fusion Audio Handler would've used the function within the definitions for easing effects. As it is, you have to define these easing effects and then tell the Fusion Audio Handler which easing effects any given object should use. I think I set the default to a + (b-a) * 0.1, which a is the current value, b is the target value, and 0.1 is the easing amount.

Realistically, I'm not sure if more than 32 samples at any given time is ideal. "Auditory vomit" may be the term I'd use in cases where more than that number is playing, though others may call it "immersion." You can assign priority to objects so that they drop in those situations not by first in, first out but by juniority. You'd do that by adding a new function for dropping channels. It's something that I considered earlier but I don't have a use for it and I couldn't think of any broad rules that would dictates that. Probably you'd have to assign the priority when you create the object by checking how many other objects of that same priority exist and then increment it (if you don't want to use the age comparator).

You can see even in that case that there's a lot of logic involved in determining which channel should drop. Obviously you should remove all completed channels first, and then if no channels are available, you'd have to call your function to determine which of the existing 32 objects (plus the new one you created) has the lowest priority. This is where I call the function to drop the oldest non-locked channel, so you'd replace this function call with a call to your new function. Music is automatically locked, so that's going to be a couple of channels. After that, player sound effects should be the next highest priority. After that, explosions. Basically ambient sounds, I imagine, should be the ones you drop in those cases, and you'll want to use the age comparator in the Fusion Audio Handler to return the oldest object of the same priority.

What that will do is drop the oldest of the 16 turret sounds when you create a new object.

Be aware that when you overload your channels (the case that I just described), any function you call that uses comparators is going to slow down the Fusion Audio Handler. While I've optimized it as well as I can, these types of functions will run extremely slowly because they have to loop through all of the scoped objects. Ways to increase performance, then, are scoping your function so that the comparator only has to check a handful of objects instead of all 33 (32 existing objects plus your new one). This is why locked objects are always excluded from the scope within any automated drop function. The default age comparator has to loop all of the unlocked objects, returning the age and ID of each one of them; when an older object is found, the comparator gets the new object's age and ID. This method is slow but I haven't thought of a way to improve it without slowing down the rest of the engine. If there's a better way to optimize the age comparator, I'd definitely like to hear or see it because I don't know of another way to find an object closest to a given value without looping through the class.

I considered using a sorted list to track age but I can't imagine that'd be faster than looping through the scoped objects since it'd require a second list or parsing the first one--either way, that method is going to be much slower in Fusion than looping through the audio objects is. Thinking about this some more: If the age comparator is running too slowly, you can offload it to 32 static variables. You'll still have to run a loop (32 times instead of the number of scoped objects) but the comparator should process that slightly faster than it can process local variables.

Quote

In a [hypothetical] world with no channel limitations at all, all that busywork would vanish: every new sound would automatically be assigned a new locked channel, and all sounds would remain conflict-free.

You're either going to program a handler yourself (C++/an extension) or you're going to use a wrapper (Fusion's with an optional handler on top of that). There's no way around this fact; the channel dropping functions are performing garbage collection. I'm using the latter approach for the Fusion Audio Handler by controlling all of the sound-related operations within Fusion but I'm still relying on Fusion to stream the channels correctly (along with frequency, pan, and volume). While it's definitely inconvenient that Fusion has a 32-channel limit, careful planning should resolve any issues there. Please login to see this link. (similar to what I described) that's only limited by the player's sound card, and most sound cards today support between 32-128 channels.

Does that shed light on why you absolutely need a handler regardless of the design program or language you're using? What I've done is automated the internal handling that Fusion provides while offering support for much more advanced control over every aspect of audio within Fusion's internal processor/handler. Like I said previously, I would've preferred making an entire extension with OpenAL so that we'd have modern sound effects but I refuse to work in Clickteam's antiquated Windows SDK. An extension still has to handle audio (basically the same as what the Fusion Audio Handler is doing) but it'd be able to do that much faster and provide modern sound effects and processing that the internal engine doesn't support. Something to think about but I don't think anyone left in this community is going to take on that task with the current SDK unless you pay by the hour.

Raising the limit to 64 or even 128 channels (I'm sure you can find sound cards that support 256 channels), while potentially alleviating premature channel dropping, does not alleviate the need to perform garbage collection.

Quote

All that stuff is complicated, time consuming, and taxing on the CPU.

I've included all of the common functions you might need already. More specific ones are going to be limited by project, so if there's something that you need, I'm happy to help out where I can. And the performance analysis I discussed previously is relative: Compared to almost everything else you're going to do in your game, the Fusion Audio Handler's extremely light and efficient. Compared to itself--where I'm looking to improve performance as much as I possibly can--some methods will be better than others are, but taxing on the CPU? No. I'm assuming you need to use an audio handler in the first place; if not, you can rely on Fusion's internal one but it offers no flexibility without adding a wrapper on top of it (whether that wrapper is your own, Nifflas's, or mine). My goal is to maximize performance, ease of use, and power of the Fusion Audio Handler, and it's entirely open-source, so you're free to try to get some extra efficiency out of it . If you do make changes to it or improve it, please post those changes so I can integrate them into the public release. I'll continue making revisions by adding the previous version into an inactive group and then uploading the new revision at the same link.

Quote

To put it a different way: imagine if, say, sprites worked the same way. Imagine that each sprite needed to live on one of 32 'channels', and that you had to manually program all these systems into your events to constantly monitor all sprites and activate, deactivate, and swap their channels all the time. Every time you added a new sprite onto the screen, you then had to stop and decide whether or not you need to delete some other sprite laying dormant in the background somewhere, and if so which one. It'd be exhausting.

Fusion's already doing that. And, guess what! You're already doing that, too. When Fusion does it, it's garbage collection. When you do it, it's a very basic form of garbage collection. Why else would you destroy particles after you create them? You don't have to destroy them but you'll eventually run out of memory if you don't.

Definitely get in touch with me about it. Mostly I just want to know who's using it and for which projects.

I don't think I was bashing it at all? That certainly wasn't my intention, so I didn't mean for you or anyone else to think that. Fusion has a very capable audio engine: It plays audio, it can control volume, frequency, and pan, and it supports up to 32 channels. I don't believe there's any bashing involved in stating those facts. If you need additional effects (DSP, VST, etc.), you have to create your own extension or use another design program or language/engine. If you need more control over the audio channels, you need to use a handler.

Should you wish to discuss that with me, please do so via private message so as not to derail this thread.