Why are fast loops so slow?

[Pixelthief]

This is something I'm having trouble understanding. I've noticed that with the Klik architecture, spanning from TGF to MMF to MMF2, fast loops are needlessly slow. Something about them, perhaps overhead in their implementation or who knows what, or a lack of optimization by the compiler, makes loops in klikware horrendous for larger scale applications. For example, in any klik program, running a loop that has 100 "on loop: set counter to random(10)" events in it, will produce no slowdown whatsoever. Create a loop that runs 10000 times with a single event as above, will produce massive slowdown, perhaps 5 FPS. This is despite the two programs doing the exact same thing, the same amount of total "operations". In a language that runs naturally at 50 FPS, anything below maybe 45 FPS is relatively unplayable.

Yet this is not restricted to a single loop running a set amount of times. Having multiple loops run each frame, totalling a high number of loops, will cause the same dramatic slowdown. Loop unwinding and such can only slightly mitigate the effects. But being restricted to under 800 loops per frame is a ridiculous limit imposed on MMF2 programmers. Kliking isn't exactly looked upon brightly compared to say C++ or Java, but without this barrier it could easily be as powerful. So what the heck is going on thats making loops so slow? Other languages look at something like concatenating two immutable strings as a "no-no" operation, because it can be so expensive on the processor, but only in Klik is a LOOP expensive. Does every single call to a loop parse the entire code stack again, instead of just the relevant events?

anywho, is there any possible way to overcome this, besides just rewriting code to use less loops (loop unwinding & other optimization). Can varied use of fast loop object, fast function, & other extensions even out the workload somehow? Is there any way to run more than 1000 loops a frame without your computer crying?

[James]

That's because MMF2 only draws the frame at the end of the event list. Loops have to execute instantly, as soon as you tell them to, so you get graphical lag if MMF2 has to wait a hell of a long time before it can draw the frame again.

You could create your own looping system with something like:

+ Counter > 0
- Do something
- Subtract 1 from Counter

And to start the loop

- Set counter to (How many times you want to loop)

Although that might have some structural problems with how you were using fastloops before. The easiest way would be to break your loops down into shorter loops each frame.

[Olivier]

Something is in the works about fastloops. François talked about implementing a new "object loop":
http://www.clickteam.com/epicenter/ubbthreads.php?ubb=showflat&Number=95487

And Yves has an idea to optimize the fastloops by storing their names and the address of each associated event in a table. The current process is to compare the fastloop names which can be slow. Yves will probably explain this better.

[Nifflas]

I've had problems with this in situations where there was absolutely no way I could reduce the numbers of fastloops I needed to use in my application. I came up with two solutions.

* Put every large nest of fastloops into it's own separate group. When you need to trigger one of the larger nests, open the group with the "on loop" events in, run the fastloop, and close the group again. This way, the other "on loop" conditions in the other groups won't be tested for a match.

* Use an extension, e.g. the function loop object for some things and the standard fastloop for others. Although they do the same thing, plain fastloop conditions won't be string compared when you call a fast function loop. It's not the prettiest method, but it was able to speed up the Knytt Stories level drawing process a whole lot.

[Dines]

I wonder if it would be possible to create a fastloop extension which had loops defined in properties, rather than by names in the event editor?

Currently, as discussed above, every fastloop uses the same condition. So every loop with every name is triggered, and those whose name doesn't match the loop we triggered will simply return false and fail to execute. Slow.

However, perhaps an extension could have loop names registered at develop time inside the object properties (so you create a list of names, much like setting up named alterable values).

The extension is coded to add a condition to itself for each loop you defined. So if you had the loops:

"Randomise"
"Render"
"Bind Physics to Objects"

Then the Loop Object would have the conditions:

On Loop: 'Randomise'
On Loop: 'Render'
On Loop: 'Bind Physics to Objects'

...each one posessing its own unique condition ID. It would be interesting to see what kind of a speed increase that gave in apps with lots of loops.

But I must admit, even with just the one loop event, it's a very slow process. It would be nice if we could at least have a condition which forcibly repeated the same event an arbitrary number of times without re-checking conditions inbetween.

ie.

Button Pressed
Repeat 20 times
--- Boil an egg

Then unlike a fastloop, which would constantly re-check the conditions, this would simply boil an egg 20 times, regardless of whether the button was no longer pressed.
(this is NOT the same as the current Repeat function!!)

[Dynasoft]

Unfortunately the loop-in-a-condition isn't possible with an extension, because of the way the selected object list works. If you try to create a loop all you get is an infinite loop. The conditions aren't evaluated in between, so you can never break out of it.

[Pixelthief]

I've had problems with this in situations where there was absolutely no way I could reduce the numbers of fastloops I needed to use in my application. I came up with two solutions.

* Put every large nest of fastloops into it's own separate group. When you need to trigger one of the larger nests, open the group with the "on loop" events in, run the fastloop, and close the group again. This way, the other "on loop" conditions in the other groups won't be tested for a match.

* Use an extension, e.g. the function loop object for some things and the standard fastloop for others. Although they do the same thing, plain fastloop conditions won't be string compared when you call a fast function loop. It's not the prettiest method, but it was able to speed up the Knytt Stories level drawing process a whole lot.

ah opening/closing groups would be very simple, I have every function in its own neat grouping already. Sadly, just the overhead from the raw # of loops I need to run, disregarding what they do, seems to be enough to tank my games. So yeah, I think I'll spread it across the function objects and such. Perhaps even multiple fast loop objects? O_O

[MechatheSlag]

Something is in the works about fastloops. François talked about implementing a new "object loop":
http://www.clickteam.com/epicenter/ubbthreads.php?ubb=showflat&Number=95487

Yeah but that won't make fastloops faster, that'll just be another way of using them.

[Pixelthief]

ok I threw together a simple looping test. As nifflas said, the culprit appears to be the way that all "on loop" actions are added to the stack, regardless of affiliation, hence the effect can essentially be mitigated by either closing groups not in use, or spreading actions out between fast loop/function objects. Loop unrolling has an actually negative effect, since other loops in the game would be needing to process the extra list, which does more damage than halving or quartering the number of a specific loop.

here:
http://claniraq.googlepages.com/loopingtest.mfa

ps: heres what I was working on that was running too slow with all loops into accounts:
http://www.youtube.com/watch?v=2DMXwEkrRgY

[Mark_Pay]

Optimization of fast loops is something I would really like to see too. If they are checking every event and doing a string comparison, it's no wonder they're so slow. I hope that Yves can implement the lookup table Olivier mentions.
At the moment, this makes it hard for a developer to do some basic heavy number/logic crunching that just can't be deferred to an extension.