Benchmarking MMF2 Performance

[MuddyMole]

In case I have mistakenly give the impression that I have experience of the Android (or any other exporters), I'd just like to clarify now - I don't.
Any comments regarding the performance of Android are based on the performance of the hardware (ie. mobile phones / tablets) and are not a criticism of the exporter itself.

However, I think it would be very helpful for users to have some idea of the relative performance they can expect from the different exporters (or rather exported games running on different devices), and so I've made a quick and simple benchmarking example.

I for one, would appreciate it if MMF2 users - and particularly owners of the various exporters (Flash, Android, iOS, etc) - could try it out and post their results here (along with the hardware it was tested on).

Download: http://sdrv.ms/15pJ8Hu

My results (3.2GHz Pentium Dual Core; 3GB RAM; Windows 7)
- Graphics: 100
- Fastloops: 98
- Collision Detection: 100
- ForEach Loops: 101

Thankyou

[Skyhunter]

My results (3.0 Ghz AMD dual core, 4GB RAM, Windows 7, Ati Radeon 5000something)

- Graphics: 259
- Fastloops: 86
- Collision Detection: 99
- ForEachLoops: 75

I tried to also test it with the swf exporter but it took several ages so i stopped it.

[Aasland]

Ran the test on my Vaio Z laptop (Intel i5 @ 2.4 Ghz, 8 GB ram, GeForce GT 330M, Windows 7)

- Graphics: 312
- Fastloops: 97
- Collision Detection: 122
- ForEach Loops: 83

[MuddyMole]

I see you have the Android exporter as well - how was performance with that? (I'm really more interested in the exporters than Windows PCs)

[Jeff]

For each isn't working correctly on Android
It should be updated in the next build as Fernando sent the changes into James for review.

Android doesn't like something in this file it doesn't run very well at all.
I am glad actual games run a lot better then this test file.
I am sure James can tweak performance a bit after he finishes up bug corrections.

[Aasland]

I tried the test on my Samsung Galaxy SIII, but it freezes on the first test.

I also tried the test on my iPhone 5. The framerate doesn't go higher than 60-62.

[ptiseigneur]

I7 870@4.1ghz/12go/HD6950 2GO/seven PRo

-GFX:495
-FL:140
-CD:176
-FE:118

[Konidias]

Intel Xeon CPU 2.4ghz+2.39ghz (2 processors)/12gb ram/win7 x64/Nvidia Quadro 4000 video card/DirectX 11

Results:
- GFX: 284
- FL: 94
- CD: 117
- FE: 93

This is my work PC... I'll have another one for android galaxy S3 and my home PC once I get home tonight or this weekend.

edit: Out of curiosity... care to share what these numbers mean? What would be a good baseline? It appears to be fps... correct?

[Konidias]

For each isn't working correctly on Android
It should be updated in the next build as Fernando sent the changes into James for review.

Android doesn't like something in this file it doesn't run very well at all.
I am glad actual games run a lot better then this test file.
I am sure James can tweak performance a bit after he finishes up bug corrections.

I'm really hoping the ForEach fix comes soon... I simply can't use this for Android games otherwise, since nearly any game I make ends up using the ForEach object in some way.

[MuddyMole]

Thanks for the responses everyone

@Jeff - Thanks, I guess I'll have to make the tests a bit less demanding.
They are unrealistic, but it's difficult to get measurable results while only testing one aspect of performance at a time (it would be no good if most desktop PCs can max-out all the scores) - and I was expecting that Android devices would perform well in some areas, but less well in others, so it's important to test them separately.

@Konidias - Yes, the values are FPS, but apart from that they're fairly arbitrary - I just took my own fairly average PC (specs in original post) as the baseline, and tweaked the tests until it scored 100 on each of them (although it only has onboard graphics, which is probably why everyone else is scoring disproportionately well on the graphics test).


NEW VERSION (v2.0): http://sdrv.ms/15pJ8Hu
The tests are far less demanding than before, so 300 is now a respectable score for a desktop PC, and any computer with a semi-decent graphics card will probably score a maximum 500 in the graphics test. I've also removed the troublesome ForEach test.
Please test - thank you