MMF3 extension/plugin design plans

[LB]

In this thread I want to discuss the needs and wants for extensions/plugins in MMF3. I think it is important to solidify the abstract ideas for this before MMF3 is too far in development so that MMF3 can be designed with the preferences of extension developers in mind. Here are some of my rough ideas for starters:

Extension Packaging
Currently in MMF2 extensions have several files that get distributed from zips to several directories in MMF2:
-The edittime MFX in \Extensions\
-The runtime MFX in \Data\Runtime\
-The various runtime versions in the various folders in \Data\Runtime\
-The help file or files in a folder in \Help\
-The example files in \Examples\Extension-Name\ (or sometimes even just \Examples\)
-The DLL dependencies in \ (root MMF2 folder)
And all these things are spread around in the MMF2 application folder.

For MMF3, I think it will be important for extensions to keep all their files together, and for them to be out of the MMF2 folder (so they're not in Program Files). Perhaps somewhere in the All Users folder or the Shared Documents folder, just somewhere easily accessible.

Consider: each extension is either a folder or zip archive (preferable folder) with this structure:
ExtensionName
├Help
│└(Help files, etc)
├Examples
│└(Examples, etc)
├Implementation
│├(various property files to define global information for the extension)
│├Edittime
││└(Implementation files for dealing with the extension within MMF3)
│├Platform-Name (one for each supported platform)
││├Debug
│││└(Platform specific implementation files for running from MMF3)
││├Release
│││└(Platform specific implementation files for built runtime files)
││├Dependencies
│││└(files to be distributed once for all instances of the extension)
A self-contained folder for the extension and its help and examples, and each runtime it supports.

Notice the edittime implementation: it is simply for the user interfaces when designing the MFAs, and contains no actual runtime code. This would be responsible for the A/C/E setups and the menus, along with the object properties and such.

Another thing to note is that I haven't given any file extensions for this - extensions may be implemented not only in native code, but also in MMF2 applications or in some text formats for very simple extensions - whatever can be done reasonably. For example the Windows runtime implementation can be a C++ DLL or a Java class, since both are supported on Windows. Or, if the Windows runtime is not provided but the Java runtime is, the Java runtime can be substituted for the Windows runtime.

Another benefit is that users can pick and choose or mix and match platform implementations for the extension and they will all always be compatible - this allows for the author to provide different versions of the runtime implementations that optimize for different things (eg only implementing a specific set of A/C/Es to optimize for space).

Finally, since this is all just some files in folders, if the structure and layout have to change the MMF3 patcher/updater can easily batch-update the structure of existing extensions (or MMF3 can do it at startup or when needed). No more being stuck with old designs that are too late to change.

Extension Implementation / MMF3 SDK/API
The current MMF2 SDKs use code and implementation layouts based on very early MMF-style products, as early as Click'n'Create. The SDK is also mainly optimized for C. While for Windows this is OK, it makes a mess for other platforms and things get really inconsistent.

For MMF3, I think it would be great if the same design for the code is used for C, C++, C#, Java, Python, Flash, etc. based on the average of what these languages can do. In general I think object-oriented programming should be encouraged, eg for the higher-level programming languages (C++, C#, Java, Python, Flash, etc), but for the lower-level language implementations (eg C) implementations it should still be able to interact with the MMF2 runtime in a fast, low-level way. In any situation, however, there should be as much flexibility as possible in the design so that new features in MMF3 can be added to the API/SDK and naturally fit in with existing code.

If breaking changes have to be made, they should update the version of the SDK so that old implementations can remain compatible. This is a real issue and is not very easy to solve without making a mess of the MMF3 runtime code, so in some cases it may be best for MMF3 to deploy the use of proxy objects that can handle older extensions. This way the MMF3 code stays clean and older incompatible extensions can be supported.

Currently in MMF2 a lot of limiting factors come from not having access to the behind-the-scenes data. In general it should be easy and common for extensions to interact with eachother, including 3rd party extensions and the 'default' extensions in MMF3. It should be encouraged that extensions give eachother access to specific implementation details and low-level/private data. While this can make the life of extension developers somewhat more challenging, it can also open up opportunities for simpler programming in MMF3 and the ability for extensions to directly transfer data in the most efficient manner.

Another problem with MMF2 is that many extensions are closed-source and the developer left the online community without giving anyone the code. To solve this, all extensions for the standard-level of MMF3 should be required to have their source published to the MMF3 extension repository before the extension or its updates can be made public on the extension repository. Extensions for the professional/developer-level of MMF3 will have to have their code submitted to Clickteam.



This sums up my major ideas and abstractions for MMF3 extension development. I know a lot of details need to be worked out and some ideas refined, so post your ideas and input so that when MMF3 comes around, extension developers will be able to quickly and easily create fantastic extensions!

Some things to refine in specific:
-How to handle inheritance and other object oriented concepts
-How to allow for the creation of proxy extensions that can, for example, pretend to be another extension (eg so old MMF2 extensions can work via proxy extensions)
-What the SDK/API design should incorporate and allow for
-How extensions can be created within MMF3 rather than another programming language

[Phi]

How extensions can be created within MMF3 rather than another programming language
if you could use LoadLibrary() and such to allow MMF3 script to interact with API calls from inside the script, there would be little issue with using script alone to create extensions. What the runtimes do, and I expect will continue to do in MMF3, is simply use a built file (ccn?) which is the same in each platform, and it's just the interpreter that needs coding for a new platform support. Extensions being open-source and closed-source could both be possible if there is an interpreter that can read plain-text code and compiled script code at edittime, and at runtime only the compiled script needs to be read.
Although such a system would not be the optimum method (that would rely on functions being directly linked as opposed to LoadLibrary calls) it would at least be functionable.

With regards to dependencies, that might not be a perfect system, mainly if there are several instances of the same file. There could be instead a way of listing the dependencies for each extension/plugin in human-readable format, but using a global dependency folder as well.

Object inheritance can simply create a C-style linked list by having an address at the end of a given property list so MMF3 can store the child and parent list in. Only the child extension would have a null address for its address, and that would provide a hierarchical structure, wherewith tracing a single object will provide a "line" of a tree diagram, like a grandchild finding all the links to his great-grandfather in a family tree.
A second suggestion would be similar to C++'s vfptr for virtual inheritance, in that the child address points to an array of addresses, allowing you to explore all the child nodes of a parent. Either scenario would be useful, but the latter provides greater functionality as the former, albeit slightly slower for MMF3 to work out what to display when someone opens the property list. However this would not suit a situation such as node D in this example:

 A
/ \
B  C
\ /
 D

Although to be honest I can't think up when that sort of inheritance would be approriate. The way to work around this would be to supply the latter suggestion for both parent and child nodes, and have two vfptr designs.

[Fanotherpg]

I don't agree LB that extensions should be put out of MMF folder. Keeping them in there saves decades on backing up all the files and examples/extensions.

[LB]

Hopefully, we shouldn't need to backup extensions or MMF3 because:
-You can always reinstall MMF3 and its addons
-You can always redownload your extensions from the MMF3 official extension repository by either clicking Download All or downloading one of your premade lists
-You can setup MMF3's extension directory path to be that of one of your dropbox folders
-You can tell MMF3 to always get the latest stable releases of extensions from the repository automatically

Basically we should have a system where the only backups you need to make are those of your projects; everything else is easily available online. Software is moving this direction and I think it is a good direction to go. What do you think?

[Fanotherpg]

Easy access to copy-paste is IMHO important. Yet I don't see a valid reason to put it in My Documents area.

However all your arguments are totally valid +1.

[LB]

I said Shared Documents, those folders that all users have access to, not My Documents, which is per-user.

[Fanotherpg]

It is same story. I dislike those areas of Windows.