[]I think it is a plan, however we may need to take some steps to prevent people from lurking in the beta forums. What I mean by that is, perhaps their level of participation should determine their access <img src="/center/images/graemlins/wink.gif" alt="" />.
If it was possible I would like to see where the testers would have to contribute in order to stay in the group (where if they contribute a lot they would stay in the group perminately).
The reasoning behind this is that it doesn't make sense to keep people as beta testers if they provide no feedback. Keeping them around in that case is just serving to recreate the situation we have currently, but on a broader scale <img src="/center/images/graemlins/wink.gif" alt="" />.
In any case testing is a vital issue and I am glad that the situation is being thought out. [/]
I can understand your point vortex, Beta testing is very time consuming, and I can understand why many people don't comment or provide feedback. Its a very time consuming process.
The key to successful beta testing is automating as much of it as possible and understanding what you are testing for.
There are level of tests or gates that need to be accomplished IMHO.
[:"blue"]
Here's what I think we need to do when developing extensions.
1. As a community we write a simple application that creates a text file about the systems being tested.
Containing information like CPU memory, hard disk, video card, os etc.
Naturally we make this little proggy open source and we insist on any debug report containing this text file.
For example I have several systems so that would make it easier for extension developers and clickteam to understand the nature of the enviroment the extension is working in.
2. When an extension is floated for the first time, identify those that a willing to.
- Level 1 - help develop the app by writing test apps, and help files.
- Level 2 - comment, suggestions and run test apps.
- Level 3 - do long term testing by actually developing applications to be used in programs.
3. At least one test (demo) program that fully test each function of any new extension.
This is critical, cos in the past developer have released extension with no test program, and crafting a test program is difficult and time consuming. (Extension developing can be quite time consuming maybe we consider a budy system where one develops the extension another develops the test programs and demos)
3a.The first and most important thing to test for is stability of different plateforms and OS setups. (This is where test prog's are really important)
3b.The second is commenting on useablity and giving suggestions.
3c.Testing for limits and assumptions ie. what happens if I put a really long string in or a really big negative number.
3d.Testing for interaction with other extensions. ie. if they work together or just don't work.
4. Clarifying the difference between an Alpha and Beta extension. ie ALPHA extension is one where the interface may change. A Beta extension, the interface is fixed and now bugs are being removed. Also an ALPHA extension should not be used for any kind of program development. Only Betas and RC should be used for developement of software.
5. A checklist & template's of things that is submitted with extensions. ie. Is there a help file? Has the properties been filled in correctly? Is the version numbering correct and updated? What is the licence? etc
6. When a extension is released to the public, it could be labelled as a release candiate and all a request be made that as many people run the test programs as possible and immidiately report bug reports. Also ask for comments. eg .Make a target of say 50 positive comments with no bugs before consider releasing an extension.
7. Consider translating the extension into other languages. ie french.
[/]
So when considering extension testers, for this forum, I believe you need to determine at what point they can contribute. For example somebody that is considering to use your extension in a production program (level 3) should be invited at the early stage to comment and indicate level of interest. Now this could be privately or as a part of this forum.
See myself I tend to hover around a level 2 for 80% of the extensions.
Reply With Quote