Open Beta Test

[Jorgee!]

Hey, just for curiosity, because i see so many "faults" or errors in the maps, and we have to do the beta test during the regular play, causing so many frustrations like Kokan, where the map is very good, but the DoDs/Combat zones are bad placed, even the assets spawn....

Seems too much work in the map but a little bit in the main bases...

So that's why im wondering it........

[ZephyrDark]

From what I've heard from the testers, they really don't have a large enough team to find EVERYTHING. If you think about it, we the players are testers as well. You can't expect a perfect map/game from the resources the DEVs have right now.

Also remember, everyone does it out of their own time, and they may not have the time to constantly search a map for bugs and errors every day without pay.

Another thing; things may have been found and fixed, but someone accidentally reverts the change (i.e. the landbridges on Al Basrah, they were supposed to be removed to make CSBs on the map have a point.)

[Jorgee!]

Hey guys, i was wondering.....

At every PR version, we complaint about things that works bad, don't work, or gameplay things and we make the question "Why didn't the Beta Test team detected so obvious fault?"

Then we get warned by "Bitching the Beta Test team" and that they do what they can, freely, only for us.

And they are right, but..... if they dont cope with the Beta Test, and we keep complaining, why don't make the Beta Test open for everyone, or at least with 200 slots for the firsts signups?

Then we would avoid the frustration and the PR team the attacks.

That would be all.

[Scot]

More people doesn't necessarily mean better quality of testing.

Testing is very specific and the testers test changes made by the DEVs and how they implement on the build etc. So for a very basic example, if a new APC is put in, that it functions properly etc etc. Obviously, there are hundreds of things, and more complex, code related and so on, but that's the basic gist of it.

What you're describing is stress testing it. The problem with stress testing is that the PR Team tried to do it, but unfortunately people don't "test". They download the build, load up and go "oooh shiny!" and that's that. The dilemma faced by the PR Team is to try and get stress testing while still testing the changes in the mod.

However, the testers always do their job. To my knowledge, there hasn't been an unfinished test list build released, so everything the testers have been told to test gets done. Bitching at the testers is completely out of order, as they always do exactly what they are told to and needed to do.

I currently think that there are problems in the PR Test team, but they are doing their jobs, so bitching at them doesn't help anyone. If you really want to help, feel free to test the build once it's released, and fully utilise the Bug Report section of the forums.

[Smegburt_funkledink]

Hi

This has been tried in the past and had mixed results. Whilst it was good to get a couple full servers to experience new game play aspects, when it came down to serious testing and taking necessary steps to look at specific bugs, the community was too caught up in the OMGZ NEW STUFFS!!!

The idea of having 'stress testers' isn't dead and has been brought up now and then but there's always been problems of people leaking files/screenshots/info etc... It has ruined the anticipation of the release previously.

If stress testing happened again, it's likely to be a small selection of trusted community members but nothing has been set in stone.

If you have any further complaints about releases, feel free to direct your complaints and frustrations at [R-DEV]crazy11. For any SP (Coop) related complaints feel free to contact [R-DEV]Outlawz

[Farks]

As far as I know, it has been tried but the wasn't worth the hassle. The more people you have, the more time has to be spent on organizing and making sure everybody is doing their job (which is easier said than done), and that in turn leads to less efficiency. Countrary to what most people think, we don't test PR by playing it like a regular round, we systematically go through all the documented changes in every build and report the results. This is usually hundreds of changes per build, so it take several hours a week. And it is by far the best way to discover any immediate problems.

Starting with large testing sessions again would just draw out the time it takes to develop a new version PR, and therefore not save any time (or frustration) in the long run.

[Jigsaw]

I'm going to close this thread firstly because this has not just been brought up, but actually tried, several times before and secondly because pretty much everything has been explained very well in the three posts above this one.



I will add one thing, that the idea of stress testing is not dead and may make a return at some point but it will likely be particularly exclusive.


I've also merged your other thread which basically covers the same thing with regard to maps being tested before release. Simply enough they are, and one method that has been tried recently with several maps is to have them tested in a closed game by a particular section of the community. So far this has been exclusive to Tactical Gamer.

Locked.

[crazy11]

I am going to comment on this Jiggy I know its closed.


The last time we tried to do stress testing it did not go very well and as Smeg said it was mainly people just doing it to download the new build and not actually stress test. We had around 200 people and could never get a full server. And as several people have mentioned the idea of stress testing is not dead and may come back in the future.

As for having a larger beta test team, that will not happen, because it just turns the test session into a clusterf**k (excuse my language) and nothing gets done.