BETA and the lost test

Many people view beta as a type of privilege where you were chosen to play the game early.  Almost as a type of super early access without any permanent character development or progress.  In fact, most of the time, people just look at beta as a means of playing their longly waited game.  This is a symptom of the times.  We live in a society with the internet and everything can be gained instantly to create an era of instant gratification.  People just want to play the games they so eagerly wait for so that they can have those experiences, see if they like it, and then move on to the next game they eagerly await.  There is definitely many problems with this type of view point and that is because beta is not early access.  Beta used to be a privilege to do exactly what beta is, and that was to test.  Beta testing has lost the word test and nobody participates in beta to test it.  Is there any way to regain that lost word or the sense of honor you got for testing out the game before it was launched?  Or how about regaining that sense of participation that you helped make your game a great?  Only time will tell to see.  Once again, Arena.Net is changing everything up with their approach to beta, so far, it is the only application-less beta in the current market.

For those of you who do not understand the game development process, there are many phases to the creation of the game.  The first step is the preplanning phase where they have a group of a few people to start brain storming ideas for their game.  This includes, settings, characters, primary story ideas, what type of game it is, new or existing IP and the list goes on.

Second is the actual development phase where the code for the game is being built, sometimes from scratch and sometimes from already existing systems.  Engines are selected to manage the game’s systems so that the game will have physics, graphics and have an overall complete feel to it.  This step is by far the longest of them all as most of the work is accomplished during this time phase.

Third phase is alpha where the game developers are done setting up the primary structure of the game (or skeleton) and begin to flesh it out.  Generally, if you see alpha footage, the game looks terrible and that is only because they usually put in fillers and rough work while the artists put together the higher quality models and landscapes.  Testing during this phase is usually done by the Quality Assurance team.

Fourth is beta testing, where the developers hand the game out to people to test and play it.  This is where the most people are involved in.  This is the phase where players test the game to extremes to find glitches, bugs and exploits to the various systems to make sure they work as it is impossible for developers and QA to actually find all the problems on their own.  By having more people involved, more issues are found so that the game can be relatively bug free by launch.  There are usually many different phases to beta testing, including closed and open.

Fifth is crunch time, where the developers begin to work long days to finish bug fixes found in beta.  This is the shortest period of time as this is the time between beta ends and the game is put in line to be manufactured and shipped.

Lastly is launch, this is where the game is sent out to the public.  After this point, most developers (particularly of console games) leaves it be unless there is some huge bug that needs to be fixed (Skyward Sword is an example of this).

There you go, that is the 6 phases of game development.  Each of those phases are important for various reasons.  The only time the public has access to any bit of the game (although limited) outside of demos at conventions.  So where did the developers and the public go wrong to lose the sanctity of the word test?  In 2011, two very large beta tests go on for two very different types of games: Star Wars, the Old Republic and Battlefield 3.  Battlefield 3 is a shooter for the PC and every major console except the wii.  They had a free, downloadable client available for open beta and many people played it and tried it out.  Problem is, nobody viewed it as a test and instead viewed it as a free copy of the game.  People played it, but many people were turned off by the fact that it was buggy and the servers did not seem stable.  Least to say, Battlefield 3 did not sell well this year because of a terrible beta season.  In fact, I am sure most of the reviews for the game that came out during the beta season even forgot it was a beta and the full game, thus crippling the games success based on the wrong perception of the game.  When you take part of beta, you have to remember that you were chosen to test the game to see if it works and to report any bugs you do find so, in detail of what you did to experience the bug, so that the developers can address it in the code and fix the problems.  Without that, developers release a buggy content that doesn’t live up to the standards of the public and the game ultimately flops.  Now, Battlefield 3 did sell quite a bit of copies.  But compared to its competition, Modern Warfare 3, the game did not sell a whole lot of copies. for further explanation.

The second major beta test that happened this last year was for the giant “WoW Killer” (because what new MMOs these days are not “WoW Killers”), and probably the most hyped up MMO of all time, Star Wars: The Old Republic.  Starwars took a different approach to their beta.  They started off having selected people do beta testing weekends (this is when I tried it out on my friends Beta) and then progressively made their way up to having everyone who signed up participate.  SWTOR’s beta was by far the most successful beta ever, a lot of people were pleased with how the game turned out, it was fun (I enjoyed it, but ultimately wasn’t my thing), it worked very well and most of all, it didn’t really have any crashes.  In fact, the game’s beta went so well, it lead to SWTOR having the most sales of any Vanilla MMO on launch day (not counting expansion) of having about 800,000 box sales in the US alone and a whopping a 1.68 million box sales around the world in its first week.  However, despite having an amazing beta and an amazingly smooth launch day as far as server stability goes.  Some major issues have surfaced with this juggernaut of a game.  Most notably is that there are major, game breaking bugs that are being encountered in the raids and some purely issues have arisen with the world pvp world Illum.  The current issue with Illum is that there is no incentive to PvP and all the quests can be accomplished via PvE means (this was a world that was heavily tested) and has actually evolved to people complaining about people PvPing in a PvP zone.  Illum isn’t so much of a bug, but there is a point to this.  The raid bugs are doing things such as bosses despawning in the middle of combat, issues with raid lock outs and according to Justin Lowe at Darth Hater, anything that can go wrong with raids is going wrong with raids.  So much to a point that the people who are playing SWTOR for raiding are actually threatening to leave the game altogether for anything else.

Now to make all of that relevant.  Great beta, but many people are finding the bugs and the way things are turning out to be very discouraging to the players.  Particularly the end game that even had servers dedicated end game content and it is still buggy.  And it is so buggy that people are jumping ship because they feel that content that buggy is a bad sign with the development team and future content being incapable with putting out content that actually works.  Things like the buggy raids and Illum are symptoms of bad beta testing.  While the beta testing went well overall, I am positive most of the people who actually participated in the beta only viewed it as a means of playing the game early.  Most of the comments I heard and read during the beta period is that the game was fun and you will like it.  Comments like that provide me with no faith that actual testing was done which means that despite having over a million people test the game, there was no actual testing done and the game actually had a failure of a beta.  That statement is supported by the fact that people tested the raid content and it is still buggy, people tested Illum but apparently, nobody realized that most players do not enjoy doing something in game unless it has a captivating story (which SWTOR has) or there is some kind of reward involved.  But how many of the people who actually tested it, submitted bug reports and such that ended up being buried under the complaints about particular minor nuances about the game?  Things like Illum and the raid issues could have been averted and Justin even said that hey submitted reports about issues like that.  Problem is, they were probably lost under people complaining about their light saber not being able to change color.

On the Guild Wars 2 front, Arena Net announced that December 16th was going to be the first day of closed Beta.  However, despite that, there were going to be no applications and the people they are choosing to participate to actually test it based on some unknown criteria to the public.  Chances are, next up will be a type of media test where larger sites will get some type of access for media purposes, but nobody can say for sure because Arena Net is being more tight lipped about this than Apple is about their latest iPhones.  Many people are even wondering why  they even announced that they were in beta to begin with.  Well, that two huge reasons.  The first being that it lets the fan base know that the game is indeed in the final stages of development which means the game is coming out soon.  The second is because they are doing things very differently and want to know that people cannot sign up for the beta so don’t trust any websites.  I am not worried about the beta, especially considering how smooth the demos have been thus far (and if the demo is very playable, than the game is probably mostly bug free).  By doing things this way, Arena Net is keeping the integrity of the beta by putting the test back in it.  They are obviously taking the testing vary seriously to make sure that the game they do release is at the highest of quality.

By doing this selection process, they are choosing people who will actually test things like wall breaks, major glitches, specific abilities and how they function in given content and numerous other things that you would expect to work in any other game.  Also, as people are selected based on some unknown parameters, it becomes an honor to actually test the game you are playing to make sure it works well and can be the best it can be.  That and nobody is going to be running about playing the game rather than testing it and with people actually testing the game, the game’s most likely only going to have very minor bugs rather than massive game breaking bugs.  Content is also most likely going to be made so that it is actually fun and played during the games live release rather than things like the Illum issue that is currently happening.  In fact, as of right now, there are currently further problems with people exploiting various things in there which is something that could have been averted had the game been thoroughly tested.  As of right now, we know that there is not a whole lot of information on things like the Tri-Server Combat.  What we do know is that completing objectives in the world PvP grants your server various bonuses that are not completely confirmed.  Even if there is no gear based incentive to take part of it, I am sure that the developers are attempting to make things like the Tri World Combat fun and interesting to the point where people actually want to participate in it.  Not create a system where people do not use the zone as it is originally intended.

I say this a lot.  But I really do have a lot of faith in Guild Wars 2.  In fact, I started playing Guild Wars last week and am loving it.  While it does have its own faults and some things seem lacking in what are viewed as common “quality of life” type features, Guild Wars is still a very function game after 7 years.  Leveling was not a chore (I hit 20 before attacking Kourna in the Nightfall campaign), the story is so interesting that want to keep playing to find out what happens next.  If they could have produced that high quality of a game with no subscription fee, than I have no doubt that Guild Wars 2 is going to be a success.  I feel that because they are taking their time (almost to the level that Blizzard does with their games) that this game will be a very solid game, have a smoother release than both SWTOR and RIFT and the Beta will actually be a test rather than early access.  Personally, I don’t care about rapid released content, the rate of content blizzard put out in WoW was perfect for me and as long as Arena Net takes the time with their content than keep releasing rehashed content to keep up with their 2 month content quota.

So in short, Beta is a testing period, not a period to play the game early.  By testing the product, you ensure that the product at release is smooth and actually functions.  Taking time to develop a game or content is a bad thing.  I like Arena Net’s approach to this.  By doing things this way, they are avoiding the Battlefield 3 beta debacle while also avoiding the SWTOR problem of having every person who signed up for it participate in the beta.  Testing is obviously their primary focus to ensure their product is actually AAA qualtiy rather than some massive publicity stunt to get people interested in their game.  Problem with SWTOR is they did get people interested in their game, but they paid a heavy cost because of it.  RIFT has rapid content but are paying the cost of it being untested and rehashed.  So keep it classy Arena Net, I have high hopes for your game.


5 responses to “BETA and the lost test

  1. To add more detail from a similar point of view, a game reaches Beta when all the issues in the QA database are closed. What this means is that QA found most of the bugs, class A (crashes, technical hangs, hangs), Bs (major graphical issues, major sound issues, major AI) and Cs (minor stuff, finishing touches, minor collisions). When a project crosses to Beta that doesn’t mean that issues found stop there. Not at all, more and more issues are found by both QA and outside people, for some projects that means closed/open Betas, random consumer tests. As Feral says here, the game isn’t finished. Actually, a game in Beta is far from being finished. It may still encounter A type bugs (see above) as knock-on when other issues are fixed. This is also the stage when regular users (that’s you), start playing and especially exploiting the system. Even after hours and hours of testing the same area or the same skill, same weapon etc., QA may still overlook some aspects or the regular user (you again) will outsmart them and the developers.

    When being part of a Beta, be grateful. You’re more than one of those cool guys or girls who get to play the game. It’s also your responsibility to track down bugs that may have been overlooked or labeled as Known Ship-able. Usually KSd issues aren’t opened after being closed, but hey, if there’s something really bad going on, report it on forums or wherever. If you do that you might push forward the release date 😛 Just saying.

  2. Thanks for the explanation, guys. I did not know all the details of beta, and it was very interesting to a layman like myself. Thanks!

  3. Great article ! Too bad not everyone does that, some actually use BETA “tests” as early access to the game.

    @Markus Clouser: I’ll rather have a “healthy” one than a “prematurely born” one. I also prefer ‘when it’s ready’ over “the point of no return” that is a release date.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s