The 'Fun' of QA

Old Article from May 2001

Last month our Gamespam editor commented on how great it is, or would be, to be paid to play games all day. How many of you know what it is like to be paid to test games all day? Could you stand putting in 60, 80, or more hours per week playing some new Interplay title? I know what you are thinking, "Golly, would be great to sit around and get paid to test games. I already like playing the Baldur's Gate, I should get paid for it." Given these thoughts I stopped in on our Quality Assurance department to find out. I ventured down into the dark recess of Interplay to talk with QA about what it is like to be a tester for Interplay.

First off, testing, real testing, is more than sitting around playing the game. It is about creating a product of known quality. What does that mean - Known Quality? It means that QA personal acknowledge that it may be impossible to find every single problem. What QA strives for is to produce a product that has a certain level of quality, and knowing, to the best of their ability what that level is. When a test lead signs his name to show that his team has thoroughly tested a product, it means that they've tried everything they could to make that game a problem free experience for you. They know the level of quality going onto store selves stores, and ultimately your computer.

What is like to test? Again, testing is more than sitting around playing the game. You've got to be watchful of missing graphics, bad sound, jumps in the story, not just the crashes and system lockups. You really do not get to sit back and enjoy the game; you are being paid to be on the look out for the wrong things. When those wrong things happen, and they do, you've got to stop, write down what happened and how you got there, in detail. You can't just record, "the game locked up when I killed an orc". The programmers need to know details. The more details you can give, the easier time they will have in tracking down the problem and getting it fixed. It takes time and practice to learn how to write good bugs. It takes even more time to know where certain bugs might crop up. Testing isn't a matter of just popping in the CD and booting up the game. Good testers aren't' spun from ethereal cloth, they are forged like a fine steel weapon.

In addition finding bugs during game play, there are the endless sessions of installing and uninstalling the game, printing, character import/export, and all sorts of other mind numbing areas that need checking, and rechecking when the programmers fix something. Imagine spending 10 hours just installing and uninstalling a game and not missing the misspelling on an install screen that wasn't there a version or two ago.

Every time a found bug is fixed by a developer, it must checked again. This is to make sure that both the original problem is really fixed, and that nothing else got broken during the repair process. Hours, days or even weeks later, you or somebody else has to retest that bug. You've have follow the same chain of events used when the bug was first found. You might have to test and retest the same bug over and over again, until the developers find and fix the problem. When all is said and done, before a product ships, a test lead might elect to test every bug ever found on any version, on any system, one last time.

Most testing at Interplay is done Ad-Hoc. That means bugs are discovered in the normal means of play. Some testing is done via script. Instead of playing the game and writing up bugs as they are found, testers have a written set of directives to follow. They must mark off each step as they do them. Their successes and failures might need to be recorded so test leads and developers know where a problem is, or where more testing needs to be done. A tester might have to do the same script several times on several different systems before a test lead is satisfied that a given problem will not occur or that sufficient testing was done.

One other thing to consider, QA departments do not make money for a company, they save money. They are an expense to reduce debt. Think about it. Programmers create a product; marketing spends money on advertising to make money, selling games. Even the web team generates revenue to justify its existence. QA is one of the few departments that doesn't generate any revenue at all, yet it is vital to any software company.

Now testing isn't all-bad. There are lots of good points about being part of the QA team. First is the ego factor. Games don't go into production without your input. The company is depending on you to find the problems. In some ways, QA is the most powerful department. Programmers might create a game, you will even see advertisements that the marketing department creates for a game, but unless QA puts it stamp of approval on a product, it doesn't leave the building. This gives them tremendous power. Depending on the QA process for a given title, there is also the chance that a testers comments and ideas about game play will make into the final, finished product. There is also the fun factor of getting to play the newest, greatest stuff, months before anybody else. In addition, don't forget about seeing your name in the final credits.

The next time you sit down to play your favorite game, and you find a problem, think about what you've read here. Think about the hours and hours of time some tester sat playing the game over and over again. That tester might not have done exactly the same thing you did, with your exact hardware. Contrary to rumor, successful game companies, do not sell beta versions so the general public can test for them. They need a staff of trained QA personal that will write up bugs in verbose detail on known hardware configurations so that developers can fix them. Sure, it's fun, but it's a job, and any job is work.