The Test department of a game publishing company is called "Quality Assurance," or "QA" for short. The term "QA" is also often used to describe the function or process of testing. In this article, the terms "test" and "QA" are sometimes used interchangeably.
Also note: This article is discussing the full-time internal job of tester (wherein the tester is an employee who comes to work daily at the game company to test games, for wages). Volunteer (unpaid) "beta testing" (wherein someone at home gets a copy of a game and provides feedback via email, usually without pay) is a separate matter entirely. Getting a job as a tester can be a good way to get started in the game biz -- volunteering to do some beta testing is more akin to simply being a customer (an end user).
Quality Assurance testing jobs are usually found at game publishing companies. Developers also do some testing (but usually not full Q.A.) - but game development companies probably don't have full-time testing jobs (unless the development company is very large and well-staffed).
Typically, someone who works at a small game development company usually performs multiple job functions. Publisher jobs are usually more specialized. So someone who starts as a tester at a publishing company might eventually move up into producing, while someone who works as a tester at a smaller development company might eventually move up into any of a number of roles.
There are also independent testing labs who hire testers. Publishers are increasingly outsourcing their Q.A. to these outside labs. Jobs at these places are okay if you just want to test and you don't have aspirations of moving up in the industry - a tester who works at one of these labs would have to quit in order to move up in the industry. It's recommended that if you want to work as a tester as a steppingstone to other jobs in the industry, that you work for a publisher or a large developer, not an independent test lab.
Tuesday, August 18, 2009
Tuesday, August 11, 2009
Video game testing
You have undoubtedly heard that a recommended way of getting started in the games biz is to get a job as a game tester. That's true, especially if you do not have a programming degree, an art degree, a business degree, etc. and if you can get the testing job with a game publisher or developer (rather than at a game testing lab located far away from game publishers or developers).
And you have undoubtedly also heard a lot of negative reactions to this advice. A lot of the negative things you have heard are probably from losers who couldn't hack it even as testers, or from guys who just approached it the wrong way. There is a common perception that testing is a "lowly entry-level job" and that testers are at the bottom of the totem pole. The fact is, testing is extremely important and the test phase is vital in polishing a game into a fun experience for the end user.
That's not to say that if you have an art degree or a programming degree, a law degree, or a business degree (or even a "game design" degree, if you find a college that offers such), that you MUST begin in the game industry in Q.A. Obviously, if you can enter the industry in a job closely related to the subject you mastered in at college, then you should target that path instead of Q.A.
But for those who have not gotten a degree in one of those areas... Testing can be an excellent way to get your foot in the door.
And you have undoubtedly also heard a lot of negative reactions to this advice. A lot of the negative things you have heard are probably from losers who couldn't hack it even as testers, or from guys who just approached it the wrong way. There is a common perception that testing is a "lowly entry-level job" and that testers are at the bottom of the totem pole. The fact is, testing is extremely important and the test phase is vital in polishing a game into a fun experience for the end user.
That's not to say that if you have an art degree or a programming degree, a law degree, or a business degree (or even a "game design" degree, if you find a college that offers such), that you MUST begin in the game industry in Q.A. Obviously, if you can enter the industry in a job closely related to the subject you mastered in at college, then you should target that path instead of Q.A.
But for those who have not gotten a degree in one of those areas... Testing can be an excellent way to get your foot in the door.
Friday, August 7, 2009
Nokia Series 30 Handsets
Nokia 1100, 1108, 2100, 2270, 2272, 2275, 2280, 2285, 2300, 3350, 3410, 3510, 3585, 3590
1100B, 2280A, 2280B, 3510i, 3530, 6210, 6250
1100B, 2280A, 2280B, 3510i, 3530, 6210, 6250
Thursday, August 6, 2009
Test plans
A test plan documents the strategy that will be used to verify and ensure that a product or system meets its design specifications and other requirements. A test plan is usually prepared by or with significant input from Test Engineers.
Depending on the product and the responsibility of the organization to which the test plan applies, a test plan may include one or more of the following:
Design Verification or Compliance test - to be performed during the development or approval stages of the product, typically on a small sample of units.
Manufacturing or Production test - to be performed during preparation or assembly of the product in an ongoing manner for purposes of performance verification and quality control.
Acceptance or Commissioning test - to be performed at the time of delivery or installation of the product.
Service and Repair test - to be performed as required over the service life of the product.
Regression test - to be performed on an existing operational product, to verify that existing functionality didn't get broken when other aspects of the environment are changed (e.g., upgrading the platform on which an existing application runs).
A complex system may have a high level test plan to address the overall requirements and supporting test plans to address the design details of subsystems and components.
Test plan document formats can be as varied as the products and organizations to which they apply. There are three major elements that should be described in the test plan: Test Coverage, Test Methods, and Test Responsibilities. These are also used in a formal test strategy.
Test coverage in the test plan states what requirements will be verified during what stages of the product life. Test Coverage is derived from design specifications and other requirements, such as safety standards or regulatory codes, where each requirement or specification of the design ideally will have one or more corresponding means of verification. Test coverage for different product life stages may overlap, but will not necessarily be exactly the same for all stages. For example, some requirements may be verified during Design Verification test, but not repeated during Acceptance test. Test coverage also feeds back into the design process, since the product may have to be designed to allow test access.
Test methods in the test plan state how test coverage will be implemented. Test methods may be determined by standards, regulatory agencies, or contractual agreement, or may have to be created new. Test methods also specify test equipment to be used in the performance of the tests and establish pass/fail criteria. Test methods used to verify hardware design requirements can range from very simple steps, such as visual inspection, to elaborate test procedures that are documented separately.
Test responsibilities include what organizations will perform the test methods and at each stage of the product life. This allows test organizations to plan, acquire or develop test equipment and other resources necessary to implement the test methods for which they are responsible. Test responsibilities also includes, what data will be collected, and how that data will be stored and reported (often referred to as "deliverables"). One outcome of a successful test plan should be a record or report of the verification of all design specifications and requirements as agreed upon by all parties.
Depending on the product and the responsibility of the organization to which the test plan applies, a test plan may include one or more of the following:
Design Verification or Compliance test - to be performed during the development or approval stages of the product, typically on a small sample of units.
Manufacturing or Production test - to be performed during preparation or assembly of the product in an ongoing manner for purposes of performance verification and quality control.
Acceptance or Commissioning test - to be performed at the time of delivery or installation of the product.
Service and Repair test - to be performed as required over the service life of the product.
Regression test - to be performed on an existing operational product, to verify that existing functionality didn't get broken when other aspects of the environment are changed (e.g., upgrading the platform on which an existing application runs).
A complex system may have a high level test plan to address the overall requirements and supporting test plans to address the design details of subsystems and components.
Test plan document formats can be as varied as the products and organizations to which they apply. There are three major elements that should be described in the test plan: Test Coverage, Test Methods, and Test Responsibilities. These are also used in a formal test strategy.
Test coverage in the test plan states what requirements will be verified during what stages of the product life. Test Coverage is derived from design specifications and other requirements, such as safety standards or regulatory codes, where each requirement or specification of the design ideally will have one or more corresponding means of verification. Test coverage for different product life stages may overlap, but will not necessarily be exactly the same for all stages. For example, some requirements may be verified during Design Verification test, but not repeated during Acceptance test. Test coverage also feeds back into the design process, since the product may have to be designed to allow test access.
Test methods in the test plan state how test coverage will be implemented. Test methods may be determined by standards, regulatory agencies, or contractual agreement, or may have to be created new. Test methods also specify test equipment to be used in the performance of the tests and establish pass/fail criteria. Test methods used to verify hardware design requirements can range from very simple steps, such as visual inspection, to elaborate test procedures that are documented separately.
Test responsibilities include what organizations will perform the test methods and at each stage of the product life. This allows test organizations to plan, acquire or develop test equipment and other resources necessary to implement the test methods for which they are responsible. Test responsibilities also includes, what data will be collected, and how that data will be stored and reported (often referred to as "deliverables"). One outcome of a successful test plan should be a record or report of the verification of all design specifications and requirements as agreed upon by all parties.
Sony PlayStation2 Game Testing
First of all, SQA testers perform a submission test on title. This submission or 'TRC' check will verify that if the game meets the requirements set by the SCEE/SCEA/SCEI in their Technical Requirements Checklist.
This test run also checks the following elements:
Naming Conventions: this test will cover the 'definitions' and 'namings' in the game in all its different languages to see if there are any violations against chapter "7.0 PlayStation Component Naming Conventions".
Memory Card Operations: every developer / publisher knows that this issue is of critical importance to Sony. SQA testers check all possible memory card messages in all available languages. As a part of this test, they will simulate any user action will be to see if the game reacts correctly to these actions. The messages will be checked for translation, naming convention standards and the correct display of the specified message.
Language check: check if your game is correctly translated into all featured languages.
At the end of the test run, SQA testers send you a detailed report on test results and TRC status.
Besides TRC testing, SQA testers can carry out other functionality tests, such as game play, balance and AI, to make sure that your PlayStation game meets the highest quality standards.
This test run also checks the following elements:
Naming Conventions: this test will cover the 'definitions' and 'namings' in the game in all its different languages to see if there are any violations against chapter "7.0 PlayStation Component Naming Conventions".
Memory Card Operations: every developer / publisher knows that this issue is of critical importance to Sony. SQA testers check all possible memory card messages in all available languages. As a part of this test, they will simulate any user action will be to see if the game reacts correctly to these actions. The messages will be checked for translation, naming convention standards and the correct display of the specified message.
Language check: check if your game is correctly translated into all featured languages.
At the end of the test run, SQA testers send you a detailed report on test results and TRC status.
Besides TRC testing, SQA testers can carry out other functionality tests, such as game play, balance and AI, to make sure that your PlayStation game meets the highest quality standards.
Microsoft XBox Game Testing
tests include:
TCR (Technical Certification Requirements). We will assess the compliance of your game with Xbox requirements. Thus, we will check if your game is ready for submission to Microsoft, helping you save both time and money.
Xbox Live. "Live TCR issues" is a separate chapter in the Xbox Guide . We will check if your game complies with it. The test includes 'Xbox live' requirements, content download, statistics, friends, voice.
Functionality. This test focuses on the general game functionality, and namely:
Menu functionality
General functionality of the buttons, options
Game play and AI
Game stability
Walkthrough
Multiplayer. We will check how your game behaves when played in all featured multiplayer modes. This can vary from multiple players on a single Xbox, system link over to LAN and of course actual online multiplayer sessions.
TCR (Technical Certification Requirements). We will assess the compliance of your game with Xbox requirements. Thus, we will check if your game is ready for submission to Microsoft, helping you save both time and money.
Xbox Live. "Live TCR issues" is a separate chapter in the Xbox Guide . We will check if your game complies with it. The test includes 'Xbox live' requirements, content download, statistics, friends, voice.
Functionality. This test focuses on the general game functionality, and namely:
Menu functionality
General functionality of the buttons, options
Game play and AI
Game stability
Walkthrough
Multiplayer. We will check how your game behaves when played in all featured multiplayer modes. This can vary from multiple players on a single Xbox, system link over to LAN and of course actual online multiplayer sessions.
Microsoft XBox Game Testing
tests include:
TCR (Technical Certification Requirements). We will assess the compliance of your game with Xbox requirements. Thus, we will check if your game is ready for submission to Microsoft, helping you save both time and money.
Xbox Live. "Live TCR issues" is a separate chapter in the Xbox Guide . We will check if your game complies with it. The test includes 'Xbox live' requirements, content download, statistics, friends, voice.
Functionality. This test focuses on the general game functionality, and namely:
Menu functionality
General functionality of the buttons, options
Game play and AI
Game stability
Walkthrough
Multiplayer. We will check how your game behaves when played in all featured multiplayer modes. This can vary from multiple players on a single Xbox, system link over to LAN and of course actual online multiplayer sessions.
TCR (Technical Certification Requirements). We will assess the compliance of your game with Xbox requirements. Thus, we will check if your game is ready for submission to Microsoft, helping you save both time and money.
Xbox Live. "Live TCR issues" is a separate chapter in the Xbox Guide . We will check if your game complies with it. The test includes 'Xbox live' requirements, content download, statistics, friends, voice.
Functionality. This test focuses on the general game functionality, and namely:
Menu functionality
General functionality of the buttons, options
Game play and AI
Game stability
Walkthrough
Multiplayer. We will check how your game behaves when played in all featured multiplayer modes. This can vary from multiple players on a single Xbox, system link over to LAN and of course actual online multiplayer sessions.
Game Testing Methodology
There is no "gold standard" method for game testing, and most methodologies are developed in-house, by individual video game developers and publishers. And, methodologies are refined from time to time and may differ for different types of games (for example, the methodology to test a MMORPG will be different from the testing for a casual game). But outlined below is a typical methodology of game testing.
Identification: Incorrect program behavior is analyzed and identified as a bug.
Reporting: The bug is reported to the developers using a defect tracking system. At minimum, the circumstances of the bug and steps to reproduce are included in the report. Developers sometimes request additional documentation such as real-time video of the bug's manifestation.
Analysis: This step of the process involves the developer responsible for the bug, such as an artist, programmer or game designer. It is outside the scope of the game tester's duties, though if there are inconsistencies in the bug report, the tester may be approached to provide more information or evidence.
Verification: After the developer claims the bug to be fixed, it is the responsibility of the tester to verify that the bug no longer occurs. Not all bugs are addressed by the developer. Reported bugs may be claimed as features (often expressed as "NAB" or "not a bug"), and may also be waived (given permission to be ignored) by producers, game designers, or even lead testers, according to company policy.
Regression testing: As game development progresses, closed bugs are checked again to ensure that they have not regressed, or re-manifested. For example, a programmer might undo a fix, having forgotten why the change was made in the first place. Though tedious, this type of testing is important since unexpected changes to code can cause old problems to resurface in the same or different ways.
Standards testing: Consoles manufacturers such as Nintendo, Sony and Microsoft have standards they require to be met for games for their platforms. Not all manufacturers standards are alike and some testers are dedicated only to ensuring these standards are met.
Identification: Incorrect program behavior is analyzed and identified as a bug.
Reporting: The bug is reported to the developers using a defect tracking system. At minimum, the circumstances of the bug and steps to reproduce are included in the report. Developers sometimes request additional documentation such as real-time video of the bug's manifestation.
Analysis: This step of the process involves the developer responsible for the bug, such as an artist, programmer or game designer. It is outside the scope of the game tester's duties, though if there are inconsistencies in the bug report, the tester may be approached to provide more information or evidence.
Verification: After the developer claims the bug to be fixed, it is the responsibility of the tester to verify that the bug no longer occurs. Not all bugs are addressed by the developer. Reported bugs may be claimed as features (often expressed as "NAB" or "not a bug"), and may also be waived (given permission to be ignored) by producers, game designers, or even lead testers, according to company policy.
Regression testing: As game development progresses, closed bugs are checked again to ensure that they have not regressed, or re-manifested. For example, a programmer might undo a fix, having forgotten why the change was made in the first place. Though tedious, this type of testing is important since unexpected changes to code can cause old problems to resurface in the same or different ways.
Standards testing: Consoles manufacturers such as Nintendo, Sony and Microsoft have standards they require to be met for games for their platforms. Not all manufacturers standards are alike and some testers are dedicated only to ensuring these standards are met.
Subscribe to:
Posts (Atom)