Does the age of test users matter? Should I get users that are around my intended age level, knowing that some games are played by younger or older people than were originally intended? Should I have my test users be all around the same age for consistency, or a variety of ages to gather more useful data?
6 Answers
Testers don't grow on trees :-). Few game coders have the luxuary of accepting or rejecting testers. Additionally, it is not at all easy to keep them for a longer period of time than maybe a few months, at most. Those who are really dedicated, of some specific reason probably, may become lifetime helpers and even friends, but most of the testers will simply disappear over time.
If you have a really good, fancy, challenging game, or a series of games, you may get such amounts of interested people that you can make choices.
Otherwise take'em all. You get all kinds of input, and it is good that way. You are yourself the one who give emphasis on different input, depending on the quality of it. You will learn to read between the lines and recognize the nicknames, the kids, the old dogs, the musicians, the technocrats. And remember to make it public. Nobody wants to chat with you alone... or very few want to.
Do not invoke testers too early - nobody gets happy of crashing and incomplete software. It must be interesting, challenging and you must create the hype factor, and even harder, keep it. Let them shine!
So take it all with whatever means you have handy, and filter the input you get.
(If you aim to pay, for eg. Lithuanian testers, it is a totally different thing. They are good and they cost money)
5 cents :-)
It depends on what you want to test.
When you want to test user acceptance and behaviour, you should of course test within your desired demographic. People from a different demographic might give you different results and mislead you about the strengths and weaknesses of your product.
But when you want to test for bugs, you need professional testers who thoroughly test all the edge-cases of your game, systematically looks for bugs, find out how to reproduce bugs they find and then write useful bug reports which contain all the information you need to fix them. This is not how a normal player plays and experiences your game. Feedback about the gameplay itself is of course welcome, but it's not what you expect from these testers. Their feedback is also not very useful, because they are testing the game and not playing it. That's a completely different perspective. So it does not matter which demographic these testers fall under.
I think if you have a target demographic in mind, you should focus on that demographic for your playtesting (that would apply to age, gender, country of origin, income, and any other demographics).
If you're looking to feel out exactly who will enjoy your game, then getting your game into the hands of as many users from all demographics as possible would be a good idea.
Usually, for a game testing, you should look for experienced players, with a good eye to spot errors or bugs in your game. People with lots of experience playing videogames will spot these things better than people who don't know much about games. Plus, if the testers happen to know something about videogames developing or game logic, can help you out pointing out technical issues and hints on where to fix bugs.
But, if what you're aiming at making a demography study (finding out which crowd you'll aim your marketing at, or what population likes your game the most, etc) then it is recommended that you get as diverse people as you can, because this will give you nice data to work on.
I think you should gather test users which are all different ages. Most of Studies are made this way and (as you already said) you gather more useful data.
Depending on jurisdiction & age, you may be subject to certain laws regarding minors, I.E. the Children's Online Privacy Protection Act in the USA might cover online / remote testing by children under 13 years of age. If you plan on testing with minors, consult a lawyer.