TALKS WITH EXPERTS

Welcome to the 5th episode of our series Talks With Experts. We are going to introduce you our Senior QA Engineer Stanley and we’ll tackle the question of the future for game testing. In case you have some further questions, you can always join our Facebook group called Free to play game developers and we will happily discuss them with you.

Tell us Stanley, why is video game QA so important?

QA acts as a counterweight to the design team because as a game developer you have a quite limited and slightly biased opinion about your game. Therefore, it is good to have a seasoned and experienced player to play your game and tell you about the obedience, performance, bug issues and design ups and downs. The main goal of QA is to ensure that the product is in the intended quality but also to bend, break, abuse and exploit the system to its limits so it can be patched and improved for the end users. Furthermore, QA acts as an information hub because QA is not only communicating with designers and programmers but also sending reports to the product managers about the state of game or update, and working closely with community management about issues, feedbacks and players’ moves.

What are the main rules or advices for being a good QA?

You need to stick to 2 basic rules of testing. First one is, do not panic! If you are dealing with server, technical or design issues and communicating them with your colleagues, it must be done in a sane, pragmatic and goal-oriented way. Also, it shows, that the situation is under control and everybody gets relaxed and can focus on the goal. Next thing is a question of trust. Trust your colleagues but do not trust the code or design. That means to always check the work if it’s doing what is it supposed to do. Avoid blaming and finger pointing and focus on the goal. As a QA, always keep the development priorities in mind. First goal is survival. Focus on the functionality and the shipping date. Second is quality. Players like smooth and bugless games but can excuse some minor hiccups. Also, you can score some bonus points with them when you prioritise fixing issues according to their feedback. Third is efficiency. Always try to improve the development process to ensure you can speed up the iteration of the updates and ensure the best quality to the players. It is important to always ask questions. If you’re not sure how some feature works from the documentation, ask the designers. If something does not work as it should, ask the programmers. Asking questions is one of the most important ways you can inform and educate yourself but also check if the other party knows what they are doing and how. Improving technical skills in the IT industry and gaming industry is a must. Deeper insight into the code, infrastructure and design principles can open a whole new level of perception on how the games work.

So, Stanley, will you share some best practices from F2P QA experience?

Risk prevention is the main goal of the QA department. QA has the best preview about the state of game, and should use this knowledge to balance what is planned for the game, what is expected and what is reality in the current timeline, and communicate this knowledge with project lead about reality and future planning.

Next advice I can give you is communication. I’m saying this again, but it is very important as most of the issues arise from bad communication between people. QA should always bring up problems until they’re resolved. They should ideally do it through official communication channels such as project management software (redmine, jira, etc.) or with people responsible for those particular issues.

Efficiency of work can be greatly improved by quality of tools & game cheats at one's disposal. It is essential for the long run to build, maintain and improve tools that can be used by you but also by not so technically skilled people, so the whole organization can reap benefits from them. In the beginning of the project it is the QA that defines the main features the tools should have. Later in the project it is good to take onboard different departments to pitch in their ideas.

When your game has planned updates, here are some of my best advices. Plan the update early in the week - so you have plenty time for monitoring, tracking and occasional hot fixing. Have an emergency plan, so you can know who to contact for an asap fix in case of a weekend outage for example. Also, be ready to compensate your players for your mistakes. If you can, test a new update publicly - your hardcore players will happily give you their feedback and bug reports. It’s also good to have some analytics and larger statistical sample of performance of software clients and server backend.

The next advice I can give you is to stop chasing quality unicorns. As you can see on this chart, it is important to define the baseline of what is important and what can “slip” away to players. Priority and severity of issues need to be defined (AKA managing risks). Don’t forget that you are also a part of the product team and nobody is happy when the update is delayed because you halted the release for issue that affects 0,001% of players (but this also depends what type of players it affects and in what way).

What are the latest trends in QA and what can we see in the near future?

First trend is automatization. This has been a hot topic in the recent years. Main reason is to make the computers do the repetitive and boring work, so people can focus on the creative side of things. If you go for automatisation, it’s ideal to start with it from the very beginning of the project, so you can eliminate hacks and workarounds, but also build a solid framework for it. Automatization s a long term investment and requires a lot of work and effort to not just build it but also maintain it. However, in the end it really pays off. If you go for it, I recommend if you to have at least one dedicated person for this endeavor.

Then, there’s the holy grail of testing - Artificial Intelligence. A.I. and machine learning have been probably the most frequently used buzzwords in the IT industry lately. A.I. will surely find its way to be used in the video game testing. For instance, game logs (information about player actions) are data rich. In the right format, they can be fed through a machine learning model so the A.I. bots can learn a wide range of play styles and simulate a player-like behaviour. It can even come up with new tactics and strategies as we can see in recent years, for example: Alpha GO, or OpenAI in the game Dota. Through A.I., we can test hundreds or maybe thousands test cases at once and it is great for ever evolving games, where the number of combinations are too big for manual testing. The A.I. is certainly the future of testing. Then, the QA can focus on managing and tinkering with the A.I. to do the repetitive tedious combinatorial work, while focusing on the exploratory testing and laying foundation with precious data for A.I.

If you guys have any further questions for Stanley, feel free to visit our Facebook group called Free to play game developers. See ya there!