Software quality assurance in practice

Introduction

Every software developer has some idea of a good quality project (bug free, fast, easy to adapt, readable code base etc.). Some of the developers also have an idea how to achieve it (reviews, TDD, proper design, proper testing etc.). All these practices need to be collected and followed is a way.

Software quality assurance is something that is on the side of any development, regardless of the development process. Some way of quality assurance is used by all big companies. The goal of software quality assurance is to have a clear picture about the quality and try to reach and maintain a predefined level of quality.

Theory of software quality assurance

Most of the companies have a dedicated quality assurance team or at least one dedicated person who is responsible for the quality of the projects. That means it should be someone who is not involved in the project, who makes no development on that, this person rather makes sure that the development team does everything in the correct way. Quality assurance is involved in each and every step of the development process: planning and tracking of the project, requirements engineering, software design, implementation and testing.

The very first point is to set up rules to be followed by the whole development team: how should a requirement be documented, which level of software design is required in which situations, who and how should review the work, which coding guideline is to be followed etc.

Once it is done the role of the quality team is to make sure that during the project everyone is following the predefined rules. To be able to achieve it the activities should be documented in a clear way. For example, if someone did a review of a piece of code it should be documented in a way, that it can be always proved later that the given version of the given code has been reviewed.

There are predefined software quality standards and frameworks, like the different versions of SPICE or CMMI, which have several predefined rules, but every project and organization is free to set up their own ruleset.

Software quality assurance in practice

Setup rules and performance indicators

In practice the very first step to set up the rules for the development: coding guidelines, testing guidelines, responsibility matrices etc.

Some of them should be introduced right at the beginning of the project. But introducing all of them at the beginning can really go against performance and the success of the project. So some of these rules can be introduced in a later stage of the project.

Some of these rules are pretty binary: either they are followed or not. Like the rules in the coding guideline: “don’t use variable names longer than 15 character”. It is very easy to decide if this rule is followed or not.

There are other cases where the answer is not so clear. A good example is test coverage. Most of the projects require unit tests, but it is not needed that each and every line is covered by these tests. In these cases so called key performance indicators (KPI) should be set up. This thing should be quantified and measurable. For example for code coverage there are multiple possible KPI’s: line coverage, branch coverage, etc. It should be decided what is the official measurement method in the project.

These rules can be relevant for any step of the development (planning, requirements, design, coding, testing etc.).

Measure your indicators

Once we know the rules to be followed we have to figure out a way to measure them. The best way is if we can automate these measurements by some tools and integrate them to the continuous integration system of the project. So that you will have continuous feedback about the state of the project, you can follow how its quality changes. It is good practice to introduce these KPIs to the whole team. Most of the tools which can do the measurement on different KPIs are easy to integrate, in other cases you can use some scripts to do the job.

Setup goals

Once the KPIs are measured and known by the team setup some goals. Like “the code coverage should reach 80%” or the number of compiler warnings should be 0, every component has to be linked to requirements etc. Let the team know these goals and let them work on them. Give regular feedback to the team about the distance from the goals and the achieved changes. It can be done by daily report emails. All these goals need to be documented in a clear way.

The most typical goals are the following:

• Design has to be linked to requirements
• Code needs to be linked to design
• Positive review should be done before merging
• Test should be linked to requirements
• Coding guidelines should be followed. (for that a proper coding guideline is required)
• Code should be covered by tests
• Memory assumption/runtime should be in a dedicated range
• The piece of change (commit) should be connected to a ticket in the ticket system
• Tickets in the ticket system should be connected to requirements

Setup gates

Finally, you can set up some built-in gates to the CI system. That means it doesn’t allow to merge code which is violating any of the rules, like: contains compiler warnings, failing some checks, not covered by tests, has failing unit tests etc. This can reduce the speed of development, but increase its quality.

In general too strict rules can be against productivity, pay attention! You should always pass the rules to the project needs. So most likely the expected quality in case of a mobile game will be much lower than in the case of an aeroplane software.

Summary

Quality assurance is nothing complicated. I tried to describe it in a practical manner and not in a principled way. But one thing is important: you have to be strict, the rules which have been set up, need to be followed.