Saturday, January 15, 2011

Missing Views of Software Quality

As some might know, I've recently defended my thesis on describing how to improve models to assess the quality of software. However, my definition of quality was limited and precise (how can you improve something that is vague). Since, I'm currently job hunting, I wanted to know what are the definitions of quality that are in use in industry. I did what everyone does and I Googled Software quality and found the article on Wikipedia. Please read it. It is really informative. Why do I say that it is informative? Because we must learn from our mistakes. Now, I'll only discuss what is lacking in terms of views on software.

Quality Views
The general definition of software quality is pretty much limited to a developer's perspective (of course there is a subsection on the user perspective). There are multiple views of quality as mentioned by Garvin of the Harvard Business Review, and this non-software notion of quality was adapted to software in Software Engineering: Theory and Practice by Kitchenham and Pfleeger and Software Engineering by Pressman. These views are not exclusive
  1. An obviously missing view is value-based quality, or how much would someone pay for the software. When someone buying software would like to know if it provides good value for its cost. Yet, even without being able to properly evaluate its value, cost is often used as a surrogate of quality. One implicitly expects that the market will adjust prices to reflect their value (or quality). For example, the top athletes on a sport's team will tend to be the best paid.
  2. The transcendental view is also ignored. In this view, you would compare software to the perfect (impossible to achieve) software system. This type of judgment is actually relatively common as most decisions are based on comparisons. For example, to choose a framework, you could investigate existing frameworks and mentally construct an idealized system with all possible features. However, when deciding which framework is best, it would only provide a subset of all these features.
  3. The user is mentioned, not as a view, but rather as an introduction to discuss usability. The user view is much more global than usability: it assesses the fitness of use. Whereas usability corresponds to the ease of use, fitness corresponds to how well a system can respond to user needs (functional or non-functional). A user would not appreciate software that does not provide features that correspond to his needs.
  4. The manufacturing views are well covered: software should conform to a specification. This is a view that is aptly named manufacturing because it evaluates whether or not variations exists in the manufacturing process.
  5. The product view judges the quality of a system by the quality of its components. A system would be judged on the quality of the code, its design and its architecture.
When building software, it is always interesting to take into account how others perceive quality to manage their expectations. In fact, in Software Quality: The Elusive Target, Kitchenham and Pfleeger present the different which stakeholders are interested in which view: production teams are interested in the manufacturing view, researchers in the product view, and a marketing team in the user view. When a new system is developed, these views can be harmonised; however, as development continues, there can be divergences. That is where the value-view should be used to resolve disputes.


Generally speaking, when talking about quality, there are different parts of organisations with various views. We will describe them now.

A SEPG (software engineering process group) exists in certain organisations. Their focus in tend to focus on improving the quality of a software product by improving its process. This corresponds clearly to the manufacturing view. A dysfunctional process will likely produce more bugs (or specification deviations). The solutions proposed are typically addressed by standardisation (eg., adopting CMMI level), or by adopting a development methodology.

Marketing/sales teams are more interested in the user-view. They want to make sure that users are satisfied with the product.

Researchers are interested in the fundamental characteristics of the software. Why? They do not have access to the context necessary to properly analyse anything beyond the product.  This view is also shared by development teams who want to deal with nice looking code.

Managers and clients are focused on value. Although improvements focused on the different views will improve quality, but ultimately, the person paying for the software has the final word.

That was a quick rant, I'll have to amend the Wikipedia article some time. I'll probably amend this entry as well.