Hubbry Logo
search
logo

Software testing controversies

logo
Community Hub0 Subscribers
Write something...
Be the first to start a discussion here.
Be the first to start a discussion here.
See all
Software testing controversies

There is considerable variety among software testing writers and consultants about what constitutes responsible software testing. Proponents of a context-driven approach consider much of the writing about software testing to be doctrine, while others believe this contradicts the IEEE 829 documentation standard.

Proponents of the context-driven approach believe that there are no best practices of testing, but rather that testing is a set of skills that allow the tester to select or invent testing practices to suit each unique situation. James Marcus Bach wrote "...there is no practice that is better than all other possible practices, regardless of the context." However, some testing practitioners do not see an issue with the concept of "best practices" and do not believe that term implies that a practice is universally applicable.

Around 1990, a new style of writing about testing began to challenge previous approaches. The seminal work in this area is often regarded as Testing Computer Software by Cem Kaner. Instead of the assumption that testers have full access to source code and complete specifications, these writers, including Kaner and James Bach, argued that testers must learn to work under conditions of uncertainty and constant change. Meanwhile, an opposing trend toward process "maturity" also gained ground, in the form of the Capability Maturity Model. The agile testing movement, which includes but is not limited to testing methods practiced on agile development projects, is popular mainly in commercial circles, whereas CMM was embraced by government and military software providers.

However, saying that "maturity models" like CMM gained ground against or opposing Agile testing may not be right. Agile movement is a 'way of working', while CMM is a process improvement idea.

But another point of view must be considered: the operational culture of an organization. While it may be true that testers must have an ability to work in a world of uncertainty, it is also true that their flexibility must have direction. In many cases test cultures are self-directed and as a result fruitless, unproductive results can ensue. Furthermore, providing positive evidence of defects may either indicate that you have found the tip of a much larger problem, or that you have exhausted all possibilities. A framework is a test of Testing. It provides a boundary that can measure (validate) the capacity of our work. Both sides have continued to debate the merits of their approaches, but the true measure lies in assessing delivery quality. Testing systematically without broader focus can be ineffective, while finding numerous errors does not necessarily mean Agile methods were the cause; it may simply indicate poor initial work.

Exploratory testing means simultaneous test design and test execution with an emphasis on learning. Scripted testing means that learning and test design happen prior to test execution, and quite often the learning has to be done again during test execution. Exploratory testing is very common, but in most writing and training about testing it is barely mentioned and generally misunderstood. Some writers consider it a primary and essential practice. Structured exploratory testing is a compromise when the testers are familiar with the software. A vague test plan, known as a test charter, is written up, describing what functionalities need to be tested but not how, allowing the individual testers to choose the method and steps of testing.

There are two main disadvantages associated with a primarily exploratory testing approach. The first is that there is no opportunity to prevent defects, which can happen when the designing of tests in advance serves as a form of structured static testing that often reveals problems in system requirements and design. The second is that, even with test charters, demonstrating test coverage and achieving repeatability of tests using a purely exploratory testing approach is difficult. For this reason, a blended approach of scripted and exploratory testing is often used to reap the benefits while mitigating each approach's disadvantages.

Some writers believe that test automation is so expensive relative to its value that it should be used sparingly. Others, such as advocates of agile development, recommend automating 100% of all tests. A challenge with automation is that automated testing requires automated test oracles (an oracle is a mechanism or principle by which a problem in the software can be recognized). Such tools have value in load testing software (by signing on to an application with hundreds or thousands of instances simultaneously), or in checking for intermittent errors in software.

See all
User Avatar
No comments yet.