Tag Archives: testing

Software Development Models: Waterfall

Waterfall model

The waterfall model originated in manufacturing and construction where changes are costly and investment in design of the production line is often much less than potential loss if the actual production fails. It is based on idea that planning and design costs are much lower than those used in the actual production.

## Software development life-cycle (SDLC)

The software waterfall model often uses some variation of following phases:

Requirements specification (Requirements Analysis) resulting in Requirements Document
Software design resulting in Software Design (SD) document
Implementation resulting in the actual software
Integration
Testing (or Validation)
Deployment (or Installation)
Maintenance
Continue reading →

How can functional test coverage help rescue waterfall projects?

2 Replies

A typical waterfall project has well defined phases that go from the idea conception until putting the solution in production, being the most typical of them:

Requirements gathering
Design of the solution / Architecture
Implementation
Testing phase (including all kinds of tests)
User Acceptance Test
Go Live (put in production)
Post Go-Live support

It’s very common in big companies that there’s a functional team responsible for each phase. Thus, we get:

Business Analysts to gather requirements
Architects, Systems Engineers or Software Analysts to design the solutions
Programmers or coders to implement it
Quality Testers or Quality Assurance engineers to check its quality
The customer to test the delivered solution

However, one of the big problems from this approach is that usually these teams work in silos. The right hand doesn’t know what the left hand is doing and this causes inefficiencies. To make it worse, sometimes the way to try to reduce silos is through an immense increase in bureaucracy, forcing teams to communicate through documentation.
Continue reading →

Code and Test Coverage

5 Replies

When moving from the traditional waterfall process with manual testing towards BDD with automated execution of scenarios, one of the often asked questions is “how do you measure your test coverage?”. What it really wants to say is “we have high test coverage with manual testing, can you guarantee the same or more with the BDD?”. From there on usually comes the suggestion to transfer one-to-one manual tests to the BDD format. That’s not a good idea since the result would be less than adequate transformation to the different format without any tangible change. BDD is not only about automation but about the way we construct and deliver software.

Code Coverage Analysis

Main usages of code coverage analysis is to:

Find areas of software under test that were not exercised by a set of test cases.
Help in creation of additional test cases to increase the coverage.
Provide quantitative measure of test coverage as indirect measure of quality.
Identify redundant test cases that do not increase coverage
Identify parts of the code that is not in use

Continue reading →

Quality Assurance is not Quality Checking

4 Replies

Since years, many software companies have their Quality department, which in many cases implies that they have the responsibility to assess the quality of the releases. This department is typically called QA department (standing for Quality Assurance), but I rather think that they are doing much more of checking releases than to assure the quality and I see a huge difference these two concepts, that is, between a QA engineer and a QC engineer:

QA stands for Quality Assurance, therefore an engineer that ensures the quality. Every time a new release is being built, the QA engineer ensures that the quality is built in it. The QA is responsible for improving the processes and participates actively in the creation of the testing tools.

QC stands for Quality Checking or Quality Control. This engineer typically waits for the release to be finished in order to check if the quality is built there. A Quality Checker typically creates a list of test cases based on the requirements themselves and puts them in a tool to plan each of the BTCs. Later he executes all the test cases, in most cases manually and tracks their execution.

Continue reading →

Black-box vs White-box Testing

9 Replies

Testing shows the presence, not the absence of bugs. Edsger W. Dijkstra

Two common types of testing are black-box and white-box testing. Both can drive or be driven by development.

Black-box testing

Black-box testing (also known as functional testing) treats software under test as a black-box without knowing its internals. Tests are using software interfaces and trying to ensure that they work as expected. As long as functionality of interfaces remains unchanged, tests should pass even if internals are changed. Tester is aware of what the program should do but does not have the knowledge of how it does it. Black-box testing is most commonly used type of testing in traditional organizations that have testers as a separate department, especially when they are not proficient in coding and have difficulties to understand the code. It provides external perspective of the software under test.
Continue reading →

Behavior Driven Development (BDD): Value Through Collaboration (Part 4: Automation)

2 Replies

This post is part of the “Behavior Driven Development (BDD): Value Through Collaboration” series.

Automation

The first 3 posts explored the process to define project requirements as behaviors. Once you have mastered writing BDD stories you are ready to start the journey towards the automation of scenarios.

Execution of BDD scenarios can be accomplished using many different frameworks. Some of them are JBehave (Java), Cucumber (Ruby), RSpec (Ruby), SpecFlow (.Net), and Jasmine (JavaScript). This post will use examples from JBehave. With slight modifications, they can be applied to any other framework. The above-mentioned frameworks provide installation instructions and documentation on how to write code that supports BDD. This article focuses mostly on the best practice and ways to avoid some of the common pitfalls. For detailed installation instructions and documentation, please consult the website of the selected framework.

Decision on which framework to choose should be based on which technology your team is most comfortable with, as well as the support it provides for the implementation of the phases described below. Next post in this series will provide comparison of different BDD frameworks.

Successful implementation of the BDD automation should be done in three phases.

Create library of normalized steps.
Combine steps into composites.
Empower scenarios with examples tables.

You should start with the first phase. As soon as enough steps have been created to support the first scenario, you can start with execution of the other two phases. From there on, work on all three phases should be done in parallel.

Continue reading →

Behavior Driven Development (BDD): Value Through Collaboration (Part 3: Scenarios)

3 Replies

This post is part of the “Behavior Driven Development (BDD): Value Through Collaboration” series.

Scenarios

The previous post explained narrative as a format to describe feature or requirement. Even though narratives can be written by anyone, it is often the result of conversations between the product owner or the business analyst and business stakeholder.

Scenarios describe interactions between user roles and the system. They are written in plain language with minimal technical details so that all stakeholders (customer, developers, testers, designers, marketing managers, etc) can have a common base for use in discussions, development, and testing.

Scenarios are the acceptance criteria of the narrative. They represent the definition of done. Once all scenarios have been implemented, the story is considered finished. Scenarios can be written by anyone, with testers leading the effort.

The whole process should be iterative within the sprint; as the development of the BDD story progresses, new scenarios can be written to cover cases not thought of before. The initial set of scenarios should cover the “happy path”. Alternative paths should be added progressively during the duration of the sprint.

Continue reading →

Behavior Driven Development (BDD): Value Through Collaboration (Part 2: Narrative)

Narrative

“User stories are a promise for a conversation” (Ron Jeffries)

A BDD story consists of a narrative and one or more scenarios. A narrative is a short, simple description of a feature told from the perspective of a person or role that requires the new functionality. The intention of the narrative is NOT to provide a complete description of what is to be developed but to provide a basis for communication between all interested parties (business, analysts, developers, testers, etc.) The narrative shifts the focus from writing features to discussing them.

Even though it is usually very short, it tries to answer three basic questions that are often overlooked in traditional requirements. What is the benefit or value that should be produced (In order to)? Who needs it (As a)? And what is a feature or goal (I want to)? With those questions answered, the team can start defining the best solution in collaboration with the stakeholders. The narrative is further defined through scenarios that provide a definition of done, and acceptance criteria that confirm that narrative that was developed fulfills expectations.

Continue reading →

This post is part of the “Behavior Driven Development (BDD): Value Through Collaboration” series.

Introduction

The goal of software projects is to deliver value to stakeholders. Even though that might sound an obvious statement, it can be easily forgotten in traditional Waterfall projects. The very nature of the Waterfall process fosters the creation of different departments whose job is to receive work from the previous stage, produce something, and pass it on.

A simplified Waterfall model would be:

Requirements specification resulting in requirements document
Design resulting in software architecture document
Development resulting in actual software
Integration
Testing
Installation
Maintenance

Each of these phases tends to have separate teams and departments. BAs work with requirements, architects write design documents, developers code, integration engineers integrate, testers test, and someone installs the software. Each of these departments is waiting for others to produce something, to start working on their artifacts and, once finished, to pass it on to someone else. The value for the stakeholders is, in many cases, unknown or forgotten. Most people involved in such projects are concerned about “doing their part” and throw it “over the wall” to those coming next. Moreover, artifacts produced by one team are often not used or understood by another.

Continue reading →