-
3881
Benefits of software testing using the Scrum framework. How to avoid the Cargo Cult?
1. Short description of the Scrum framework
Fundamentals, roles, artifacts and values of the Scrum framework have been described in the Scrum Guide, however, there are many other publications that illustrates the framework in a more detailed way. Following the Scrum Guide: “Scrum is not a process, technique, or definitive method. […] The Scrum framework consists of Scrum Teams and their associated roles, events, artifacts, and rules. Each component within the framework serves a specific purpose and is essential to Scrum’s success and usage. The rules of Scrum bind together the roles, events, and artifacts, governing the relationships and interaction between them.” [1]
The Scrum Team consists of a Development Team, a Scrum Master and a Product Owner. The Scrum Master is neither a project manager nor a team leader; he is rather a servant-leader and a facilitator for the Scrum Team. His main responsibilities include:
- Removing barriers between development and the customer/ Product Owner
- Improving the productivity of the Development Team
- Coaching the Development Team and other stakeholders in the scope of understanding and using Scrum
- Facilitating Scrum meetings
The Product Owner is responsible for the Product Backlog. One of his responsibilities is presenting Product Backlog items to the Scrum Team. He ensures common understanding of Product Backlog items by the Scrum Team. He decides on the priorities of the items.
The Product Backlog is a prioritized list of tasks that need to be completed for a product, including features, enhancements, fixes and improvement actions. It should be continuously refined to be as precise as possible, which is the main aim of the Product Backlog Refinement meetings.
There is also a lower level backlog- Sprint Backlog. It consists of Product Backlog items for certain Sprint, which provides a new working functionality for the product- the Sprint Goal. The main goal of Sprint Backlog is visibility of the status of the work that needs to be done in a Sprint. The status is updated during the Daily Scrum.
Scrum defines a couple of solutions to maximize the productivity and regularity, while at the same time minimizing the number of meetings which need to be organized for the Scrum Team.
The main solution is the Sprint. It’s a relatively a short period of time during which the increment of a product is created. The increment is commonly understood as a sum of work done by a team fulfilled Definition of Done criteria, which provides releasable feature.
Sprint Planning meeting help to define the Sprint goal, which is a short description of what the Scrum Team wants to achieve during the particular Sprint. It allows to understand WHY team is building the next increment. The team translates high-level user stories into smaller tasks, which go to the Product Backlog. Sprint goal is achieved by completion of Product Backlog items.
Figure 2 The team’s Scrum board[3]
The next important part of the Scrum framework is the Daily Scrum. It is about 15-minutes long daily meetings. Each team member can share information regarding:
- What did I do yesterday that helped the Development Team meet the Sprint Goal?
- What will I do today to help the Development Team meet the Sprint Goal?
- Do I see any impediment that prevents me or the Development Team from meeting the Sprint Goal?
These questions help to plan the next 24 hours of the Sprint; how- ever, sometimes additional, more detailed questions should be asked. It is a very valuable meeting, which improves knowledge sharing inside a team.
At the end of a Sprint, the increment should be verified by stakeholders. This is achieved during a Review meeting. Then assessment about what has been done during the Sprint is made. Development Team may provide demo of the delivered increment. Based on the feedback during the Review meeting, a Product Owner can adapt Product Backlog for the next Sprint. An item is considered “Done” when the set of rules called Definition of Done is fulfilled.
As continuous improvement is a part of Scrum, feedback about the most recent Sprint is highly desirable. Sprint Retrospective Meetings help to identify and improve weak points of interpersonal relations, processes or tools inside the team. They are an occasion to discuss technical and interpersonal issues and to comment if something went well.
2. How does the testing process work in Scrum and what are the benefits of having a tester in a Scrum Team?
Test management process using the V-model or the Waterfall model is quite different than with Scrum. In traditional methods the leading tester, test manager or test architect is responsible for planning the test strategy, scope, etc. He is leading the team and managing testing tasks. The main disadvantage of these models is that they are not flexible – especially the Waterfall. The next drawback is that SW bugs are detected at the late state of product development. Nonetheless, these methodologies are still in use in some projects and fit well for them.
A Scrum Team has no dedicated test manager, because it should be a self-organized and cross-functional group of people. Cross-functional doesn’t mean that each team member has all competences needed for developing a product end to end. It rather means that all team members are focused on delivering a working product and synergy of their skills from different fields improves the product quality. So, testers in a Scrum Team can be an added value and they can help to improve quality of the final product from the early phase of product life cycle.
One of the key roles of a tester in a Scrum Team is helping with defining user stories, Definition of Done and Acceptance Criteria. If a team is working in ATDD (Acceptance Test Driven Development) methodology, then during the planning and design meeting, at the very beginning of the Sprint, all Scrum Team members should take part in defining of Acceptance Criteria (AC). They are considered as a set of requirements that SW must meet, to be accepted by the customer or the Product Owner. New features are developed and test cases are designed based on these requirements. The most popular template of defining Acceptance Criteria is: “Given… When… Then”. This is the generic form of AC definition: with given context or circumstances, when an action is carried out, then a result is observed. The biggest advantage of this method is simple language which guarantees common understanding of the criteria. There is no split between the testers and the developers in the Scrum Team, however somebody within the development team may have testing skills and then take on the tester’s role. Often testers are in possession of detailed domain knowledge which can significantly improve quality of the created criteria.
Figure 3 Waterfall model[4]
Quick information about the level of quality of the developed software is very important. In the traditional model, where separate development and testing team are working on a product, tests are run after the software is ready. It takes some time before a tester can analyze test results, find the responsible development team and give them feedback. In Scrum Team feedback is much faster. It is possible due to team members’ location but also thanks to test automation and Continuous Integration practice. Tests runs even after each commit to the common code repository. There are several levels of testing under the Continuous Integration process: unit tests, component tests, integration tests and system tests. Results from the execution of them are clearly visible on the dashboard and can be tracked. There is no need to wait for the end of execution of test cases on the previous stage. Tests might be executed simultaneously on all levels. Test cases created by testers are often more complex than those designed by developers as unit tests. Integration test cases might discover bugs in interactions with other system components and interface misalignments. Automation gives quick daily feedback about code quality, but not everything can be automated.
In practice, when a tester raises a bug report, an investigation on the development side needs to be performed. Quality of the bug report has significant impact on the problem resolution time. Relevant information about an observed issue needs to be provided for the developer. For that reason, many organizations introduce a template for bug reports. It forces good practices and improves common understating of the problem on tester’s and developer’s side. Sometimes it is not enough and additional questions need to be asked. When there is communication between two or more teams, there is always some time waste regarding answering the questions, especially when considered teams are located apart from each other. In a Scrum Team this problem is significantly reduced. Due to the shared location of the team, all misunderstandings regarding test environment and scenario can be resolved immediately. Moreover, delivered corrections might be tested rapidly, so feedback to the developer is given straight away. It significantly improves the velocity of SW developing process and reduces costs for bug fixing. I’ve encountered a quote, which gives a sense of test automation process, regarding the team where it is performed: “Also common is the test automation group zombie. This zombie is the practice of assigning test automation to a dedicated team of test automators. The appeal is that we can keep developers focused on writing new code instead of writing and maintaining automated tests. The danger is that test automation inevitably lags development, so feedback from testing is delayed in a way that significantly reduces its value.”[5]
The next great benefit of the presence of a tester in a Scrum Team is his knowledge of software testing and strategies. Based on testing training like ISTQB and experience, testers can train the rest of the team in the quality assurance process. Developers often don’t have detailed knowledge about system operations or hardware. Testers may bring this information to the team and thanks to that quality of the final product is better.
While it is an advantage, it might be also a drawback. One of the biggest drawback is that developers and testers make the same assumptions in their work. They sometimes have the same attitude to challenges in daily tasks. It is not a problem when they need to order a pizza for a lunch, but it is when they must design code and test cases to check this code. In software testing it is expected from testers that they are independent and objective. It isn’t so simple when they are working in a Development Team.
They may unconsciously try to fit to the developer’s vision of the product. For that reason, some test scenarios might not be covered due to influence of developers, or even not recognized. Testers may also affect decisions about software design within a team. For ex- ample, they can suggest selection of a simpler algorithm regarding a procedure, because of the smaller testing effort which must be spent on test automation. Sometimes it could be dictated by environmental issues, like tool availability. As the next disadvantage of testing without dedicated team we can identify the additional effort for synchronization of testing tasks across Scrum Teams. In this case some additional meeting might be required for assurance, that there is no gap in test coverage and teams are synchronized.
3. The Spotify model as a next step in managing the workflow?
From time to time new solutions for making project management more effective, or improving organizational structure of the company are emerging. Sometimes they are entirely new, sometimes just modification of older solutions. One of them is the so-called Spotify model. It was developed in the Swedish entertainment company Spotify Technology SA. As it was said by Marcin Floryan, one of the chapter leaders in Spotify, the Spotify model should be considered rather as an organizational snapshot of the engineering culture, than a model as such. But where does this idea come from? How has it evolved into the today’s form? At the beginning, Spotify employees used the Scrum methodology to manage their workflow, but they have abandoned this framework.
Figure 4 Spotify organizational structure [6]
Spotify adapts Agile for their needs and, as it is visible in the Figure 4 , some new terms have been introduced regarding organization of teams. All Scrum practices became optional. Scrum Master is transformed in this organization into an Agile Coach to indicate his role as a servant leader. A Scrum Team became a Squad, also cross-functional and self-organized, but a more independent team, with end to end responsibility for their part of the product. The Product Owner prioritizes their work like how it was done in Scrum.
These changes shall entail more autonomy for teams, which results in better motivation of employees and improves teams’ velocity. Because of such set-up, it is important to communicate the common vision of the product and ensure that high level of alignment between Squads is kept. Otherwise each Squad will go in a different direction and ultimately organization will fail. This is the role of a Tribe leader in cooperation with Product Owner. The Tribe is a set of Squads and is applied in larger organizations.
Independency gives an opportunity to use multiple tools and practices across the Tribe. Each Squad within the Tribe can develop different methods and tools in their work. Then they may discuss it with other Squads and propagate their solutions across the Tribe. It is kind of a lightweight process of examining and selection of optimal tools and practices.
There are also mysterious names for other groups called Chapter and Guild. The Chapter should be considered as a group of people having responsibilities from a certain area, for example Developers or Testers. They meet to synchronize and discuss challenges regarding their competence areas. The Guild is a kind of a virtual team or task force. It brings together people from different Squads and Chapters, and their aim is to provide a solution for specific problem or deliver some improvement. It is one of the mechanisms of information flow between Squads working on a bigger feature together. One of the biggest problems in an autonomous team is knowledge sharing with other teams. It is a big issue especially, while these teams are working on one big feature. Bringing them together in a Guild partially solves this problem.
As it was mentioned before, there is a kind of independency and it also applies to team composition. If there is are testers, they should have specific knowledge and skills.
Test automation skills are essential for fast delivery of test results. As continuous testing is considered fundamental for this “model”, manual testing is restricted to exploratory tests. Working in close cooperation with developers, knowledge about acceptance test driven development and white box testing is also more than welcome. As frequent feedback is expected, high level of communication skills is essential for good cooperation within a team. High autonomy of a team must be balanced by a result-oriented mindset and high discipline in organization of their own work.
All these solutions are continuously changing and should be tuned to current business needs. As a matter of fact, even Spotify abandoned this model and isn’t using it anymore in such form.
4. What is the Cargo Cult and why may it have an adverse impact on testers and their teams?
The Cargo Cult began in Fiji in the 19th and early 20th century when local tribal societies met with western people. Then the same phenomenon occurred in Papua New Guinea and other islands of Melanesia. The clearest examples, in my opinion, were described by American soldiers during World War II. As time went by, U.S. Army moved across the Pacific. As more and more troops were going ahead, they had to be supplied. Once Americans have taken island of strategic importance, they started to build a landing strip. Thanks to that, aircrafts could land on the island and deliver supplies. Then another building structures and when the natives first saw all the goods brought by the foreigners, they were shocked and impressed by Soldiers sometimes shared their canned food or other small items with the locals. These gifts were of considerable value to them.
This process was observed by the natives. They tought that all this cargo was sent by gods and it only took acting like foreigners to get it. So, they started building a makeshift landing stripes, fake air- crafts, antennas made of bamboo, etc. They also noticed soldiers’ uniforms, various signs on them and tried to copy them. The drill was considered by Melanesians to be a kind of a ritual, and they also tried to imitate that. Local people blindly emulated incomprehensible behaviors and believed that they could succeed.
We can observe similar behavior in many aspects of our life. From daily routines to advanced scientific research. Even Nobel’s Prize laureate Richard P. Feynman described this phenomenon regarding research techniques.[7] We still expect that if some mechanisms which are working in other organizations will be imported to our organization, we will succeed. Going back to it is for example deployment of a new bug tracking tool, because it is doing well in other companies. After some time, that there is no place for additional information which we are using for generating our statistics etc. We must realize that without deep analysis of our working environment and comprehension of the new way of working, we won’t succeed. Many companies tried to deploy Agile or Scrum and failed. We mainly identify deployment as the next factor of success in scrum an entire organization. For example: it is not possible to continuously
deliver software without continuous testing, so all work using the same methodology. As it was mentioned before, regarding the Spotify model, sometimes it is not necessary and such setup may work in some way, however it is not We are hearing from time to time that some company failed to deploy. There are several reasons why it may happen.
Breaching fundamental principles of Scrum, we cannot expect success at the end, especially with an inexperienced team. For example, if a few past sprints had different length each, it is hard to estimate tasks for the next one, because we do not have a common benchmark. If Scrum meetings are not facilitated by the Scrum Master and other team members, it is destructive for the whole team. Some- times it is because of lack of the Scrum Master’s confidence and respect within the team. Nobody then knows what is the progress and what is going on in the Sprint. On the other hand, if a Scrum Master tries to micromanage the team, then there is no Scrum.
“Stop trying to borrow wisdom and think for yourself. Face your difficulties and think and think and think and solve your problems yourself. Suffering and difficulties provide opportunities to become better. Success is never giving up.” It was said by Taiichi Ohno. He was a Japanese industrial engineer and businessman. Author of the “Toyota Production System: Beyond Large-Scale Production”. He is the father of the Toyota Production System, adapted and transformed in the United States to a Lean Manufacturing. It is not a plug and play solution and Motorola and BlackBerry found out about it. It is hard to start with Scrum without relevant people. Both engineers and managers must be ready for changes.
5. Why testing in the Scrum methodology is not relevant for some projects?
Scrum is a framework for developing complex products, which can- not be planned and controlled in a traditional way, or it is not effective. It suits fast-changing projects well because it allows it quick response to the customer’s needs. Change of scope or requirements is nothing new and each task in the backlog can be re-prioritized.
I would point out two main reasons not to use Scrum: people and project type. If people are against the changes or do not have comprehensive knowledge about Scrum, it is not a good idea to introduce a new framework. Team members must understand Scrum principles and apply them in daily work. Due to close cooperation inside a team, people should be open-minded, communicative, and have relevant technical skills. If we already have the relevant people, we need to consider if the project is worth the changes. So, if the scope of the project is well known and most of the expected changes are almost certain, then there is no need to use the Scrum framework. There is no need to introduce Scrum if your organization is dealing with repetitive projects.
Figure 5 Agile application in the context of project type [8]
There is also a technical aspect. As Continuous Integrations is an important part of the software development process, there must be an environment allowing to make this happen. Before the first sprint starts, relevant tools and hardware must be provided. If there is no possibility to meet this requirement or there is lack of knowledge how to set up the environment, the project will fail immediately.
As the level of uncertainty in the project is rising, there is a higher recommendation to use one of the Agile techniques. It allows better adaptation to the changing environment.
6. Summary
Software testing methodologies have evolved in the past years and continue to evolve further. Good management of the testing process is crucial to provide high quality features. It is hard to describe what an important and resource-consuming activity it is. Good cooperation across the Development Team and continuous synchronization with the Product Owner helps to keep everything up to date and respond to the customer’s needs. Testing takes more than half development time. Lone of the most significant people in the IT world be a kind of description of this process: “We have as many testers as we have developers. And testers spend all their time testing, and developers spend half their time testing. We’re more of a testing, a quality software organization than we’re a software organization.”[9] These words have been spoken by Bill Gates regarding the testing practices in the Microsoft.
Scrum relies on feedback, and testers are the most important source of feedback. It is very important to provide high quality test cases and for that reason they should also be under the review process.
Some may say that developers have unit tests which are sufficient, and testers are not an added value in a development team. Follow- ing the V-Model, there are test levels corresponding to development stages and unit tests do not guarantee a high quality of code. Testers have a considerable influence on many aspects of the software development life cycle. Shortening software quality feedback time and participating actively in relevant team activities has a significant role in an organization. For that reason, test environment and methodology should be continuously improved inside organization, considering all technical and project-related aspects. As IT industry is changing fast and many new solutions are presented, it is not a good idea to introduce each of them in every organization. Any change must be preceded by a deep analysis of the organizational and engineering culture within the company, and then after a positive verification, changes may be incorporated.
References
[1] The Scrum Guide ©2017 Ken Schwaber and Jeff Sutherland
[2] Testing in Scrum: A Guide for Software Quality Assurance in the Agile World Tilo Linz Rocky Nook, Inc., 28 mar 2014 – 240
[3] www.inflectra.com
[4] http://www.softwaretestinghelp.com/what-is-sdlc-waterfall-model/
[5] Dale Emery
[6] https://spotifylabscom.files.wordpress.com/2014/03/spotify- engineering-culture-part1.jpeg
[7] http://calteches.library.caltech.edu/51/2/CargoCult.htm
[8] Strategic Management and Organisational Dynamics: The challenge of complexity to ways of thinking about organizations
[9] Bill Gates
